Skip to content

Commit

Permalink
Merge pull request #3 from akrobi/mydev
Browse files Browse the repository at this point in the history
pull changes from mydev branch to main branch
  • Loading branch information
akrobi authored Oct 6, 2022
2 parents 32be638 + 1f04af2 commit 4db8f6e
Show file tree
Hide file tree
Showing 5 changed files with 60 additions and 69 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ jobs:
- name: Run a one-line script
run: echo Hello, world!

# Runs a set of commands using the runners shell
- name: Run a multi-line script
# Runs testing: checks to see if the run clears the unittests
- name: run testing
run: |
cd testing
python -m unittest discover
90 changes: 58 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,78 @@

![Logo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/th5xamgrr6se0x5ro4g6.png)

![Logo](./img/logo.png)

# Speech-to-Text Data Collection

A tool that can be deployed to process posting and receiving text and audio files from and into a data lake, apply transformation in a distributed manner, and load it into a warehouse in a suitable format to train a speech-to-text model.

## Data Capture Pipeline

![Pipeline Diagram](https://via.placeholder.com/468x300?text=App+Screenshot+Here)
![Pipeline Diagram](./img/pipelineDiagram.png)

## Directory Structure

```bash
├── .github/workflows
│ └── main.yml
.
├── airflow
│ ├── file1.ext
│ └── file2.ext
│ ├── dags
│ │ ├── extract_load.py
│ │ └── scripts
│ │ ├── dataloader.py
│ │ ├── db_connection.py
│ │ ├── __init__.py
│ │ └── schema
│ │ └── amharicnews.sql
│ ├── data
│ │ └── AmharicNewsDataset.csv
│ ├── docker-compose.yaml
│ └── logs
│ └── scheduler
│ └── latest -> /opt/airflow/logs/scheduler/2022-10-05
├── backend
│ ├── file3.ext
│ ├── file4.ext
│ └── file5.ext
│ └── dummy.txt
├── frontend
│ ├── file6.ext
│ └── file7.ext
├── logging
│ ├── file8.ext
│ ├── file9.ext
│ └── file10.ext
├── screenshots
│ ├── file11.ext
│ └── file12.ext
├── testing
│ ├── file13.ext
│ ├── file14.ext
│ └── file15.ext
├── .gitignore
├── EDA.ipynb
│ ├── dummy.txt
│ ├── frontend
│ │ ├── package.json
│ │ ├── package-lock.json
│ │ ├── public
│ │ │ ├── favicon.ico
│ │ │ ├── index.html
│ │ │ ├── logo192.png
│ │ │ ├── logo512.png
│ │ │ ├── manifest.json
│ │ │ └── robots.txt
│ │ ├── README.md
│ │ └── src
│ │ ├── App.css
│ │ ├── App.js
│ │ ├── App.test.js
│ │ ├── index.css
│ │ ├── index.js
│ │ ├── logo.svg
│ │ ├── reportWebVitals.js
│ │ └── setupTests.js
│ └── proto.png
├── img
│ ├── logo.png
│ └── pipelineDiagram.png
├── LICENSE
├── logging
│ └── dummy.txt
├── notebook
│ └── Amharic_news_Classification.ipynb
├── README.md
└── requirements.txt
├── requirements.txt
├── screenshots
│ ├── airflowscreenshoot.png
│ └── design diagram.png
└── testing
├── dummy.txt
└── test_dataloading.py

17 directories, 39 files

```

## Run Locally

Clone the project
Expand All @@ -68,12 +99,10 @@ Set up pipeline
python3 setup.py
```


## Screenshots

![App Screenshot](https://via.placeholder.com/468x300?text=App+Screenshot+Here)


## Authors

- [Haylemicheal](https://github.com/Haylemicheal)
Expand All @@ -83,6 +112,3 @@ Set up pipeline
- [Patrick](https://github.com/prubayita)
- [Wangui](https://github.com/akrobi)
- [Yonas](https://github.com/yonamg)



Binary file added img/logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added img/pipelineDiagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
35 changes: 0 additions & 35 deletions testing.ipynb

This file was deleted.

0 comments on commit 4db8f6e

Please sign in to comment.