Week: Weeks 1-4, 7/20-8/14
Overview of Things Learned
- Web Scraping using BeautifulSoup
- Data Pre-Processing (Data Cleaning, Data Transformation, Data Reduction)
- Bert/Simple Transformers (Ideology and coding implementation)
Technical Area
- Data analysis with Python
- Web Scraping with Python (BeautifulSoup)
- Using Bert
Tools
- Github
- Google Collab
- Bert
- Asana
Soft Skills
- Communicating with a team to work on a project
Achievement Highlights
- Wrote an analysis on the cons of CodeChef to be used as data for the machine learning model
- Used BeautifulSoup to scrap data from Codecademy
- Understood BERT and learned how to use BERT
Meetings Attended
- Attended most meetings or watched the recording if I couldn’t make it
Goals for the Upcoming Week
- Make a machine learning algorithm with the right hyperparameters to best fit the data
Detailed Statement of Completed Tasks
- With a small team, we analyzed that CodeChef wasn’t a suitable website to collect data from for our machine learning model
- Used BeautifulSoup to web scrape from the code academy forums, compiled the forum title, upvotes, tags, and comments, and store the data in a CSV
- Used data pre-processing techniques like tokenization to get the data ready to train a machine learning algorithm