Overview of things learned:
- Technical Area:
- Learned how to use web scraping tools like BeautifulSoup and Selenium
- Learned about cleaning data as well as data pre-processing
- Understood how to store data in a CSV or JSON file
- Tools:
- BeautifulSoup
- Jupyter Notebooks
- Selenium
- VSCode
- Soft Skills:
- Gained an understanding of common bugs encountered while web scraping and learned how to debug my code faster using online resources
- Enhanced my knowledge about HTML, CSS, and XPath
Achievements:
- Successfully scraped data from the Kubernetes Community Forums
- Successfully cleaned and pre-processed the scraped data
- Stored the data into a DataFrame using Pandas and created a .csv file
- Gained a basic understanding of cosine similarity
Tasks:
- Picked a community forum from DiscourseHub: Kubernetes
- Scraped the data from the forum and stored it in a .csv file
- Performed basic cleaning of the data
- Performed some basic EDA on the data