Web scraping, database and related analytics

Dr. Tirthajyoti Sarkar (You can connect with me on LinkedIn)

---

Requirements

• Python 3.5+
• NumPy ($ pip install numpy)
• Pandas ($ pip install pandas)
• requests ($ pip install requests)
• BeautifulSoup4 ($ pip install beautifulsoup4)
• MatplotLib ($ pip install matplotlib)

---

My new book on Data wrangling with Python

---

What type of Notebooks are here?

• Web scraping and related analytics using Python tools
• Fundamentals of Regular expressions (Regex)
• Application of urllib
• Application of BeautifulSoup for HTML parsing
• Application of ElementTree for XML parsing
• Application of Python json library for JSON parsing
• Application of Python sqlite library (building a personal movie database)

---

How to design your own mini-IMDB movie database by scraping web?

---

Check out this article I wrote on Medium about this topic

---

How to scrape data from CIA website (this is harmless, I promise) about simple facts on various nations?

Check out this article I wrote on Medium about this topic

---

How to build a Yelp crawler which can generate interesting word cloud based on a particular city's food cuisine and taste?

---

How to crawl the Project Gutenberg portal and download 100 most popular books automatically?

---

How to use a free API to download basic information about countries around the world and build a database?