Scrape All The Things
Recently I’ve got into quite a few scraping projects to turn HTML into structured data. I’ve been using Scrapy, a Python library. The project posted above grabs finance data from finviz.com and yahoo.com, neatly storing it into .csv files for doing some data science on it.
There are fantastic resources to use Scrapy, and it makes grabbing large amounts of data possible and even fun once you get the hang of it.