Scraper for the Bureua of Transportation Statistics website which runs on a APS.net framework, likely applicable to other parts of the site
This is my first attempt at scraping a website outside of class, first time using scrapy, and first time working with an APS.net framework, so it probably isn't to pretty, but learned a lot building it over the last couple of hours and thankful to have it for my DS4A project.
libraries:
- scrapy
- pandas
Bureau Website: https://transtats.bts.gov/ONTIME/OriginDestination.aspx
APS.net framweork scraping guide: https://blog.scrapinghub.com/2016/04/20/scrapy-tips-from-the-pros-april-2016-edition
Scrapy documentation: https://docs.scrapy.org/en/latest/index.html