Python Web ScrapingЭтот исчерпывающий список содержит библиотеки python, связанные с веб-парсингом и обработкой данных.Web Scraping: Frameworksscrapy - web-scraping framework (twisted based).
•pyspider - A powerful spider system.
•autoscraper - A smart, automatic and lightweight web scraper
•grab - web-scraping framework (pycurl/multicurl based)
•ruia - Async Python 3.6+ web scraping micro-framework based on asyncio
•cola - A distributed crawling framework.
•frontera - A scalable frontier for web crawlers
•dude - A simple framework for writing web scrapers using decorators.
•ioweb - Web scraping framework based on gevent and lxml
Web Scraping : Tools•portia - Visual scraping for Scrapy.
•restkit - HTTP resource kit for Python. It allows you to easily access to HTTP resource and build objects around it.
•requests-html - Pythonic HTML Parsing for Humans.
•ScrapydWeb - A full-featured web UI for Scrapyd cluster management, which supports Scrapy Log Analysis & Visualization, Auto Packaging, Timer Tasks, Email Notice and so on.
•Starbelly - Starbelly is a user-friendly and highly configurable web crawler front end.
•Gerapy - Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
Web Scraping : Bypass Protection•cloudscraper - A Python module to bypass Cloudflare's anti-bot page.
GIthub
@pythonl
