网页爬虫设计

创建项目

Scrapy startproject DgSpider

Name	Name	Last commit message	Last commit date
Latest commit zjfGit Create README.md May 3, 2017 d14ea6b · May 3, 2017 History 8 Commits
__pycache__	__pycache__	Scrapy Spider based on Python3	May 3, 2017
spiders	spiders	Update ContentSpider.py	May 3, 2017
PostHandle.py	PostHandle.py	Update PostHandle.py	May 3, 2017
README.md	README.md	Create README.md	May 3, 2017
__init__.py	__init__.py	Scrapy Spider based on Python3	May 3, 2017
check_post.py	check_post.py	Update check_post.py	May 3, 2017
commonUtils.py	commonUtils.py	Scrapy Spider based on Python3	May 3, 2017
contentSettings.py	contentSettings.py	Scrapy Spider based on Python3	May 3, 2017
items.py	items.py	Scrapy Spider based on Python3	May 3, 2017
middlewares.py	middlewares.py	Scrapy Spider based on Python3	May 3, 2017
mysqlUtils.py	mysqlUtils.py	Scrapy Spider based on Python3	May 3, 2017
pipelines.py	pipelines.py	Update pipelines.py	May 3, 2017
postSettings.py	postSettings.py	Scrapy Spider based on Python3	May 3, 2017
settings.py	settings.py	Scrapy Spider based on Python3	May 3, 2017
uploadUtils.py	uploadUtils.py	Scrapy Spider based on Python3	May 3, 2017
urlSettings.py	urlSettings.py	Update urlSettings.py	May 3, 2017