爬虫之爬取小说
展示
源码地址
https://github.com/Liar0320/spider
依赖
- cherrio Fast, flexible & lean implementation of core jQuery designed specifically for the server.
- inquirer A collection of common interactive command line user interfaces.
安装
Clone the repo using Git:
git clone https://github.com/Liar0320/spider.git
Alternatively you can download this repository and then:
npm i
npm run start
配置参数
请求的超时时间
Timeout for each request.
default value: 15000
最大同时下载多少章节
The maximum number of threads that exist at one time, that is, the number of asynchronous downloads that are started at the same time.
default value: 20
请求失败时，重新发起请求的次数
The number of times the request was re initiated if there was a request failure.
default value: 4
下载的目录
Root of download directory
default value: join(__dirname, ‘./download’)