|
|
@@ -5,19 +5,19 @@
|
|
|
2. sh ./main/scheduling_main.sh ${crawler_dir} ${log_type} ${crawler} ${env} ${machine} >>${nohup_dir} 2>&1 &
|
|
|
```
|
|
|
参数说明
|
|
|
-${crawler_dir}: 爬虫执行路径，如: ./youtube/youtube_main/run_youtube_follow.py
|
|
|
-${log_type}: 日志命名格式，如: follow，则在 youtube/logs/目录下，生成 2023-02-08-follow.log
|
|
|
+${crawler_dir}: 爬虫执行路径，如: scheduling/scheduling_main/run_write_task.py
|
|
|
+${log_type}: 日志命名格式，如: scheduling-task，则在 scheduling/logs/目录下，生成 2023-02-08-scheduling-task.log
|
|
|
${crawler}: 哪款爬虫，如: youtube / kanyikan / weixinzhishu
|
|
|
${env}: 爬虫运行环境，正式环境: prod / 测试环境: dev
|
|
|
-${machine}: 爬虫运行机器，阿里云服务器: aliyun_hk / aliyun / macpro / macair / local
|
|
|
-${nohup_dir}: nohup日志存储路径，如: ./youtube/nohup.log
|
|
|
+${machine}: 爬虫运行机器，阿里云服务器: aliyun_hk / aliyun / local
|
|
|
+${nohup_dir}: nohup日志存储路径，如: shceduling/nohup-task.log
|
|
|
```
|
|
|
|
|
|
#### 运行命令
|
|
|
```
|
|
|
阿里云 102 服务器
|
|
|
sh ./main/scheduling_main.sh scheduling/scheduling_main/run_write_task.py --log_type="scheduling-write" --crawler="scheduling" --env="prod" --machine="aliyun" shceduling/nohup-write.log
|
|
|
-sh ./main/scheduling_main.sh scheduling/scheduling_main/run_scheduling_task.py --log_type="scheduling-task" --crawler="scheduling" --env="prod" --machine="aliyun" shceduling/nohup-write.log
|
|
|
+sh ./main/scheduling_main.sh scheduling/scheduling_main/run_scheduling_task.py --log_type="scheduling-task" --crawler="scheduling" --env="prod" --machine="aliyun" shceduling/nohup-task.log
|
|
|
|
|
|
香港服务器
|
|
|
sh ./main/scheduling_main.sh scheduling/scheduling_main/run_write_task.py --log_type="scheduling-write" --crawler="scheduling" --env="prod" --machine="aliyun_hk" shceduling/nohup-write.log
|
