我目前在docker-compose上运行一个8核实例(docker_serivce、dramatiq_service、rabbitmq_service、postgres_service等):django_apscheduler==0.5.2 dramatiq [rabbitmq, watch]==1.8.1 RabbitMQ (rabbitmq: 3.8-management-alpine image)
错误在于许多进程都在执行相同的任务。任务只创建一次(BackgroundScheduler调度程序),但是有多个进程尝试执行它,并且不同进程之间的一些线程是相同的。
下面是一个例子。该作业只在队列中出现一次,但当它被执行时,许多进程都会使用它。139698903181056是线程,21是进程
"INFO | 10/06/2021 10:30:09 | 139698903181056 | app.orders.tasks | tasks | tasks.py | 22 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:09 | 139698894788352 | app.orders.tasks | tasks | tasks.py | 22 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:08 | 139698911573760 | app.orders.tasks | tasks | tasks.py | 18 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:07 | 139699123865344 | app.orders.tasks | tasks | tasks.py | 19 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:03 | 139698903181056 | app.orders.tasks | tasks | tasks.py | 21 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:02 | 139698374702848 | app.orders.tasks | tasks | tasks.py | 16 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:01 | 139698903181056 | app.orders.tasks | tasks | tasks.py | 19 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:01 | 139698903181056 | app.orders.tasks | tasks | tasks.py | 21 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:01 | 139698307593984 | app.orders.tasks | tasks | tasks.py | 21 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:00 | 139698861217536 | app.orders.tasks | tasks | tasks.py | 22 | 39 | send_menu | Sending menu #87..."发布于 2021-06-18 04:05:14
这个问题与Django-apscheduler有关。
dramatiq容器(8个worker)在调度程序暂停的情况下启动,但在我的代码的一部分中,有一段代码恢复了调度程序,因此所有8个worker同时开始触发任务,即使作业只创建了一次,也会导致重复的任务。
寓意:始终检查您的调度程序是否只启动一次。
https://stackoverflow.com/questions/67928552
复制相似问题