Atom worker service - add 'max-job-count' and 'max-mem-limit' options
|Assignee:||Steve Breker||% Done:|
|Target version:||Release 2.7.0|
|Google Code Legacy ID:||Tested version:|
Add a new option to the jobs:worker task to accept a max-job-count value. This max-job-count value will indicate to the atom-worker the number of jobs it should process before shutting down. The atom-worker will return a specific value (return code 111) when it shuts down due to the maximum number of jobs being reached. This return can be checked by systemd to trigger a restart of the atom-worker.
This option can be used when configuring the atom-worker service (/usr/lib/systemd/system/atom-worker.service).
e.g. Set to shutdown after 10 jobs completed:
ExecStart=/usr/bin/php7.4 -d memory_limit=1000M -d error_reporting="E_ALL" -dextension=memprof.so symfony jobs:worker --max-job-count=10
Similarly, add a new --max-mem-usage option that will cause the worker to exit with code '111' when the atom-worker hits this configured limit. Jobs must not be stopped midway, but must be allowed to complete after the mem limit is breached.
e.g. Set to shutdown after 10 jobs completed OR if the atom-worker res memory exceeds 500MB:
ExecStart=/usr/bin/php7.4 -d memory_limit=1000M -d error_reporting="E_ALL" -dextension=memprof.so symfony jobs:worker --max-job-count=10 --max-mem-limit=500000
#7 Updated by Steve Breker about 1 month ago
TEAM LEAK notes doc: https://docs.google.com/document/d/1sm4gABvoqRouzoUX3CBmRTy8Vg9WwU78XAY00aZ9reY/edit#
#11 Updated by Steve Breker about 1 month ago
The max-mem-usage option will not shutdown immediately. This limit is checked between jobs so that they have a chance to complete before the worker shuts down with exit code 111. This means that --max-mem-usage must be set LOWER (25-50% lower?) than the PHP memory limit to account for large jobs completing.
#13 Updated by Steve Breker about 1 month ago
- Status changed from New to QA/Review
Merged --max-mem-usage feature to qa/2.x.