Commit graph

3 commits

Author SHA1 Message Date
Shinya Maeda
36b30cf18e Revert concurrent pipeline schedule creation
This commit reverts the previously introduced concurrent pipeline
schedule creation which was a viable solution for mitigating
inconsistent pipeline schedule by Sidekiq Memory Killer.
2019-06-18 21:56:11 +07:00
Shinya Maeda
7ac5dfc1e9 Fix pipeline schedule when owner is nil
Fixing the bug
2019-06-11 18:03:40 +07:00
Shinya Maeda
6a18a411a3 Make pipeline schedule worker resilient
Currently, pipeline schedule worker is unstable because it's
sometimes killed by excessive memory consumption.
In order to improve the performance, we add the following fixes:

1. next_run_at is always real_next_run, which means the value
   always takes into account of worker's cron schedule
1. Remove exlusive lock. This is already covered by real_next_run
   change.
1. Use RunPipelineScheduleWorker for avoiding memory killer.
   Memory consumption is spread to the multiple sidekiq worker.
2019-06-03 10:04:57 +07:00