1
0
Fork 0
mirror of https://github.com/mperham/sidekiq.git synced 2022-11-09 13:52:34 -05:00
Commit graph

852 commits

Author SHA1 Message Date
Mike Perham
cc2a07d45e
Adjust API a bit, 2022-11-06 15:04:16 -08:00
Marcelo Lauxen
ef8d4425b6
use start_with? instead of starts_with? which is only available on Rails () 2022-11-05 08:09:33 -07:00
David Wessman
09f2a4ca3b
Renames test files to *_test.rb () 2022-11-04 01:12:31 -07:00
Mike Perham
636fa9a06a
Fix lingering threads in tests
Also, allow embedding process to provide their own heartbeat thread.
2022-11-01 12:32:31 -07:00
Mike Perham
b1fc7fa8be
Upgrade test suite to use maxitest 2022-11-01 12:17:13 -07:00
David Runger
f2b9191367
Change Rails log level to WARN in before_setup rather than setup ()
`setup` is too late; by then, some unwanted log output will already have
been written to stdout. `before_setup` runs early enough to suppress it
(since it is `ActiveSupport::Testing::TaggedLogging#before_setup` which
writes the undesired log output at log level `info`).
2022-10-29 05:44:33 -07:00
Lutz Lengemann
148b2603f5
Fix natural sorting for processes ()
* Fix natural sorting for processes

* Remove pry
2022-10-26 09:45:08 -07:00
Teo Ljungberg
c72f0d2e81
Rename worker references to job ()
* standardrb

* Rename all `*_worker` references to `*_job`

* Rename Worker references to Job in test suite

* Rename ApiJob to ApiAjJob to avoid collisions

* Rename remaining Worker to Job references

* Rename WorkController to JobController

Including routes

* fixup! Rename WorkController to JobController
2022-10-11 08:38:53 -07:00
Abdelkader Boudih
936846ad70
Remove deprecated worker reference. ()
* Remove deprecated worker reference.

* fix build
2022-10-08 15:36:53 -07:00
Mike Perham
ad9442f116
Merge main, adjust WorkSet#each to not greed parse job_hash JSON 2022-10-04 08:44:47 -07:00
Mike Perham
f466a035cf
Remove unused parameter in Fetch/bulk_requeue API 2022-09-30 08:57:00 -07:00
Bobby McDonald
26fed38a32
Add ability to parse capsules from a config file () 2022-09-28 13:47:17 -07:00
Mike Perham
354f368054
Minimize metrics overhead, disable hourly/daily aggregates for now since we dont use them, expire histograms quicker 2022-09-22 11:19:12 -07:00
Mike Perham
d0e0e545a3
Adjust CurrentAttributes to use String, fixes 2022-09-20 14:10:26 -07:00
Adam McCrea
b19260f2f7
Fix flaky test () 2022-09-20 14:08:11 -07:00
Mike Perham
17e2858385
Adjust CurrentAttributes to use String, fixes 2022-09-20 13:58:35 -07:00
Mike Perham
9ee55e9a02
fix tests 2022-09-15 13:07:43 -07:00
Mike Perham
8259783f3c
Add tests for sharding support 2022-09-15 13:01:58 -07:00
Adam McCrea
fafefc4a47
Fix flaky test () 2022-09-12 05:55:57 -07:00
Mike Perham
fca1d5e7a5
Move deploy API outside of metrics 2022-09-09 14:23:49 -07:00
Mike Perham
30bd88bf73
Fix NameError when running test solo 2022-09-09 10:42:51 -07:00
Mike Perham
46de75ea12
Remove slow test which was timing out 2022-09-09 10:41:27 -07:00
Mike Perham
d0f03fcbbb
refactor fetch timeout to avoid warning on Ruby 2.7 2022-09-09 10:41:07 -07:00
Mike Perham
39e87f7b10
use global redis with web UI 2022-09-08 08:47:11 -07:00
Vikram B Kumar
6fc666644e
fix 5514 - handle consecutive delimiters in hostnames while sorting them () 2022-09-08 07:29:31 -07:00
Heinrich Lee Yu
8dc28030da
Fix poll interval not changing based on process count ()
We should not memoize the value of `scaled_poll_interval` because this
changes based on the number of Sidekiq processes

Co-authored-by: Heinrich Lee Yu <heinrich@gitlab.com>
2022-09-07 16:03:52 -07:00
Mike Perham
b64ce75f1c
merge main 2022-09-02 10:29:29 -07:00
Mike Perham
337f65c1bd
capsule polish 2022-09-02 07:37:29 -07:00
Mike Perham
b7d57d2ed8
more test coverage 2022-09-01 15:47:24 -07:00
Mike Perham
0395eea2a4
Remove much of the redis-client compat layer 2022-09-01 14:51:16 -07:00
Mike Perham
1b83a15278
bugfixes for Pro 6.0 2022-08-30 11:51:23 -07:00
Mike Perham
aecf95fea1
Workaround some flaky tests 2022-08-25 11:03:28 -07:00
Mike Perham
29dca70e24
Introduce Sidekiq::Capsule ()
* Initial work on Sidekiq::Config

* Initial work on Sidekiq::Config

* reduce dependencies in deploy marks

* bare sidekiq and webapp

* Modify runtime to work with Capsules

* Cleanup

* Rename test files to remove test_ prefix

* Update test suite and standard rules to be more compliant

* Move constant definition outside code, per standard formatting

* Loads of changes for introduction of Capsules

* Remove Redis adapter abstraction

* update capsule overview

* Ensure Sidekiq.redis uses the correct pool for jobs running within a Capsule

* Use default_capsule for safety

* Slow down the beat to halve its Redis overhead

* move config fixtures into cfg/

* Add capsule middleware test

* use accessor
2022-08-25 10:15:11 -07:00
Mike Perham
09dacfed8f Redis-rb 4.8.0 compatibility, fixes 2022-08-22 13:27:37 -07:00
Mike Perham
3c93f643af Merge branch 'main' into 7-0 2022-08-22 10:27:53 -07:00
Adam McCrea
0b3751bf29
Histogram chart for job-specific metrics ()
* Refactor job query

* First attempt at a histogram chart

* Explore a box plot chart

* Show 3 chart variations

* Outline boxes instead of solid boxes

* Remove box plot chart

* Use linear y-axis

This matches the axis for the metrics overview page, and it clarifies the data.

* Data tables for job metrics

* Add histogram totals chart

* Move things around

* Tooltip for histogram chart

* Fix deploy tooltip

* Extract marks query

* Extract chart base class

* Renaming

* Ensure a min radius for histogram bubbles

High job counts can result in a very small multiplier, which was making some of the bubbles too small to be visible.

* Round everything to two decimals for consistency

* styling for metrics headers

* Show emdash when timing info is n/a

* No job results found message

* No need for metrics header
2022-08-16 10:56:36 -07:00
Adam McCrea
6f34717aef
[WIP] Improve metrics UI with charts ()
* Add multi-line chart for total execution time by job

* Fiddling with the UX

* Refactor metrics `top_jobs` query

* debugging

* revert debugging

* revert debugging

* Add failed and avg time, just one data table

* Add color swatch in data table

* Measure in seconds

* Fix duplicate color

* standard style

* Rename for clarity

* Bring back empty metrics test

* Execution time is not consistent, assert processed counts instead

* Only include top 5 in chart, change swatch element to checkbox

* Wire up the checkboxes to show/hide each job class on the chart

* The checkboxes should not appear disabled

* Ensure seconds for y-axis to match table and UX improvements

- All data shown on metrics page is now in seconds
- Tooltip now includes "UTC" with the time
- Tooltip rounds the number of seconds and includes "seconds"

* Show deploy marks in metrics chart

* Fix annotation position when updating datasets

* Remove deploy labels on chart

We shouldn't assume the first word of the label is the git SHA, and the label annotations were hacky anyway.

* tweaks

Co-authored-by: Mike Perham <mperham@gmail.com>
2022-08-12 09:53:00 -07:00
Mike Perham
1d1e166fc0 merge main 2022-08-03 10:28:05 -07:00
Sam
459530bd96
Code coverage improvements ()
* Add coverage for helpers.rb

* Add simplecov to git actions

* Test simplecov

* Changed this to run on all pull requests, not just main and 7-0

* Fix yaml spacing

* Create simplecov file and pointer for file

* Add empty hash to json file

* See if we need simplecov gem enabled

* Update coverage file

* Add in covered_percent

* Move simplecov_file out

* Remove token

* Add token back in

* Remove unnecessary code

* Add comment for running simplecov in the rake file

* Fix typo in spec file

* Remove references to codecov, add minimum requirement of 90% to simplecov
2022-08-03 09:45:06 -07:00
Mike Perham
f220897db9
Per-job execution metrics ()
* New execution stats data format, 

We store time and counts per-queue, per-class and totals, daily.

* break metrics into separate jobs and queues hashes

* tweak

* Move metrics tracking to middleware, start work on Query API

* Add support for labeled points in time

* Add fetch method for deploy marks

* rejigger metrics file layout

* fix tests

* Remove per-queue metrics, adds a lot of complexity with little value IMO

* store per-minute histograms

* Keep basic stats hardwired as is

The idea being that metrics are optional middleware as they have significantly more overhead in CPU time and Redis space.

* Implement top N metrics dashboard

* Add topN and job-specific metric pages

* Supply histogram data to job metrics page

* cant use local time as CI is in a different tz

* Add basic metrics graph, refactor dashboard JS to make Rickshaw reuseable

* prepare for public beta
2022-07-28 12:21:42 -07:00
Sam
dcd6d32c35
Create a timeout that happens before cleanup ()
* Create a timeout that happens before cleanup

* Remove unnecessary code

* Move a redis call into the method that is running redis:

* Add timelock to beginning of cleanup
2022-07-19 14:11:06 -07:00
Mike Perham
6efebec90f fix broken tests 2022-07-14 16:05:25 -07:00
Mike Perham
0676a5202e
Enable sidekiq_retry_in to signal :kill or :discard dynamically, fixes () 2022-07-14 15:56:58 -07:00
Kevin Bongart
b9d9bfe7df
Sort processes in UI ()
* Sort processes in UI using natural sort order

* Rename variable for consistency

* Fix natural sort algorithm and add more test examples

* Don't sort processes unless all process have a hostname

* Object#present? is a Rails method, not a Ruby method

* Flush Redis database before each helper test
2022-07-14 12:46:52 -07:00
Mike Perham
1b4ffdb06f merge main 2022-07-12 13:37:45 -07:00
Jean byroot Boussier
2b58b74344
redis-client: Fix ZADD compatibility () 2022-06-13 06:27:57 -07:00
Mike Perham
2ee7e27697 merge 6.5 2022-06-09 14:06:33 -07:00
Mike Perham
d985aa35f0 Adjust middleware, fixes
Middleware now has access to the `config` object so we can provide local helper methods.
2022-06-08 12:31:25 -07:00
Mike Perham
55ced28181
Update standard rules ()
* update standard rules and run standard:fix

* Fix more standard errors

* standardize
2022-06-05 07:44:52 -07:00
Sam
0724848bf1
Update last of spec files to use it and describe, instead od minitest () 2022-06-02 10:31:42 -07:00