1
0
Fork 0
mirror of https://github.com/mperham/sidekiq.git synced 2022-11-09 13:52:34 -05:00
Commit graph

110 commits

Author SHA1 Message Date
Mike Perham
29dca70e24
Introduce Sidekiq::Capsule ()
* Initial work on Sidekiq::Config

* Initial work on Sidekiq::Config

* reduce dependencies in deploy marks

* bare sidekiq and webapp

* Modify runtime to work with Capsules

* Cleanup

* Rename test files to remove test_ prefix

* Update test suite and standard rules to be more compliant

* Move constant definition outside code, per standard formatting

* Loads of changes for introduction of Capsules

* Remove Redis adapter abstraction

* update capsule overview

* Ensure Sidekiq.redis uses the correct pool for jobs running within a Capsule

* Use default_capsule for safety

* Slow down the beat to halve its Redis overhead

* move config fixtures into cfg/

* Add capsule middleware test

* use accessor
2022-08-25 10:15:11 -07:00
Mike Perham
3c93f643af Merge branch 'main' into 7-0 2022-08-22 10:27:53 -07:00
Adam McCrea
0b3751bf29
Histogram chart for job-specific metrics ()
* Refactor job query

* First attempt at a histogram chart

* Explore a box plot chart

* Show 3 chart variations

* Outline boxes instead of solid boxes

* Remove box plot chart

* Use linear y-axis

This matches the axis for the metrics overview page, and it clarifies the data.

* Data tables for job metrics

* Add histogram totals chart

* Move things around

* Tooltip for histogram chart

* Fix deploy tooltip

* Extract marks query

* Extract chart base class

* Renaming

* Ensure a min radius for histogram bubbles

High job counts can result in a very small multiplier, which was making some of the bubbles too small to be visible.

* Round everything to two decimals for consistency

* styling for metrics headers

* Show emdash when timing info is n/a

* No job results found message

* No need for metrics header
2022-08-16 10:56:36 -07:00
Adam McCrea
6f34717aef
[WIP] Improve metrics UI with charts ()
* Add multi-line chart for total execution time by job

* Fiddling with the UX

* Refactor metrics `top_jobs` query

* debugging

* revert debugging

* revert debugging

* Add failed and avg time, just one data table

* Add color swatch in data table

* Measure in seconds

* Fix duplicate color

* standard style

* Rename for clarity

* Bring back empty metrics test

* Execution time is not consistent, assert processed counts instead

* Only include top 5 in chart, change swatch element to checkbox

* Wire up the checkboxes to show/hide each job class on the chart

* The checkboxes should not appear disabled

* Ensure seconds for y-axis to match table and UX improvements

- All data shown on metrics page is now in seconds
- Tooltip now includes "UTC" with the time
- Tooltip rounds the number of seconds and includes "seconds"

* Show deploy marks in metrics chart

* Fix annotation position when updating datasets

* Remove deploy labels on chart

We shouldn't assume the first word of the label is the git SHA, and the label annotations were hacky anyway.

* tweaks

Co-authored-by: Mike Perham <mperham@gmail.com>
2022-08-12 09:53:00 -07:00
Mike Perham
7a4118dfd1 merge main 2022-08-11 07:24:18 -07:00
Mike Perham
f97d16acad Rack 2/3 compat: header keys must be lower-case 2022-08-11 07:22:29 -07:00
Mike Perham
1d1e166fc0 merge main 2022-08-03 10:28:05 -07:00
Mike Perham
7edf82fe1b safe_load our language files 2022-07-29 08:19:47 -07:00
Mike Perham
f220897db9
Per-job execution metrics ()
* New execution stats data format, 

We store time and counts per-queue, per-class and totals, daily.

* break metrics into separate jobs and queues hashes

* tweak

* Move metrics tracking to middleware, start work on Query API

* Add support for labeled points in time

* Add fetch method for deploy marks

* rejigger metrics file layout

* fix tests

* Remove per-queue metrics, adds a lot of complexity with little value IMO

* store per-minute histograms

* Keep basic stats hardwired as is

The idea being that metrics are optional middleware as they have significantly more overhead in CPU time and Redis space.

* Implement top N metrics dashboard

* Add topN and job-specific metric pages

* Supply histogram data to job metrics page

* cant use local time as CI is in a different tz

* Add basic metrics graph, refactor dashboard JS to make Rickshaw reuseable

* prepare for public beta
2022-07-28 12:21:42 -07:00
Mike Perham
6efebec90f fix broken tests 2022-07-14 16:05:25 -07:00
Kevin Bongart
b9d9bfe7df
Sort processes in UI ()
* Sort processes in UI using natural sort order

* Rename variable for consistency

* Fix natural sort algorithm and add more test examples

* Don't sort processes unless all process have a hostname

* Object#present? is a Rails method, not a Ruby method

* Flush Redis database before each helper test
2022-07-14 12:46:52 -07:00
Mike Perham
2ee7e27697 merge 6.5 2022-06-09 14:06:33 -07:00
Mike Perham
67daa7a408
Prepare for upcoming Sidekiq::Config redesign ()
* Prepare for upcoming Sidekiq::Config redesign

Adjust the server internals to use a config object rather than refering directly to the Sidekiq module.
2022-05-31 13:37:31 -07:00
Mike Perham
f3a9819a34 Start redis adapter cleanup 2022-05-12 08:42:07 -07:00
Mike Perham
f22a1c27c7 Remove support for redis-namespace and old Redis 2022-05-11 12:24:22 -07:00
Mike Perham
890c75bb6e Update standard to use ruby 2.7 rules 2022-05-10 14:29:44 -07:00
Dominik Bauernfeind
572f7035b3 Use Base64.urlsafe_* like Rake::Protection::AuthenticityToken 2022-04-02 19:47:26 +02:00
Mike Perham
e7d154eeb5 Remove "worker" from codebase where possible 2022-03-03 12:37:25 -08:00
Mike Perham
b65a932ae6 Include display_class in set of UI-ignored keys 2022-01-26 17:02:02 -08:00
Mike Perham
7785ac1399 Validate days parameter to avoid possible DoS in Web UI
Thank you to Sergey Shpakov of http://tutum.space for reporting.
2022-01-20 10:42:26 -08:00
Mike Perham
63f6e68777 standard formatting 2022-01-05 21:01:32 -08:00
Mike Perham
4d303904e5
Refresh javascripts ()
Refactor Web UI javascripts:

- Remove jQuery
- Remove dropdown.js, unused
- Update app JS to use vanilla JS
2021-11-01 09:55:17 -07:00
Stephen Humphries
84dd20d397
Persist live poll status in browser localStorage ()
* Persist live poll status in browser localStorage

* Prefix localStorage variables with 'sidekiq'

* Minor fix for default handling in localStorage timeInterval parsing
2021-08-30 12:31:41 -07:00
Mike Perham
f1b24da9c2 Rename Sidekiq::Job, reserve for future use 2021-08-20 10:43:55 -07:00
Shaun Bennett
13e2b564c8
Add private to Sidekiq Web UI Cache-Control to prevent request collapsing () 2021-08-14 09:46:15 -07:00
Shaun Bennett
20f4babd2d
Use no-store for web UI to prevent intermediate caching () 2021-08-14 09:12:04 -07:00
Mike Perham
a2c7541518 Make jobtag higher priority than bootstrap classes 2021-07-06 09:14:27 -07:00
Mike Perham
d86fe256e8 Update standard gem 2021-05-24 12:30:03 -07:00
Mike Perham
24c35e64fd Add note about sessions and Rails apps in API mode 2021-04-06 08:02:26 -07:00
Mike Perham
64f70339d1 Add pessimistic regexp on queue name input to avoid XSS, fixes 2021-03-25 16:16:55 -07:00
Mike Perham
9aeba313a9 Some mobile UI polish, fixes 2021-03-12 16:05:40 -08:00
j-sieg
561f0bfae8
Don't hardcode URI scheme of redis connection url () 2021-03-12 07:44:19 -08:00
Mike Perham
f0ddebc740
Add process/thread count summary to Busy page ()
* Add process/thread count summary to Busy page

* pr

* Move Busy status numbers to stats tiles

* style nazi

* Add RSS stat box
2021-02-18 16:28:44 -08:00
Mike Perham
968bc81043
Improve Web UI session experience ()
* Simplify Web UI sessions

Remove all of the hacks and support infrastructure around Rack sessions. Rails provides this by default so we don't need it for 95% of users. The other 5% need to provide a Rack session.

This is a big change and has the potential to break installs so it deserves at least a minor version bump.

See also ,  and many others.
2021-02-12 14:50:51 -08:00
Mike Perham
9998a59b6d Bump standard, code format updates 2020-12-16 11:07:31 -08:00
Jared Szechy
09305a0909
Fix handling of no session () 2020-11-04 09:09:30 -08:00
Mike Perham
3b5ae30c4e Add process RSS to the Busy page, fixes 2020-10-14 15:11:40 -07:00
Mike Perham
2325b1a475 Upgrade standard, code style fixes 2020-08-28 12:40:19 -07:00
basherru
df702a16ce
check if session object is not empty in csrf protection ()
Co-authored-by: basherru <alexander.baz@okwork.io>
2020-08-19 10:39:30 -07:00
Mike Perham
14d31269bc Explicitly scope to ::Rack, fixes 2020-07-27 16:42:27 -07:00
Mike Perham
4338695727 Bump standard, style fixes 2020-06-17 15:15:20 -07:00
Mike Perham
47bc87034f Refactor and clean up CSRF protection code
1. Remove unused methods
2. Remove unused and unnecessary features
3. Add basic tests

The session token is stored server-side and never given to the client.
A masked token is generated for each request which can be inserted into forms to be POSTed.
2020-06-03 16:06:36 -07:00
Abdelkader Boudih
111d9d5369
Vendor rack-protection/authenticity_token middleware ()
* [COVID-19] Vendor rack-protection/authenticity_token middleware

* Include MIT license as required

* blasted rubocop

Co-authored-by: Mike Perham <mperham@gmail.com>
2020-06-03 10:21:58 -07:00
David Santos Merino
797578c591
Optimize stringify item before pushing it ()
* Optimize stringify hashes

* Optimize symbolize hashes
2020-05-22 15:38:52 -07:00
Mike Perham
6bd4eaffdc Bump standard, fix style issues 2020-03-17 13:38:48 -07:00
Akshay Birajdar
2bfd41117c
Prefer Hash#transform_keys & Regex#named_captures ()
* Prefer Hash#transform_keys to stringify keys

* Prefer Regexp#named_captures to create hash for match data
2020-03-15 13:32:25 -07:00
Mike Perham
675e40c326 Don't calculate Content-Length, let Rack do that for us; fixes 2020-01-30 08:52:31 -08:00
Mike Perham
4f630adb21 fix env in web title 2020-01-13 17:03:18 -08:00
Mike Perham
2657803817 fix style 2019-11-14 14:03:28 -08:00
Shayon Mukherjee
0ddc6c9858 Introduce the ability to pause queue from the UI ()
* Introduce the ability to pause queue from the UI

Considering sidekiq web UI allow to perform complex tasks with ease.
Having the ability to pause a queue during an emergency (or operational intense)
scenarios would come in quite handy.

This PR introduces a new button to under the "Actions" column on `sidekiq/queues` pages,
right next to the `Delete` button. Depending on the state of the queue, it will according
show the `Pause` or `Unpause` text, with appropriate form element that `POST`s to the same
endpoint. The endpoint logic is updated to handle the new cases. Added some relevant unit tests.

Adding mock `Sidekiq::Queue#pause!` and `Sidekiq::Queue#unpause!` functions to
the API, which I believe will be overridden by sidekiq pro.

* Remove confirmation text and switch ordering of new copy in en.yml

* Only allow pause/unpause operations from sidekiq pro

This means the button from UI is hidden and api route has validation
in place such that it will only perform the respective pause/unpause
operation on proper payload, in sidekiq pro.

Its using `Sidekiq::Pro` constant as a check to detect whether or
not pro is present.

* Initialize Queue only once in api router action
2019-11-14 10:06:53 -08:00