1
0
Fork 0
mirror of https://github.com/mperham/sidekiq.git synced 2022-11-09 13:52:34 -05:00
Commit graph

215 commits

Author SHA1 Message Date
Mike Perham
f220897db9
Per-job execution metrics ()
* New execution stats data format, 

We store time and counts per-queue, per-class and totals, daily.

* break metrics into separate jobs and queues hashes

* tweak

* Move metrics tracking to middleware, start work on Query API

* Add support for labeled points in time

* Add fetch method for deploy marks

* rejigger metrics file layout

* fix tests

* Remove per-queue metrics, adds a lot of complexity with little value IMO

* store per-minute histograms

* Keep basic stats hardwired as is

The idea being that metrics are optional middleware as they have significantly more overhead in CPU time and Redis space.

* Implement top N metrics dashboard

* Add topN and job-specific metric pages

* Supply histogram data to job metrics page

* cant use local time as CI is in a different tz

* Add basic metrics graph, refactor dashboard JS to make Rickshaw reuseable

* prepare for public beta
2022-07-28 12:21:42 -07:00
Sam
dcd6d32c35
Create a timeout that happens before cleanup ()
* Create a timeout that happens before cleanup

* Remove unnecessary code

* Move a redis call into the method that is running redis:

* Add timelock to beginning of cleanup
2022-07-19 14:11:06 -07:00
Mike Perham
b73994056d formatting 2022-07-12 13:35:39 -07:00
Mike Perham
cdb1eac495 cleanup stats yard 2022-07-12 13:23:35 -07:00
Mike Perham
ec589de5ef better yard for middleware 2022-07-12 13:15:10 -07:00
SamArdrey
8559fc1d7b Get opts working: 2022-06-15 10:06:37 -07:00
SamArdrey
67283ce73b Add yarddoc files to gitignore, remove yardoc from rake tasks, update api file 2022-06-14 17:26:58 -07:00
SamArdrey
be7b305e15 Make sidekiq api module public, so the classes show up 2022-06-14 09:15:56 -07:00
Mike Perham
16b51f3961 switch from rdoc to yard 2022-06-13 15:22:26 -07:00
Mike Perham
4508477bf5 more doc 2022-06-13 10:43:52 -07:00
Mike Perham
9102c38735 Better rdoc for public APIs 2022-06-13 10:03:12 -07:00
Mike Perham
4026085d35 standardize 2022-06-09 13:52:17 -07:00
Mike Perham
f8c7579920 rdoc'ing 2022-06-08 13:02:24 -07:00
SamArdrey
6eadad86f9 Add a bit of documentation, install yard 2022-06-08 12:26:06 -07:00
Mike Perham
277ea1888f Fix 2022-05-31 13:41:37 -07:00
Mike Perham
67daa7a408
Prepare for upcoming Sidekiq::Config redesign ()
* Prepare for upcoming Sidekiq::Config redesign

Adjust the server internals to use a config object rather than refering directly to the Sidekiq module.
2022-05-31 13:37:31 -07:00
Jean byroot Boussier
aaac999c6d
Add a compatibility layer for redis-client ()
* Add a compatibility layer for `redis-client`

As discussed in https://github.com/mperham/sidekiq/pull/5253

Switching entirely to redis-client is deemed risky, so instead
we can support both.

All is needed is a small translation layer, and some very minimal
adjustments in the callers.

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
Co-authored-by: Mike Perham <mperham@gmail.com>
2022-05-10 12:25:04 -07:00
Mike Perham
152c3d8104 fix code formatting 2022-04-08 12:08:50 -07:00
Mike Perham
e7d154eeb5 Remove "worker" from codebase where possible 2022-03-03 12:37:25 -08:00
Thach Chau
10e7feb312
Add keyword arguments handler in Sidekiq::JobRecord#display_args ()
* Add keyword arguments handler in Sidekiq::JobRecord#display_args

* Update Sidekiq::JobRecord#display_args kwargs's presence check so that we are not dependent on Rails.
2022-01-29 09:04:29 -08:00
Jean byroot Boussier
c792342753
Fix deprecated uses of Redis#pipelined and Redis#multi ()
Context: https://github.com/redis/redis-rb/pull/1059

The following is deprecated
```ruby
redis.pipelined do
  redis.get(key)
end
```

And should be rewritten as:
```ruby
redis.pipelined do |pipeline|
  pipeline.get(key)
end
```

Functionally it makes no difference.

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2022-01-24 08:18:16 -08:00
Mike Perham
7785ac1399 Validate days parameter to avoid possible DoS in Web UI
Thank you to Sergey Shpakov of http://tutum.space for reporting.
2022-01-20 10:42:26 -08:00
Mike Perham
63f6e68777 standard formatting 2022-01-05 21:01:32 -08:00
Michael Hauser-Raspe
051555e55c
Preserve behaviour of fetch_stats! returning stats. () 2021-09-27 08:43:15 -07:00
Mike Perham
f1b24da9c2 Rename Sidekiq::Job, reserve for future use 2021-08-20 10:43:55 -07:00
Mike Perham
465c5c610c refactor stat method 2021-07-06 09:18:57 -07:00
Michael Hauser-Raspe
47ce2a0b38
Quick stats api. ()
* Lazily fetch slow stats later.

* Put calls to lazy load into stat method.

* Add test for workers_size.
2021-07-06 09:13:52 -07:00
Matt Duszynski
02348cdc97
Add Sidekiq::Process#queues method () 2021-06-03 10:02:48 -07:00
Mike Perham
6adde6b346 standard fmt 2021-05-26 13:33:05 -07:00
Mike Perham
614d41585b Improve logging of .delay jobs, closes 2021-05-24 12:29:45 -07:00
Lutz Lengemann
997b02963b
Minor text change to make the description correct. () 2021-05-14 05:24:33 -07:00
Mike Perham
53999adc2c Minor API updates for the blog post 2021-04-19 16:19:45 -07:00
Mike Perham
e0f7d2d95f Not true, not sure when this changed 2021-04-18 14:12:55 -07:00
Mike Perham
285d6541a5 Log poor Redis RTT, fixes 2021-02-22 15:48:38 -08:00
Mike Perham
f0ddebc740
Add process/thread count summary to Busy page ()
* Add process/thread count summary to Busy page

* pr

* Move Busy status numbers to stats tiles

* style nazi

* Add RSS stat box
2021-02-18 16:28:44 -08:00
Mike Perham
6c94a9dd9f Bump standard 2020-10-21 08:18:49 -07:00
Mike Perham
3b5ae30c4e Add process RSS to the Busy page, fixes 2020-10-14 15:11:40 -07:00
Mike Perham
2139254f77 Sort busy workers by run_at, fixes
We sort oldest first so long-running jobs will appear at the top; those jobs are typically the ones that developers want to see and diagnose.
2020-07-13 12:42:36 -07:00
Mike Perham
6a334042ee Migrate to exists? for redis-rb 4.2, fixes 2020-06-09 15:14:02 -07:00
Mike Perham
6bd4eaffdc Bump standard, fix style issues 2020-03-17 13:38:48 -07:00
Mike Perham
e44d5b8f75
Use unlink, not del, for performance ()
* Use `unlink`, not `del`, for performance

* issue no.
2020-02-05 14:05:17 -08:00
Mike Perham
83c9524fb4 payload should exposed as a Hash, fixes 2019-11-22 09:53:44 -08:00
fatkodima
6a4fe1ab8d Fix SortedEntry#reschedule when called with non float as time () 2019-10-17 06:09:00 -07:00
fatkodima
4803d87969 Use idiomatic ruby Array methods () 2019-10-16 08:42:32 -07:00
fatkodima
d6b00967ec Avoid costly json parsing in JobSet#delete_by_jid for wrong values () 2019-10-09 06:45:23 -07:00
fatkodima
d9d4728d0b Fix returned Enumerator for SortedSet#scan () 2019-10-09 06:44:02 -07:00
fatkodima
59df40520b Release redis connection in ProcessSet#each when not needed () 2019-10-09 06:43:18 -07:00
fatkodima
fc6bad64e9 Replace #inject/#reduce with #sum () 2019-10-08 14:50:26 -07:00
fatkodima
ce7d1661c4 Dump error backtraces as json before compressing () 2019-10-08 13:45:17 -07:00
fatkodima
01b4976c7b Improve fault tolerance of rescheduling jobs using API 2019-10-02 13:54:18 -07:00