* Update test_integration files per PR #1956
test_integration_cluster.rb
Request handling during server TERM - two tests
`#test_term_closes_listeners_tcp`
`#test_term_closes_listeners_unix`
using `#term_closes_listeners`
Send requests 10 per second. Send 10, then :TERM server, then send another 30.
No more than 10 should throw Errno::ECONNRESET.
Request handling during phased restart - two tests
`#test_usr1_all_respond_tcp`
`#test_usr1_all_respond_unix`
using `#usr1_all_respond`
Send requests 1 per second. Send 1, then :USR1 server, then send another 24.
All should be responded to, and at least three workers should be used
Stuck worker tests - two tests
`#test_stuck_external_term_spawn`
Tests whether externally TERM'd 'stuck' workers are proper re-spawned.
`#test_stuck_phased_restart`
Tests whether 'stuck' workers are properly shutdown during phased-restart.
helper files/methods changes
1. helper file changes to allow binding to TCP or UNIX, see kwarg unix:
2. Skip on Windows for signal TERM
* Misc updates, debug output, cleanup
* Add comments
* fix test_int_signal_with_background_thread_in_jruby per review
* TestIntegrationCluster#term_closes_listeners - add interleaved assert
* cluster.rb - remove duplicate Worker#term? method
* Allow extra runtime deps to be defined when using prune_bundler
* Check extra_runtime_dependencies is set before iterating over them
* Load additional paths for extra runtime dep gems
* Don't load extra dependencies, just add their paths to $LOAD_PATH
* Fix typos and extraneous checks and rescues
* Use Gem::Specification#full_require_paths when available
* Prevent use of prune_bundler and extra_runtime_dependencies with early versions of RubyGems
* Ensure LOAD_PATH is modified by extra_runtime_dependencies
* Refactor prune_bundler in launcher.rb and write some unit tests
Currently when a SIGTERM is sent to a puma cluster, the signal is trapped, then sent to all children, it then waits for children to exit and then the parent process exits. The socket that accepts connections is only closed when the parent process calls `exit 0`. The problem with this flow is there is a period of time where there are no child processes to work on an incoming connection, however the socket is still open so clients can connect to it. When this happens, the client will connect, but the connection will be closed with no response. Instead, the desired behavior is for the connection from the client to be rejected. This allows the client to re-connect, or if there is a load balance between the client and the puma server, it allows the request to be routed to another node.
This PR fixes the existing behavior by manually closing the socket when SIGTERM is received before shutting down the workers/children processes. When the socket is closed, any incoming requests will fail to connect and they will be rejected, this is our desired behavior. Existing requests that are in-flight can still respond.
## Test
This behavior is quite difficult to test, you'll notice that the test is far longer than the code change. In this test we send an initial request to an endpoint that sleeps for 1 second. We then signal to other threads that they can continue. We send the parent process a SIGTERM, while simultaneously sending other requests. Some of these will happen after the SIGTERM is received by the server. When that happens we want none of the requests to get a `ECONNRESET` error, this would indicate the request was accepted but then closed. Instead we want `ECONNREFUSED`.
I ran this test in a loop for a few hours and it passes with my patch, it fails immediately if you remove the call to close the listeners.
```
$ while m test/test_integration.rb:235; do :; done
```
## Considerations
This PR only fixes the problem for "cluster" (i.e. multi-worker) mode. When trying to reproduce the test with single mode, on (removing the `-w 2` config) it already passes. This leads us to believe that either the bug does not exist in single threaded mode, or at the very least reproducing the bug via a test in the single threaded mode requires a different approach.
Co-authored-by: Danny Fallon <Danny.fallon.ie+github@gmail.com>
Co-authored-by: Richard Schneeman <richard.schneeman+foo@gmail.com>
running `rake` loads all files starting with `test_` which is not supposed to also load the helper,
since the helper should be loaded by each test and is not a test itself.
