Meter calling accept(2) with available pool capacity (#1278)

* Meter calling accept(2) with available pool capacity

Talking through this with Nate and Richard, we realized that accepting
new clients without taking account for the available capacity of the
thread pool doesn't improve throughput, it only hurts it in the case of
multiple workers. If a worker pauses (or starts up before other
workers), then a worker can accidentally suck up a high number of
clients and leave unused capacity inside the other workers.

This change will smooth out this issue, with a minor penalty to maximum
throughput.

* Rewrite the conditional to be less confusing

lib/puma/server.rb

@@ -363,7 +363,7 @@ module Puma
                     end
 
                     pool << client
-                    pool.wait_until_not_full unless queue_requests
+                    pool.wait_until_not_full
                   end
                 rescue SystemCallError
                   # nothing

lib/puma/thread_pool.rb

@@ -155,7 +155,15 @@ module Puma
 
     def wait_until_not_full
       @mutex.synchronize do
-        until @todo.size - @waiting < @max - @spawned or @shutdown
+        while true
+          return if @shutdown
+          return if @waiting > 0
+
+          # If we can still spin up new threads and there
+          # is work queued, then accept more work until we would
+          # spin up the max number of threads.
+          return if @todo.size < @max - @spawned
+
           @not_full.wait @mutex
         end
       end