[ruby/net-http] Do not set SNI hostname if connecting to IP address

RFC 6066, section 3, explicitly disallows the use of an IP address
as an SNI server name.  So check if the connection is being made
to an IP address using the resolv regexps, and do not set an SNI
hostname in that case.

Recent changes to LibreSSL make it more strictly follow RFC 6066,
resulting an s.hostname= raising an error if passed an IP address.
When such verions of LibreSSL are used, this change not only fixes
the net/http tests, it also fixes tests for webrick and open-uri,
which both make SSL connections to 127.0.0.1 using net/http in
their tests.

Avoid warning in the openssl extension by unsetting
@ssl_context.verify_hostname if connecting to an IP address.
Make changes so that the post_connection_check still happens
when connecting to an IP address, which is necessary to keep
checking that the certificate returned includes the IP address,
which one of the tests depends on.

Revert the previous change that modified the regexp used for
checking the error message.

https://github.com/ruby/net-http/commit/fa68e64bee

lib/net/http.rb

@@ -22,6 +22,7 @@
 
 require 'net/protocol'
 require 'uri'
+require 'resolv'
 autoload :OpenSSL, 'openssl'
 
 module Net   #:nodoc:
@@ -1036,17 +1037,34 @@ module Net   #:nodoc:
           OpenSSL::SSL::SSLContext::SESSION_CACHE_CLIENT |
           OpenSSL::SSL::SSLContext::SESSION_CACHE_NO_INTERNAL_STORE
         @ssl_context.session_new_cb = proc {|sock, sess| @ssl_session = sess }
+
+        # Still do the post_connection_check below even if connecting
+        # to IP address
+        verify_hostname = @ssl_context.verify_hostname
+
+        # Server Name Indication (SNI) RFC 3546/6066
+        case @address
+        when Resolv::IPv4::Regex, Resolv::IPv6::Regex
+          # don't set SNI, as IP addresses in SNI is not valid
+          # per RFC 6066, section 3.
+
+          # Avoid openssl warning
+          @ssl_context.verify_hostname = false
+        else
+          ssl_host_address = @address
+        end
+
         debug "starting SSL for #{conn_addr}:#{conn_port}..."
         s = OpenSSL::SSL::SSLSocket.new(s, @ssl_context)
         s.sync_close = true
-        # Server Name Indication (SNI) RFC 3546
-        s.hostname = @address if s.respond_to? :hostname=
+        s.hostname = ssl_host_address if s.respond_to?(:hostname=) && ssl_host_address
+
         if @ssl_session and
            Process.clock_gettime(Process::CLOCK_REALTIME) < @ssl_session.time.to_f + @ssl_session.timeout
           s.session = @ssl_session
         end
         ssl_socket_connect(s, @open_timeout)
-        if (@ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE) && @ssl_context.verify_hostname
+        if (@ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE) && verify_hostname
           s.post_connection_check(@address)
         end
         debug "SSL established, protocol: #{s.ssl_version}, cipher: #{s.cipher[0]}"

test/net/http/test_https.rb

@@ -255,7 +255,7 @@ class TestNetHTTPS < Test::Unit::TestCase
     ex = assert_raise(OpenSSL::SSL::SSLError){
       http.request_get("/") {|res| }
     }
-    re_msg = /certificate verify failed|hostname \"#{HOST_IP}\" does not match|ssl3 ext invalid servername/
+    re_msg = /certificate verify failed|hostname \"#{HOST_IP}\" does not match/
     assert_match(re_msg, ex.message)
   end