1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2022-11-09 12:17:21 -05:00
ruby--ruby/lib/net/http.rb

1135 lines
25 KiB
Ruby
Raw Normal View History

=begin
= net/http.rb version 1.2.0
Copyright (C) 1999-2001 Yukihiro Matsumoto
written & maintained by Minero Aoki <aamine@dp.u-netsurf.ne.jp>
This file is derived from "http-access.rb".
This program is free software. You can re-distribute and/or
modify this program under the same terms as Ruby itself,
GNU General Public License or Ruby License.
Japanese version of this document is in "net" full package.
You can get it from RAA (Ruby Application Archive). RAA is:
http://www.ruby-lang.org/en/raa.html
== class Net::HTTP
=== Class Methods
: new( address = 'localhost', port = 80, proxy_addr = nil, proxy_port = nil )
creates a new Net::HTTP object.
If proxy_addr is given, this method is equals to
Net::HTTP::Proxy(proxy_addr,proxy_port).
: start( address = 'localhost', port = 80, proxy_addr = nil, proxy_port = nil )
: start( address = 'localhost', port = 80, proxy_addr = nil, proxy_port = nil ) {|http| .... }
is equals to
Net::HTTP.new( address, port, proxy_addr, proxy_port ).start(&block)
: get( address, path, port = 80 )
gets entity body from path and returns it.
return value is a String.
: get_print( address, path, port = 80 )
gets entity body from path and print it.
return value is an entity body (a String).
: Proxy( address, port )
creates a HTTP proxy class.
Arguments are address/port of proxy host.
You can replace HTTP class by this proxy class.
# example
proxy_http = HTTP::Proxy( 'proxy.foo.org', 8080 )
:
proxy_http.start( 'www.ruby-lang.org' ) do |http|
# connecting proxy.foo.org:8080
:
end
: proxy_class?
If self is HTTP, false.
If self is a class which was created by HTTP::Proxy(), true.
: port
HTTP default port (80).
=== Instance Methods
: start
: start {|http| .... }
creates a new Net::HTTP object and starts HTTP session.
When this method is called with block, gives HTTP object to block
and close HTTP session after block call finished.
: proxy?
true if self is a HTTP proxy class
: proxy_address
address of proxy host. If self is not a proxy, nil.
: proxy_port
port number of proxy host. If self is not a proxy, nil.
: get( path, header = nil, dest = '' )
: get( path, header = nil ) {|str| .... }
gets data from "path" on connecting host.
"header" must be a Hash like { 'Accept' => '*/*', ... }.
Response body is written into "dest" by using "<<" method.
This method returns Net::HTTPResponse object.
# example
response = http.get( '/index.html' )
If called with block, give a part String of entity body.
: head( path, header = nil )
gets only header from "path" on connecting host.
"header" is a Hash like { 'Accept' => '*/*', ... }.
This method returns a Net::HTTPResponse object.
You can http header from this object like:
response['content-length'] #-> '2554'
response['content-type'] #-> 'text/html'
response['Content-Type'] #-> 'text/html'
response['CoNtEnT-tYpe'] #-> 'text/html'
: post( path, data, header = nil, dest = '' )
: post( path, data, header = nil ) {|str| .... }
posts "data" (must be String) to "path".
If the body exists, also gets entity body.
Response body is written into "dest" by using "<<" method.
"header" must be a Hash like { 'Accept' => '*/*', ... }.
This method returns Net::HTTPResponse object.
If called with block, gives a part of entity body string.
: request( request, [src] )
: request( request, [src] ) {|response| .... }
sends REQUEST to (remote) http server. This method also writes
string from SRC before it if REQUEST is a post/put request.
(giving SRC for get/head request causes ArgumentError.)
If called with block, gives a HTTP response object to the block.
== class Net::HTTP::Get, Head, Post
HTTP request classes. These classes wraps request header and
entity path. All arguments named "key" is case-insensitive.
=== Class Methods
: new
creats HTTP request object.
=== Methods
: self[ key ]
returns the header field corresponding to the case-insensitive key.
For example, a key of "Content-Type" might return "text/html"
: self[ key ] = val
sets the header field corresponding to the case-insensitive key.
== class Net::HTTPResponse
HTTP response class. This class wraps response header and entity.
All arguments named KEY is case-insensitive.
=== Methods
: self[ key ]
returns the header field corresponding to the case-insensitive key.
For example, a key of "Content-Type" might return "text/html".
A key of "Content-Length" might do "2045".
More than one fields which has same names are joined with ','.
: self[ key ] = val
sets the header field corresponding to the case-insensitive key.
: key?( key )
true if key exists.
KEY is case insensitive.
: each {|name,value| .... }
iterates for each field name and value pair.
: canonical_each {|name,value| .... }
iterates for each canonical field name and value pair.
: code
HTTP result code string. For example, '302'.
: message
HTTP result message. For example, 'Not Found'.
: read_body( dest = '' )
gets response body and write it into DEST using "<<" method.
If this method is called twice or more, nothing will be done
and returns first DEST.
: read_body {|str| .... }
gets response body little by little and pass it to block.
: body
response body. If #read_body has been called, this method
returns arg of #read_body, DEST. Else gets body as String
and returns it.
== Switching Net::HTTP versions
You can use Net::HTTP 1.1 features by calling HTTP.version_1_1 .
And calling Net::HTTP.version_1_2 allows you to use 1.2 features
again.
# example
HTTP.start {|http1| ...(http1 has 1.2 features)... }
HTTP.version_1_1
HTTP.start {|http2| ...(http2 has 1.1 features)... }
HTTP.version_1_2
HTTP.start {|http3| ...(http3 has 1.2 features)... }
=end
require 'net/protocol'
module Net
class HTTPBadResponse < StandardError; end
class HTTPHeaderSyntaxError < StandardError; end
class HTTP < Protocol
HTTPVersion = '1.1'
#
# connection
#
protocol_param :port, '80'
def initialize( addr = nil, port = nil )
super
@proxy_address = nil
@proxy_port = nil
@curr_http_version = HTTPVersion
@seems_1_0_server = false
end
private
def conn_command( sock )
end
def do_finish
end
#
# proxy
#
public
class << self
def Proxy( p_addr, p_port = nil )
ProxyMod.create_proxy_class( p_addr, p_port || self.port )
end
alias orig_new new
def new( address = nil, port = nil, p_addr = nil, p_port = nil )
c = p_addr ? self::Proxy(p_addr, p_port) : self
i = c.orig_new( address, port )
setvar i
i
end
def start( address = nil, port = nil, p_addr = nil, p_port = nil, &block )
new( address, port, p_addr, p_port ).start( &block )
end
def proxy_class?
false
end
def proxy_address
nil
end
def proxy_port
nil
end
end
def proxy?
false
end
def proxy_address
nil
end
def proxy_port
nil
end
def edit_path( path )
path
end
module ProxyMod
class << self
def create_proxy_class( p_addr, p_port )
mod = self
klass = Class.new( HTTP )
klass.module_eval {
include mod
@proxy_address = p_addr
@proxy_port = p_port
}
def klass.proxy_class?
true
end
def klass.proxy_address
@proxy_address
end
def klass.proxy_port
@proxy_port
end
klass
end
end
def initialize( addr, port )
super
@proxy_address = type.proxy_address
@proxy_port = type.proxy_port
end
attr_reader :proxy_address, :proxy_port
alias proxyaddr proxy_address
alias proxyport proxy_port
def proxy?
true
end
private
def conn_socket( addr, port )
super @proxy_address, @proxy_port
end
def edit_path( path )
'http://' + addr_port + path
end
end # module ProxyMod
#
# for backward compatibility
#
@@newimpl = true
class << self
def version_1_2
@@newimpl = true
end
def version_1_1
@@newimpl = false
end
private
def setvar( obj )
f = @@newimpl
obj.instance_eval { @newimpl = f }
end
end
#
# http operations
#
public
def self.def_http_method( nm, hasdest, hasdata )
name = nm.id2name.downcase
cname = nm.id2name
lineno = __LINE__ + 2
src = <<" ----"
def #{name}( path, #{hasdata ? 'data,' : ''}
u_header = nil #{hasdest ? ',dest = nil, &block' : ''} )
resp = nil
request(
#{cname}.new( path, u_header ) #{hasdata ? ',data' : ''}
) do |resp|
resp.read_body( #{hasdest ? 'dest, &block' : ''} )
end
if @newimpl then
resp
else
resp.value
#{hasdest ? 'return resp, resp.body' : 'resp'}
end
end
def #{name}2( path, #{hasdata ? 'data,' : ''}
u_header = nil, &block )
request( #{cname}.new(path, u_header),
#{hasdata ? 'data,' : ''} &block )
end
----
module_eval src, __FILE__, lineno
end
def_http_method :Get, true, false
def_http_method :Head, false, false
def_http_method :Post, true, true
def_http_method :Put, false, true
def request( req, *args )
common_oper( req ) {
req.__send__( :exec,
@socket, @curr_http_version, edit_path(req.path), *args )
yield req.response if block_given?
}
req.response
end
private
def common_oper( req )
req['connection'] ||= 'keep-alive'
if not @socket then
start
req['connection'] = 'close'
elsif @socket.closed? then
re_connect
end
if not req.body_exist? or @seems_1_0_server then
req['connection'] = 'close'
end
req['host'] = addr_port
yield req
req.response.__send__ :terminate
@curr_http_version = req.response.http_version
if not req.response.body then
@socket.close
elsif keep_alive? req, req.response then
D 'Conn keep-alive'
if @socket.closed? then # (only) read stream had been closed
D 'Conn (but seems 1.0 server)'
@seems_1_0_server = true
@socket.close
end
else
D 'Conn close'
@socket.close
end
req.response
end
def keep_alive?( req, res )
/close/i === req['connection'].to_s and return false
@seems_1_0_server and return false
/keep-alive/i === res['connection'].to_s and return true
/close/i === res['connection'].to_s and return false
/keep-alive/i === res['proxy-connection'].to_s and return true
/close/i === res['proxy-connection'].to_s and return false
@curr_http_version == '1.1' and return true
false
end
#
# utils
#
public
def self.get( addr, path, port = nil )
req = Get.new( path )
resp = nil
new( addr, port || HTTP.port ).start {|http|
resp = http.request( req )
}
resp.body
end
def self.get_print( addr, path, port = nil )
print get( addr, path, port )
end
private
def addr_port
address + (port == HTTP.port ? '' : ":#{port}")
end
def D( msg )
if @dout then
@dout << msg
@dout << "\n"
end
end
end
HTTPSession = HTTP
class Code
def http_mkchild( bodyexist = nil )
c = mkchild(nil)
be = if bodyexist.nil? then @body_exist else bodyexist end
c.instance_eval { @body_exist = be }
c
end
def body_exist?
@body_exist
end
end
HTTPInformationCode = InformationCode.http_mkchild( false )
HTTPSuccessCode = SuccessCode .http_mkchild( true )
HTTPRedirectionCode = RetriableCode .http_mkchild( true )
HTTPRetriableCode = HTTPRedirectionCode
HTTPClientErrorCode = FatalErrorCode .http_mkchild( true )
HTTPFatalErrorCode = HTTPClientErrorCode
HTTPServerErrorCode = ServerErrorCode.http_mkchild( true )
HTTPSwitchProtocol = HTTPInformationCode.http_mkchild
HTTPOK = HTTPSuccessCode.http_mkchild
HTTPCreated = HTTPSuccessCode.http_mkchild
HTTPAccepted = HTTPSuccessCode.http_mkchild
HTTPNonAuthoritativeInformation = HTTPSuccessCode.http_mkchild
HTTPNoContent = HTTPSuccessCode.http_mkchild( false )
HTTPResetContent = HTTPSuccessCode.http_mkchild( false )
HTTPPartialContent = HTTPSuccessCode.http_mkchild
HTTPMultipleChoice = HTTPRedirectionCode.http_mkchild
HTTPMovedPermanently = HTTPRedirectionCode.http_mkchild
HTTPMovedTemporarily = HTTPRedirectionCode.http_mkchild
HTTPNotModified = HTTPRedirectionCode.http_mkchild( false )
HTTPUseProxy = HTTPRedirectionCode.http_mkchild( false )
HTTPBadRequest = HTTPClientErrorCode.http_mkchild
HTTPUnauthorized = HTTPClientErrorCode.http_mkchild
HTTPPaymentRequired = HTTPClientErrorCode.http_mkchild
HTTPForbidden = HTTPClientErrorCode.http_mkchild
HTTPNotFound = HTTPClientErrorCode.http_mkchild
HTTPMethodNotAllowed = HTTPClientErrorCode.http_mkchild
HTTPNotAcceptable = HTTPClientErrorCode.http_mkchild
HTTPProxyAuthenticationRequired = HTTPClientErrorCode.http_mkchild
HTTPRequestTimeOut = HTTPClientErrorCode.http_mkchild
HTTPConflict = HTTPClientErrorCode.http_mkchild
HTTPGone = HTTPClientErrorCode.http_mkchild
HTTPLengthRequired = HTTPClientErrorCode.http_mkchild
HTTPPreconditionFailed = HTTPClientErrorCode.http_mkchild
HTTPRequestEntityTooLarge = HTTPClientErrorCode.http_mkchild
HTTPRequestURITooLarge = HTTPClientErrorCode.http_mkchild
HTTPUnsupportedMediaType = HTTPClientErrorCode.http_mkchild
HTTPNotImplemented = HTTPServerErrorCode.http_mkchild
HTTPBadGateway = HTTPServerErrorCode.http_mkchild
HTTPServiceUnavailable = HTTPServerErrorCode.http_mkchild
HTTPGatewayTimeOut = HTTPServerErrorCode.http_mkchild
HTTPVersionNotSupported = HTTPServerErrorCode.http_mkchild
###
### header
###
net_private {
module HTTPHeader
def size
@header.size
end
alias length size
def []( key )
@header[ key.downcase ]
end
def []=( key, val )
@header[ key.downcase ] = val
end
def each( &block )
@header.each( &block )
end
def each_key( &block )
@header.each_key( &block )
end
def each_value( &block )
@header.each_value( &block )
end
def delete( key )
@header.delete key.downcase
end
def key?( key )
@header.key? key.downcase
end
def to_hash
@header.dup
end
def canonical_each
@header.each do |k,v|
yield canonical(k), v
end
end
def canonical( k )
k.split('-').collect {|i| i.capitalize }.join('-')
end
def range
s = @header['range']
s or return nil
arr = []
s.split(',').each do |spec|
m = /bytes\s*=\s*(\d+)?\s*-\s*(\d+)?/i.match( spec )
m or raise HTTPHeaderSyntaxError, "wrong Range: #{spec}"
d1 = m[1].to_i
d2 = m[2].to_i
if m[1] and m[2] then arr.push (d1 .. d2)
elsif m[1] then arr.push (d1 .. -1)
elsif m[2] then arr.push (-d2 .. -1)
else
raise HTTPHeaderSyntaxError, 'range is not specified'
end
end
return *arr
end
def range=( r, fin = nil )
if fin then
r = r ... r+fin
end
case r
when Numeric
s = r > 0 ? "0-#{r - 1}" : "-#{-r}"
when Range
first = r.first
last = r.last
if r.exclude_end? then
last -= 1
end
if last == -1 then
s = first > 0 ? "#{first}-" : "-#{-first}"
else
first >= 0 or raise HTTPHeaderSyntaxError, 'range.first is negative'
last > 0 or raise HTTPHeaderSyntaxError, 'range.last is negative'
first < last or raise HTTPHeaderSyntaxError, 'must be .first < .last'
s = "#{first}-#{last}"
end
else
raise TypeError, 'Range/Integer is required'
end
@header['range'] = "bytes=#{s}"
r
end
alias set_range range=
def content_length
s = @header['content-length']
s or return nil
m = /\d+/.match(s)
m or raise HTTPHeaderSyntaxError, 'wrong Content-Length format'
m[0].to_i
end
def chunked?
s = @header['transfer-encoding']
(s and /(?:\A|[^\-\w])chunked(?:[^\-\w]|\z)/i === s) ? true : false
end
def content_range
s = @header['content-range']
s or return nil
m = %r<bytes\s+(\d+)-(\d+)/(?:\d+|\*)>i.match( s )
m or raise HTTPHeaderSyntaxError, 'wrong Content-Range format'
m[1].to_i .. m[2].to_i + 1
end
def range_length
r = content_range
r and r.length
end
def basic_auth( acc, pass )
@header['authorization'] = ["#{acc}:#{pass}"].pack('m').gsub(/\s+/, '')
end
end
}
###
### request
###
net_private {
class HTTPRequest
include ::Net::NetPrivate::HTTPHeader
def initialize( path, uhead = nil )
@path = path
@header = tmp = {}
return unless uhead
uhead.each do |k,v|
key = k.downcase
if tmp.key? key then
$stderr.puts "WARNING: duplicated HTTP header: #{k}" if $VERBOSE
end
tmp[ key ] = v.strip
end
tmp['accept'] ||= '*/*'
@socket = nil
@response = nil
end
attr_reader :path
attr_reader :response
def inspect
"\#<#{type}>"
end
def body_exist?
type::HAS_BODY
end
private
#
# write
#
def exec( sock, ver, path )
ready( sock ) {
request ver, path
}
@response
end
def ready( sock )
@response = nil
@socket = sock
yield
@response = get_response
@socket = nil
end
def request( ver, path )
@socket.writeline sprintf('%s %s HTTP/%s', type::METHOD, path, ver)
canonical_each do |k,v|
@socket.writeline k + ': ' + v
end
@socket.writeline ''
end
#
# read
#
def get_response
begin
resp = read_response
end while ContinueCode === resp
resp
end
def read_response
resp = get_resline
while true do
line = @socket.readuntil( "\n", true ) # ignore EOF
line.sub!( /\s+\z/, '' ) # don't use chop!
break if line.empty?
m = /\A([^:]+):\s*/.match( line )
m or raise HTTPBadResponse, 'wrong header line format'
nm = m[1]
line = m.post_match
if resp.key? nm then
resp[nm] << ', ' << line
else
resp[nm] = line
end
end
resp
end
def get_resline
str = @socket.readline
m = /\AHTTP(?:\/(\d+\.\d+))?\s+(\d\d\d)\s*(.*)\z/i.match( str )
m or raise HTTPBadResponse, "wrong status line: #{str}"
httpver = m[1]
status = m[2]
discrip = m[3]
::Net::NetPrivate::HTTPResponse.new(
status, discrip, @socket, type::HAS_BODY, httpver )
end
end
class HTTPRequestWithBody < HTTPRequest
private
def exec( sock, ver, path, str = nil )
check_arg str, block_given?
if block_given? then
ac = Accumulator.new
yield ac # must be yield, DO NOT USE block.call
data = ac.terminate
else
data = str
end
@header['content-length'] = data.size.to_s
@header.delete 'transfer-encoding'
ready( sock ) {
request ver, path
@socket.write data
}
@response
end
def check_arg( data, blkp )
if data and blkp then
raise ArgumentError, 'both of data and block given'
end
unless data or blkp then
raise ArgumentError, 'str or block required'
end
end
end
class Accumulator
def initialize
@buf = ''
end
def write( s )
@buf.concat s
end
alias << write
def terminate
ret = @buf
@buf = nil
ret
end
end
}
class HTTP
class Get < ::Net::NetPrivate::HTTPRequest
HAS_BODY = true
METHOD = 'GET'
end
class Head < ::Net::NetPrivate::HTTPRequest
HAS_BODY = false
METHOD = 'HEAD'
end
class Post < ::Net::NetPrivate::HTTPRequestWithBody
HAS_BODY = true
METHOD = 'POST'
end
class Put < ::Net::NetPrivate::HTTPRequestWithBody
HAS_BODY = true
METHOD = 'PUT'
end
end
###
### response
###
net_private {
class HTTPResponse < Response
include ::Net::NetPrivate::HTTPHeader
CODE_CLASS_TO_OBJ = {
'1' => HTTPInformationCode,
'2' => HTTPSuccessCode,
'3' => HTTPRedirectionCode,
'4' => HTTPClientErrorCode,
'5' => HTTPServerErrorCode
}
CODE_TO_OBJ = {
'100' => ContinueCode,
'101' => HTTPSwitchProtocol,
'200' => HTTPOK,
'201' => HTTPCreated,
'202' => HTTPAccepted,
'203' => HTTPNonAuthoritativeInformation,
'204' => HTTPNoContent,
'205' => HTTPResetContent,
'206' => HTTPPartialContent,
'300' => HTTPMultipleChoice,
'301' => HTTPMovedPermanently,
'302' => HTTPMovedTemporarily,
'303' => HTTPMovedPermanently,
'304' => HTTPNotModified,
'305' => HTTPUseProxy,
'400' => HTTPBadRequest,
'401' => HTTPUnauthorized,
'402' => HTTPPaymentRequired,
'403' => HTTPForbidden,
'404' => HTTPNotFound,
'405' => HTTPMethodNotAllowed,
'406' => HTTPNotAcceptable,
'407' => HTTPProxyAuthenticationRequired,
'408' => HTTPRequestTimeOut,
'409' => HTTPConflict,
'410' => HTTPGone,
'411' => HTTPFatalErrorCode,
'412' => HTTPPreconditionFailed,
'413' => HTTPRequestEntityTooLarge,
'414' => HTTPRequestURITooLarge,
'415' => HTTPUnsupportedMediaType,
'500' => HTTPFatalErrorCode,
'501' => HTTPNotImplemented,
'502' => HTTPBadGateway,
'503' => HTTPServiceUnavailable,
'504' => HTTPGatewayTimeOut,
'505' => HTTPVersionNotSupported
}
def initialize( stat, msg, sock, be, hv )
code = CODE_TO_OBJ[stat] ||
CODE_CLASS_TO_OBJ[stat[0,1]] ||
UnknownCode
super code, stat, msg
@socket = sock
@body_exist = be
@http_version = hv
@header = {}
@body = nil
@read = false
end
attr_reader :http_version
def inspect
"#<#{type} #{code}>"
end
def value
SuccessCode === self or error! self
end
#
# header (for backward compatibility)
#
def response
self
end
alias header response
alias read_header response
#
# body
#
def read_body( dest = nil, &block )
if @read and (dest or block) then
raise IOError, "#{type}\#read_body called twice with argument"
end
unless @read then
to = procdest( dest, block )
stream_check
if @body_exist and code_type.body_exist? then
read_body_0 to
@body = to
else
@body = nil
end
@read = true
end
@body
end
alias body read_body
alias entity read_body
private
def terminate
read_body
end
def read_body_0( dest )
if chunked? then
read_chunked dest
else
clen = content_length
if clen then
@socket.read clen, dest, true # ignore EOF
else
clen = range_length
if clen then
@socket.read clen, dest
else
@socket.read_all dest
end
end
end
end
def read_chunked( dest )
len = nil
total = 0
while true do
line = @socket.readline
m = /[0-9a-fA-F]+/.match( line )
m or raise HTTPBadResponse, "wrong chunk size line: #{line}"
len = m[0].hex
break if len == 0
@socket.read( len, dest ); total += len
@socket.read 2 # \r\n
end
until @socket.readline.empty? do
;
end
end
def stream_check
@socket.closed? and raise IOError, 'try to read body out of block'
end
def procdest( dest, block )
if dest and block then
raise ArgumentError, 'both of arg and block are given for HTTP method'
end
if block then
::Net::NetPrivate::ReadAdapter.new block
else
dest || ''
end
end
end
}
end # module Net