Fix bug with fragments in referer headers in RackTest

We discovered that if the RackTest browser has a current_url that
includes a fragment, then when a request is made that includes
current_url as a referrer (eg clicking a link, submitting a form) then
it will include the fragment. Accordng to the HTTP spec this is not
allowed:

     user agent MUST NOT include the fragment and userinfo components of the
     URI reference [RFC3986], if any, when generating the Referer field value.

See: https://httpwg.org/specs/rfc7231.html#header.referer

This seems to have been introduced when support for including the
fragment in #current_url was added in 1a48bc7716

Co-authored-by: Thao Vo <thao.vo@futurelearn.com>

lib/capybara/rack_test/browser.rb

@@ -33,13 +33,13 @@ class Capybara::RackTest::Browser
     path = request_path if path.nil? || path.empty?
     uri = build_uri(path)
     uri.query = '' if method.to_s.casecmp('get').zero?
-    process_and_follow_redirects(method, uri.to_s, attributes, 'HTTP_REFERER' => current_url)
+    process_and_follow_redirects(method, uri.to_s, attributes, 'HTTP_REFERER' => referer_url)
   end
 
   def follow(method, path, **attributes)
     return if fragment_or_script?(path)
 
-    process_and_follow_redirects(method, path, attributes, 'HTTP_REFERER' => current_url)
+    process_and_follow_redirects(method, path, attributes, 'HTTP_REFERER' => referer_url)
   end
 
   def process_and_follow_redirects(method, path, attributes = {}, env = {})
@@ -141,4 +141,10 @@ private
   def fragment_or_script?(path)
     path.gsub(/^#{Regexp.escape(request_path)}/, '').start_with?('#') || path.downcase.start_with?('javascript:')
   end
+
+  def referer_url
+    build_uri(last_request.url).to_s
+  rescue Rack::Test::Error
+    ''
+  end
 end

spec/rack_test_spec.rb

@@ -216,6 +216,12 @@ RSpec.describe Capybara::RackTest::Driver do
       expect(driver.current_url).to match %r{/landed$}
     end
 
+    it 'should not include fragments in the referer header' do
+      driver.visit('/header_links#an-anchor')
+      driver.find_xpath('.//input').first.click
+      expect(driver.request.get_header("HTTP_REFERER")).to eq('http://www.example.com/header_links')
+    end
+
     it 'is possible to not follow redirects' do
       driver = described_class.new(TestApp, follow_redirects: false)