forks/archivebox
1
0
Fork 0
Code Releases Activity

Adds HEADLESS_USER_AGENT variable

Browse source 
Allows setting Headless Chrome's User-Agent to bypass rudimentary anti-scraper/anti-bot checks by sites. https://intoli.com/blog/making-chrome-headless-undetectable/ has more detections if there is desire to get serious about anti-detection
This commit is contained in:
noncetonic 2019-03-19 05:30:06 -07:00 committed by GitHub
parent 57c91e900c
commit 127c72bd79
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
etc/ArchiveBox.conf.default
View file

@ -41,6 +41,7 @@
#FETCH_WGET_REQUISITES=True
#RESOLUTION="1440,900"
#WGET_USER_AGENT="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"
#HEADLESS_USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36"
#GIT_DOMAINS="github.com,bitbucket.org,gitlab.com"
#COOKIES_FILE="path/to/cookies.txt"
#CHROME_USER_DATA_DIR="~/.config/google-chrome/Default"