Nick Sweeting
49c428cb6c
make both user agents equal
2019-03-19 11:16:00 -04:00
noncetonic
8b648696fc
Changes HEADLESS_USER_AGENT to CHROME_USER_AGENT
2019-03-19 08:14:06 -07:00
noncetonic
127c72bd79
Adds HEADLESS_USER_AGENT variable
...
Allows setting Headless Chrome's User-Agent to bypass rudimentary anti-scraper/anti-bot checks by sites. https://intoli.com/blog/making-chrome-headless-undetectable/ has more detections if there is desire to get serious about anti-detection
2019-03-19 05:30:06 -07:00
Nick Sweeting
c729a80eae
Update ArchiveBox.conf.default
2019-03-18 23:13:40 -04:00
Nick Sweeting
fdf1aab1e9
nicer config header
2019-03-18 23:12:23 -04:00
Nick Sweeting
0552ce81ed
better ArchiveBox.conf.default file
2019-02-21 15:56:31 -05:00
Nick Sweeting
1834acfb5f
add cookies example config to conf.default file
2019-02-21 12:58:51 -05:00
Nick Sweeting
5a7d00a639
fetch page title during archiving process
2019-02-19 01:44:54 -05:00
Nick Sweeting
20de451515
add linux config example
2019-01-23 01:48:04 -05:00
Nick Sweeting
95301c9306
better default config
2019-01-23 01:42:55 -05:00
Nick Sweeting
a5fe91da9d
add default config
2019-01-23 01:30:10 -05:00
