Change user agent for selenium driver

Question

I have the following code in Python:

from selenium.webdriver import Firefox

from contextlib import closing

with closing(Firefox()) as browser:

browser.get(url)

I would like to print the user-agent HTTP header and possibly change it. Is it possible? 

Answer 1

There is no way in Selenium to read the request or response headers. You could do it by instructing your browser to connect through a proxy that records this kind of information.

Setting the User-Agent in Firefox

The usual way to change the user agent for Firefox is to set the variable "general.useragent.override" in your Firefox profile. Note that this is independent of Selenium.

You can direct Selenium to use a profile different from the default one, like this:

from selenium import webdriver

profile = webdriver.FirefoxProfile()

profile.set_preference("general.useragent.override", "whatever you want")

driver = webdriver.Firefox(profile) 

 Setting the User-Agent in Chrome

With Chrome, what you want to do is use the user-agent command line option. Again, this is not a Selenium thing. You can invoke Chrome at the command line with chrome --user-agent=foo to set the agent to the value foo.

With Selenium you set it like this:

from selenium import webdriver

from selenium.webdriver.chrome.options import Options

opts = Options()

opts.add_argument("user-agent=whatever you want")

driver = webdriver.Chrome(chrome_options=opts)

Both methods above were tested and found to work. I don’t know about other browsers.

Getting the User-Agent

Selenium does not have methods to query the user agent from an instance of WebDriver. Even in the case of Firefox, you cannot discover the default user agent by checking what general.useragent.override would be if not set to a custom value. (This setting does not exist before it is set to some value.)

Once the browser is started, however, you can get the user agent by executing: 

agent = driver.execute_script("return navigator.userAgent")

The agent variable will contain the user agent. 

To learn in-depth about Selenium, sign up for an industry based Selenium certification.

Comments

Nice explanation, Thanks

---

Very well explained, thanks!

---

Thanks for the detailed answer! This was very helpful.

---

Great answer! Explained very well.

---

Thanks for the well-explained answer!

---

OK, I combined it with my code. So, for setting user-agent is there a set_preference method. Is there something like get_preference as well, to know what it was before?

Answer 2

building on prabhpreet's wonderful explanation, if someone is looking for an answer on  Setting the User Agent in PhantomJS you can use the following

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
...
caps = DesiredCapabilities.PHANTOMJS
caps["phantomjs.page.settings.userAgent"] = "whatever you want"
driver = webdriver.PhantomJS(desired_capabilities=caps)

Comments

user-agent is not a request or response header, but I think it is a general header

Answer 3

If you want to change user agent for selenium driver the use the below-mentioned code which changes the UserAgent of a request but this will only work with Chrome.

from selenium import webdriver from selenium.webdriver.common.desired_capabilities import DesiredCapabilities driver = webdriver.Chrome(driver_path) driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent":"python 2.7", "platform":"Windows"}) driver.get('http://amiunique.org')

The above code will return your user agent which you can get by using the below mentioned piece of code:-

agent = driver.execute_script("return navigator.userAgent")

Comments

This worked for me