BrowserMob Proxy Python - How to get response body?

Question

I need to get the response body content for a POST request using Selenium Chrome driver and browsermob proxy. Currently this content is not included in my file HAR output when i read it although i can see the response in the browser network traffic. How can i make it so response traffic is captured? (sorry new to programming and can't see much python documentation for BMP)

    server.start()
    proxy = server.create_proxy()
    chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--proxy-server={0}".format(proxy.proxy)) 
    driver = webdriver.Chrome(executable_path="chromedriver.exe", chrome_options=chrome_options)

    proxy.new_har("req", options={'captureHeaders': True,'captureContent':True})
    driver.get('https://www.example.com/something')


    result_har = json.dumps(proxy.har, ensure_ascii=False)
    with open("haroutput.har", "w") as harfile:
        harfile.write(result_har)

    server.stop()
    driver.quit()

Ludhiana

Answer 1

You can get request and response by key with the identical name in proxy.har['log']['entries'] . Response's content is under entry['response']['content']

But before you have to add 'captureContent':True into the option dict of proxy.new_har call.

Example:

from browsermobproxy import Server

server = Server("./bin/browsermob-proxy")

server.start()
proxy = server.create_proxy()

from selenium import webdriver
co = webdriver.ChromeOptions()
co.add_argument('--proxy-server={host}:{port}'.format(host='localhost', port=proxy.port))

driver = webdriver.Chrome(executable_path="chromedriver", chrome_options=co)

proxy.new_har('req',options={'captureHeaders': True,'captureContent':True})

driver.get(url)
proxy.har  # returns a HAR

for ent in proxy.har['log']['entries']:
    _url = ent['request']['url']
    _response = ent['response']
    _content = _response['content']['text']

Ashtonashtonunderlyne

That doesn't seem to be working, entries is a list of dictionary with keys like "method", "url", "cookies", "queryString", etc. But it seems like there is no content (even with 'captureContent':True) – Nata

This worked for me. I was able to find the response data in proxy.har['log']['entries'][0]['response']['content']['text'] – Roz

That's strange, did you manage to have it with the firefox driver too? – Nata

@Nata "method", "url", "cookies", "queryString"... are keys of the request object, but this is a topic about response body – Mc