Mocking urllib2.urlopen().read() for different responses

Question

I am trying to mock the urllib2.urlopen library in a way that I should get different responses for different urls I pass into the function.

The way I am doing it in my test file now is like this

@patch(othermodule.urllib2.urlopen)
def mytest(self, mock_of_urllib2_urllopen):
    a = Mock()
    a.read.side_effect = ["response1", "response2"]
    mock_of_urllib2_urlopen.return_value = a
    othermodule.function_to_be_tested()   #this is the function which uses urllib2.urlopen.read

I expect the the othermodule.function_to_be_tested to get the value "response1" on first call and "response2" on second call which is what side_effect will do

but the othermodule.function_to_be_tested() receives

<MagicMock name='urlopen().read()' id='216621051472'>

and not the actual response. Please suggest where I am going wrong or an easier way to do this.

Fleda

You can just patch @patch(urllib2.urlopen) directly.. – Forbearance

I am trying to patch the copy which is imported in my other module. I guess thats the way it's supposed to be done – Fleda

I would personally just redesign my code to not be hardcoded to use urllib.urlopen at all; e.g. it would be calling self.urlopen_fn whose default value is urllib.urlopen but which you can just set to your_mock_urlopen during testing; it can even be a module-level parameter you can set from the outside. – Lashundalasker

You imported urllib2, so both othermodule.urllib2.urlopen and urllib2.urlopen refer to the exact same function object. – Forbearance

@ErikAllik: mock should make this easy. – Forbearance

@MartijnPieters: yeah, I know, but I think designing your code to not need such patch-mocking make it better overall. – Lashundalasker

I don't agree; you still need to use a standard library call somewhere at some point. – Forbearance

See also: How can one mock/stub python module like urllib – Rathbone

Answer 1

The argument to patch needs to be a description of the location of the object, not the object itself. So your problem looks like it may just be that you need to stringify your argument to patch.

Just for completeness, though, here's a fully working example. First, our module under test:

# mod_a.py
import urllib2

def myfunc():
    opened_url = urllib2.urlopen()
    return opened_url.read()

Now, set up our test:

# test.py
from mock import patch, Mock
import mod_a

@patch('mod_a.urllib2.urlopen')
def mytest(mock_urlopen):
    a = Mock()
    a.read.side_effect = ['resp1', 'resp2']
    mock_urlopen.return_value = a
    res = mod_a.myfunc()
    print res
    assert res == 'resp1'

    res = mod_a.myfunc()
    print res
    assert res == 'resp2'

mytest()

Running the test from the shell:

$ python test.py
resp1
resp2

Edit: Whoops, initially included the original mistake. (Was testing to verify how it was broken.) Code should be fixed now.

Seam