Python: trying to raise multi-purpose exception for multiple error types

Question

I'm working with urllib and urllib2 in python and am using them to retrieve images from urls.

Using something similar to :

try:
    buffer=urllib2.url_open(urllib2.Request(url))
    f.write(buffer)
    f.close
except (Errors that could occur):    #Network Errors(?)
    print "Failed to retrieve "+url
    pass

Now what happens often is that the image does not load/is broken when using the site via a normal web browser this is presumably because of high server load or because the image does not exist or could not be retrieved by the server.

Whatever the reason may be, the image does not load and a similar situation can also/is likely to occur when using the script. Since I do not know what error it might it throw up how do I handle it?

I think mentioning all possible errors in the urllib2,urllib library in the except statement might be overkill so I need a better way.

(I also might need to/have to handle broken Wi-Fi, unreachable server and the like at times so more errors)

Indiraindirect

Doing [e for e in dir(urllib2) if 'rror' in e] gives me ['HTTPDefaultErrorHandler', 'HTTPError', 'HTTPErrorProcessor', 'URLError']. Does that help at all? – Baggy

So this means it will always either be an HTTPError or URLError? – Indiraindirect

Those are only errors in urllib2. So if you want to catch "all" errors explicitly, those are the ones that you should list. If you're having trouble catching multiple exceptions in one exception block, look at this question – Baggy

btw, if any of our answers answered your question you should accept the one that did. – Coble

Answer 1

There are only two exceptions you'll see, HTTPError (HTTP status codes) and URLError (everything that can go wrong), so it's not like it's overkill handling both of them. You can even just catch URLError if you don't care about status codes, since HTTPError is a subclass of it.

Rhodos

grepping "error" (grep -in 'class .*error' ...) in the source directory confirms this :-). – Flavorful

That's not true. Try something like: try: urlopen('') except Exception, e: print type(e), and you will get ValueError, for example. – Growing

Is an HTTPError caught by the except URLError statement? And if it is, might it be possible to use the Exception base class to catch all possible exceptions? – Indiraindirect

Yes, HTTPError is derived from URLError. cababunga, I suppose that's true, but irrelevant if sanity checking has been done beforehand. – Rhodos

@Ayos the Exception base class is redundant, except: catches all errors. – Coble

@Coble except Exception: is not redundant; most people who want to "catch all errors" don't actually want to catch KeyboardInterrupt or SystemExit, which inherit directly from BaseException – Pepito

@AirThomas thanks for catching that. I didn't know about BaseException back when I wrote that, kind of like basestring. – Coble

Answer 2

If you just want to print explanation of what happened, just print the exception itself.

except Exception, e: 
    print e

Exception object's str() method will retrieve human readable message for you.

Growing

OP wants to catch only network errors. This will catch all errors (eg 1+'' results in an error that will be caught here) – Baggy

Answer 3

This is a great wiki article on handling exceptions in python. You can use SilverbackNet's Answer or, if you want to catch all the errors there is a nice recipe in here. It's also great for logging a traceback and finding out any other possible errors that were raised.

import sys
try:
    buffer=urllib2.url_open(urllib2.Request(url))
    f.write(buffer)
    f.close() #you didn't have the parentheses in your sample
except: # catch *all* exceptions
    e = sys.exc_info()[0] #this returns a 3-tuple; (type, value, traceback)
    print 'Error: %s' % e

Coble

I used to do that, and still do occasionally for "debug" logging, but that kind of overkill logging gets uselessly spammy for most purposes. Excellent tool for when you aren't sure what to expect out of a statement, though. – Rhodos

Yeah, we're about to deploy servers and it's our first time using python, so we aren't sure what errors to expect. – Coble