Is there a predefined class for URL in Python?

Question

I am looking for something like java.net.URL in python-modules, Django, Zope or wherever in Python. I want it preferably from the semantics reason, because the result of analysis of concerned program implies that the URL plays an essential role in it. The consequence is that such URL class also will have great practical usage in that program.

Of course I could write such class on my own, but I'd like to look around before I start to reinvent the wheel.

I did look at urllib2 and urlparse. The urlparse basically has the functionality I need, but it doesn't encapsulate it into a class like java.net.URL. Regarding my analysis of my program it works upside-down.

I looked also into the source code of urlparse at the classes SplitResult and ParseResult. They have some basic functionality and they can be used for subclassing. But I'll have to rewrite rest of the urlparse functions as the subclass methods.

I found also mxURL - Flexible URL Datatype for Python. It is very close to what I really want. Only it seems to be quite an overkill for my purpose.

Can anyone suggest another option? Should I proceed with reinventing the wheel?

My solution:

To get my URL class I did basically two things:

1. Inherit from urlparse.ResultMixin.
2. Define function which only calls urlparse.urlparse() and transforms results to parameters of URL instance.

Menispermaceous

Why do you need a class for that? – Eck

@Cat Plus Plus: having a class for URLs can be very convenient. So convenient in fact, that the Python standard library includes one. – Bowerman

@larsmans: It's not that much more than a named tuple, really. – Eck

@Cat Plus Plus: what more would you expect from a URL class? ;) – Bowerman

I think having a URL class is a great idea. A URL is a value object much like any other. Maybe it's more Java philosophy than Python, but, for example, constructing one from a string then getting scheme, host, path etc is a very strong case. Looks like urlparse does this job fine, but doesn't undermine the case for the Java class. – Giesecke

@larsmans I would expect setting GET arguments using a url[key] = value syntax. – Outrigger

Answer 1

You might want consider having a look at furl because it might be an answer to your needs.

Sharma

Thanks. I really like it! I'll give it a try ;-) – Menispermaceous

Downside of furl is that is doesn't (and won't ) handle params. But the question is who needs params. – Menispermaceous

@Menispermaceous from the GitHub url the maintainer says "If their use grows, I'll happily add them (or accept a pull request)." If you need params, I'm sure you could contribute it! :) – Marinna

Answer 2

urlparse does encapsulate URLs into a class, called ParseResult, so it can be considered a factory function for these. Straight from the Python docs:

>>> urlparse('http://www.cwi.nl:80/%7Eguido/Python.html')
ParseResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
            params='', query='', fragment='')

If you desperately want a class called URL to encapsulate your URLs, use an alias (URL = urlparse.ParseResult) or create an adapter.

Bowerman

ParseResult is namedtuple. I can't figure out why. I want to change existing url but there's no clear way to do this with urllib. – Unwisdom

@Unwisdom Now, in future we have pathlib in standard library which is immutable. Although you can easily produce new Path changed somehow. – Selfpity

Answer 3

~10 years late to the party here, but today, pydantic provides several URL types that might be helpful for validating, storing and passing around URLs; with type hints and mypy becoming more and more prevalent nowadays, some might consider this some kind of standard.

Zestful

Pydantic network types: docs.pydantic.dev/latest/api/networks/#pydantic.networks – Aachen

Answer 4

What we have as of 2018:

• purl,
• YURL,
• furl.

Only furl is being maintained today but its major disadvantage is that it's mutable, that doesn't encourage best practices, of course. (There is good modern reference — pathlib which consists of immutable classes.)

Overall, having a painless OO way to parse and construct URLs is graeat.

Update

yarl is worth looking at.

Selfpity

Answer 5

urlpath is my go-to for a URL object. It mirrors the pathlib Path object.

Oratorical

Answer 6

>>> import urllib3
>>> help(urllib3.util.url)
...
CLASSES
Url(builtins.tuple)
    Url

class Url(Url)
 |  Url(scheme=None, auth=None, host=None, port=None, path=None, query=None, fragment=None)
 |  
 |  Data structure for representing an HTTP URL. Used as a return value for
 |  :func:`parse_url`. Both the scheme and host are normalized as they are
 |  both case-insensitive according to RFC 3986.
 |  
 |  Method resolution order:
 |      Url
 |      Url
 |      builtins.tuple
 |      builtins.object

Cammiecammy