Retrieving parameters from a URL

Question

Given a URL like the following, how can I parse the value of the query parameters? For example, in this case I want the value of some_key .

/some_path?some_key=some_value'

I am using Django in my environment; is there a method on the request object that could help me?

I tried using self.request.get('some_key') but it is not returning the value some_value as I had hoped.

Northampton

Answer 1

This is not specific to Django, but for Python in general. For a Django specific answer, see this one from @jball037

Python 2:

import urlparse

url = 'https://www.example.com/some_path?some_key=some_value'
parsed = urlparse.urlparse(url)
captured_value = urlparse.parse_qs(parsed.query)['some_key'][0]

print captured_value

Python 3:

from urllib.parse import urlparse
from urllib.parse import parse_qs

url = 'https://www.example.com/some_path?some_key=some_value'
parsed_url = urlparse(url)
captured_value = parse_qs(parsed_url.query)['some_key'][0]

print(captured_value)

parse_qs returns a list. The [0] gets the first item of the list so the output of each script is some_value

Here's the 'parse_qs' documentation for Python 3

Within

In python3 import urllib.parse as urlparse – Panda

Note that parse_qs will return you a list object. You need to get the first element of it if you want a string urlparse.parse_qs(parsed.query)['def'][0] – Helsa

Note that if url contain '#' character just like this: foo.appspot.com/#/abc?def=ghi, you must disallow fragments: urlparse(url, allow_fragments=False), otherwise query will return empty str. – Murchison

Py3 docs: urlparse; parse_qs. Py2 docs: urlparse; parse_qs. – God

in python3, must use from urllib.parse import urlparse, parse_qs and parse_qs(parsed.query) – Armlet

@FrancescoFrassinelli it's valid. urlparse.urlparse(url) --> urlparse(url) – Anklet

@Anklet you are right of course. I do not know why that evening I wrote that :) Thanks. – Stratify

What happens if the URL Is not valid (just a string) ... oR ... if the param is not present? – Stans

Getting AttributeError: 'ParseResult' object has no attribute 'decode', unsolved – Protozoology

Answer 2

I'm shocked this solution isn't on here already. Use:

request.GET.get('variable_name')

This will "get" the variable from the "GET" dictionary, and return the 'variable_name' value if it exists, or a None object if it doesn't exist.

Cloudlet

I can't get this to work. Maybe some more details. How are you getting request? Is this python3 compatible? – Boyette

@Boyette this answer (like the question) is for django, it is a django request object. if you are not using django, use one of the other answers. – Cp

def parse_category_page(self, response): product_id = response.GET.get('number') doesnt work neither does request – Stans

I'm shocked that someone actually added this answer and has so many likes (just kidding ;) ). One thing is getting query params from a random URL, and a very different thing is getting those params from inside an endpoint that you are serving using Django or another framework. This answer is in the case your URL is precisely the one you are serving with Django. The question is about some random internet URL... should parse it instead. – Incorrupt

Answer 3

for Python > 3.4

from urllib import parse
url = 'http://foo.appspot.com/abc?def=ghi'
query_def=parse.parse_qs(parse.urlparse(url).query)['def'][0]

Sera

Best way to do it IMO. Doesn't need extra packages and works A+ on Python 3.5. – Slavocracy

Answer 4

import urlparse
url = 'http://example.com/?q=abc&p=123'
par = urlparse.parse_qs(urlparse.urlparse(url).query)

print par['q'][0], par['p'][0]

Florencio

Answer 5

There is a new library called furl. I find this library to be most pythonic for doing url algebra. To install:

pip install furl

Code:

from furl import furl
f = furl("/abc?def='ghi'") 
print f.args['def']

Predisposition

Great library! Didn't know about it but works like a charm :) – Trimer

:) I have wasted my share of time trying to make urlparse work for me. No more. – Predisposition

Answer 6

I know this is a bit late but since I found myself on here today, I thought that this might be a useful answer for others.

import urlparse
url = 'http://example.com/?q=abc&p=123'
parsed = urlparse.urlparse(url)
params = urlparse.parse_qsl(parsed.query)
for x,y in params:
    print "Parameter = "+x,"Value = "+y

With parse_qsl(), "Data are returned as a list of name, value pairs."

Derna

Answer 7

In pure Python:

def get_param_from_url(url, param_name):
    return [i.split("=")[-1] for i in url.split("?", 1)[-1].split("&") if i.startswith(param_name + "=")][0]

Operator

Answer 8

The url you are referring is a query type and I see that the request object supports a method called arguments to get the query arguments. You may also want try self.request.get('def') directly to get your value from the object..

Demurrage

Answer 9

def getParams(url):
    params = url.split("?")[1]
    params = params.split('=')
    pairs = zip(params[0::2], params[1::2])
    answer = dict((k,v) for k,v in pairs)

Hope this helps

Bonds

This should not be necessary inside a webapp. – Idolum

Answer 10

The urlparse module provides everything you need:

urlparse.parse_qs()

Yolandayolande

Does this answer provide anything not provided by other answers? – Samarasamarang

Answer 11

There's not need to do any of that. Only with

self.request.get('variable_name')

Notice that I'm not specifying the method (GET, POST, etc). This is well documented and this is an example

The fact that you use Django templates doesn't mean the handler is processed by Django as well

Vernita

Answer 12

import cgitb
cgitb.enable()

import cgi
print "Content-Type: text/plain;charset=utf-8"
print
form = cgi.FieldStorage()
i = int(form.getvalue('a'))+int(form.getvalue('b'))
print i

Olivarez

Answer 13

Most answers here suggest using parse_qs to parse an URL string. This method always returns the values as a list (not directly as a string) because a parameter can appear multiple times, e.g.:

http://example.com/?foo=bar&foo=baz&bar=baz

Would return:

{'foo': ['bar', 'baz'], 'bar' : ['baz']}

This is a bit inconvenient because in most cases you're dealing with an URL that doesn't have the same parameter multiple times. This function returns the first value by default, and only returns a list if there's more than one element.

from urllib import parse

def parse_urlargs(url):
    query = parse.parse_qs(parse.urlparse(url).query)
    return {k:v[0] if v and len(v) == 1 else v for k,v in query.items()}

For example, http://example.com/?foo=bar&foo=baz&bar=baz would return:

{'foo': ['bar', 'baz'], 'bar': 'baz'}

Easley

Answer 14

There is a nice library w3lib.url

from w3lib.url import url_query_parameter
url = "/abc?def=ghi"
print url_query_parameter(url, 'def')
ghi

Queri

Answer 15

Parsing it in raw python:

path = '/some_path?k1=v1&k2=v2&k3=v3'
_, query_string = path.split('?')
params  = dict(param.split('=') for param in query_string.split('&'))
print(params)

Output:

{'k1': 'v1', 'k2': 'v2', 'k3': 'v3'}

Exude

Answer 16

Btw, I was having issues using parse_qs() and getting empty value parameters and learned that you have to pass a second optional parameter 'keep_blank_values' to return a list of the parameters in a query string that contain no values. It defaults to false. Some crappy written APIs require parameters to be present even if they contain no values

for k,v in urlparse.parse_qs(p.query, True).items():
  print k

Dispersal

Answer 17

parameters = dict([part.split('=') for part in get_parsed_url[4].split('&')])

This one is simple. The variable parameters will contain a dictionary of all the parameters.

Boeotia

Answer 18

I see there isn't an answer for users of Tornado:

key = self.request.query_arguments.get("key", None)

This method must work inside an handler that is derived from:

tornado.web.RequestHandler

None is the answer this method will return when the requested key can't be found. This saves you some exception handling.

Strephonn

Answer 19

I didn't want to mess with additional libraries. Simple ways suggested here didn't work out either. Finally, not on the request object, but I could get a GET parameter w/o all that hassle via self.GET.get('XXX'):

...
def get_context_data(self, **kwargs):
    context = super(SomeView, self).get_context_data(**kwargs)
    context['XXX'] = self.GET.get('XXX')
...

Python 2.7.18, Django 1.11.20

Kleper

Answer 20

NOT using Django and tried on Python 3.9. It can also be retrieved using the below code snippet in the handler:

considering URL as-

http://localhost:8081/api/v1/users?query_key=abcdef12345

and handler method as:

@routes.get('/api/v1/users')
async def handler_users(request):
    query_key = request.url.query['query_key']

Pretzel

Answer 21

You can use only request.GET.get() method

urls.py:

    path('some_path', views.func),

views.py:

def func(request):
    some_key = request.GET.get('some_key', None)
    ...

Phonemic