Is there any way to do HTTP PUT request in Python?

Question

I need to upload some data to a server using HTTP PUT method in Python. From my brief reading of the urllib2 docs, it only does HTTP POST.

Is there any way to do an HTTP PUT in Python?

Meshach

Answer 1

I've used a variety of python HTTP libs in the past, and I've settled on requests as my favourite. Existing libs had pretty useable interfaces, but code can end up being a few lines too long for simple operations. A basic PUT in requests looks like:

payload = {'username': 'bob', 'email': '[email protected]'}
>>> r = requests.put("http://somedomain.org/endpoint", data=payload)

You can then check the response status code with:

r.status_code

or the response with:

r.content

Requests has a lot synactic sugar and shortcuts that'll make your life easier.

Goble

Even though the above code looks extremely simple, don't infer that 'requests' is in any way lacking or under-powered. It is extremely capable, just with a highly tidy interface. – Guardado

I wonder how long it'll take this answer to gradually accumulate votes until it becomes the new highest-voted answer? – Guardado

You don't understand how much this is awesome!!! I was struggling with a crummy java library! ... I think I kinda love you for pointing to "Requests"! – Benito

Didn't see your answer till now but yeah I agree this is the simplest approach. I posted an answer with slightly different details. Cheers – Mines

Use json=payload parameter if you want the data to be in the body. – Fichu

If I want to put file then ? filname is in variable say tailf ! – Maidenhair

bty what is difference between python requests.post & requests.put can both be used to upload file to server same like curl -X POST -d @$myfilename lsd-qa-web-3.56m.vgtf.net/nba/$myfilename – Maidenhair

@AshishKarpe How the server treats it. You're talking to the server, and telling it what you want it to do, but the server chooses how to interpret it. – Kept

So with your example you replace the username and email value with the one you just defined in the dict? Is that what the put method does? And what happens if the key username does not exist? Does it create a new one? @JohnCarter – Cassandracassandre

I want to quickly point out that this interface is better than the other answer, because it's functional programming. Why create an object when a function with simple data structures as parameters will do. I wish other python libraries would follow suit. – Carbajal

Answer 2

import urllib2
opener = urllib2.build_opener(urllib2.HTTPHandler)
request = urllib2.Request('http://example.org', data='your_put_data')
request.add_header('Content-Type', 'your/contenttype')
request.get_method = lambda: 'PUT'
url = opener.open(request)

Co

Looks like a bit of a dirty hack, but it seems to work elegantly and completly – Meshach

It would be less of a hack if you were to subclass urllib2.Request instead of monkey-patching it. – Deniable

This answer was brilliant when it was written, but nowadays it's a lot easier to use the 'requests' package instead, see John Carter's answer. 'Requests' is in no way a toy - it is extremely capable. – Guardado

This answer is canonical, but outdated. Please, consider using requests library instead. – Byng

Thank you, I didn't want to use anything outside of the standard python library and this works perfect. I don't quite understand why the urllib2 is designed to only cater for GET and POST as standard, but this work around is champion. Can the urllib2.build_opener(urllib2.HTTPHandler) be reused in multiple calls? – Nadeen

You could use urllib2.urlopen instead of opener.open if you do not need to use an opener. – Raine

Can this work with https Auth key too? i.e. using urlib2 with a put – Faison

Since I asked this question, I have also discovered the excellent requests library. I have changed the accepted answer to the one that had that. – Meshach

Warning... Doesn't seem to work well for use with the webHDFS API. This answer looked promising, as I can only use the standard python libraries for my task, however, I couldn't get it to work well when redirects are involved. Specifically when trying to write a new file using webHDFS APIs, a two-put process is required with the first PUT command returning a redirect URL which isn't handled well by this solution. All I would get back was an HTTP 307 redirect error without any header information to grab the new redirect URL from. I'm also an idiot and could be missing something here too. – Sestos

For any folks not wanting to use a separate library... the last line should be as follows if you want to get back the body: response = opener.open(request) return response.read() – Palaearctic

Answer 3

Httplib seems like a cleaner choice.

import httplib
connection =  httplib.HTTPConnection('1.2.3.4:1234')
body_content = 'BODY CONTENT GOES HERE'
connection.request('PUT', '/url/path/to/put/to', body_content)
result = connection.getresponse()
# Now result.status and result.reason contains interesting stuff

Picco

Don't use httplib if you (or any of your potential users) need proxy support. See this article for details. – Deniable

Why would you say that? Your article clearly states that it works. See Rolf Wester's answer, he says that urllib fails but httplib works. – Picco

httplib only works when the user explicitly connects to the proxy and modifies the request to include the full URL in the GET parameter. The reason urllib failed was because the http_proxy wasn't set properly. urllib uses httplib under the scenes, but also handles redirects, proxies, etc. – Deniable

While this solution would work, the one using requests is much simpler, elegant, and in my opinion better. – Reverie

@Reverie I agree. This answer was posted before requests existed. As of now, the only case where this solution would be better is if only Python Standard Library would be available. Otherwise, I'd also recommend to use requests. – Picco

Answer 4

You can use the requests library, it simplifies things a lot in comparison to taking the urllib2 approach. First install it from pip:

pip install requests

More on installing requests.

Then setup the put request:

import requests
import json
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}

# Create your header as required
headers = {"content-type": "application/json", "Authorization": "<auth-key>" }

r = requests.put(url, data=json.dumps(payload), headers=headers)

See the quickstart for requests library. I think this is a lot simpler than urllib2 but does require this additional package to be installed and imported.

Mines

Doesnt requests support PUT as well? – Bates

requests supports get, put, post, delete head and options. Fixed the example to use put. Check out requests quickstart. – Mines

@RPradeep Thanks for that. – Mines

Answer 5

This was made better in python3 and documented in the stdlib documentation

The urllib.request.Request class gained a method=... parameter in python3.

Some sample usage:

req = urllib.request.Request('https://example.com/', data=b'DATA!', method='PUT')
urllib.request.urlopen(req)

Gastronomy

Answer 6

You should have a look at the httplib module. It should let you make whatever sort of HTTP request you want.

Persiflage

Nice solution, quiet pythonic but a bit too close to the metal and involving writing a lot of other code already – Meshach

Answer 7

I needed to solve this problem too a while back so that I could act as a client for a RESTful API. I settled on httplib2 because it allowed me to send PUT and DELETE in addition to GET and POST. Httplib2 is not part of the standard library but you can easily get it from the cheese shop.

Investigator

httplib2 is borderline abandonware. It has a long list of bugs that go unfixed despite community contributions (patches). I suggest thinking twice before using httplib2 in any production environments. – Deniable

Answer 8

I also recommend httplib2 by Joe Gregario. I use this regularly instead of httplib in the standard lib.

Yaron

Answer 9

Have you taken a look at put.py? I've used it in the past. You can also just hack up your own request with urllib.

Kyser

I don't really wanna use some random guys http library – Meshach

trying import put getting # pip install put Downloading/unpacking put Could not find any downloads that satisfy the requirement put Cleaning up... No distributions at all found for put – Maidenhair

# tail -f /root/.pip/pip.log Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main status = self.run(options, args) File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1178, in prepare_files – Maidenhair

url = finder.find_requirement(req_to_install, upgrade=self.upgrade) File "/usr/lib/python2.7/dist-packages/pip/index.py", line 277, in find_requirement raise DistributionNotFound('No distributions at all found for %s' % req) DistributionNotFound: No distributions at all found for put – Maidenhair

Answer 10

You can of course roll your own with the existing standard libraries at any level from sockets up to tweaking urllib.

http://pycurl.sourceforge.net/

"PyCurl is a Python interface to libcurl."

"libcurl is a free and easy-to-use client-side URL transfer library, ... supports ... HTTP PUT"

"The main drawback with PycURL is that it is a relative thin layer over libcurl without any of those nice Pythonic class hierarchies. This means it has a somewhat steep learning curve unless you are already familiar with libcurl's C API. "

Manta

I'm sure it would work, but I want something a bit more pythonic – Meshach

Answer 11

If you want to stay within the standard library, you can subclass urllib2.Request:

import urllib2

class RequestWithMethod(urllib2.Request):
    def __init__(self, *args, **kwargs):
        self._method = kwargs.pop('method', None)
        urllib2.Request.__init__(self, *args, **kwargs)

    def get_method(self):
        return self._method if self._method else super(RequestWithMethod, self).get_method()


def put_request(url, data):
    opener = urllib2.build_opener(urllib2.HTTPHandler)
    request = RequestWithMethod(url, method='PUT', data=data)
    return opener.open(request)

Threefold

Answer 12

You can use requests.request

import requests

url = "https://www.example/com/some/url/"
payload="{\"param1\": 1, \"param1\": 2}"
headers = {
  'Authorization': '....',
  'Content-Type': 'application/json'
}

response = requests.request("PUT", url, headers=headers, data=payload)

print(response.text)

Hexone

Answer 13

A more proper way of doing this with requests would be:

import requests

payload = {'username': 'bob', 'email': '[email protected]'}

try:
    response = requests.put(url="http://somedomain.org/endpoint", data=payload)
    response.raise_for_status()
except requests.exceptions.RequestException as e:
    print(e)
    raise

This raises an exception if there is an error in the HTTP PUT request.

Grasshopper

Answer 14

Using urllib3

To do that, you will need to manually encode query parameters in the URL.

>>> import urllib3
>>> http = urllib3.PoolManager()
>>> from urllib.parse import urlencode
>>> encoded_args = urlencode({"name":"Zion","salary":"1123","age":"23"})
>>> url = 'http://dummy.restapiexample.com/api/v1/update/15410' + encoded_args
>>> r = http.request('PUT', url)
>>> import json
>>> json.loads(r.data.decode('utf-8'))
{'status': 'success', 'data': [], 'message': 'Successfully! Record has been updated.'}

Using requests

>>> import requests
>>> r = requests.put('https://httpbin.org/put', data = {'key':'value'})
>>> r.status_code
200

Alishiaalisia