How to fix <Response 500> error in python requests?

Question

I am using an API, which receives a pdf file and does some analysis, but I am receiving Response 500 always

Have initially tested using Postman and the request goes through, receiving response 200 with the corresponding JSON information. The SSL security should be turned off.

However, when I try to do request via Python, I always get Response 500

Python code written by me:

import requests

url = "https://{{BASE_URL}}/api/v1/documents"
fin = open('/home/train/aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf', 'rb')
files = {'file': fin}
r = requests.post(url, files=files, verify=False)
print (r)
#r.text is empty

Python code, produced by the Postman:

import requests

url = "https://{{BASE_URL}}/api/v1/documents"

payload = "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"file\"; filename=\"aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf\"\r\nContent-Type: application/pdf\r\n\r\n\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--"
headers = {
    'content-type': "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW",
    'Content-Type': "application/x-www-form-urlencoded",
    'cache-control': "no-cache",
    'Postman-Token': "65f888e2-c1e6-4108-ad76-f698aaf2b542"
    }

response = requests.request("POST", url, data=payload, headers=headers)

print(response.text)

Have masked the API link as {{BASE_URL}} due to the confidentiality

Response by Postman:

{
    "id": "5e69058e2690d5b0e519cf4006dfdbfeeb5261b935094a2173b2e79a58e80ab5",
    "name": "aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf",
    "fileIds": {
        "original": "5e69058e2690d5b0e519cf4006dfdbfeeb5261b935094a2173b2e79a58e80ab5.pdf"
    },
    "creationDate": "2019-06-20T09:41:59.5930472+00:00"
}

Response by Python:

Response<500>

UPDATE:

Tried the GET request - works fine, as I receive the JSON response from it. I guess the problem is in posting pdf file. Is there any other options on how to post a file to an API?

Postman Response RAW:

POST /api/v1/documents
Content-Type: multipart/form-data; boundary=--------------------------375732980407830821611925
cache-control: no-cache
Postman-Token: 3e63d5a1-12cf-4f6b-8f16-3d41534549b9
User-Agent: PostmanRuntime/7.6.0
Accept: */*
Host: {{BASE_URL}}
cookie: c2b8faabe4d7f930c0f28c73aa7cafa9=736a1712f7a3dab03dd48a80403dd4ea
accept-encoding: gzip, deflate
content-length: 3123756

file=[object Object]

HTTP/1.1 200
status: 200
Date: Thu, 20 Jun 2019 10:59:55 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Location: /api/v1/files/95463e88527ecdc94393fde685ab1d05fa0ee0b924942f445b14b75e983c927e
api-supported-versions: 1.0
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Referrer-Policy: strict-origin

{"id":"95463e88527ecdc94393fde685ab1d05fa0ee0b924942f445b14b75e983c927e","name":"aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf","fileIds":{"original":"95463e88527ecdc94393fde685ab1d05fa0ee0b924942f445b14b75e983c927e.pdf"},"creationDate":"2019-06-20T10:59:55.7038573+00:00"}

CORRECT REQUEST

So, eventually - the correct code is the following:

import requests

files = {
    'file': open('/home/train/aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf', 'rb'),
}
response = requests.post('{{BASE_URL}}/api/v1/documents', files=files, verify=False)
print (response.text)

Pileum

500 errors are server errors. It is possible that the endpoint is looking for Request headers like User-Agent.. Can you try the same by adding User-Agent in headers for request.post? – Sauerbraten

@MadhanVaradhodiyil, have added headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0'}, but still same response. – Pileum

postman does not seem to desactivate Ssl, it is? What if you set verify to True? – Galligan

@Galligan it is possible to deactivate the SSL Certificate Verification form File>Settings in postman – Pileum

Yeah, but I meant what if you remove verify=False from your python script? Because even if you desactivated ssl verification in postman settings, it does not appear in the python script... – Galligan

@olinox14, it won't work then, as I'll receive SSLError (just checked). Have a look at the edit in the post,as the problem seems to be in posting file. – Pileum

Have you tried running the postman generated code in python? I would start there and then start removing parts from the payload / header until you find the minimum code that works. 'content-type': "multipart/form-data;" seems important but the boundary can be any string, maybe you can even leave it out. – Geraldo

Actually ignore that, requests should handle the headers for you. Run the postman generated code in python and, assuming it works, check the exact string request that was sent by postman. Chances are there is some weird nonsense like a filename that has to be called twice or something annoyingly specific the server wants. – Geraldo

Maybe the postman request is just sending the file location name=\"file\"; filename=\"aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf\" where as you seem to be sending a stream of the file. What happens if you try files = {"file": "aab2wieuqcnvn3g6syadumik4bsg5.0062.pdf"}? – Geraldo

@Dan, thanks for the suggestions. Tried with the file, but did not work. I have added the request details from postman in RAW – Pileum

Can't you print out the same RAW response from your python request? Then try spot the differences. It's probably something inane like files = {'file': {'file': [fin]}} – Geraldo

Answer 1

A 500 error indicates an internal server error, not an error with your script.

If you're receiving a 500 error (as opposed to a 400 error, which indicates a bad request), then theoretically your script is fine and it's the server-side code that needs to be adjusted.

In practice, it could still be due a bad request though.

If you're the one running the API, then you can check the error logs and debug the code line-by-line to figure out why the server is throwing an error.

In this case though, it sounds like it's a third-party API, correct? If so, I recommend looking through their documentation to find a working example or contacting them if you think it's an issue on their end (which is unlikely but possible).

Verbify

Answer 2

psycopg2-binary sometimes falling.

See your stderr.log file in the website's directory on your server. If psycopg2-binary module load error detected, go to terminal, execute

pip install --update pip pip uninstall psycopg2-binary pip install psycopg2-binary

Lone

Answer 3

Below code worked for me:

uploadURL = "https://some-good-url.com/contracts/upload"
headers = {"Authorization": "Bearer eyJ0eXA.big-long-token.123"}
filepath =os.path.join(os.path.dirname(os.path.abspath(__file__)), "my-wonderful-file.pdf")
files = [('file',('my-wonderful-file.pdf', open(filepath,'rb'),'application/pdf'))]
response = requests.post(
    url = uploadURL,
    files = files,
    headers = headers,
    data={},
    verify=False)

Bantustan