powershell invoke-restmethod multipart/form-data

Question

I'm currently trying to upload a file to a Webserver by using a REST API. And as mentioned I'm using PowerShell for this. With curl this is no problem. The call looks like this:

curl -H "Auth_token:"$AUTH_TOKEN -H "Content-Type:multipart/form-data" -X POST -F appInfo='{"name": "test","description": "test"}' -F uploadFile=@/test/test.test https://server/api/

But I'm completely helpless when it comes to exporting this to powershell with a Invoke-Restmethod command. As far as I searched it is not possible to use the Invoke-Restmethod for this. https://www.snip2code.com/Snippet/396726/PowerShell-V3-Multipart-formdata-example But even with that Snipped I'm not smart enough to get this working since I don´t want to upload two files but instead one file and some arguments.

I would be very thankful if someone could get me back on the track with this :o Thanks!

Sargent

Answer 1

It should be pretty straight forward. Taking from this answer:

$Uri = 'https://server/api/';
$Headers = @{'Auth_token'=$AUTH_TOKEN};
$FileContent = [IO.File]::ReadAllText('C:\test\test.test');
$Fields = @{'appInfo'='{"name": "test","description": "test"}';'uploadFile'=$FileContent};

Invoke-RestMethod -Uri $Uri -ContentType 'multipart/form-data' -Method Post -Headers $Headers -Body $Fields;

You may want to use [IO.File]::ReadAllBytes() if the file isn't a text file.

This also may not work well if you're uploading a huge file.

Piccadilly

Could you tell me something in my this post? – Alpinist

I propose deletion of this answer as in the latest version there is more elegant way to do thi.s – Mastersinger

This didn't work for me in PowerShell 5.1. The body is sent as x-www-form-urlencoded not as multipart/form-data, checked in Fiddler. – Sucker

Answer 2

@Bacon-Bits answer didn't seem to work for me. My server rejected it with a potentially malformed form-data body :-(

I found this gist, and trimmed it up a bit for my purposes. Here's my end result:

$FilePath = 'c:\temp\temp.txt';
$URL = 'http://your.url.here';

$fileBytes = [System.IO.File]::ReadAllBytes($FilePath);
$fileEnc = [System.Text.Encoding]::GetEncoding('UTF-8').GetString($fileBytes);
$boundary = [System.Guid]::NewGuid().ToString(); 
$LF = "`r`n";

$bodyLines = ( 
    "--$boundary",
    "Content-Disposition: form-data; name=`"file`"; filename=`"temp.txt`"",
    "Content-Type: application/octet-stream$LF",
    $fileEnc,
    "--$boundary--$LF" 
) -join $LF

Invoke-RestMethod -Uri $URL -Method Post -ContentType "multipart/form-data; boundary=`"$boundary`"" -Body $bodyLines

Nonce

Worked perfectly for me, exactly what I needed. That said, having to draw the whole file into memory before submitting... Does give me pause – Coumarone

@Coumarone So I wonder if you could pipe in a stream... into the middle of some other static text? Seems do-able. Wasn't a concern for me, since my files were < 50KiB generally. – Nonce

I had to change the encoding to ISO-8859-1 ([System.Text.Encoding]::GetEncoding("ISO-8859-1").GetString($fileBytes)) as per this post #25469618 . I hope this saves others time. – Swastika

@GradyGCooper What type of server were you posting to? I was uploading to IIS. Might be good to clear up "why" you had to change the encoding. – Nonce

My upvote as it helped me resolve my problem here #55425132. – Mossy

Works great with the Python Flask example. I was using with gitlab-runner specific runner and power-shell version 5.1 – Nystagmus

@Nonce I needed to update encoding due to fact, that UTF8 uses variable char length. My 61MB binary file was shortened to 58MB using UTF8 conversion. – Bandoline

I gave up using multipart/form-data with PowerShell Invoke-RestMethod. I just call curl.exe in my PS script and enjoy life more :) – Adaline

This worked beautifully for me when trying to upload documents via Docusign API – Lyric

@Nonce How did you send the appInfo='{"name": "test","description": "test"} in your question? – Susa

I propose deletion of this answer as in the latest version there is more elegant way to do thi.s – Mastersinger

@Nonce I don't think it requires much explanation. The standard suggestion is UTF-8 which is multibyte and includes far more characters than ISO-8859-1 single-byte. – Eddie

I had to remove double quotes from boundary in content-type then it worked for me. -ContentType "multipart/form-data; boundary=$boundary" – Sucker

Answer 3

With PowerShell Core this should work out of the box with the new -Form parameter.

See: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-restmethod?view=powershell-7

$Uri = 'https://api.contoso.com/v2/profile'
$Form = @{
    firstName  = 'John'
    lastName   = 'Doe'
    email      = '[email protected]'
    avatar     = Get-Item -Path 'c:\Pictures\jdoe.png'
    birthday   = '1980-10-15'
    hobbies    = 'Hiking','Fishing','Jogging'
}
$Result = Invoke-RestMethod -Uri $Uri -Method Post -Form $Form

Welcher

The example was copied from official docs, which has explanations. – Stephainestephan

what the version 5.1 form parameter passed in -Body for a Get. Get allows for a body form data in postman for access with cloudflare – Exoskeleton

Answer 4

I needed to pass both the header and some more parameters (insert=true and debug=true) along with the file content. Here's my version which extends the script by @jklemmack.

param([string]$path)

$Headers = @{Authorization = "Bearer ***************"}
$Uri = 'https://host:8443/api/upload'

$fileBytes = [System.IO.File]::ReadAllBytes($path);
$fileEnc = [System.Text.Encoding]::GetEncoding('ISO-8859-1').GetString($fileBytes);
$boundary = [System.Guid]::NewGuid().ToString(); 
$LF = "`r`n";

$bodyLines = ( 
    "--$boundary",
    "Content-Disposition: form-data; name=`"insert`"$LF",
    "true$LF",
    "--$boundary",
    "Content-Disposition: form-data; name=`"debug`"$LF",
    "true$LF",    
    "--$boundary",
    "Content-Disposition: form-data; name=`"file`"; filename=`"$path`"",
    "Content-Type: application/octet-stream$LF",
    $fileEnc,
    "--$boundary--$LF" 
) -join $LF

Invoke-RestMethod -Uri $Uri -Headers $Headers -Method Post -ContentType "multipart/form-data; boundary=`"$boundary`"" -Body $bodyLines

Kaleidoscope

Awesome! thanks I was trying to figure out which encoding to use, as UTF8 nor UTF7 would work, nor did default. Now I know its ISO-8859-1 – Karlenekarlens

Thank you so much for this! I have been trying on and off to figure this out for around a year now, and this was finally what worked for me! – Permanent

how should we do if no file upload but only form data e.g. name and value? – Generative

I propose deletion of this answer as in the latest version there is more elegant way to do thi.s – Mastersinger

ISO-8859-1 fixed it for me as well. – Eddie

Answer 5

So, I've battled with this quite a bit lately and discovered it is indeed possible to match curl functionality, but it's not immediately obvious how to do multipart/form-data correctly. All the responses above have covered important pieces of the puzzle, but I'm going to try and tie it all together here for the next sorry fellow who is trying to implement curl functionality in native Powershell.

@jklemmack's solution is the one that put me on the right track, and is the most flexible, because it allows you to construct the form-data content specifically, controlling both the boundaries, along with how the data gets formatted within it.

For anyone trying to do this, I think it's important that you arm yourself with a proper web debugging proxy like Fiddler (.net) or Burp Suite (java), so that you can inspect each of the REST calls in detail to understand the specific format of the data being passed across to the API.

In my specific case, I noticed that curl was inserting a blank line above each part of the form data - so to extend @jklemmack's example, it would look like the following:

    $bodyLines = (
        "--$boundary",
        "Content-Disposition: form-data; name=`"formfield1`"",
        '',
        $formdata1,
        "--$boundary",
        "Content-Disposition: form-data; name=`"formfield2`"",
        '',
        $formdata2,
        "--$boundary",
        "Content-Disposition: form-data; name=`"formfield3`"; filename=`"$name_of_file_being_uploaded`"",
        "Content-Type: application/json",
        '',
        $content_of_file_being_uploaded,
        "--$boundary--"
    ) -join $LF

Hope this saves someone a lot of time in the future!

I also still agree that if you need to do this from scratch, and have the option of using the curl native binary directly (while ensuring due-diligence around security and compliance), that you take advantage of it's maturity and the conveniences that it provides. Use curl. It is better that this multipart logic be vigourously tested and maintained by the curl community at large, vs the onus being on your internal dev or operations teams.

Monstrosity

Saved. My. Bacon. Thank you very much for this, helped me out immensely at work. – Chiffon

I propose deletion of this answer as in the latest version there is more elegant way to do thi.s – Mastersinger

@Mastersinger please elaborate or share a link to your reference. PS 6+ definitely improves upon much of this functionality, however much like the lack of curl, many organizations do not yet have that installed across the many servers they need to manage. – Jilly

@rbleattler, honestly nothing new really but here is my answer https://mcmap.net/q/587739/-powershell-invoke-restmethod-multipart-form-data – Mastersinger

Answer 6

There was lot of hacky code required to do this and most of the answers here are just stating them. These answers should be either deleted or archived along with all hacky blogs over the internet.

With latest version (PowerShell Core v7.2.6 as of this writing). All you need is to give Path using Get-Item-Path.

    $Form = @{ 
        document=  Get-Item -Path .\image.png # no leading @ sign.
    }    

  $Result = Invoke-RestMethod -Uri $Uri -Method Post -Form $Form

Please note that there is no @ sign before get item like you put in curl. I was putting @ sign and breaking my request.

reference : https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-restmethod?view=powershell-7#example-4--simplified-multipart-form-data-submission

Mastersinger

The -Form parameter is not available on PS 5.1. While many other answers are "hacky" or "not as elegant", they function (and don't pose any security issues AFAIK). They also suggest using alternate encoding. Although it's a separate issue, it seems to affect many people here (including me). – Eddie

@TylerMontney, if you have to use older version of powershell go ahead. I want to make it loud and clear for everyone, If you have choice to use powershell core, then you don't need any hacky code. Powershell is anyway used for temp PoC code so why not use the latest? – Mastersinger

Answer 7

I had some troubles trying to do the following curl command using Invoke-RestMethod:

curl --request POST \
  --url https://example.com/upload_endpoint/ \
  --header 'content-type: multipart/form-data' \
  --form '[email protected]'
  -v

In my case, it turned out to be easier to use curl with powershell.

$FilePath = "C:\example.csv"
$CurlExecutable = "C:\curl-7.54.1-win64-mingw\bin\curl.exe"

$CurlArguments = '--request', 'POST', 
                'https://example.com/upload_endpoint/',
                '--header', "'content-type: multipart/form-data'",
                '--form', "file=@$FilePath"
                '-v',

# Debug the above variables to see what's going to be executed
Write-Host "FilePath" $FilePath
Write-Host "CurlExecutable" $FilePath
Write-Host "CurlArguments" $CurlArguments

# Execute the curl command with its arguments
& $CurlExecutable @CurlArguments

Download the executable for your os on curl's website.

Here are some reasons that could make you pick curl instead of powershell's invoke-restmethod

• Many of the tools out there can generate curl commands
• curl supports uploading files larger than 2GB (see Shukri Adams comment)

Both Curl and Powershell's invoke-restmethod are fine solutions. You might want to consider curl if none of the other answers worked for you. It is usually better to stick with built-in solutions, but sometimes alternatives can be useful.

Drawing

Yes, but I don't have curl. I'm executing on a remote system (over 300 in fact), and don't have it installed. I'd like to embed this in a Invoke-Script script block to upload log files. Pure PowerShell is the winner in my scenario. – Nonce

I see, pure powershell solution would be the best indeed. I personally had no luck with other answers in here at the time of writing my answer. In the case where you manage 300 servers, I’d suggest using a provisioning tool (ansible, salt, etc) to manage them. Sending powershell script on all these machines would be same effort as getting curl executable on all of these. For sure I don’t know your exact use case. If you already have remote execution on the servers, nothing blocks you from downloading curl from same script if executable can’t be found. Hope you find a solution, good luck 👍 – Drawing

Our scenario is locked down remote kiosks with limited over-the-air capabilities - one of which is the direct execution of PS scripts provided by the central management software (ScreenConnect, FYI). I ended up using a GIST and trimming it way down - gist.github.com/weipah/19bfdb14aab253e3f109. This script isn't run on all kiosks at once, but rather on any for which we need to zip & upload log files for debugging / analysis. Anyway - I'll keep curl in mind - certainly powerful, but not one I think of immediately coming from a hard .NET background. – Nonce

"Curl is free and open source software" is no longer a valid criticism. Powershell, and Invoke-RestMethod with it, are also free and open source: github.com/powershell/powershell – Piccadilly

Answer 8

Such pain trying to get a powershell v4 on windows 8.1 to upload files to my upload.php

# This code works and matches to a Firefox 78.6.0esr upload transmission verified via wireshark

$FilePath = 'c:\Temp\file-to-upload.txt';
$URL = 'http://127.0.0.1/upload.php';

$fileBytes = [System.IO.File]::ReadAllBytes($FilePath);
$fileEnc = [System.Text.Encoding]::GetEncoding('UTF-8').GetString($fileBytes);
$boundary = [System.Guid]::NewGuid().ToString(); 
$LF = "\r\n";

$bodyLines = "--$boundary $LF Content-Disposition: form-data; name='file'; filename='file-to-upload.txt' $LF Content-Type: application/octet-stream $LF $fileEnc $LF --$boundary-- $LF";

Invoke-RestMethod -Uri $URL -Method Post -ContentType "multipart/form-data; boundary=$boundary" -Body $bodyLines

For reference, the upload.php is:

<?php
    $uploaddir = '/var/www/uploads/';
    $uploadfile = $uploaddir . $_FILES['file']['name'];
    move_uploaded_file($_FILES['file']['tmp_name'], $uploadfile)
?>

Wireshark Sample

POST /upload.php HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT; Windows NT 6.3; en-US) WindowsPowerShell/4.0
Content-Type: multipart/form-data; boundary=96985b62-451a-41fa-9eca-617e3599797c
Host: 127.0.0.1
Content-Length: 284
Connection: Keep-Alive

--96985b62-451a-41fa-9eca-617e3599797c \r\n Content-Disposition: form-data; name='file'; filename='ftp.txt' \r\n Content-Type: application/octet-stream \r\n open 127.0.0.1 21
anonymous
anonymous
bin
put file-to-upload.txt
quit
 \r\n --96985b62-451a-41fa-9eca-617e3599797c-- \r\nHTTP/1.1 200 OK
Date: Sat, 02 Jan 2021 22:11:03 GMT
Server: Apache/2.4.46 (Debian)
Content-Length: 0
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8

Ms