Powershell Invoke-RestMethod over HTTPS

Question

I'm trying to communicate with a service over powershell but I am failing miserably. I suspect it is the certificate and I have googled for the answer and found two options, none of which worked for me. I have also tried to combine the two unsuccessfully.

Option 1:

add-type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
        public bool CheckValidationResult(
            ServicePoint srvPoint, X509Certificate certificate,
            WebRequest request, int certificateProblem) {
            return true;
        }
    }
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

$urlJSON = "https://internal.ad.local/path/api_jsonrpc.php"

#Create authentication JSON object using ConvertTo-JSON
$objAuth = (New-Object PSObject | Add-Member -PassThru NoteProperty jsonrpc '2.0' |
Add-Member -PassThru NoteProperty method 'user.authenticate' |
Add-Member -PassThru NoteProperty params @{user="user";password="password"} |
Add-Member -PassThru NoteProperty id '2') | ConvertTo-Json


Invoke-RestMethod -Uri $urlJSON -body $objAuth -method "Post"

Option 2:

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}

$urlJSON = "https://internal.ad.local/path/api_jsonrpc.php"

#Create authentication JSON object using ConvertTo-JSON
$objAuth = (New-Object PSObject | Add-Member -PassThru NoteProperty jsonrpc '2.0' |
Add-Member -PassThru NoteProperty method 'user.authenticate' |
Add-Member -PassThru NoteProperty params @{user="user";password="password"} |
Add-Member -PassThru NoteProperty id '2') | ConvertTo-Json


Invoke-RestMethod -Uri $urlJSON -body $objAuth -method "Post"

Here's the error message:

Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send.
At C:\Users\user\AppData\Local\Temp\46eaa6f7-62a0-4c10-88d1-79212d652bc9.ps1:24 char:1
+ Invoke-RestMethod -Uri $urlJSON -body $objAuth -method "Post"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

I might add:

• surfing directly to the service works with a web browser
• I tried opening up to HTTP as well, and that worked
• The certificate used by the service is self-signed but trusted by my machine via a root certficate (no warnings is issues in IE or Chrome)
• I have done network captures and made sure that a packets does indeed reach the server.

Any suggestions appreciated!

Kind regards, Patrik

Updated post as to suggestions made by Mr Tree below:

Name                       : lambda_method
DeclaringType              :
ReflectedType              :
Module                     : RefEmit_InMemoryManifestModule
MethodHandle               :
Attributes                 : PrivateScope, Public, Static
CallingConvention          : Standard
IsSecurityCritical         : False
IsSecuritySafeCritical     : False
IsSecurityTransparent      : True
ReturnType                 : System.Boolean
ReturnParameter            :
ReturnTypeCustomAttributes : System.Reflection.Emit.DynamicMethod+RTDynamicMethod+EmptyCAHolder
MemberType                 : Method
MethodImplementationFlags  : NoInlining
IsGenericMethodDefinition  : False
ContainsGenericParameters  : False
IsGenericMethod            : False
IsPublic                   : True
IsPrivate                  : False
IsFamily                   : False
IsAssembly                 : False
IsFamilyAndAssembly        : False
IsFamilyOrAssembly         : False
IsStatic                   : True
IsFinal                    : False
IsVirtual                  : False
IsHideBySig                : False
IsAbstract                 : False
IsSpecialName              : False
IsConstructor              : False
CustomAttributes           :
MetadataToken              :

Update 2 based on a comment by Mr Tree:

Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send.
At C:\Users\user\AppData\Local\Temp\ff47910e-fd8e-4be8-9241-99322144976a.ps1:13 char:1
+ Invoke-RestMethod -Uri $urlJSON -body $objAuth -method "Post"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

Vagrant

Try {$true} -as [Net.Security.RemoteCertificateValidationCallback] in the second example... Do you have the CA installed in the LocalMachine or CurrentUser store? – Maidinwaiting

Thank you for your help. The CA is installed under both my CurrentUser and Computer account in Trusted Root Certification. Will add the result of the command you posted in my post above. – Vagrant

(not sure what to look for there) – Vagrant

What do you get if you remove the following from Option 2 [System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true} . Then restart Powershell and re-run the script? – Maidinwaiting

Thank you for your suggestion. I get the same error as before. The post has been updated with the details. – Vagrant

Answer 1

I solved the mystery while troubleshooting another thing. The web server in question only supported TLS1.1 and TLS1.2. Powershell does not seem to support that by default.

To force TLS1.2 you can use this line:

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Vagrant

New DellEMC Isilon simulator from version 8.2.1 seem to have TLS1.2 for the built-in apache web server. This correction provided by PatrickJ make them running on the new platform. Thanks! – Lavina

Answer 2

It appears you're trying to invoke a json API with Invoke-RestMethod. According to the documentation:

-ContentType

Specifies the content type of the web request.

If this parameter is omitted and the request method is POST, Invoke-RestMethod sets the content type to "application/x-www-form-urlencoded". Otherwise, the content type is not specified in the call.

To use a json body, you will need to use Invoke-RestMethod -ContentType 'application/json' <other args>

Helvetic

Answer 3

I went through a lot of pain recently in order to get past a similar situation. I was creating a proof of concept using a trial SaaS service. The service had a self-signed SSL certificate so I wanted to ignore certificate errors while attempting to invoke a POST method to it (similar to the "-k" parameter for curl). After much struggle, I found that it needed both - (a) the call to ignore cert validation errors and (b) an explicit setting for TLS 1.2 as the security protocol. I think the latter because the service was perhaps declining attempts to connect using any of the other protocols. (I spent too much time trying the different variations for doing each as suggested on various SOF threads but independently...)

Here's the code that worked...

Important: The cert validation bypass is purely for the prototype/PoC. We do not intent to do this in production (and you shouldn't either!).

$defaultSecurityProtocol = $null
try
{   
    #BUGBUG, TODO: Disabling cert validation for the duration of this call...('trial' version cert is self-signed.)
    #Remove this after the PoC.
    [System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true } 
    #Cache the previous protocol setting and explicitly require TLS 1.2 
    $defaultSecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol
    [System.Net.ServicePointManager]::SecurityProtocol = `
                [System.Net.SecurityProtocolType]::Tls12

    if (-not [String]::IsNullOrWhiteSpace($authZHeaderValue))
    {
        $response = Invoke-WebRequest `
                    -Uri $webhookUrl `
                    -Method "Post" `
                    -Body $eventJson `
                    -Header @{ $authZHeaderName = $authZHeaderValue} 
    }
    else 
    {
        $response = Invoke-WebRequest `
                    -Uri $webhookUrl `
                    -Method "Post" `
                    -Body $eventJson
    }
}
catch 
{
    $msg = $_.Exception.Message
    $status = $_.Exception.Status
    $hr = "{0:x8}" -f ($_.Exception.HResult)
    $innerException = $_.Exception.InnerException
    #Just issue a warning about being unable to send the notification...
    Write-Warning("`n`t[$status] `n`t[0x$hr] `n`t[$msg] `n`t[$innerException]")
}
finally 
{
    # Set securityProtocol and CertValidation behavior back to the previous state.
    [System.Net.ServicePointManager]::SecurityProtocol = $defaultSecurityProtocol
        [System.Net.ServicePointManager]::ServerCertificateValidationCallback = $null
}

Also want to add that the preferred security protocols keep changing as various vulnerabilities are discovered and fixes implemented. Moreover, different systems (SSL/TLS stacks in client OSes and servers/services) often have their own catching up to do with the latest/most secure options. Thus exactly which flag might work will be a function of the client and server systems and also of time (in that TLS1.2 might not remain preferred a few months later). Ideally one shouldn't need to specify a flag at all. Please see the "Remarks" section in this MSDN document for more.

Buie

Answer 4

This is a complete Powershell script that worked for me:

try {
    # Configure API request payload
    $url = "https://my.server.com/api/countries"
    $method = "POST"
    $headers = @{
                  "Content-Type" = "application/json"
               }
    $body = '{
              "name": "India"
            }'
    # Reading request body from json file
    # $body = Get-Content request.json | Out-String
    $certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 "C:\ClientCert.pfx", "Password1"
 
    # Call the API
    Invoke-RestMethod -Uri $url -Method $method -Headers $headers -Body $body -Certificate $certificate
    
    Write-Host "`n#################### SUCCESS ####################`n"
} catch {
    Write-Host "`n#################### ERROR ####################`n"

    # Getting response code
    $responseCode = $_.Exception.Message
    
    # Getting response body
    if ($_.Exception.Response -ne $null) {
        $responseStream = $_.Exception.Response.GetResponseStream()
        $streamReader = New-Object System.IO.StreamReader($responseStream)
        $streamReader.BaseStream.Position = 0
        $streamReader.DiscardBufferedData()
        $responseBody = $streamReader.ReadToEnd()
    }
    
    Write-Host "Response Code - " $responseCode
    Write-Host "Response Body - " $responseBody
    
    Write-Host "`n###############################################`n"
}

Dubbing