How to show TLS handshake information and CONNECT request in Invoke-WebRequest

Question

When I'm accessing a site through HTTPS and/or with HTTP proxy, cURL in Linux provides the -v/--verbose flag to show the CONNECT request to the proxy, as well as the SSL/TLS handshake process (including certificate), like

* Rebuilt URL to: https://www.example.com/
*   Trying 192.168.2.1...
* Connected to my-proxy.local (192.168.2.1) port 8080 (#0)
* Establish HTTP proxy tunnel to www.example.com:443
> CONNECT www.example.com:443 HTTP/1.1
> Host: www.example.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 597 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
*        server certificate verification OK
*        server certificate status verification SKIPPED
*        common name: www.example.org (matched)
*        server certificate expiration date OK
*        server certificate activation date OK
*        certificate public key: RSA
*        certificate version: #3
*        subject: C=US,ST=California,L=Los Angeles,O=Internet Corporation for Assigned Names and Numbers,OU=Technology,CN=www.example.org
*        start date: Tue, 03 Nov 2015 00:00:00 GMT
*        expire date: Wed, 28 Nov 2018 12:00:00 GMT
*        issuer: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert SHA2 High Assurance Server CA
*        compression: NULL
* ALPN, server accepted to use http/1.1
> GET / HTTP/1.1
> Host: www.example.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: max-age=604800
< Content-Type: text/html
< Date: Mon, 27 Nov 2017 23:08:55 GMT
< Etag: "359670651+gzip+ident"
< Expires: Mon, 04 Dec 2017 23:08:55 GMT
< Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
< Server: ECS (ord/4C84)
< Vary: Accept-Encoding
< X-Cache: HIT
< Content-Length: 1270
<
< (body...)

Is there anyway to get similar information when using Invoke-WebRequest? Or should I use another CmdLet? I tried -Debug and -Verbose, both of which shows neither information. Even the raw content only contains the actual request after proxy, that is, in the above example the GET / HTTP/1.1.

In brief, I want to see something like the line

> CONNECT www.example.com:443 HTTP/1.1

and

* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256

Marie

This is a feature request (although not easy to implement). – Marie

Only way so far seems to be to use some tracing tool, eg: serverfault.com/a/882682/148275 – Snappish

Answer 1

You can get some of this information from the .RawContent property of Invoke-WebRequest. Unfortunately if you opt for Invoke-RestMethod, PowerShell basically discards all of the HTTP information which you're interested.

For this example, I'll use https://jsonplaceholder.typicode.com/posts, which is a good test REST Endpoint for fiddling with things like this.

First, I'll make a connection to the site and store it in a Variable, $response.

$response = Invoke-WebRequest -uri https://jsonplaceholder.typicode.com/posts 

Now I can interrogate and pull out some of the useful fields to get some of the info you're looking for.

$response.BaseResponse 


IsMutuallyAuthenticated : False
Cookies                 : {__cfduid=d84018de2d621df9d53eb52d97cd33a651511881763}
Headers                 : {Transfer-Encoding, Connection, Vary, Access-Control-Allow-Credentials...}
SupportsHeaders         : True
ContentLength           : -1
ContentEncoding         : 
ContentType             : application/json; charset=utf-8
CharacterSet            : utf-8
Server                  : cloudflare-nginx
LastModified            : 11/28/2017 10:17:27 AM
StatusCode              : OK
StatusDescription       : OK
ProtocolVersion         : 1.1
ResponseUri             : https://jsonplaceholder.typicode.com/posts
Method                  : GET
IsFromCache             : False

We can also get some good info in the first 25 lines or so of the RawContent property, as seen here. RawContent is, well, raw, so I apply a split on new-lines then use array indexing as depicted by [0..20] to select the first 21 lines.

$response.RawContent.Split("`n")[0..20]
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Origin, Accept-Encoding
Access-Control-Allow-Credentials: true
Pragma: no-cache
X-Content-Type-Options: nosniff
CF-Cache-Status: HIT
CF-RAY: 3c4e3f804f9d82f7-ATL
Cache-Control: public, max-age=14400
Content-Type: application/json; charset=utf-8
Date: Tue, 28 Nov 2017 15:09:23 GMT
Expires: Tue, 28 Nov 2017 19:09:23 GMT
ETag: W/"6b80-Ybsq/K6GwwqrYkAsFxqDXGC7DoM"
Set-Cookie: __cfduid=d84018de2d621df9d53eb52d97cd33a651511881763; expires=Wed, 28-Nov-18 15:09:23 GMT; path=/; domain=.typicode.com; HttpOnly
Server: cloudflare-nginx
Via: 1.1 vegur
X-Powered-By: Express

I agree that it would be nice to be able to get this information back too. I'll open up an issue on the github.com/PowerShell repo and see if we can get something like this added in the future, and the link will be added to this answer.

Dimercaprol

Sorry, I think I didn't explain well in question. I know how to get the headers and status code for ordinary request. The raw content does not contain TLS handshake information, or the CONNECT request when I access the site through proxy. I updated the question in case of confusion. – Marie

Unfortunately our request fails with a Could not establish trust relationship for the SSL/TLS secure channel so we're unable to see the raw message body. Any tips appreciated. – Pileous

Try this [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12 at the top of your script. Also you should try asking a new question to get better response. – Dimercaprol

Actually, I think the question was perfectly clear, but this doesn't answer it. I had hoped that Powershell was more capable than this, but as it uses .NET it's just as limited. One solution is to download portable OpenSSL and use the s_client command. – Toland