How to programmatically replicate a request found in Chrome Developer Tools?

Question

I'm looking at my balance on Venmo.com but they only show you 3 months at a time and I'd like to get my entire transaction history.

Looking at the Chrome Developer Tools, under the network tab, I can see the request to https://api.venmo.com/v1/transaction-history?start_date=2017-01-01&end_date=2017-01-31 which returns JSON.

I'd like to programmatically iterate through time and make several request and aggregate all of the transactions. However, I keep getting 401 Unauthorized.

My initial approach was just using Node.js. I looked at the cookie in the request and copied it into a secret.txt file and then sent the request:

import fetch from 'node-fetch'
import fs from 'fs-promise'

async function main() {
  try {
    const cookie = await fs.readFile('secret.txt')  
    const options = {
      headers: {
        'Cookie': cookie,
      }, 
    }
    try {
      const response = await fetch('https://api.venmo.com/v1/transaction-history?start_date=2016-11-08&end_date=2017-02-08', options)
      console.log(response)
    } catch(e) {
      console.error(e)
    }
  } catch(e) {
    console.error('please put your cookie in a file called `secret.txt`')
    return
  }
}

That didn't work do I tried copying all of the headers over:

const cookie = await fs.readFile('secret.txt')  
const options = {
  headers: {
    'Accept-Encoding': 'gzip, deflate, sdch, br',
    'Accept-Language': 'en-US,en;q=0.8',
    'Cache-Control': 'no-cache',
    'Connection': 'keep-alive',
    'Cookie': cookie,
    'Host': 'api.venmo.com',
    'Origin': 'https://venmo.com',
    'Pragma': 'no-cache',
    'Referer': 'https://venmo.com/account/settings/balance/statement?end=02-08-2017&start=11-08-2016',
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36',
  }, 
}
try {
  const response = await fetch('https://api.venmo.com/v1/transaction-history?start_date=2016-11-08&end_date=2017-02-08', options)
  console.log(response)
} catch(e) {
  console.error(e)
}

This also did not work.

I even tried making the request from the console of the website and got a 401:

fetch('https://api.venmo.com/v1/transaction-history?start_date=2016-11-08&end_date=2017-02-08', {credentials: 'same-origin'}).then(console.log)

So my question here is this: I see a network request in Chrome Developer Tools. How can I make that same request programmatically? Preferably in Node.js or Python so I can write an automated script.

Sadiron

Chrome lets you copy network requests as fetch() calls now. – Institutional

If you'd like to add a "Copy as Python (requests)" to your Chrome's Network tab, see this GitHub thread: github.com/curlconverter/curlconverter/issues/… – Institutional

Answer 1

In the Network tab of the Chrome Developer Tools, right click the request and click "Copy" > "Copy as cURL (bash)". You can then either write a script using the curl command directly, or use https://curlconverter.com/ to convert the cURL command to Python, JavaScript, PHP, R, Go, Rust, Elixir, Java, MATLAB, Dart or JSON.

Ironworks

So... you're recommending that the OP should copy their cookie's secret value into some third-party website? – Enterovirus

@ParthianShot curlconverter.com works client-side using JavaScript. Unless the website gets hacked or sold to someone nefarious, the curl commands will never leave your browser. There's also a command-line tool that doesn't make any network calls whatsoever, which you can install with npm install curlconverter. Disclaimer: I maintain curlconverter. – Institutional

There are a couple of websites that do the same thing as curlconverter that do upload your command to themselves though, such as ReqBin. I would never use them. – Institutional