Nodejs - Expressjs - Verify shopify webhook

Question

I am trying to verify the hmac code sent from a shopify webhook on a dev environment. However shopify will not send a post request for a webhook to a non live endpoint, so I am using requestbin to capture the request and then use postman to send it to my local webserver.

From shopify documentation, I seem to be doing everything right and have also tried applying the method used in node-shopify-auth verifyWebhookHMAC function. But none of this has worked so far. The codes are never a match. What am I doing wrong here?

My code to verify the webhook:

 function verifyWebHook(req, res, next) {
      var message = JSON.stringify(req.body);
    //Shopify seems to be escaping forward slashes when the build the HMAC
        // so we need to do the same otherwise it will fail validation
        // Shopify also seems to replace '&' with \u0026 ...
        //message = message.replace('/', '\\/');
        message = message.split('/').join('\\/');
    message = message.split('&').join('\\u0026');
      var signature = crypto.createHmac('sha256', shopifyConfig.secret).update(message).digest('base64');
      var reqHeaderHmac = req.headers['x-shopify-hmac-sha256'];
      var truthCondition = signature === reqHeaderHmac;

      winston.info('sha256 signature: ' + signature);
      winston.info('x-shopify-hmac-sha256 from header: ' + reqHeaderHmac);
      winston.info(req.body);

      if (truthCondition) {
        winston.info('webhook verified');
        req.body = JSON.parse(req.body.toString());
        res.sendStatus(200);
        res.end();
        next();
      } else {
        winston.info('Failed to verify web-hook');
        res.writeHead(401);
        res.end('Unverified webhook');
      }
    }

My route which receives the request:

router.post('/update-product', useBodyParserJson, verifyWebHook, function (req, res) {
  var shopName = req.headers['x-shopify-shop-domain'].slice(0, -14);
  var itemId = req.headers['x-shopify-product-id'];
  winston.info('Shopname from webhook is: ' + shopName + ' For item: ' + itemId);
});

Sapsucker

Answer 1

I do it a little differently -- Not sure where I saw the recommendation but I do the verify in the body parser. IIRC one reason being that I get access to the raw body before any other handlers are likely to have touched it:

app.use( bodyParser.json({verify: function(req, res, buf, encoding) {
    var shopHMAC = req.get('x-shopify-hmac-sha256');
    if(!shopHMAC) return;
    if(req.get('x-kotn-webhook-verified')) throw "Unexpected webhook verified header";
    var sharedSecret = process.env.API_SECRET;
    var digest = crypto.createHmac('SHA256', sharedSecret).update(buf).digest('base64');
    if(digest == req.get('x-shopify-hmac-sha256')){
        req.headers['x-kotn-webhook-verified']= '200';
    }
 }})); 

and then any web hooks just deal with the verified header:

if('200' != req.get('x-kotn-webhook-verified')){
    console.log('invalid signature for uninstall');
    res.status(204).send();
    return;
}
var shop = req.get('x-shopify-shop-domain');
if(!shop){
    console.log('missing shop header for uninstall');
    res.status(400).send('missing shop');
    return;
}

Felty

Note: technically the 204 should be a 403 but I prefer to just silently let any callers think they succeeded. – Felty

This still didn't work for me :( How do you test it on a dev environment, as shopify doesn't send requests for webdhooks to dev endpoints? – Sapsucker

Also I log the req.body, just before the point where you define shopHAMC, its an empty object. Should it be empty at this point still? – Sapsucker

the body isn't defined at at that point. it's actually set by the body parser which runs after verify. Use the buffer. As far as testing goes I just register the webook when the app is installed.Since my app wasn't published on the app store I just used my live url. You can also use ngrok for this. – Felty

I am using request bin to capture and register the webhook and then postman to send the request to my dev. I wonder if there is not something wrong in that process!? – Sapsucker

Where are you planning on hosting the live app? If you can't use that try ngrok instead of requestb.in and postman. – Felty

Also did you follow my advice with the verify method? When you alter the body you have a very high likelihood of breaking the hmac. – Felty

I did use the verify method. I will have to give ngrok a try. what I am doing is obviously not working. All I was doing was copy pasting the body as is in requestb.in into the body box in postman and ahd it set to raw. – Sapsucker

So by using ngrok I got the proper raw body. requestb.in was altering the raw body somewhat. Once I got the proper raw body, the verification worked! Thanks! – Sapsucker

@Felty I'm unable to find the "sharedSecret" key. All I have is the public app's secret key. Are they the same? If not where can I find it? I don't see any answer anywhere on this. All I see is the key that is shown in "webhooks" setting under Admin, which I believe is not used in public app. If it is in fact the actual key, where and how will my public app get access to that key from each of the stores that installed my app.. Maybe you can shed some light on this mystery.. :\ – Pain

Yes the public app's secret key is the "shared secret". Shopify signs the webhook with the app's secret key. It has nothing to do with the shops the app is installed in. – Felty

Answer 2

Short Answer

The body parser in express does not handle BigInt well, and things like order number which are passed as integer get corrupted. Apart from that certain values are edited such as URLs are originally sent as "https://...", which OP also found out from the other code.

To solve this, do not parse the data using body parser and instead get it as raw string, later on you can parse it with json-bigint to ensure none of it has been corrupted.

Long Answer

Although the answer by @bknights works perfectly fine, it's important to find out why this was happening in the first place.

For a webhook I made on the "order_created" event from Shopify I found out that the id of the request being passed to the body was different than what I was sending from my test data, this turned out to be an issue with body-parser in express which did not play nice with big integers.

Ultimately I was deploying something to Google cloud functions and the req already had raw body which I could use, but in my test environment in Node I implemented the following as a separate body parser as using the same body parser twice overwrote the raw body with JSON

var rawBodySaver = function (req, res, buf, encoding) {
    if (buf && buf.length) {
      req.rawBody = buf.toString(encoding || 'utf8');
    }
}
app.use(bodyParser.json({verify: rawBodySaver, extended: true}));

Based on this answer

I later on parse the rawBody using json-bigint for use in code elsewhere as otherwise some of the numbers were corrupted.

Chancellor

I used body-parser.text() and it solved this problem github.com/tiendq/shopify-promobar/blob/master/server/webhooks/… – Salpa

Answer 3

// Change the way body-parser is used
const bodyParser = require('body-parser');

var rawBodySaver = function (req, res, buf, encoding) {
    if (buf && buf.length) {
        req.rawBody = buf.toString(encoding || 'utf8');
    }
}
app.use(bodyParser.json({ verify: rawBodySaver, extended: true }));


// Now we can access raw-body any where in out application as follows
// request.rawBody in routes;

// verify webhook middleware
const verifyWebhook = function (req, res, next) {
    console.log('Hey!!! we got a webhook to verify!');

    const hmac_header = req.get('X-Shopify-Hmac-Sha256');
    
    const body = req.rawBody;
    const calculated_hmac = crypto.createHmac('SHA256', secretKey)
        .update(body,'utf8', 'hex')
        .digest('base64');

    console.log('calculated_hmac', calculated_hmac);
    console.log('hmac_header', hmac_header);

    if (calculated_hmac == hmac_header) {
        console.log('Phew, it came from Shopify!');
        res.status(200).send('ok');
        next();
    }else {
        console.log('Danger! Not from Shopify!')
        res.status(403).send('invalid');
    }

}

Electronarcosis

Answer 4

Had the same issue. Using request.rawBody instead of request.body helped:

import Router from "koa-router";
import koaBodyParser from "koa-bodyparser";
import crypto from "crypto";

...

koaServer.use(koaBodyParser()); 

...

koaRouter.post(
    "/webhooks/<yourwebhook>",
    verifyShopifyWebhooks,
    async (ctx) => {
      try {
        ctx.res.statusCode = 200;
      } catch (error) {
        console.log(`Failed to process webhook: ${error}`);
      }
    }
);

...

async function verifyShopifyWebhooks(ctx, next) {
  const generateHash = crypto
    .createHmac("sha256", process.env.SHOPIFY_WEBHOOKS_KEY) // that's not your Shopify API secret key, but the key under Webhooks section in your admin panel (<yourstore>.myshopify.com/admin/settings/notifications) where it says "All your webhooks will be signed with [SHOPIFY_WEBHOOKS_KEY] so you can verify their integrity
    .update(ctx.request.rawBody, "utf-8")
    .digest("base64");

  if (generateHash !== shopifyHmac) {
    ctx.throw(401, "Couldn't verify Shopify webhook HMAC");
  } else {
    console.log("Successfully verified Shopify webhook HMAC");
  }
  await next();
}

Inelegant