Can't verify Webhooks in Node.js! (serverless AWS lambda)

[Zolbayar]

That exclamation point used in the title is not an accident. 

1. The official doc doesn't have an example for Node.js 

2. On this Shopify Partners' blog post, it says: 

exports.handler = (event, context, callback) => {
  var client_secret = event.client_secret;
  delete event.client_secret;

  //calculate the hash
  var calculated_hash = crypto.createHmac("sha256", client_secret).update(new Buffer(event.body, "base64")).digest("base64");

  //reject the message if the hash doesn't match
  if (event["X-Shopify-Hmac-SHA256"] != calculated_hash) {
    console.log("calculated_hash: (" + calculated_hash + ") != X-Shopify-Hmac-SHA256: (" + event["X-Shopify-Hmac-SHA256"] + ")");
    return;
  }
}

This post was specifically for an AWS lambda function. But when I generate the hash like this:

var calculated_hash = crypto.createHmac("sha256", client_secret).update(new Buffer(event.body, "base64")).digest("base64");

It's still different from the `X-Shopify-Hmac-Sha256` header value. 

3. I tried it with the following different ways: 

• update(Buffer.from(event.body, "base64"))

• update(Buffer.from(JSON.stringify(event.body), 'utf8', 'hex'))

• update(event.body)

4. I replaced `client_secret` with the following values: 

• "API secret key" from partners.shopify.com → Apps → App
• "API key" from partners.shopify.com → Apps → App
• "Access token" I got from the store when we installed the app

All the docs say it's either "client secret" or "shared secret". But I can't find any value with that exact name. Am I using the wrong secret? Or am I using a wrong method to generate the hash? 

[Zolbayar]

I tried again with the following code snippet. Even the buffer lengths are different.

const message = querystring.stringify(map);
    const providedHmac = Buffer.from(hmac, 'utf-8');
    const generatedHash = Buffer.from(
      crypto
        .createHmac('sha256', shopifyApp.apiSecret)
        .update(message)
        .digest('hex'),
      'utf-8',
    );
    let hashEquals = false;
    try {
      hashEquals = crypto.timingSafeEqual(generatedHash, providedHmac);
    } catch (e) {
      console.log('Error while timingSafeEqual', e);
      hashEquals = false;
    }

[SBD_]

Hey @Zolbayar,

Here's an Express example (where secretKey is the API secret key), see if you can get this working then work backwards to remove Express.

const express = require('express')
const app = express()
const crypto = require('crypto')
const secretKey = '<your secret key>'
const bodyParser = require('body-parser')

app.use('/webhooks', bodyParser.raw({ type: 'application/json' }))
app.use(bodyParser.json())

app.post('/webhooks/orders/create', async (req, res) => {
  const hmac = req.get('X-Shopify-Hmac-Sha256')

  // create a hash using the body and our key
  const hash = crypto
    .createHmac('sha256', secretKey)
    .update(req.body, 'utf8', 'hex')
    .digest('base64')

  // Then compare hash to hmac.
  // ...
})

app.listen(3000, () => console.log('Example app listening on port 3000!'))

Let me know if you get stuck.

[Zolbayar]

Hey @SBD_ ,

Thanks for replying 🙂

I did it like that and the hash differs from the hmac:

hash jVjYxZpx4qaDLjW7g62RLaOKSs2H87861p7/nWuz150=
hmac EKYVgsZJ4DQo5jBSMbPrUSGN6rGRWOQbYhKVnuwvpQ0=

I think the cause might be the following line: 

app.use('/webhooks', bodyParser.raw({ type: 'application/json' }))
app.use(bodyParser.json())

Since I'm using serverless.com to deploy to AWS lambda, I'm not sure how can I do this.

[AlexH316]

Hi,

Did you ever happen to get this working?  I am trying to use AWS Lambda/API Gateway and like you, I am pulling my hair out trying to get the hmac to validate.

Thanks!

Alex

[Luis45]

Hi,

Did you ever happen to get this working? , I have the same problem

[Zolbayar]

Hi @Luis45,

I made it work with this method: 

const verifyHMAC = (shopifyHmac, shopifyPayload) => {
  const hash = crypto
    .createHmac('sha256', process.env.SHOPIFY_PUBLIC_APP_SECRET)
    .update(shopifyPayload, 'utf8')
    .digest('base64');

  const isEqual = crypto.timingSafeEqual(
    Buffer.from(hash),
    Buffer.from(shopifyHmac),
  );

  return isEqual;
};

The key was using providing only `utf8` to the `.update(shopifyPayload, 'utf8')`. Not '`base64' `or `'utf8', 'hex'`

[Mete]

app.use(bodyParser.json({
        verify: (req, res, buf) => {
            req.rawbody = buf
        }
    })
);

Then you can use req.rawbody to generate hash instead of req.body

https://github.com/expressjs/body-parser/pull/24
https://github.com/expressjs/body-parser#verify

[krumholz]

When I add the bodyParser like this:

app.use(bodyParser.json({
    verify: (req: any, res: any, buf: any) => {
        req.rawbody = buf
    }
  })
);

I get an error when I try to use the rawbody like this:

app.post('/uninstall', async (req, res) => {
    req.rawbody
};

Is there something I'm doing obviously wrong?

[Zolbayar]

Hi @krumholz ,

I'm confused. Seems like you're trying to get the raw body by using the json body parser. I think you should use

bodyParser.raw([options])

instead 🙂

[SBD_]

Hey @Zolbayar + @AlexH316,

Finally got to the bottom of this. The body contains backslashes, so some\/string becomes some/string.

I escaped the backslashes themselves (some\\/string) and had success with the following code:

const hash = crypto
  .createHmac('sha256', secret)
  .update(body, 'utf8', 'hex')
  .digest('base64')

So you can manually escape the backslashes, or try to get the raw body before it's turned into a string.

Let me know how you go.

[razzededge]

There is a blog post about how to make it work with node 12 lambda on aws here:

https://medium.com/@damianpieszczynski/validating-shopify-webhooks-with-aws-lambda-and-node-58a8da00...

[longinoa]

Hey SBD_  (and anyone else really)

I am running into the same issue here. I can not get my node.js 12 aws lambda to verify.

I have tried doing the manual escaping as well:

module.exports.validateHMACForWebhook = (event) => {
    const hmac =
        event.headers['X-Shopify-Hmac-Sha256'].toLowerCase();
    const topic =
        event.headers['X-Shopify-Topic'].toLowerCase();
    const domain =
        event.headers['X-Shopify-Shop-Domain'].toLowerCase();


    const body = Buffer.from(event.body, "utf8").toString();
    var bodyForVerification = body.replace('/\\/g', '\\\\');

    //let's calculate the hash of the content that webhook has sent to us
    const content_hmac_hash = crypto.createHmac("sha256", process.env.SHOPIFY_API_SECRET)
        .update(bodyForVerification)
        .digest("base64");
    console.log("calculated: " + content_hmac_hash + " given: " + hmac); 

     return {
        verified: content_hmac_hash === hmac,
        topic,
        domain,
        body: JSON.parse(body)
    };
}

When I return out of this function verified is false

2021-02-01T05:09:39.258Z 80bbfa7f-58d6-4afb-baae-e948e351ed68 INFO calculated: IGg6dc2sTh3jMbAJwPKhn+EY/fV32CuZyyG3gwMN8j4= given: jtp9alrzvckasuywzpr/tvucirowrhljur192p4g1ri=

Has anyone gotten this working?

[razzededge]

You can try my tutorial here

https://link.medium.com/mxx2QpZ8vdb

[longinoa]

@razzededge I saw your tutorial (thanks for putting it together!) 

I get the same problem with yours. I had chalked it up to the difference between the store's notification webhook and the app's subscribed webhook that I was using. However I just tested it with my development store's notification and had the same issue 😞

[razzededge]

Well it certainly should work - but I think you have problem here:

SHOPIFY_API_SECRET

if you're using api secret then it will not work - you need to use the webhook secret  

find it under the:  myshopify.com/admin/settings/notifications

at the bottom here:

[longinoa]

Yeah I used that value...  I am going to keep digging ...

[longinoa]

Okay interesting - when I create a lambda directly from aws's console and use the code it works correctly - however when I am using my serverless.yml to deploy the exact same code it is failing.

What is happening is that the lambda created directly from aw's console is getting escaped json as the body

"body": "{\"id\":\"eeafa272cebfd4b22385bc4b645e762c\",\"token\":\"eeafa272cebfd4b22385bc4b645e762c\",\"line_items\":[{\"id\":704912205188288575,\"properties\":{},\"quantity\":3,\"variant_id\":704912205188288575,\"key\":\"704912205188288575:33f11f7a1ec7d93b826de33bb54de37b\",\"discounted_price\":\"19.99\",\"discounts\":[],\"gift_card\":false,\"grams\":200,\"line_price\":\"59.97\",\"original_line_price\":\"59.97\",\"original_price\":\"19.99\",\"price\":\"19.99\",\"product_id\":788032119674292922,\"sku\":\"example-shirt-s\",\"taxable\":true,\"title\":\"Example T-Shirt - \",\"total_discount\":\"0.00\",\"vendor\":\"Acme\",\"discounted_price_set\":{\"shop_money\":{\"amount\":\"19.99\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"19.99\",\"currency_code\":\"USD\"}},\"line_price_set\":{\"shop_money\":{\"amount\":\"59.97\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"59.97\",\"currency_code\":\"USD\"}},\"original_line_price_set\":{\"shop_money\":{\"amount\":\"59.97\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"59.97\",\"currency_code\":\"USD\"}},\"price_set\":{\"shop_money\":{\"amount\":\"19.99\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"19.99\",\"currency_code\":\"USD\"}},\"total_discount_set\":{\"shop_money\":{\"amount\":\"0.0\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"0.0\",\"currency_code\":\"USD\"}}}],\"note\":null,\"updated_at\":\"2021-02-02T04:49:40.193Z\",\"created_at\":\"2021-02-02T04:49:40.193Z\"}",
    "isBase64Encoded": false

where as the version using my serverless implementation based on this github: https://github.com/maxkostinevich/Shopify-Serverless-Starter-App is getting base 64 encoded:

 "body": "eyJpZCI6ImVlYWZhMjcyY2ViZmQ0YjIyMzg1YmM0YjY0NWU3NjJjIiwidG9rZW4iOiJlZWFmYTI3MmNlYmZkNGIyMjM4NWJjNGI2NDVlNzYyYyIsImxpbmVfaXRlbXMiOlt7ImlkIjo3MDQ5MTIyMDUxODgyODg1NzUsInByb3BlcnRpZXMiOnt9LCJxdWFudGl0eSI6MywidmFyaWFudF9pZCI6NzA0OTEyMjA1MTg4Mjg4NTc1LCJrZXkiOiI3MDQ5MTIyMDUxODgyODg1NzU6MzNmMTFmN2ExZWM3ZDkzYjgyNmRlMzNiYjU0ZGUzN2IiLCJkaXNjb3VudGVkX3ByaWNlIjoiMTkuOTkiLCJkaXNjb3VudHMiOltdLCJnaWZ0X2NhcmQiOmZhbHNlLCJncmFtcyI6MjAwLCJsaW5lX3ByaWNlIjoiNTkuOTciLCJvcmlnaW5hbF9saW5lX3ByaWNlIjoiNTkuOTciLCJvcmlnaW5hbF9wcmljZSI6IjE5Ljk5IiwicHJpY2UiOiIxOS45OSIsInByb2R1Y3RfaWQiOjc4ODAzMjExOTY3NDI5MjkyMiwic2t1IjoiZXhhbXBsZS1zaGlydC1zIiwidGF4YWJsZSI6dHJ1ZSwidGl0bGUiOiJFeGFtcGxlIFQtU2hpcnQgLSAiLCJ0b3RhbF9kaXNjb3VudCI6IjAuMDAiLCJ2ZW5kb3IiOiJBY21lIiwiZGlzY291bnRlZF9wcmljZV9zZXQiOnsic2hvcF9tb25leSI6eyJhbW91bnQiOiIxOS45OSIsImN1cnJlbmN5X2NvZGUiOiJVU0QifSwicHJlc2VudG1lbnRfbW9uZXkiOnsiYW1vdW50IjoiMTkuOTkiLCJjdXJyZW5jeV9jb2RlIjoiVVNEIn19LCJsaW5lX3ByaWNlX3NldCI6eyJzaG9wX21vbmV5Ijp7ImFtb3VudCI6IjU5Ljk3IiwiY3VycmVuY3lfY29kZSI6IlVTRCJ9LCJwcmVzZW50bWVudF9tb25leSI6eyJhbW91bnQiOiI1OS45NyIsImN1cnJlbmN5X2NvZGUiOiJVU0QifX0sIm9yaWdpbmFsX2xpbmVfcHJpY2Vfc2V0Ijp7InNob3BfbW9uZXkiOnsiYW1vdW50IjoiNTkuOTciLCJjdXJyZW5jeV9jb2RlIjoiVVNEIn0sInByZXNlbnRtZW50X21vbmV5Ijp7ImFtb3VudCI6IjU5Ljk3IiwiY3VycmVuY3lfY29kZSI6IlVTRCJ9fSwicHJpY2Vfc2V0Ijp7InNob3BfbW9uZXkiOnsiYW1vdW50IjoiMTkuOTkiLCJjdXJyZW5jeV9jb2RlIjoiVVNEIn0sInByZXNlbnRtZW50X21vbmV5Ijp7ImFtb3VudCI6IjE5Ljk5IiwiY3VycmVuY3lfY29kZSI6IlVTRCJ9fSwidG90YWxfZGlzY291bnRfc2V0Ijp7InNob3BfbW9uZXkiOnsiYW1vdW50IjoiMC4wIiwiY3VycmVuY3lfY29kZSI6IlVTRCJ9LCJwcmVzZW50bWVudF9tb25leSI6eyJhbW91bnQiOiIwLjAiLCJjdXJyZW5jeV9jb2RlIjoiVVNEIn19fV0sIm5vdGUiOm51bGwsInVwZGF0ZWRfYXQiOiIyMDIxLTAyLTAyVDA1OjEzOjI2LjIwMloiLCJjcmVhdGVkX2F0IjoiMjAyMS0wMi0wMlQwNToxMzoyNi4yMDJaIn0=",
    "isBase64Encoded": true

when when run through a `Buffer.from(event.body, 'utf8')` does not produce the  escaped quotes. 

I am assuming there are other things that are not escaped as well - my hunch would be \ itself. 

Does anyone know if I have to go the full route of https://community.shopify.com/c/Shopify-APIs-SDKs/webhook-verification/m-p/665201/highlight/true#M45... or can I achieve this another way? possibly through an escaping helper in node.js?

[razzededge]

Well it's not weird - Api Gateway is configured to digest binary transfers - thus expecting they will be Base64 Encoded - this part in serverless config is the culprit or should I say the effect something no "outof the box" - just get rid of the definition that everything is Binary 😄
"*/*" this fck all up imho - I just cannot comprehend why any header is defined as binary 😉😉

custom:
  apiGateway:
    binaryMediaTypes:
      - image/png
      - image/jpeg
      - "*/*"
  apigwBinary:
    types:
      - 'image/jpeg'
      - 'image/png'
  defaultStage: dev
  environment: ${file(env.yml):${self:provider.stage}, file(env.yml):default}

[longinoa]

That was it! Thanks @razzededge! I'm still getting used to understanding all things server development - I come from a mobile background 🙂

[razzededge]

No problem - good that it resolved it for you.

[ushooa]

I got this to work, the key is to create an HTTP API on API gateway (not REST API), on HTTP API, create a POST route and on its integration method, hook it up to your javascript lambda function.

here is the code snippet:

exports.handler = async (event, context, callback) => {

const { webhook_verify_hash } = process.env;

//get the header with validation hash from webhook
const shopify_hmac_hash = event.headers ?
event.headers['X-Shopify-Hmac-Sha256'] || event.headers['x-shopify-hmac-sha256']
: "";


const content_hmac_hash = crypto.createHmac("sha256", webhook_verify_hash)
.update(event.body, "utf8")
.digest("base64");


if(content_hmac_hash !== shopify_hmac_hash) {
console.log('Integrity of request compromised, aborting');

....
} else {

.......

}

}

[theschoolofux]

Had the same issue. Using request.rawBody instead of request.body helped:

import Router from "koa-router";
import koaBodyParser from "koa-bodyparser";
import crypto from "crypto";

...

koaServer.use(koaBodyParser()); 

...

koaRouter.post(
    "/webhooks/<yourwebhook>",
    verifyShopifyWebhooks,
    async (ctx) => {
      try {
        ctx.res.statusCode = 200;
      } catch (error) {
        console.log(`Failed to process webhook: ${error}`);
      }
    }
);

...

async function verifyShopifyWebhooks(ctx, next) {
  const generateHash = crypto
    .createHmac("sha256", process.env.SHOPIFY_WEBHOOKS_KEY) // that's not your Shopify API secret key, but the key under Webhooks section in your admin panel (<yourstore>.myshopify.com/admin/settings/notifications) where it says "All your webhooks will be signed with [SHOPIFY_WEBHOOKS_KEY] so you can verify their integrity
    .update(ctx.request.rawBody, "utf-8")
    .digest("base64");

  if (generateHash !== shopifyHmac) {
    ctx.throw(401, "Couldn't verify Shopify webhook HMAC");
  } else {
    console.log("Successfully verified Shopify webhook HMAC");
  }
  await next();
}

[Luis45]

Hi, I have the same problem. but i am not using "crypto", I'm using "crypto-js" Search the module "'crypto" but I did not find it and the module "crypto-js"doesn't have the func "createHmac"..

could you tell me how to find the module "crypto" and upload a lambda aws..?