Can't verify Webhooks in Node.js! (serverless AWS lambda)

Zolbayar
Shopify Partner
46 1 8

That exclamation point used in the title is not an accident. 

 

1. The official doc doesn't have an example for Node.js 

 

2. On this Shopify Partners' blog post, it says: 

 

exports.handler = (event, context, callback) => {
  var client_secret = event.client_secret;
  delete event.client_secret;

  //calculate the hash
  var calculated_hash = crypto.createHmac("sha256", client_secret).update(new Buffer(event.body, "base64")).digest("base64");

  //reject the message if the hash doesn't match
  if (event["X-Shopify-Hmac-SHA256"] != calculated_hash) {
    console.log("calculated_hash: (" + calculated_hash + ") != X-Shopify-Hmac-SHA256: (" + event["X-Shopify-Hmac-SHA256"] + ")");
    return;
  }
}

This post was specifically for an AWS lambda function. But when I generate the hash like this:

 

 

var calculated_hash = crypto.createHmac("sha256", client_secret).update(new Buffer(event.body, "base64")).digest("base64");

It's still different from the `X-Shopify-Hmac-Sha256` header value. 

 

3. I tried it with the following different ways: 

  • update(Buffer.from(event.body, "base64"))
  • update(Buffer.from(JSON.stringify(event.body), 'utf8', 'hex'))
  • update(event.body)

4. I replaced `client_secret` with the following values: 

  • "API secret key" from partners.shopify.com → Apps → App
  • "API key" from partners.shopify.com → Apps → App
  • "Access token" I got from the store when we installed the app

All the docs say it's either "client secret" or "shared secret". But I can't find any value with that exact name. Am I using the wrong secret? Or am I using a wrong method to generate the hash? 

 

Working remotely from Mongolia ~ www.gereesee.com
Replies 23 (23)

Zolbayar
Shopify Partner
46 1 8

I tried again with the following code snippet. Even the buffer lengths are different.

 

const message = querystring.stringify(map);
    const providedHmac = Buffer.from(hmac, 'utf-8');
    const generatedHash = Buffer.from(
      crypto
        .createHmac('sha256', shopifyApp.apiSecret)
        .update(message)
        .digest('hex'),
      'utf-8',
    );
    let hashEquals = false;
    try {
      hashEquals = crypto.timingSafeEqual(generatedHash, providedHmac);
    } catch (e) {
      console.log('Error while timingSafeEqual', e);
      hashEquals = false;
    }

 

Working remotely from Mongolia ~ www.gereesee.com
SBD_
Shopify Staff
1829 269 405

Hey @Zolbayar,

 

Here's an Express example (where secretKey is the API secret key), see if you can get this working then work backwards to remove Express.

 

 

const express = require('express')
const app = express()
const crypto = require('crypto')
const secretKey = '<your secret key>'
const bodyParser = require('body-parser')

app.use('/webhooks', bodyParser.raw({ type: 'application/json' }))
app.use(bodyParser.json())

app.post('/webhooks/orders/create', async (req, res) => {
  const hmac = req.get('X-Shopify-Hmac-Sha256')

  // create a hash using the body and our key
  const hash = crypto
    .createHmac('sha256', secretKey)
    .update(req.body, 'utf8', 'hex')
    .digest('base64')

  // Then compare hash to hmac.
  // ...
})

app.listen(3000, () => console.log('Example app listening on port 3000!'))

Let me know if you get stuck.

 

Scott | Developer Advocate @ Shopify 

Zolbayar
Shopify Partner
46 1 8

Hey @SBD_ ,

Thanks for replying 🙂

 

I did it like that and the hash differs from the hmac:

hash jVjYxZpx4qaDLjW7g62RLaOKSs2H87861p7/nWuz150=
hmac EKYVgsZJ4DQo5jBSMbPrUSGN6rGRWOQbYhKVnuwvpQ0=

I think the cause might be the following line: 

app.use('/webhooks', bodyParser.raw({ type: 'application/json' }))
app.use(bodyParser.json())

Since I'm using serverless.com to deploy to AWS lambda, I'm not sure how can I do this.

Working remotely from Mongolia ~ www.gereesee.com
AlexH316
Tourist
5 0 5

Hi,

 

Did you ever happen to get this working?  I am trying to use AWS Lambda/API Gateway and like you, I am pulling my hair out trying to get the hmac to validate.

 

Thanks!

 

Alex

Luis45
New Member
6 0 0

Hi,

 

Did you ever happen to get this working? , I have the same problem

Zolbayar
Shopify Partner
46 1 8

Hi @Luis45,

 

I made it work with this method: 

 

 

const verifyHMAC = (shopifyHmac, shopifyPayload) => {
  const hash = crypto
    .createHmac('sha256', process.env.SHOPIFY_PUBLIC_APP_SECRET)
    .update(shopifyPayload, 'utf8')
    .digest('base64');

  const isEqual = crypto.timingSafeEqual(
    Buffer.from(hash),
    Buffer.from(shopifyHmac),
  );

  return isEqual;
};

The key was using providing only `utf8` to the `.update(shopifyPayload, 'utf8')`. Not '`base64' `or `'utf8', 'hex'`

 

Working remotely from Mongolia ~ www.gereesee.com
Mete
Shopify Partner
132 0 37
app.use(bodyParser.json({
verify: (req, res, buf) => {
req.rawbody = buf
}
})
);

Then you can use req.rawbody to generate hash instead of req.body

 

https://github.com/expressjs/body-parser/pull/24
https://github.com/expressjs/body-parser#verify

Co-Founder / Developer at: merchbees
Merchbees Low Stock Alert - Keep track of your low stock items by email and slack
Merchbees Inventory Value - Know your inventory value and quantity in real-time
Push Down & Hide Out of Stock - Move out of stock products to the bottom of the collection to improve SEO & hide/ unhide automatically
krumholz
Shopify Partner
6 0 1

When I add the bodyParser like this:

app.use(bodyParser.json({
    verify: (req: any, res: any, buf: any) => {
        req.rawbody = buf
    }
  })
);

I get an error when I try to use the rawbody like this:

app.post('/uninstall', async (req, res) => {
    req.rawbody
};

Is there something I'm doing obviously wrong?

Zolbayar
Shopify Partner
46 1 8

Hi @krumholz ,

I'm confused. Seems like you're trying to get the raw body by using the json body parser. I think you should use

bodyParser.raw([options])

instead 🙂

Working remotely from Mongolia ~ www.gereesee.com
SBD_
Shopify Staff
1829 269 405

Hey @Zolbayar + @AlexH316,

Finally got to the bottom of this. The body contains backslashes, so some\/string becomes some/string.

 

I escaped the backslashes themselves (some\\/string) and had success with the following code:

const hash = crypto
.createHmac('sha256', secret)
.update(body, 'utf8', 'hex')
.digest('base64')

So you can manually escape the backslashes, or try to get the raw body before it's turned into a string.

 

Let me know how you go.

Scott | Developer Advocate @ Shopify 

razzededge
Tourist
6 0 3

There is a blog post about how to make it work with node 12 lambda on aws here:

https://medium.com/@damianpieszczynski/validating-shopify-webhooks-with-aws-lambda-and-node-58a8da00...

longinoa
Excursionist
16 1 0

Hey SBD_  (and anyone else really)

 

I am running into the same issue here. I can not get my node.js 12 aws lambda to verify.

 

I have tried doing the manual escaping as well:

 

module.exports.validateHMACForWebhook = (event) => {
    const hmac =
        event.headers['X-Shopify-Hmac-Sha256'].toLowerCase();
    const topic =
        event.headers['X-Shopify-Topic'].toLowerCase();
    const domain =
        event.headers['X-Shopify-Shop-Domain'].toLowerCase();


    const body = Buffer.from(event.body, "utf8").toString();
    var bodyForVerification = body.replace('/\\/g', '\\\\');

    //let's calculate the hash of the content that webhook has sent to us
    const content_hmac_hash = crypto.createHmac("sha256", process.env.SHOPIFY_API_SECRET)
        .update(bodyForVerification)
        .digest("base64");
    console.log("calculated: " + content_hmac_hash + " given: " + hmac); 

     return {
        verified: content_hmac_hash === hmac,
        topic,
        domain,
        body: JSON.parse(body)
    };
}

 

 

When I return out of this function verified is false

2021-02-01T05:09:39.258Z 80bbfa7f-58d6-4afb-baae-e948e351ed68 INFO calculated: IGg6dc2sTh3jMbAJwPKhn+EY/fV32CuZyyG3gwMN8j4= given: jtp9alrzvckasuywzpr/tvucirowrhljur192p4g1ri=

 

Has anyone gotten this working?

razzededge
Tourist
6 0 3
You can try my tutorial here

https://link.medium.com/mxx2QpZ8vdb
longinoa
Excursionist
16 1 0

@razzededge I saw your tutorial (thanks for putting it together!) 

 

I get the same problem with yours. I had chalked it up to the difference between the store's notification webhook and the app's subscribed webhook that I was using. However I just tested it with my development store's notification and had the same issue 😞

razzededge
Tourist
6 0 3

Well it certainly should work - but I think you have problem here:

SHOPIFY_API_SECRET

 

if you're using api secret then it will not work - you need to use the webhook secret  

find it under the:  myshopify.com/admin/settings/notifications

at the bottom here:

Screenshot 2021-02-01 at 18.16.15.png

longinoa
Excursionist
16 1 0

Yeah I used that value...  I am going to keep digging ...

 

longinoa
Excursionist
16 1 0

Okay interesting - when I create a lambda directly from aws's console and use the code it works correctly - however when I am using my serverless.yml to deploy the exact same code it is failing.

What is happening is that the lambda created directly from aw's console is getting escaped json as the body

"body": "{\"id\":\"eeafa272cebfd4b22385bc4b645e762c\",\"token\":\"eeafa272cebfd4b22385bc4b645e762c\",\"line_items\":[{\"id\":704912205188288575,\"properties\":{},\"quantity\":3,\"variant_id\":704912205188288575,\"key\":\"704912205188288575:33f11f7a1ec7d93b826de33bb54de37b\",\"discounted_price\":\"19.99\",\"discounts\":[],\"gift_card\":false,\"grams\":200,\"line_price\":\"59.97\",\"original_line_price\":\"59.97\",\"original_price\":\"19.99\",\"price\":\"19.99\",\"product_id\":788032119674292922,\"sku\":\"example-shirt-s\",\"taxable\":true,\"title\":\"Example T-Shirt - \",\"total_discount\":\"0.00\",\"vendor\":\"Acme\",\"discounted_price_set\":{\"shop_money\":{\"amount\":\"19.99\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"19.99\",\"currency_code\":\"USD\"}},\"line_price_set\":{\"shop_money\":{\"amount\":\"59.97\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"59.97\",\"currency_code\":\"USD\"}},\"original_line_price_set\":{\"shop_money\":{\"amount\":\"59.97\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"59.97\",\"currency_code\":\"USD\"}},\"price_set\":{\"shop_money\":{\"amount\":\"19.99\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"19.99\",\"currency_code\":\"USD\"}},\"total_discount_set\":{\"shop_money\":{\"amount\":\"0.0\",\"currency_code\":\"USD\"},\"presentment_money\":{\"amount\":\"0.0\",\"currency_code\":\"USD\"}}}],\"note\":null,\"updated_at\":\"2021-02-02T04:49:40.193Z\",\"created_at\":\"2021-02-02T04:49:40.193Z\"}",
    "isBase64Encoded": false

where as the version using my serverless implementation based on this github: https://github.com/maxkostinevich/Shopify-Serverless-Starter-App is getting base 64 encoded:

 "body": "eyJpZCI6ImVlYWZhMjcyY2ViZmQ0YjIyMzg1YmM0YjY0NWU3NjJjIiwidG9rZW4iOiJlZWFmYTI3MmNlYmZkNGIyMjM4NWJjNGI2NDVlNzYyYyIsImxpbmVfaXRlbXMiOlt7ImlkIjo3MDQ5MTIyMDUxODgyODg1NzUsInByb3BlcnRpZXMiOnt9LCJxdWFudGl0eSI6MywidmFyaWFudF9pZCI6NzA0OTEyMjA1MTg4Mjg4NTc1LCJrZXkiOiI3MDQ5MTIyMDUxODgyODg1NzU6MzNmMTFmN2ExZWM3ZDkzYjgyNmRlMzNiYjU0ZGUzN2IiLCJkaXNjb3VudGVkX3ByaWNlIjoiMTkuOTkiLCJkaXNjb3VudHMiOltdLCJnaWZ0X2NhcmQiOmZhbHNlLCJncmFtcyI6MjAwLCJsaW5lX3ByaWNlIjoiNTkuOTciLCJvcmlnaW5hbF9saW5lX3ByaWNlIjoiNTkuOTciLCJvcmlnaW5hbF9wcmljZSI6IjE5Ljk5IiwicHJpY2UiOiIxOS45OSIsInByb2R1Y3RfaWQiOjc4ODAzMjExOTY3NDI5MjkyMiwic2t1IjoiZXhhbXBsZS1zaGlydC1zIiwidGF4YWJsZSI6dHJ1ZSwidGl0bGUiOiJFeGFtcGxlIFQtU2hpcnQgLSAiLCJ0b3RhbF9kaXNjb3VudCI6IjAuMDAiLCJ2ZW5kb3IiOiJBY21lIiwiZGlzY291bnRlZF9wcmljZV9zZXQiOnsic2hvcF9tb25leSI6eyJhbW91bnQiOiIxOS45OSIsImN1cnJlbmN5X2NvZGUiOiJVU0QifSwicHJlc2VudG1lbnRfbW9uZXkiOnsiYW1vdW50IjoiMTkuOTkiLCJjdXJyZW5jeV9jb2RlIjoiVVNEIn19LCJsaW5lX3ByaWNlX3NldCI6eyJzaG9wX21vbmV5Ijp7ImFtb3VudCI6IjU5Ljk3IiwiY3VycmVuY3lfY29kZSI6IlVTRCJ9LCJwcmVzZW50bWVudF9tb25leSI6eyJhbW91bnQiOiI1OS45NyIsImN1cnJlbmN5X2NvZGUiOiJVU0QifX0sIm9yaWdpbmFsX2xpbmVfcHJpY2Vfc2V0Ijp7InNob3BfbW9uZXkiOnsiYW1vdW50IjoiNTkuOTciLCJjdXJyZW5jeV9jb2RlIjoiVVNEIn0sInByZXNlbnRtZW50X21vbmV5Ijp7ImFtb3VudCI6IjU5Ljk3IiwiY3VycmVuY3lfY29kZSI6IlVTRCJ9fSwicHJpY2Vfc2V0Ijp7InNob3BfbW9uZXkiOnsiYW1vdW50IjoiMTkuOTkiLCJjdXJyZW5jeV9jb2RlIjoiVVNEIn0sInByZXNlbnRtZW50X21vbmV5Ijp7ImFtb3VudCI6IjE5Ljk5IiwiY3VycmVuY3lfY29kZSI6IlVTRCJ9fSwidG90YWxfZGlzY291bnRfc2V0Ijp7InNob3BfbW9uZXkiOnsiYW1vdW50IjoiMC4wIiwiY3VycmVuY3lfY29kZSI6IlVTRCJ9LCJwcmVzZW50bWVudF9tb25leSI6eyJhbW91bnQiOiIwLjAiLCJjdXJyZW5jeV9jb2RlIjoiVVNEIn19fV0sIm5vdGUiOm51bGwsInVwZGF0ZWRfYXQiOiIyMDIxLTAyLTAyVDA1OjEzOjI2LjIwMloiLCJjcmVhdGVkX2F0IjoiMjAyMS0wMi0wMlQwNToxMzoyNi4yMDJaIn0=",
    "isBase64Encoded": true

when when run through a `Buffer.from(event.body, 'utf8')` does not produce the  escaped quotes. 

 

I am assuming there are other things that are not escaped as well - my hunch would be \ itself. 

 

Does anyone know if I have to go the full route of https://community.shopify.com/c/Shopify-APIs-SDKs/webhook-verification/m-p/665201/highlight/true#M45... or can I achieve this another way? possibly through an escaping helper in node.js?

razzededge
Tourist
6 0 3

Well it's not weird - Api Gateway is configured to digest binary transfers - thus expecting they will be Base64 Encoded - this part in serverless config is the culprit or should I say the effect something no "outof the box" - just get rid of the definition that everything is Binary 😄
"*/*" this fck all up imho - I just cannot comprehend why any header is defined as binary 😉😉

 

custom:
  apiGateway:
    binaryMediaTypes:
      - image/png
      - image/jpeg
      - "*/*"
  apigwBinary:
    types:
      - 'image/jpeg'
      - 'image/png'
  defaultStage: dev
  environment: ${file(env.yml):${self:provider.stage}, file(env.yml):default}

 

 

longinoa
Excursionist
16 1 0

That was it! Thanks @razzededge! I'm still getting used to understanding all things server development - I come from a mobile background 🙂

razzededge
Tourist
6 0 3

No problem - good that it resolved it for you.

ushooa
Visitor
1 0 0

I got this to work, the key is to create an HTTP API on API gateway (not REST API), on HTTP API, create a POST route and on its integration method, hook it up to your javascript lambda function.

here is the code snippet:

exports.handler = async (event, context, callback) => {

const { webhook_verify_hash } = process.env;

//get the header with validation hash from webhook
const shopify_hmac_hash = event.headers ?
event.headers['X-Shopify-Hmac-Sha256'] || event.headers['x-shopify-hmac-sha256']
: "";


const content_hmac_hash = crypto.createHmac("sha256", webhook_verify_hash)
.update(event.body, "utf8")
.digest("base64");


if(content_hmac_hash !== shopify_hmac_hash) {
console.log('Integrity of request compromised, aborting');

....
} else {

.......

}

}

theschoolofux
Shopify Partner
10 0 3

Had the same issue. Using request.rawBody instead of request.body helped:

 

import Router from "koa-router";
import koaBodyParser from "koa-bodyparser";
import crypto from "crypto";

...

koaServer.use(koaBodyParser()); 

...

koaRouter.post(
    "/webhooks/<yourwebhook>",
    verifyShopifyWebhooks,
    async (ctx) => {
      try {
        ctx.res.statusCode = 200;
      } catch (error) {
        console.log(`Failed to process webhook: ${error}`);
      }
    }
);

...

async function verifyShopifyWebhooks(ctx, next) {
  const generateHash = crypto
    .createHmac("sha256", process.env.SHOPIFY_WEBHOOKS_KEY) // that's not your Shopify API secret key, but the key under Webhooks section in your admin panel (<yourstore>.myshopify.com/admin/settings/notifications) where it says "All your webhooks will be signed with [SHOPIFY_WEBHOOKS_KEY] so you can verify their integrity
    .update(ctx.request.rawBody, "utf-8")
    .digest("base64");

  if (generateHash !== shopifyHmac) {
    ctx.throw(401, "Couldn't verify Shopify webhook HMAC");
  } else {
    console.log("Successfully verified Shopify webhook HMAC");
  }
  await next();
}

 

Sergei Golubev | Devigner @ The School of UX | schoolofux.com
Luis45
New Member
6 0 0

Hi, I have the same problem. but i am not using "crypto", I'm using "crypto-js" Search the module "'crypto" but I did not find it and the module "crypto-js"doesn't have the func "createHmac"..

 

 

could you tell me how to find the module "crypto" and upload a lambda aws..?