API CHANGED - carts/create webhook structure suddenly changed

[Levi3]

About 20 minutes ago (5:40pm EDT) we started getting hundreds of invalid carts/create webhook calls across multiple shopify accounts. It looks exactly like the last issue I posted on here where it started ramping up like a recent roll-out, and the rate is still increasing.

The payloads look like:

{"cart":{"id":"a3f1...","token":"a3f1....","line_items":[],"note":null,"updated_at":"2018-10-23T21:44:49.031Z","created_at":"2018-10-23T21:44:49.031Z"}}

The documentation (and historical events) do not have the data nested in a "cart" object. this is a sudden undocumented breaking change as far as I can tell. see the documentation: https://help.shopify.com/en/api/reference/events/webhook

Help. Can you guys roll back whatever deploy caused this, or does everyone need to suddenly handle a new format?

This also affects the carts/update webhook in the same way.

[Levi3]

When I said hundreds, I meant hundreds per minute. We've seen over 13k of these requests in the last 25 minutes and the rate is still increasing.

[FiveThrive]

We are having the same issue. From preliminary triage it looks like the carts/create and carts/update webhooks have had an extra node added to the webhook data.

[Levi3]

I've been unable to get help through shopify support. They could not confirm the issue, but escalated my concern and will be in touch in 1 to 3 business days... in the meantime we're deploying a work around to our servers.

Hopefully our past experience helps you out. The last time something like this happened, shopify was able to roll back a deploy causing the issue within a few hours. however, even after rolling back, the invalid requests continued for days because of their backoff/retry logic. So in that case, we had to deploy a fix on our end as well to drop the bad requests and return a 200 status code so they would stop retrying. They did not resend the data in the correct format.

Here's that thread in case it helps: https://ecommerce.shopify.com/c/shopify-apis-and-technology/t/product-webhook-sending-variants-as-a-...

[FiveThrive]

Thanks for the update Levi, I've had no luck other than getting one response on Twitter. I've deployed a workaround as well that is backwards compatible if they do roll it back.

[Mike_Arsenault_]

Hello everyone! We were able to roll back the change that broke the WebHooks integration, that *just* happened. Can you confirm things are looking normal for you again?

[Levi3]

I can confirm new cart events are normal again for us. Thanks.