Billing Attempt with no completion, error or next action

[Brian_S]

We've seen at least one BillingAttempt that did not get processed by Shopify and I assume there are others. I'm curious if it could be related to this reported incident because the dates line up: https://www.shopifystatus.com/incident-details/530

Here's the BillAttempt id: gid://shopify/SubscriptionBillingAttempt/10826809442

Any help would be appreciated.

[J-ROM]

Hey Brian,

I can confirm the billing attempt job never enqueued properly due to the production incident you linked.

I would suggest to retry billing the subscription contract with a different idempotency key. This particular billing attempt will never resolve, so there won't be any risk of double-charging the customer for that order.

Hope this helps.

Cheers!
J-ROM

[Brian_S]

Thanks @J-ROM - what is your guidance in situations like these? Should subscription apps have a reconciliation process X hours after the billing attempt was created and if there was no response from Shopify then try again?   Maybe 24 hours?   The obvious risk is that Shopify is actually going to bill but is behind or down and we retry and end up double billing the customer

[J-ROM · Accepted solution]

Hi Brian,

My guidance would be not to implement any automated reconciliation process.

Billing attempts that fail to enqueue are normally deleted from our system so there is no ambiguity whether you can retry without risking to double-charge a customer. This incident exposed an edge case in our code where the billing attempt wasn't deleted properly and we're going to address that. This is an exceptional, one-off situation, and not the normal for you to account for.

Hope this address your concerns.

Cheers,
J-ROM

[Brian_S]

Ok - thats very helpful.  

Though what I'm hearing is that it's possible that a BA fails to enqueue (ie attempt to bill), in which case we definitely do want to retry.  So maybe we should have an automatic reconciliation process that checks if the BA id from the response that we get from shopify when calling `subscriptionBillingAttemptCreate` even exists and if shopify returns nothing back, only then do we retry the attempt? 

[J-ROM]

You're right, let me clarify:

• if the billing attempt creation fails, you'll get an error back right away and the billing attempt will not exist in our DB. In this case you should definitively retry with the same idempotency key.
• if the billing attempt creation succeeds, you'll get an ID back. At this point, we guarantee the billing attempt will be processed in a timely manner, and you should not retry yet with a different idempotency key as this could lead to a double-charge like you mentioned.
• once the attempt is processed and the payment fails, then it is safe for you to retry another billing attempt with a different idempotency key.

Again, this bug broke our promised behaviour, and we are addressing it so it doesn't happen in the future. 

Hope this is clearer.

Cheers,
J-ROM