[Bug] App Subscription "price exceeds balance remaining" validation not allowing Subscription Update

dylanpierce
Shopify Partner
210 3 94

Here's a fun bug with the App Usage API.

There's a race condition between AppUsageCharge recordings & the AppSubscription validation.

For example, I had a merchant that had a usage cap of $1,000 USD. However, the AppUsageCharges must charge before checking the capped amount for the billing cycle.

This means my app was able to charge more than what the usage cap allowed.

Here's a snapshot of the merchant's subscription status at the time of this bug occurring. You'll see the balance exceeds the monthly usage cap, which shouldn't be possible:

 

{
    "data": {
        "node": {
            "id": "gid://shopify/AppSubscription/22148513870",
            "createdAt": "2021-10-14T15:46:59Z",
            "name": "A.I. powered ID checks at $1.25 per check",
            "status": "ACTIVE",
            "test": false,
            "lineItems": [
                {
                    "id": "gid://shopify/AppSubscriptionLineItem/22148513870?v=1&index=0",
                    "plan": {
                        "pricingDetails": {
                            "balanceUsed": {
                                "amount": "1015.75"
                            },
                            "cappedAmount": {
                                "amount": "1000.0"
                            },
                            "interval": "EVERY_30_DAYS",
                            "terms": "$1.25 per ID check, up to 800 ID checks per month."
                        }
                    }
                }
            ]
        }
    },

 



Whoops, so as an app developer I reach out to the merchant that we need to increase their monthly cap so they can continue to use the app.

However, the App Subscription update API rejects my request to update the Subscription monthly cap amount because the current balance exceeds the usage cap.

This means I can't create an update mutation to only change the usage cap:

 

{
    "data": {
        "appSubscriptionLineItemUpdate": {
            "appSubscription": null,
            "confirmationUrl": null,
            "userErrors": [
                {
                    "field": null,
                    "message": "Validation failed: Capped amount cannot be lower than balance used",
                    "__typename": "UserError"
                }
            ],
            "__typename": "AppSubscriptionLineItemUpdatePayload"
        }
    },
    "extensions": {
        "cost": {
            "requestedQueryCost": 10,
            "actualQueryCost": 10,
            "throttleStatus": {
                "maximumAvailable": 2000,
                "currentlyAvailable": 1990,
                "restoreRate": 100
            }
        }
    }
}

 

 

So thanks to a race condition bug introduced when applying charges, the account is now in a state where I can't update it's usage billing without cancelling and making a brand new subscription.

I ended up having to make a brand new subscription for this merchant.

Could we have the validations on usage balances during appSubscriptionLineItems looked at? It's not possible to recover from this bug as a developer currently.

Founder of Real ID - Verify your customer's real IDs easily & securely with modern A.I.

Want to see it in action? Check out our demo store.

Replies 7 (7)

Luke_K
Shopify Staff
402 66 98

Hey Dylan,

 

Thanks for raising this! By chance do you have the request ID from these call(s) / the store name? Feel free to shoot me the details in a DM if it's easier. Thanks!

| Shopify |
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution!
dylanpierce
Shopify Partner
210 3 94

Hi @Luke_K 

Thanks for the offer, but I'd rather keep the conversation in public in case another Partner has this issue too and other Shopify support staff and reference it to the engineers.

The old subscription ID is included in the code I shared, which you can infer the shop name from. Here's the newest subscription ID: `gid://shopify/AppSubscription/22119317569`

Here's the scenario you can use to recreate the bug:

As a merchant
Given I have an active AppUsage subscription with an app
Given my balance on the AppUsage subscription exceeds the monthly cap on the AppUsageSubscription
I expect a partner to be able to run an updateSubscriptionLineItem mutation to upgrade my monthly cap
BUT instead the mutation returns a validation error "Balance exceeds monthly cap" and is unable to create a confirmation link to update the monthly cap

The real root of this bug is a race condition between AppUsageCharges and checking the monthly cap. There must be a small window of time when the monthly cap isn't considered when creating a new AppUsageCharge.

A balance shouldn't be able to exceed the monthly cap anyway.

I hope these details help.

Founder of Real ID - Verify your customer's real IDs easily & securely with modern A.I.

Want to see it in action? Check out our demo store.

Luke_K
Shopify Staff
402 66 98

Thanks @dylanpierce  for the extra information, I've escalated this for further investigation.

 

Generally we will ask for at least an x-request-id if you are facing unexpected API functionality. The x-request-id shared publically, contains no store or app specific info, and helps us locate any unique logs around errors. 

 

Searching internally with unique resource IDs and trying find out what happened specifically at runtime can be a much more involved process due to many factors (data, log retention, etc). Additionally, we may run into limits based on authentication, which in turn limits what we can pass back to partners in the Community Forums, especially if we require explicit authenticated permission to access to an app or store specifically.

 

While our inability to authenticate here on the forums isn't ideal, we can work around this using an x-request-id's (if you have them). It's also worth mentioning at this point, that we do have a Partner Bug Reporting Form in Beta - once you are authenticated it can be accessed at https://help.shopify.com/en/questions/partners#/contact - it sidesteps the above limitation of the forums. 

 

Thanks!

 

| Shopify |
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution!
dylanpierce
Shopify Partner
210 3 94

Thanks for escalating @Luke_K 

I wish there was such thing as unlimited storage. But the scenario I left will let a developer recreate the problem. 

I wasn't aware of a special header that was being included. Is there a tutorial or blog post that instructs partners on how to find this header and best practices on storing it so we can reference problem API requests easier?


Founder of Real ID - Verify your customer's real IDs easily & securely with modern A.I.

Want to see it in action? Check out our demo store.

Luke_K
Shopify Staff
402 66 98

Hey @dylanpierce 

 

There actually isn’t an official guide that I'm aware of, more’s the pity. Noted, I can understand it would be helpful and will make it known internally here. Yes, responses from calls made against the Shopify API include the x-request-id response field, and so our Technical support team(s) will use that for pinpointing what’s happening.

 

I note our libraries would handle the headers (say for example, Shopify-Node-API's http_client.ts would in the case of error return the x-request-id of the failing request). Thanks!

| Shopify |
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution!
dylanpierce
Shopify Partner
210 3 94

@Luke_K  a guide would be very helpful. I think I get where that is stored though, thanks for the pointer.


Founder of Real ID - Verify your customer's real IDs easily & securely with modern A.I.

Want to see it in action? Check out our demo store.

Jeff-Blake
Shopify Partner
43 0 9

I've been experiencing this bug for over a year, and it's been happening a lot more in the past month.