I thought OAuth + billing would be easy :-) then I started mapping the states

Shopify Apps
,
1

Hi all,

I wanted to share one thing I underestimated while building a Shopify app.

At the beginning I thought the install flow would be quite simple:

install → OAuth → billing approval → initial sync → ready

At least on paper it looked like this :slight_smile:

My setup is not very exotic: embedded UI, API, worker and a database.

Because the app stores Shopify data locally, I also need an initial sync after installation, and there are background jobs running in the worker.

This is the point where it became more complex than I expected.

Very quickly I noticed that I do not really have one app state.

I actually have several state dimensions at the same time, for example installation, billing, data readiness, sync and later also uninstall/reinstall and recovery.

A shop can be installed, but billing is not active yet.

Or billing is active, but the initial sync is still running.

Or the app is installed and billing is fine, but the local data is only partial because a sync failed.

Then there are billing states like cancelled at period end, expired, frozen or declined.

And uninstall made the whole thing even more interesting.

The app can already be uninstalled while local data is still waiting for cleanup or deletion.

And then I started thinking about reinstall.

What happens if the merchant installs again while something from the previous installation is still being cleaned up?

What happens to background work which belongs to the old installation?

Can I still trust the existing local data, do I need a new sync, or do I first need a new authorization?

At some point I stopped thinking about one single installed = yes/no state and started creating a state matrix instead.

Very simplified, I now have separate areas like:

  • Installation: installed / uninstalled

  • Billing: none / active / cancelled at period end / expired / frozen / declined

  • Data: partial / active / pending redact / deleted

  • Initial sync: pending / running / success / failed

  • Reinstall / recovery: verification pending / OAuth required / reauth required / support required, etc.

The interesting part is not really the individual states.

It is their combination.

For example:

installed + billing active + data partial + initial sync running

is a completely different situation than:

installed + billing active + data active + sync successful

even though both shops are technically “installed”.

Because UI, API and worker run separately, I persist the relevant state in the database and let every part decide from this state what it is currently allowed to do.

Another lesson for me was not to build the logic around the assumption that every webhook or event will always arrive in exactly the order I would prefer.

So the diagram below is intentionally not a complete state machine. It is more a picture of the different state dimensions which can exist at the same time.

What surprised me most is how much lifecycle logic came out of something that initially sounded like a pretty small feature.

I would be really interested how this developed in other apps.

Did you also end up with much more lifecycle state than you expected at the beginning?

What approach worked well for you to keep all these moving parts manageable?

And is there any edge case you only discovered later and wish you had thought about earlier? :slight_smile:

Would be interesting to hear your experiences.

states
states1619×971 261 KB

2

The dimension I would add to that matrix is scopes. It bit me on the second version of my app. A shop can be installed, billing active, sync healthy, and the stored token still does not carry the scope you added last week. Nothing in the install state or the billing state tells you. It surfaces as a 403 on one specific API call, usually inside a background job at 3am, and the merchant just sees a feature that quietly does nothing. I now store the granted scopes at auth time and compare them against the required set on every app load, then send the shop through reauth when they differ.

The other thing that helped more than the matrix itself was demoting webhooks. I stopped treating them as state transitions and started treating them as “something changed, go re-read the source”. For billing that means querying currentAppInstallation activeSubscriptions when the merchant opens the app instead of trusting whatever the last app subscription webhook wrote into my database. Out of order delivery mostly stops mattering once the payload is only a hint rather than the truth.

Reinstall got simpler for me after I realised the old token just dies. A new install issues a new token and any call with the old one comes back 401. So “reauth required” is not something I have to predict from webhook ordering, it is something I discover on the next API call and recover from. Also worth knowing that Shopify holds shop/redact for about 48 hours after an uninstall and skips it entirely if the shop reinstalls inside that window, so a pending redact state can end up waiting on an event that is never going to arrive.

How are you handling the initial sync failing halfway through on a big shop? That is the one I still do not have a clean answer for. Partial data plus billing active plus a worker that already gave up is the combination that generates my support tickets.

3

Uninstall was the one I got wrong. My webhook handler authenticated requests the normal way, which needs a valid token for that shop, and a shop that just uninstalled does not have one, so app/uninstalled returned 500 on exactly the shops it was written for. I verify the HMAC by hand now and never touch a session inside a webhook.

The other thing worth deciding early is which state you own and which one you are only caching from Shopify. I kept the plan on the shop row and refreshed it from Shopify on every load, so anything I wrote there by hand quietly disappeared on the next page view

4

That scope point is a very good one. I store the granted scopes already, but I have to check if I really treat a difference between required and granted scopes as its own readiness condition everywhere. I did not include this in my matrix and I think you are right, it belongs there.

I also like your description of webhooks as “something changed, go re-read the source”. For billing I ended up quite close to this. I persist the webhook state, but I also have a reconcile path against Shopify because I did not want the local billing state to depend only on the last webhook which arrived.

On reinstall I went a little more defensive because of the worker. I have background jobs which may still belong to the previous installation, so I tag an installation generation and don’t want an old job to become valid again only because the same shop was installed again. For a simpler app I agree that discovering the dead token via 401 may be enough, but with async jobs I got nervous about this :slight_smile:

Regarding the initial sync: this was also one of my painful combinations. I keep data readiness separate from billing and from the sync job itself. So if a sync dies halfway through, billing can still be active but the data remains partial and the app is not considered fully ready. The failed sync can be retried, but I don’t promote the existing partial dataset to normal operation just because some entities were already written.

What I am still thinking about is how far to go with resumable syncs for really large shops versus just restarting a failed section. Your support-ticket example is a good argument for making this more explicit.

5

The thing that collapsed most of that matrix for us was making the initial sync resumable instead of stateful. A cursor per resource, written after every page, means partial is never a state you have to classify, the data is just behind, and a worker that died at 3am is a job you run again rather than a shop in a special condition.

The edge case we found late was on the other side: Shopify retries a failing webhook endpoint for about two days and then removes the subscription, so a shop can look installed, billed and healthy in your database while receiving nothing at all.

We run a cheap periodic reconciliation against the Admin API now, mostly so that gap gets discovered by us and not by the merchant. Agreed on demoting webhooks to a signal to go and read the source, that one change removed most of our ordering bugs too.

6

That distinction between state you own and state you only cache from Shopify is a really good way to describe it.

I ended up with a very similar rule, just with different wording: for things like billing and authorization I treat Shopify as the single source of truth and only persist the state locally because UI, API and worker need a stable value to work with.

Other states are really owned by the app itself. Initial sync, local data readiness, installation generation or recovery state are things Shopify cannot tell me, so those have to stay authoritative in my own database.

Your uninstall example is a good reminder too. The webhook itself has to be authenticated independently via Shopify’s HMAC and must not depend on a shop token/session that may already be invalid after uninstall.

I like the “who owns this state?” wording though. That is probably a cleaner way to think about the matrix than just looking at the state values themselves.

7

The resumable-sync distinction is interesting. I currently keep data readiness separate from the sync job, because I still need UI/API to know whether the local dataset can be trusted as complete. But I can see the advantage of combining that with per-resource checkpoints, so a failed worker continues instead of restarting a large sync.

Your webhook-subscription example is a nasty one too. I ended up adding a remote verification/reconcile path for business webhooks for a similar reason: local state alone cannot prove that Shopify is still actually delivering them.

Interesting that this comes back to the same pattern again — local operational state is useful, but for Shopify-owned state you still need some way to reconcile against the source.

8

the cached copy is where it bit us. two code paths read it differently: the UI showed the computed plan, the send path read the raw cached value, so a merchant looked upgraded but was still capped. since then every cached state has exactly one accessor and nothing else touches the raw field.

9

One state nobody mentions until it burns them: the error screen itself. If your failure page is rendered with the same embedded stack that just failed - App Bridge, your API, your session - the merchant sees a blank iframe exactly when you most need to tell them something. We moved ours to plain static HTML with zero dependencies after watching polished error components turn to mush the moment App Bridge wasn’t there.

And on billing: if two things on your first screen can both trigger a plan lookup, make one of them the owner. We had two loaders racng on the home screen, and the paid plan handle occasionally lost the race to the free default - merchant pays, sees Free. Found it only because a human noticed the wrong plan name on a receipt.

10

That is a good distinction. I was thinking mostly about “who owns the truth”, but your example adds another rule on top of it: even if the ownership is clear, every consumer still needs to resolve the cached state in exactly the same way.

I already treat Shopify as the source of truth for billing and keep a local operational copy, but I think the stronger rule is that UI, API and worker should never interpret that raw cached value independently. They should all go through one canonical resolver/accessor.

Otherwise you can have one source of truth and still end up with two different truths inside the application :slight_smile:

I am adding that one to my notes.

11

The static error page is a really good point. I had been treating failures mostly as application states, but there is clearly another category: failures where the normal UI stack itself cannot be trusted anymore.

A recovery screen which depends on App Bridge, a valid session and the API is not much help if one of exactly those dependencies caused the failure. I think a dependency-minimal fallback for bootstrap/auth failures is a good design rule.

Your billing race also connects nicely to another comment here about cached state. I am starting to think that “Shopify is the source of truth” is only the first rule. You also need exactly one path which is allowed to reconcile/write that truth locally, and one canonical way for the rest of the app to read it.

Otherwise even one external source of truth can become several internal versions of the truth :slight_smile:

Both points are going into my notes. Thanks!