Duplicate type definition with GraphQL and Sidekiq

Excursionist
17 0 6

This error has been plaguing us for months and we've tried everything we can think of to mitigate it but nothing has worked so far. This morning it happened 50 times in a row, each one requiring an app reboot because it's a runtime error.

Several times a day, we'll get an error that looks like this: `Duplicate type definition found for name 'CustomerVisit' at 'CustomerVisit' (#<Class:0x0000558a99049498>, #<Class:0x0000558a99049498>)`. The duplicate definition is random and just depends on which definition happened to get defined twice when the app boots.

It seems to always happen in the background workers that are triggered by Shopify webhooks that we get many thousands of times an hour. Those webhooks trigger workers that are running via Sidekiq.

The GraphQL ruby gem has an issue related to this here: rmosolgo/graphql-ruby#1505.

And here's another possibly related issue regarding memory bloat in Sidekiq, which is another problem we run into regularly, when using graphql-client and graphql ruby's older legacy APIs: rmosolgo/graphql-ruby#2883.

We're at a loss for what to do here. The GraphQL API is obviously out of our hands so we can't go make the suggested changes in those above issues to try and stop this problem. And so far, all mitigation strategies we've tried haven't worked.

Using GraphQL is a necessity so we can't rely on the REST API, and Sidekiq is also a necessity so that the thousands of webhooks we see every hour aren't spawning workers in the web process and blocking other network requests.

Has anyone seen anything like this before? Or have any idea what might help?

1 Like