Duplicate type definition with GraphQL and Sidekiq

ellitt
Excursionist
18 0 10

This error has been plaguing us for months and we've tried everything we can think of to mitigate it but nothing has worked so far. This morning it happened 50 times in a row, each one requiring an app reboot because it's a runtime error.

Several times a day, we'll get an error that looks like this: `Duplicate type definition found for name 'CustomerVisit' at 'CustomerVisit' (#<Class:0x0000558a99049498>, #<Class:0x0000558a99049498>)`. The duplicate definition is random and just depends on which definition happened to get defined twice when the app boots.

It seems to always happen in the background workers that are triggered by Shopify webhooks that we get many thousands of times an hour. Those webhooks trigger workers that are running via Sidekiq.

The GraphQL ruby gem has an issue related to this here: rmosolgo/graphql-ruby#1505.

And here's another possibly related issue regarding memory bloat in Sidekiq, which is another problem we run into regularly, when using graphql-client and graphql ruby's older legacy APIs: rmosolgo/graphql-ruby#2883.

We're at a loss for what to do here. The GraphQL API is obviously out of our hands so we can't go make the suggested changes in those above issues to try and stop this problem. And so far, all mitigation strategies we've tried haven't worked.

Using GraphQL is a necessity so we can't rely on the REST API, and Sidekiq is also a necessity so that the thousands of webhooks we see every hour aren't spawning workers in the web process and blocking other network requests.

Has anyone seen anything like this before? Or have any idea what might help?

Replies 4 (4)

MarkoRakic
Tourist
5 0 9

Hi, @ellitt were you able to fix this issue?

I am having the same issue and I am weird since it doesn't work when retrying a Sidekiq worker even it worked before.

Looking forward to hearing from you.

Thank you.

NorioUmata
Shopify Partner
3 0 4

Hi @ellitt have you resolved this issue? Thank you!

ellitt
Excursionist
18 0 10

@NorioUmata @MarkoRakic We haven't really corrected the issue, at least not in any direct way. It hasn't happened nearly as much in the past couple of months, but I'm not really sure why to be honest.

 

David_Crowder1
Shopify Expert
7 0 0

Same problem popping up here:

Duplicate type definition found for name 'Location' at 'Field QueryRoot.location's return type' (#<Class:0x0000562851454c08>, #<Class:0x0000562851454c08>)

With
Sidekiq 6.0.6
graphql (1.10.6)
graphql-client (0.16.0)

The log and metrics suggest that it's a memory(bloat) related issue