GraphQL Bulk Operations randomly cancelling

_JB
Shopify Staff
Shopify Staff
750 86 163

Hey @Louise_Elmose_H,

Sorry for the radio silence here. I've checked in and can assure our developers are aware of the issue, and have prioritized it accordingly. While I can't offer a timeline on a fix, I've done some more digging on the cancellations happening for your app to try and narrow down why it's happening for you so frequently. Most of the cancelled bulk operations occur on shops with a larger volume of orders, and when the fulfillment_status: unfulfilled filter is used. Based on this, I can suggest 2 potential workarounds that can ensure your bulk ops queries complete while our team works to fix this on our end.

The first thing you can try isn't guaranteed to work, but will be a quick fix if it does. You can try adding an additional argument to the query for status:open, with the idea being this will limits the dataset the system has to iterate over, and could make the probability of a timeout much less likely. 

Otherwise, a more involved step that will definitely work is to remove the fulfillment_status: unfulfilled argument from your query. This will require you to iterate through orders in the response and pull out the unfulfilled ones, but this query won't be susceptible to the unexpected cancellations as the query containing the argument.

If neither of these options work for you, I'd be happy to discuss the functionality of your app further, and help potentially come up with a different method for retrieving this data. Let me know if you'd like to do that, since this issue appears to be mostly affecting just your app we can move the convo to DM.

JB | Developer Support @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit Shopify.dev or the Shopify Web Design and Development Blog

Louise_Elmose_H
Shopify Partner
65 2 7

Hi @_JB ,

Thank you for your answer.

I have now implemented the workaround which does not use the query for unfulfilled orders, and it seems to fix my problems - thanks a lot!!

Is the issue for the order status query alone, or does the cancel operation happen for all types of queries?

 

I would still like to be updated when the bug is fixed though, is it possible to be added to a notification list for this or similar?

 

Thanks,

-Louise

0 Likes
_JB
Shopify Staff
Shopify Staff
750 86 163

Hi @Louise_Elmose_H,

From what I can tell, that fulfillment_status filter was responsible for almost every unexpected bulkOp cancellation I was able to find, so you shouldn't see any cancellations if you're no longer using the filter. I'll post here once we have an update regarding the fix. 

JB | Developer Support @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit Shopify.dev or the Shopify Web Design and Development Blog

0 Likes
Kris_Doyle
Shopify Partner
87 0 19

Hi @_JB ,

I've been having a similar problem with Bulk Operations for one particular client. I believe it's due to the number of SKUs per product (they have almost 100 variants per product and 6000+ products in the collection).

The last time I ran it, I received the CANCELED status. I'm currently running it now, and it's still running after almost 2 hours. I suspect it's going to fail or be canceled again.

The current operation ID is gid://shopify/BulkOperation/199671447623. Based on the objectCount and rootObjectCount (which haven't changed in over an hour), it looks like it should be complete, so I'm not sure why it's still running.

Any thoughts on what might be happening? Thanks very much for your help!

Quick Update - The job mentioned above did ultimately come back with "CANCELED" after running for about 2.5 hours.

0 Likes
Kris_Doyle
Shopify Partner
87 0 19

Hi @_JB ,

I've unfortunately run into a new problem on top of the previous one. I re-ran one of the earlier jobs that was canceled and this time it completed, but was missing 2000+ products. Not sure what's going on but something's definitely not right.

The ID of the export that completed but was missing 2000+ products is:
gid://shopify/BulkOperation/199851409479

My client is quite frustrated at the moment so if there's any way you can help, I'd greatly appreciate it

Thanks!
Kris

0 Likes
Louise_Elmose_H
Shopify Partner
65 2 7

Hi @Kris_Doyle,

Maybe you can show your GraphQL query?

Thanks, 

-Louise

0 Likes
Kris_Doyle
Shopify Partner
87 0 19

Hi @Louise_Elmose_H ,

The query is:

 

 

mutation {
	bulkOperationRunQuery(query:
	"""
	{
	  collection(id: "Z2lkOi8vc2hvcGlmeS9Db2xsZWN0aW9uLzk0ODAyNTc1NDIy") {
		products {
		  edges {
			node {
			  id
			  legacyResourceId
			  publishedAt
			  vendor
			  title
			  handle
			  productType
			  variants {
				edges {
				  node {
					id
					sku
					inventoryQuantity
				  }
				}
			  }
			}
		  }
		}
	  }
	}
	""")
	{
		bulkOperation{id,status}userErrors{field,message}
	}
}

 

 

 The odd thing is that when I run it, sometimes it cancels (with CANCELED status), sometimes it returns fewer products than the collection has, and sometimes it completes, but returns a different number of products each time (typically a few thousand or more less than the collection has).

It has succeeded in the past though; this seems to be a recent issue. I believe it's due to the number of SKUs in each product (between 50 and 100 per product and there are 6000+ products). This is creating a situation where there are over 500,000 (total objectCount) rows being exported. Sometimes the rootObjectCount has the correct number of products, and then the objectCount doesn't change for 2 hours. It's almost like it's getting the products, and then hanging when trying to fetch the variants. I suspect the CANCELED status is due to a timeout.

In any case, it's definitely not working normally.

Thanks,
Kris

 

0 Likes
Louise_Elmose_H
Shopify Partner
65 2 7

Hi @Kris_Doyle ,

 

My CANCELED issue was fixed by removing my query filter in the graphql and filtering programmatically instead.

I cannot tell from JB's answer whether the bulk job CANCELED problem is query filtering in general, or only the fulfillment_status filter I was using.

But I think it's worth trying out to remove your filter in the graphql query, and then programmatically filter out all products that does not match your collection id.

 

-Louise

0 Likes
Kris_Doyle
Shopify Partner
87 0 19

Thanks @Louise_Elmose_H, by filter criteria, do you mean the collection ID?

In our case, the purpose of this query is to get the products for the collection in sorted order. If we didn't use the collection ID, the query wouldn't have any value since the products wouldn't be in order, and we'd have to go through potentially many more thousands of products to find the collection members (which wouldn't be in order anyways).

Unless I'm misunderstanding your suggestion?

@_JB would it be possible to check those IDs I mentioned? My first query used to work so I'm not sure what's changed. I've tried both the 2020-04 and 2020-10 versions of the API with the same result. The more troubling issue is where the query returns random numbers of products (with COMPLETED status), but not the actual number in the collection. I'll try it again today to see if anything's changed.

Update: We just ran the original query again this morning with odd results. The first time, it completed with 1000 products. The second time it completed with 3800 products. The collection actually has 6000 products. Something's definitely wonky here. Not sure what could be causing it to fail after random numbers of products.

I'm pretty sure it's the variants that are causing it to fail. Unfortunately I need those because of the SKUs.

Thanks very much
Kris

0 Likes
_JB
Shopify Staff
Shopify Staff
750 86 163

Hey @Kris_Doyle,

Thanks for providing those details. I've just provided all that to our developers who are investigating now, I'll post back here once we have a fix in place.

JB | Developer Support @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit Shopify.dev or the Shopify Web Design and Development Blog

0 Likes