Solved

GraphQL Bulk Operation mutation file size limit

cloudev0
Shopify Partner
7 0 6

Does the import api have a file size limit? I'm trying to upload a ~36mb file, and I keep getting broken pipe in ruby (Faraday) and curl almost instantly when I initiate the multipart form POST request. Any advice?

Thanks in advance.

Accepted Solution (1)

GrahamS
Shopify Staff (Retired)
193 36 54

This is an accepted solution.

Hey @cloudev0 

 

At the moment, the file size limit is 20 megabytes for bulk mutations. We're going to review the documentation to have this made more clear in the future.

 

Best,

To learn more visit the Shopify Help Center or the Community Blog.

View solution in original post

Replies 7 (7)

alex0000
Shopify Partner
10 0 2

I have the same problem, does somebody have an answer ? (seem to be S3 limitation)

 

<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>EntityTooLarge</Code>
    <Message>Your proposed upload exceeds the maximum allowed size</Message>
    <ProposedSize>20975508</ProposedSize>
    <MaxSizeAllowed>20971520</MaxSizeAllowed>
    <RequestId>N5FTYTNP6MPP1VB9</RequestId>
    <HostId>Nlu1xkUPTWKP345Uma79b2hQzFzqDJ3fBrZN7g1iM552PHuf0uif/gM4Cqg3UxXp7KVi+b5AIvI=</HostId>
</Error>
cloudev0
Shopify Partner
7 0 6

I ended up rotating files when writing the mutation data (think log rotation, with each file having 20mb max), then chain upload the files. Upload first file. Wait for Webhook. When Webhook comes, respond and upload the next file and so on.

Wilhelm_Uschtr1
Shopify Partner
47 0 3

@cloudev0 If I may ask, what's the average import speed/rate you get doing that?

cloudev0
Shopify Partner
7 0 6

Sorry for the late reply, I can't give a specific metric since it really depends on the kind of data. But for a specific example,
a chained mutation that manages to update price, cost, compare at price, description, tags, weight and 1 product level metafield of around 12k variants, that had the size of ~21.2MB (so 2 bulk operations), took around 11 minutes (started at 2024-05-06T13:06:40+03:00 and completed at2024-05-06T13:17:30+03:00 according to Shopify webhook)

Wilhelm_Uschtr1
Shopify Partner
47 0 3

Awesome, thanks for the late reply! 🙂

GrahamS
Shopify Staff (Retired)
193 36 54

This is an accepted solution.

Hey @cloudev0 

 

At the moment, the file size limit is 20 megabytes for bulk mutations. We're going to review the documentation to have this made more clear in the future.

 

Best,

To learn more visit the Shopify Help Center or the Community Blog.

DavidT
Shopify Partner
23 1 6

So how are we supposed to get around this problem without having to do multiple bulk mutations???

QuickEdit - Bulk Product Edit - Quick and easy bulk editor for products and variants.
SafeShip - Address Validator - International address validation and PO box blocking at checkout for Shopify Plus merchants.