Private uploading in GraphQL Bulk operation ?

[AnthonyAcceo]

Hi,

I'm currently working on updating/creating clients with the GraphQL API. My process follows the following Shopify documentation: https://shopify.dev/docs/api/usage/bulk-operations/queries 

So I obtain the useful information for uploading with the mutation: "stagedUploadsCreate"
Subsequently, I upload my "client.jsonl" file to the URL returned by the previous query.
This request returns me an URL where my file is stored. (This file is public to anyone having access to this url)
This URL can then be used by the "bulkOperationRunMutation" mutation to create my clients.

The problem with this process is that the URL is exposed to the public until I delete it.
Recently, in my province, a strict law was passed regarding the regulation of customers' personal data. We have confirmed with our attorney and my current process would not meet the requirements as my client's client data are exposed to the public on a server.

I would like to know if it is possible to make this file private or only readable by Shopify. If so, how can I do it?

Thanks a lot !

[Liam]

Hi Anthony,

The stagedUploadsCreate mutation is not designed for uploading any files that contain customer or client data as it's designed to always provide a publicly accessible URL for file uploads, so I'd strongly recommend to immediately stop using that approach. You could consider using an app with Admin API access to fetch customer data directly when you need. This method allows you to work with customer data directly without needing to upload it via a public URL.

Another option could be that before uploading client.jsonl, you encrypt the data. Only your application (which has the decryption key) and Shopify (after you process the data through a bulk operation) can then read it. This method ensures that even if someone accesses the URL, the data will be unintelligible without the decryption key. Before following this approach though, you should confirm with your legal expert that it would comply with your local data privacy laws. 

Hope this helps, 

[AnthonyAcceo]

Hi Liam, 

Thanks for the clear answer. I am aware of the risks associated with providing a publicly accessible URL containing customer data. I have consulted with our legal expert, and the solution you are proposing appears to be perfect for my needs. I would like to understand how this works. Is there any documentation available on this topic?

[DavidT]

I really don't understand. In my experience, the URL looks like this:

https://shopify-staged-uploads.storage.googleapis.com/tmp/{SOME_ID}/bulk/{RANDOM-UUID}/{YOUR_FILE_NAME}.jsonl

If your file links look different, please share an example.

---

This file is public to anyone having access to this url

---

Yes, but how are they going to get access to the URL? They would have to guess the random UID, as well as the filename and other ID. It's simply not possible. It's like saying "the data is public to anyone with the password". If you don't trust that the random ID is unguessable, just make the filename a secure randomly generated name. It's literally the same as having a password/secret key at that point.

But I also don't see why they can't just change it to be like the bulk operation results file, where you have a signature and an expiration date. I wasn't aware the files were never deleted.