Bulk operation url

Solved
thantine
New Member
2 0 1

I'm new on shopify app.

I've performed a bulk operation and got back the jsonl file url from storage.googleapis. How can I download and parse data from the url programmatically?

Thanks

0 Likes
HunkyBill
Shopify Expert
4490 45 485

This is an accepted solution.

You are given an URL. You can use any scripting language to open the file and parse it. As an example, in Ruby, it is this simple:

 

require 'open-uri'     # a library that allows you to open URL

download_url = "https://thecrazylongjsonurlwhereshopifyputallyourdata"
open(download_url) do |line|
    data_line = JSON.parse(line)      # so a line is a string, and JSON is a string, so we parse that string, and get back JSON, so we can programmatically deal with our data.
end

So note that in Javascript, Python or any other common scripting language, it is exactly the same! No difference. I use this pattern all over the place to bulk download data with the GraphQL API, then process it, and finally act on it. Things like massive inventory updates, or product transfers, tons of applications!

 

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
thantine
New Member
2 0 1

@HunkyBill Thanks so much for your support

aryancampervan
New Member
4 0 0

Whenever I try to access the url, I get a "Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object." error. Do you have any idea why this would be happening? When I paste the url into my browser, I can download the file fine.

0 Likes
HunkyBill
Shopify Expert
4490 45 485

If you want help, I can suggest a couple of things.

1. Tell the community what scripting language you are using

2. Show off some code. A few lines are all that is needed.

With that, you'll probably expose what you are doing wrong to someone in the community that recognizes what you are trying to do, and maybe they can help you. As you can tell, the URL is fine, and you get data. Now all you need to figure out is how to programmatically download an URL.

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
0 Likes
aryancampervan
New Member
4 0 0

I created a separate post regarding this here: https://community.shopify.com/c/Shopify-APIs-SDKs/Shopify-Bulk-Operations-Issue/td-p/1134633 and will update it with the info you asked for. Please take a look at it there! Thanks!

0 Likes