How can I effectively set up ETL for data analytics in an online store?

Shopify Apps
1

Hello!

This is my first message here; I am a beginner data analyst (retraining from a biology background!) and I started an internship at a Shopify shop in which I am asked to do a job that is more of a data engineer one:

TASKS

  • Set up data analytics warehouse → data pipeline
  • ETL (Extract-Transform-Load) from the Shopify data software tools (API) to xls/csv (Excel)
  • Track KPIs (defined thereafter) monthly then weekly (and daily?) → schedule jobs
  • Create dataviz dashboards & reports using Google Data Studio
    (+ CRM Customer Relation Management?)

KPIs

  • mailing list subscribers

  • website visits

  • orders

  • returns

  • customers placing 2+ orders

  • orders per brand → trends

  • profitability = order value €
  • products on wishlists (desired/populars)

I suppose I need to connect to the Shopify Admin API then do the ETL process, but I never did that and I am self-taught so I feel a bit lost.
I already asked the shop to create a private app and got its credentials, but is it the right option or should I create a custom app instead?
I don’t have direct access to the shop, and I have a partner account, not a staff one…
How do I proceed from there? Where can I find the relevant information?
I found some documentation but this is a bit overwhelming, there is a lot, it is quite technical and I am not sure which option applies to my case:

https://github.com/Shopify/shopify_python_api/blob/master/README.md
I am stuck at step 4, probably because I’ve not yet figured out what to put as ‘request_params’:

NameError Traceback (most recent call last)>

https://help.shopify.com/en/manual/apps/private-apps
https://shopify.dev/tutorials/build-a-shopify-app-with-node-and-react/embed-your-app-in-shopify
https://shopify.dev/tutorials
https://community.shopify.com/post/1142391

2

Hi @ClaireBLondon

You are on the right track - a private app with the right read access rights is the right way to access the data, and a partner account should suffice.

We’ve worked with multiple clients to setup the ETL flow & transformations from Shopify to a sheet, you can email me at anto@opstack.co, happy to help.

thanks

Anto

3

Thank you Anto_S, I really appreciate!

4

That looks good but there also many vendors who can sync data for you if you didn’t get this working. https://gravitydata.co also allows you sync sub minute and it takes a few min to setup. There is a free account too

5

Hi @ClaireBLondon did you figure out what to put as ‘request_params’ ? Im also stuck at this point

6

For ETL operations, you can use the simple and convenient Skyvia service. It’s a no-code cloud solution