PriceRules - Syncing of large number of records

[james148]

PriceRules - Syncing of large number of records
Does anyone have any experience syncing large number of PriceRules to Shopify?

We are wishing to sync pricing records from an ERP solution to Shopify where the PriceRules are effectively the customer pricing records in the ERP - one PriceRule in Shopify to every customer/item in the ERP.

There are about 500k records.

Any ideas of how we keep these in Sync? I.e. Particularly handling deletions (i understand we can keep an intermediate change/diff table, but this is a pfaff and can get out of date).

[Liam]

Hi James,

Syncing large of data from an ERP to Shopify can indeed be a complex process. Here are some ideas on how to handle this:

1. Batching: Shopify's APIs have rate limits, so you'll want to batch your API calls. You can make use of Shopify's Bulk Operations API which is designed to handle large volumes of data.

2. Webhooks: To keep the data in sync, you can use webhooks to notify your system of any changes made to variant prices. This way, you can update the data in your ERP system in real time as changes occur in Shopify.

3. Scheduled Sync: Depending on how frequently your data changes, you could also consider scheduling regular syncs between your ERP system and Shopify. This could be done daily, hourly, or even more frequently depending on needs.

4. Deletions: Handling deletions can be tricky. One approach is to mark deleted records in your ERP system and then reflect these deletions in Shopify during your next sync. 

5. Third-Party Integration Tools: There are some third-party apps available that can help you integrate your ERP system with Shopify. These apps can handle the syncing process for you, and many of them are designed to handle large volumes of data.

Hope these suggestions help!

[james148]

Liam, thanks.

1. I wasn't aware of the bulk operations API - this will be useful.
2. Webhooks - B2B Integrations need to be driven from ERP solution - so not useful in this context.
3.  & 5. We are the Third-Party Integration tool.
4. It would be really useful, if Shopify introduces a queryable 'external/foreign id' property for some/all records. This would allow records to be tagged with a (unique) value from the source/originating system (such as an ERP). Instead of the API needing Shopify IDs (which can be tricky to manage in a source solution), you pass the external/foreign id instead, and Shopify internally translates it to the Shopify ID. Salesforce employs a similar approach - and it's extremely useful in scenarios where there are large number of records, records need to be Synced TO SF and there's usually no natural area in the source system to record the ID of the target record (Shopify or SF).