REST Api - InventoryLevel search by updated_at_min order?

MozzoERP
Shopify Partner
83 4 20

In the event a store has thousands of InventoryLevel, we'd like to use paging and break up importing them across multiple requests spread out over time...i.e. get 1,000 records every 5 min.

For other data elements, like Products, we do this by making requests like: 

updated_at_min=2019-08-26T22:57:51.000Z&order=updated_at asc&limit=1000

We save the UpdatedAt time for the last record in our database, then when the job runs the next time, it uses this new UpdatedAt time to get the next batch of 1000. 

It seems InventoryLevel is sorted by Id and you cannot override that to updated_at. As such, we could apply the same approach as above but use the Id instead of UpdatedAt. But there is no documented way to search InventoryLevel by >= Id, i.e. Id_min.

Any suggestions?

Chad Richardson
Mozzo Software - Modular Software that grows with you from solopreneur to a 200 person mega team. Why keep outgrowing your Shopify Apps? Start with us, and just use the modules you need, then add more as you grow. http://MozzoERP.com
Replies 7 (7)

MozzoERP
Shopify Partner
83 4 20

Bump

Chad Richardson
Mozzo Software - Modular Software that grows with you from solopreneur to a 200 person mega team. Why keep outgrowing your Shopify Apps? Start with us, and just use the modules you need, then add more as you grow. http://MozzoERP.com

ryanlunka
Shopify Partner
6 0 4

It looks like there's a parameter to get InventoryLevel by updated_at_min. Does that get you where you need?

https://shopify.dev/docs/admin-api/rest/reference/inventory/inventorylevel#index-2020-10

As a general best practice though, trying to query by timestamps for large amounts of data like this is pretty inefficient. InventoryLevels will update A LOT, especially on a high volume store.

If you are using the REST API, I'd try to leverage webhooks. That will require your side of the integration to process events efficiently. I'd also look at the GraphQL API which has some interesting new ways (often not really well documented) you can process large queries like that. I'm only semi-technical, so I couldn't go much deeper into those weeds, personally.

This is all with limited context for what you're trying to do, but I can make some guesses. I hope it's helpful!

MozzoERP
Shopify Partner
83 4 20

@ryanlunka thanks, but I am aware that it has the updated_at_min filter, but the issue is the sorting of the results in a predictable way such that we can pick up where we left off on subsequent requests.

For a simplistic representation, let's say we limit our page size to 3 records and get the following results.

Inventory IdUpdatedAtQty
32020-12-01 05:00:056
12020-12-01 05:01:262
72020-12-01 05:02:363
   

 

We'd then store the LAST UpdatedAt from the first batch, 2020-12-01 05:02:36, then on our next request, use that datetime as the updated_at_min parameter to get everything changed since then. Then repeat until all rows are fetched.

Note this is for a one time inventory snapshot pull to populate our ERP with current inventory levels. Then we'll control inventory from there and will be doing pushes back to Shopify to update after that.

Chad Richardson
Mozzo Software - Modular Software that grows with you from solopreneur to a 200 person mega team. Why keep outgrowing your Shopify Apps? Start with us, and just use the modules you need, then add more as you grow. http://MozzoERP.com

robbwinkle
Tourist
5 0 3

I would suggest using the GraphQL Bulk Operation to get all the results in one query. It will allow you to eliminate the 1000 record limit and get around any leaky bucket API limitations.  You can then download the JSONL file and read it in reverse one line at a time to process each result.

The documentation can be found here https://shopify.dev/tutorials/perform-bulk-operations-with-admin-api.

Here is an example query to get InventoryItems updated after a given date/time:

query {
  inventoryItems(query:"updated_at:>2020-10-29T14:25:35Z", first:10) {
    edges {
      node {
        id
        sku
        updatedAt
      }
    }
  }
}

 

Note: the above query limits you to the first 10 results, but would be ignored by the bulk operation giving you all of the updated InventoryItems.  Documentation for what is available on the InventoryItem object is here https://shopify.dev/docs/admin-api/graphql/reference/inventory/inventoryitem.  The query syntax for the GraphQL API is here https://shopify.dev/concepts/about-apis/search-syntax

MozzoERP
Shopify Partner
83 4 20

Thanks @robbwinkle 

Our app currently has the infrastructure for REST only. Adding Graph, though in our roadmap, is a larger task than we'd like to perform for this as it is a one time event that occurs when we install our app to get a current snapshot of the inventory. After that, our app controls the inventory details and posts those updates to Shopify.

Even with Graph, I don't see a way to sort inventory... guessing that is why you suggest the bulk operation so you don't need to worry about sorting/paging?

Chad Richardson
Mozzo Software - Modular Software that grows with you from solopreneur to a 200 person mega team. Why keep outgrowing your Shopify Apps? Start with us, and just use the modules you need, then add more as you grow. http://MozzoERP.com
robbwinkle
Tourist
5 0 3

@MozzoERP Yes, that is correct. I was implying getting all the results so you could sort them yourself.  There aren't the same sorting options for inventory as there are for orders.

Gregarican
Shopify Partner
1033 86 283

@robbwinkle agreed on the idea of using GQL's bulk operation to just pull larger recordsets all in one shot. If that's not a viable solution due to an inherent requirement of solely utilizing the REST API, then larger recordsets require handling the cursor-based paginated results. And regardless of the API model, iterating the results to sort, slice, and dice however is needed.

For my own projects I find it better to receive too many records as opposed to too few. And iterating a result set to keep, to discard, and re-sort to my final set is just part of the process.