I'm a developer working on ingesting Shopify data via API to build ETL scripts for a data warehousing project of which Shopify is a key data source. I am iterating through all orders via the pagination page token. I've noticed that the Orders API is inconsistent in the fields that is returned in the response. Empty fields are fine but when fields are entirely missing, they break the ETL scripts as I am expecting those fields to be ingested. I've noticed this happening on several fields, some of which may be explained and I've opted to remove them from being ingested but others are entirely unexplainable and should be in the responses at all times.
I've noticed this issue so far with these fields:
For destination_location and payment_details, I could surmise reasons why they are not consistently returned but it is extremely frustrating to see origin_location and client_details to be missing in some responses as these are fields explicitly called out in the documentation that should be supported. If these fields are missing because there is no information regarding those fields for an order, that's fine for the fields to exist and be empty but I find it a very bad practice for the fields themselves to be missing altogether as it can have a number of consequences for ETL scripts. This puts the reliability of Shopify APIs to be doubtful. If anyone is aware of this bug, please let me know that it's being addressed.