Query with Bulk Operation not working

Solved

Query with Bulk Operation not working

softwaregravy
Shopify Partner
13 0 1

I am attempting to fetch data with createdAt or updatedAt properties in a certain window. Exactly like the example on the tutorial page for products: https://shopify.dev/tutorials/perform-bulk-operations-with-admin-api

 

mutation {
  bulkOperationRunQuery(
  query: """
  {
    orders(query: "created_at:>=2020-01-01 AND created_at:<2020-02-01") {
      edges {
        node {
          id
          createdAt
        }
      }
    }
  }
  """
  ) {
    bulkOperation {
      id
      status
    }
    userErrors {
      field
      message
    }
  }
}

I've tried lots of variations on the query string. createdAt, formatting the date in iso8601, querying with just 1 param instead of 2 (no AND)

 

The API does not give any errors, but is returning all the nodes.

 

Here's one example: gid://shopify/BulkOperation/6692732987

This returned the full set of nodes, but the query had just "created_at:<2020-01-01", but all the nodes were created in Feb of 2020.

 

Any idea what I'm doing wrong?

 

 

Accepted Solution (1)
hassain
Shopify Staff (Retired)
624 104 188

This is an accepted solution.

Hi @softwaregravy ,

Thank you for all of the additional information! Really appreciate providing me with all of this.

Taking a deeper look, the issue here is that the Shopify "customers" resource in GraphQL Admin API does not support filtering by the query argument 'created_at'.

The Shopify Bulk Operations functionality is built on top of the GraphQL Admin API, so any limitation with the Admin API will also be present in Bulk Operations. If we look at the documentation for the Query Root properties of the GraphQL Admin API (https://shopify.dev/docs/admin-api/graphql/reference/queryroot?api[version]=2020-04) we can see that the "customers" resource does not support filtering results in the query argument by the parameter of 'created_at': 

20-05-1u2qz-juka4

Because of this, if you try to use the parameter of 'created_at' (either in the Bulk Operation job or in a regular Admin API query) it will be ignored and all of the customers will be returned instead. The closest parameter you can use instead is maybe the 'updated_at' parameter or 'last_abandoned_order_date'

To learn more visit the Shopify Help Center or the Community Blog.

View solution in original post

Replies 11 (11)

softwaregravy
Shopify Partner
13 0 1

I've kept working at this, and been trying non-bulk operations, e.g. just regular queries

 

I cannot get querying by created_at working in either. I have verified that querying by updated_at is working for regular queries (I have not verified bulk)

 

So, tldr is that I am still can't get querying by created_at to work through. Examples below:

 

 

Querying by created at which is returning objects created before the date in the query

{
  customers(query: "created_at:>=2020-03-15", first: 50) {
    edges {
      cursor
      node {
        id
        createdAt
        updatedAt
      }
    }
  }
}
{
  "data": {
    "customers": {
      "edges": [
        {
          "cursor": "eyJsYXN0X2lkIjoyOTAwMjE3NzkwNTIzLCJsYXN0X3ZhbHVlIjoiMjkwMDIxNzc5MDUyMyJ9",
          "node": {
            "id": "gid://shopify/Customer/2900217790523",
            "createdAt": "2020-02-15T20:02:41Z",
            "updatedAt": "2020-02-15T20:13:48Z"
          }
        },
...
       
      ]
    }
  },
  "extensions": {
    "cost": {
      "requestedQueryCost": 52,
      "actualQueryCost": 52,
      "throttleStatus": {
        "maximumAvailable": 1000,
        "currentlyAvailable": 948,
        "restoreRate": 50
      }
    }
  }

 

whereas this query for updated_at

{
  customers(query: "updated_at:>=2020-03-15", first: 50) {
    edges {
      cursor
      node {
        id
        createdAt
        updatedAt
      }
    }
  }
}

Is correctly returning no records

{
  "data": {
    "customers": {
      "edges": []
    }
  },
  "extensions": {
    "cost": {
      "requestedQueryCost": 52,
      "actualQueryCost": 2,
      "throttleStatus": {
        "maximumAvailable": 1000,
        "currentlyAvailable": 998,
        "restoreRate": 50
      }
    }
  }
}

 

hassain
Shopify Staff (Retired)
624 104 188

Hi @softwaregravy ,

 

Try putting single quotes (' ') around the timestamps in your query. So for example: 

 

mutation {
  bulkOperationRunQuery(
  query: """
  {
    orders(query: "created_at:>='2020-01-01' AND created_at:<'2020-02-01'") {
      ....
      ....

If that does not work for you, could you please provide the value of the X-Request-ID response header from the Shopify API response when you create this Bulk Operations Query?

 

To learn more visit the Shopify Help Center or the Community Blog.

softwaregravy
Shopify Partner
13 0 1

adding the quotes had no effect

 

How do I enable the logging to get a request id? I'm using the ruby client.

hassain
Shopify Staff (Retired)
624 104 188

Check out how the shopify_api ruby gem gets the response header for 'X-Shopify-API-Call-Limit', it'll give you an idea on how to get other response headers too: https://github.com/Shopify/shopify_api/blob/master/lib/shopify_api/limits.rb

 

To learn more visit the Shopify Help Center or the Community Blog.

softwaregravy
Shopify Partner
13 0 1

Hi Hassain,

 

Can you give me some more direction here? the item you referenced seemed to apply to the REST api vs graphql. I've traded the query from ShopifyAPI gem here: https://github.com/Shopify/shopify_api/blob/master/lib/shopify_api/graphql/http_client.rb to the GraphQL gem here https://github.com/github/graphql-client/blob/master/lib/graphql/client/http.rb#L73 and the response seems to be totally transient with no headers captured or passed back. Monkey-patching this to get a header will be pretty in depth. 

 

Maybe I am making a mistake and there's an easier tay forward here. 

 

Any chance you can find by gid? gid://shopify/BulkOperation/6859718715

hassain
Shopify Staff (Retired)
624 104 188

Hey @softwaregravy ,

 

I cannot track down the correct logs with just the ID alone. If you are still unable to get the X-Request-ID response header, you can also provide me with the Bulk Operation ID + the store's myshopify.com URL + the time the request was made (with timezone provided).

 

Feel free to DM me this information if you do not feel comfortable sharing it on this thread.

 

To learn more visit the Shopify Help Center or the Community Blog.

softwaregravy
Shopify Partner
13 0 1

No problem. This is a test store: gratzi-test-store.myshopify.com

gid://shopify/BulkOperation/7140573243

submitted Thu, 21 May 2020 between 13:40 and 13:43 UTC.

 

The bulk operation returned all objects in the store. It should have returned 0.

softwaregravy
Shopify Partner
13 0 1

FYI, I have submitted a pull request to try and make getting the request-id possible for graphql queries. If you have someone at Shopify who works on your clients, I'd love to get some feedback  on it.

 

https://github.com/github/graphql-client/pull/249

softwaregravy
Shopify Partner
13 0 1

FYI, I used the patch I mentioned to submit a request. (previous example may have had a typo)

the x-request-id for this job is

f8397cfb-218c-4562-998d-79c02cf66377

 

Submitted just before: Thu, 21 May 2020 18:34:20 UTC +00:00

 

Job returned 96 objects. Expected 0

hassain
Shopify Staff (Retired)
624 104 188

This is an accepted solution.

Hi @softwaregravy ,

Thank you for all of the additional information! Really appreciate providing me with all of this.

Taking a deeper look, the issue here is that the Shopify "customers" resource in GraphQL Admin API does not support filtering by the query argument 'created_at'.

The Shopify Bulk Operations functionality is built on top of the GraphQL Admin API, so any limitation with the Admin API will also be present in Bulk Operations. If we look at the documentation for the Query Root properties of the GraphQL Admin API (https://shopify.dev/docs/admin-api/graphql/reference/queryroot?api[version]=2020-04) we can see that the "customers" resource does not support filtering results in the query argument by the parameter of 'created_at': 

20-05-1u2qz-juka4

Because of this, if you try to use the parameter of 'created_at' (either in the Bulk Operation job or in a regular Admin API query) it will be ignored and all of the customers will be returned instead. The closest parameter you can use instead is maybe the 'updated_at' parameter or 'last_abandoned_order_date'

To learn more visit the Shopify Help Center or the Community Blog.

softwaregravy
Shopify Partner
13 0 1

Thanks. I appreciate the help. I feel a bit silly I never thought to check that.

 

FWIW, I would suggest that better behavior would have been an error from the API.