Bulk Query Internal Server Error

[achempak]

I'm trying to do a bulk order query using shopify's graphql API. I've been running the same script for months, and it's been working fine. Every day, I request orders that have been updated in the last 2 weeks. However, on May 19 12:12PM UTC, the request repeatedly failed (hence the orders data in my db is stale as of that date). I tried querying for orders beginning May 20, and the request worked fine. I'm assuming the internal server error is related to order(s) that were placed between May 19 and May 20. Ideally I don't want to completely rewrite my script to use the regular graphql API (non-bulk API) to fetch this data. Can anyone help diagnose the issue? It's quite urgent. Here is the query I used

mutation {
    bulkOperationRunQuery (
      query: """
        {
          orders(query:"updated_at:>'2021-05-19T07:10:00'"){
            edges {
              node {
                id
                name
                email
                phone
                createdAt
                updatedAt
                closedAt
                processedAt
                cancelledAt
                customer {
                  id
                }
                billingAddress {
                  id
                }
                shippingAddress {
                  id
                }
                currencyCode
                totalPriceSet {
                  shopMoney {
                  amount
                  }
                }
                subtotalPriceSet {
                  shopMoney {
                  amount
                  }
                }
                totalDiscountsSet {
                  shopMoney {
                    amount
                  }
                }
                totalTaxSet {
                  shopMoney {
                    amount
                  }
                }
                displayFinancialStatus
                displayFulfillmentStatus
                note
                cancelReason
                discountCode
                transactions {
                  id
                  receipt
                  accountNumber
                  amountSet{
                    shopMoney{
                    amount
                    }
                  }
                  totalUnsettledSet{
                    shopMoney{
                    amount
                    }
                  }
                  createdAt
                  processedAt
                  status
                  errorCode
                  gateway
                }

                discountApplications(first: 5) {
                  edges{
                    node{
                    allocationMethod
                    targetSelection
                    targetType
                    value {
                      __typename
                    }
                    }
                  }
                }

                lineItems {
                  edges {
                    node {
                      id
                      name
                      product {
                        id
                      }
                      variantTitle
                      variant {
                        id
                      }
                      sku
                      vendor
                      quantity
                      unfulfilledQuantity
                      fulfillableQuantity
                      refundableQuantity
                      discountedTotalSet {
                        shopMoney {
                          amount
                        }
                      }
                      originalTotalSet {
                        shopMoney {
                          amount
                        }
                      }
                      totalDiscountSet {
                        shopMoney {
                          amount
                        }
                      }
                      unfulfilledDiscountedTotalSet {
                        shopMoney {
                          amount
                        }
                      }
                      unfulfilledOriginalTotalSet {
                        shopMoney {
                          amount
                        }
                      }

                      fulfillmentStatus            
                      fulfillmentService {
                        serviceName
                      }
                    }
                  }
                }
                customerLocale
                tags
                confirmed
              }
            }
          }
        }
      """
    ) {
      bulkOperation {
        id
        status
      }
      userErrors {
        field
        message
      }
    }
  }

And below is the failed response

{
  "data": {
    "currentBulkOperation": {
      "id": "gid:\/\/shopify\/BulkOperation\/435233816638",
      "status": "FAILED",
      "errorCode": "INTERNAL_SERVER_ERROR",
      "createdAt": "2021-06-02T01:07:16Z",
      "completedAt": null,
      "objectCount": "0",
      "fileSize": null,
      "url": null,
      "partialDataUrl": null
    }
  },
  "extensions": {
    "cost": {
      "requestedQueryCost": 1,
      "actualQueryCost": 1,
      "throttleStatus": {
        "maximumAvailable": 2000.0,
        "currentlyAvailable": 1999,
        "restoreRate": 100.0
      }
    }
  }
}

These are a few other failed response IDs:

gid://shopify/BulkOperation/434631934014

gid://shopify/BulkOperation/434626592830

gid://shopify/BulkOperation/434622464062

gid://shopify/BulkOperation/433406345278

gid://shopify/BulkOperation/433406345278

[GrahamS]

Hey @achempak ,

Thanks for checking in about this. I would like to have this diagnosed further, but a more extensive investigation will require some authorization in order to access the necessary data. Can I please ask you to contact our support team directly and pursue an escalation? We should be able to look at the underlying cause of these returned server error messages.