We recently had this timeout issue occur for one of our customer's saved search. So from our side, we fetch the whole saved search page by page using the REST API and save it on our end but sometimes the saved search API returns 502 Bad Gateway error thereby forcing us to implement a retry logic. It is all well and working but the only downside is that it affects our SLAs negatively. So what should run for 3 hours sometimes runs for 4 to 5 hours instead.
Here's a sample X-Request-ID that triggered this issue: 5b40104c-17cd-4ace-9029-448e0d1f9160
We are hoping to get suggestions from you on how we can finally have a better solution for this.
Thanks for providing the x-request-id for us! I was able to take a look through the logs, and found that the most likely reason for the timeouts that you're running into is that you have the request results limit set for 250, which we don't generally recommend. I would recommend setting the limit to the default of 50 (you can remove the limit field from the query altogether) and see how long it takes to get your full dataset that way. It's unlikely that you would run into timeouts when requesting in batches of 50, and each response would be returned more quickly than the current responses containing 250 results. I hope this is helpful!