Shopify Flow to put high-risk orders on hold

We have a Shopify Flow set up to automatically put high-risk orders on hold, so our support team can look into these and cancel if necessary.

It is working for some (but not the majority of orders). We have seen a big ramp in fraudulent transactions recently, and as this has been failing a ton of orders have gone through and been fulfilled. We are now dealing with the chargebacks and fall out from them.

Is anyone able to help me figure out why this is not working correctly?

Many thanks in advance.

Looks like your trigger could be order created, so all orders go through the workflow. Then you can see what step is failing in the recent runs.

Also, why not change the payment capture to after fulfillment?

Switch the starting trigger to “Order risk analysed.” This ensures the Flow only runs once Shopify has assigned a final risk score (Low, Medium, or High). Go into the Flow app, click on your workflow, and look at the “Recent Runs” list. Check the runs for orders that should have been held. If they show as “Succeeded” but didn’t take action, your Conditions like order / riskLevel is HIGH are likely checking the data before it’s ready. If you don’t see them at all, the trigger didn’t fire. If you are losing money on transaction fees for these fraudulent orders, manual capture is a lifesaver. Shopify doesn’t charge the transaction fee until you capture the funds, so canceling a high-risk order before capture saves you the 2.9% + fee.