I just launched RTS Guardian, a Shopify app built for merchants in the Philippines dealing with RTS (Return to Sender) orders.
The problem it tackles: even when an order looks fine at checkout, some of them quietly turn into RTS after they’ve shipped — by the time you find out, you’ve already lost the shipping cost and the product is stuck in transit. RTS Guardian flags orders that look likely to become RTS post-shipment and lets you follow up with the customer via Messenger or WhatsApp before the delivery attempt fails, instead of finding out after the fact.
How it’s different from existing COD/fraud-check apps: most COD verification apps (like COD King and similar) work at the order stage — they screen orders before you fulfill them to catch risky ones upfront. RTS Guardian picks up where those leave off. It’s built for the orders that pass that initial check but still go sideways after shipment, which is a gap most COD apps don’t cover since they stop watching once the order is confirmed.
A couple of things worth knowing if you’re checking it out:
It’s built around AfterShip for tracking data, so right now it’s really only useful if you’re already using AfterShip for order tracking.
It also works as a PWA, so you can add it to your phone’s home screen and check RTS status/notifications without opening a full app.
I’m the solo developer behind it, and it’s genuinely early — I’d love feedback from anyone shipping COD orders in the Philippines, even if you don’t end up using it. If it sounds relevant to what you’re dealing with, feel free to reply here or DM me and I’m happy to walk through how it works.
The part I would stress-test first is the messaging leg, not the scoring.
Both channels you are sending on close 24 hours after the customer’s last message to you. Meta’s Messenger policy is a 24 hour standard messaging window, and outside it you need a message tag or a one-time notification. WhatsApp has the same shape, outside the 24 hour customer service window template messages are the only thing you can send, and templates have to be pre-approved and are priced per message. A customer whose parcel is out for delivery ordered three or four days ago and almost certainly has not messaged the store since, so your normal case is the outside-the-window case. Every nudge is then a pre-approved utility template with fixed copy and variables, not a free-form “is this address still right?”. Worth pricing in before a merchant asks why the bill moves.
Second thing is label lag. You only learn whether an order really was RTS when the parcel comes back, which is days after the score fired. So the loop from changing a rule to knowing the rule got better runs a couple of weeks, and early on you will be judging the score on merchant reactions instead of outcomes. Storing the raw feature values you scored on, per order, is what saves you later when you want to re-test a threshold against labels you did not have at the time.
Last one, small but it bites. Shopify has no state that means returning to sender. FulfillmentEventStatus gives you ATTEMPTED_DELIVERY, DELAYED, IN_TRANSIT, OUT_FOR_DELIVERY and DELIVERED, and FAILURE there means the fulfillment request failed, not the delivery. So an RTS order sits in the admin looking like it is still in transit forever and your app becomes the only place that truth lives. Writing back an attempted delivery event or even just an order tag keeps the admin agreeing with you.
How are you labelling RTS today, is the merchant marking it by hand or are you reading it off AfterShip checkpoints?
Hi @uma0306 Welcome To Shopify Community So The gap you’re targeting is smart, most COD-check apps stop watching once an order passes the initial screen, but plenty of legitimate-looking orders still go sideways after shipment (customer becomes unreachable, changes their mind, address issues surface during delivery), so catching that post-shipment risk window before the failed delivery attempt actually happens could save real shipping cost. Being built around AfterShip is a fair tradeoff for v1 since it lets you focus on the flagging logic rather than building tracking infrastructure from scratch, though it does mean adoption is capped to merchants already using AfterShip, worth being upfront about that dependency early like you already are. Curious what signals the flagging is actually based on, is it tracking status patterns (like “in transit too long” or repeated failed attempts), or does it also factor in order-level signals (first-time customer, order value, address quality) alongside the shipment data? Hope this helps solve your problem, and if it does, don’t forget to like and mark it as the solution. Thank you!
@uma0306 I think the biggest opportunity here is to turn the RTS score into an actual post-shipment workflow, rather than just a risk alert.
For example, once an order gets flagged, you could trigger different actions based on the tracking state:
In transit for too long → send a proactive WhatsApp/Messenger confirmation
Out for delivery + previous delivery issue → prioritize the customer for follow-up
Attempted delivery → immediately notify the merchant and customer
Repeated failed attempts → increase the RTS risk score and trigger a stronger follow-up
Confirmed RTS → save that outcome against the order/customer so the next order can be scored more accurately
One thing I’d especially recommend is storing the tracking signals + score at the moment the order was flagged. Once you eventually get the actual RTS outcome, you can compare the prediction with the real result and continuously improve your thresholds.
You could even make this actionable for merchants with something like “High RTS Risk → Contact Customer → Confirmed → Resolved/RTS” instead of making them manually figure out what to do after receiving the alert.
Since you’re already using AfterShip for the tracking layer, this could make the app less of a “prediction tool” and more of a complete RTS prevention workflow.
@uma0306 I’d be really interested to know what signals you’re currently using for the initial RTS score and whether you’re already storing the eventual delivery/RTS outcome for each order.
Really appreciate the depth here — this is exactly the kind of stress-testing I was hoping for. Let me go point by point.
Messaging window: You’re right that this matters a lot for API-based sends, but the current implementation uses deep links (fb-messenger://user/[ID], wa.me/[number]) rather than the Send API. The merchant taps a button in the dashboard, their own personal/business Messenger or WhatsApp app opens with the customer’s chat pre-loaded, and they send it manually themselves. Since it’s not a programmatic send, the 24-hour session window / template requirement doesn’t apply — it’s just a human sending a message from their own app. The trade-off I’m aware of: I lose delivery confirmation on my end (I only know the button was tapped, not whether the message was actually sent), so it’s a UX convenience layer rather than a tracked notification system.
Labeling / scoring: This one turned out to not apply the way I described it — my fault for using “score” loosely earlier. RTS Guardian isn’t running a prediction model. There’s no threshold I’m tuning (like “3 days stalled = high risk”). The risk levels are a direct mapping of AfterShip’s own status values — FailedAttempt, Exception, Expired, Undelivered get color-coded, Delivered doesn’t. So there’s no “wait for the parcel to come back and check if the prediction was right” loop, because I’m not predicting anything — I’m surfacing a status AfterShip has already determined. If the labeling is inaccurate, that’s an AfterShip-side signal quality issue, not something I need to backtest.
Shopify state gap: Completely agree, and this one stands as-is regardless of the above. Shopify has no RTS-equivalent status, so right now this risk info only lives in my dashboard. Writing an order tag back via the Admin API when a high-risk status is detected is on my list — that closes the gap you pointed out.
To directly answer your question: I’m reading it straight off AfterShip checkpoints (FailedAttempt/Exception/Expired/Undelivered), not merchant manual tagging, and not a scored/ML model.
Thanks again for taking the time to write this out — feedback this specific is rare and genuinely useful at this stage.
Thank you so much for this — genuinely, comments this thoughtful are exactly why I decided to post here instead of just quietly launching and hoping for the best.
To answer your question directly: right now the flagging is based purely on AfterShip’s own tracking status patterns — things like FailedAttempt, Exception, Expired, and Undelivered get flagged and color-coded, while Delivered doesn’t. It does NOT currently factor in order-level signals like first-time customer, order value, or address quality — that’s shipment data only, v1 keeps it simple on purpose.
That said, your question is making me think order-level signals could genuinely sharpen this down the line — first-time customer + COD + a shipment status like FailedAttempt is probably a very different risk profile than a repeat customer with the same status. Appreciate you planting that seed.
And yes, the AfterShip dependency is very much a conscious v1 tradeoff, exactly as you described — didn’t want to build tracking infra from scratch when there’s already a solid signal source to build on top of. Long-term I’d like to loosen that requirement, but it’s not there yet.
Really grateful for the time you put into this — thank you!
This is a great reframe, thank you — “prediction tool” vs “complete prevention workflow” is exactly the right way to think about where this should go.
Quick clarification on the current state, since a couple of comments here have assumed a scoring model: there isn’t one yet. Right now the “risk level” is a direct pass-through of AfterShip’s own tracking status — FailedAttempt, Exception, Expired, Undelivered get flagged and color-coded, Delivered doesn’t. No custom scoring, no thresholds I’m tuning, no order-level signals factored in. So it’s closer to “AfterShip status, made visible and actionable” than a prediction engine today.
That said, the workflow you laid out (in-transit-too-long → proactive message, repeated failures → escalate, confirmed RTS → save outcome against the customer) is basically the roadmap I should be building toward, and it’s a much better mental model than what I had. The state-machine framing (“High Risk → Contact Customer → Confirmed → Resolved/RTS”) in particular is something I want to build directly into the dashboard instead of leaving merchants to figure out next steps themselves.
And yes — storing tracking signals against the order at flag-time, then reconciling against the eventual outcome, is on my list now specifically because of this thread. Even without a scoring model today, having that history is what would let me eventually build toward customer-level RTS history, which is honestly one of the more interesting ideas above.
Really appreciate you laying this out so concretely — this gives me a lot to work with.
Hi @uma0306 So That makes sense as a v1 scope decision, shipment-status-only flagging is a solid, testable foundation before layering in more variables that could muddy the signal early on. The first-time customer + COD + FailedAttempt combination you mentioned does sound like a meaningfully sharper risk signal than status alone, since a repeat customer hitting the same status is far more likely to just be a temporary delivery hiccup rather than an actual RTS risk. Good luck with it, this is exactly the kind of focused, real-problem-first app that tends to find its footing once a few merchants start using it in production. Hope this helps solve your problem, and if it does, don’t forget to like and mark it as the solution. Thank you!
Thank you — really appreciate the encouragement, and the whole thread honestly. Between the messaging window point, the labeling question, and this one, I’ve got a much clearer picture of where v1 stands and where to take it next. Will keep this thread in mind as it moves into production. Thanks again!
