Re: inconsistent behaviour with new multi-location shipping rules and CarrierServiceAPI

inconsistent behaviour with new multi-location shipping rules and CarrierServiceAPI

Parcel_Intellig
Shopify Partner
107 1 50

Hi,

 

There seems to be some inconsistencies with the way Shopify treats manually created rates and CCS rates when the store has multi-origin shipping activated.

 

Say i had

product A - list price $8 - only available at location A

product B - list price $8 - only available at location B

 

and setup a single delivery profile which contains all products and all locations, in this profile i setup 3 rates

1. standard shipping, for orders $0-$10 - $10

2. free shipping, for orders > $10 - $0

3. App implemented using carrier service API

 

If i place 1x product A and 1x product B in the cart, Shopify will send two requests to the App, one for each product as they are in different locations. However, the two rates setup in Shopify will be calculated off the entire cart total and show free shipping.

 

Few issues here:

1. any apps using CCS won't ever see the entire cart contents now in this situation and unable to apply a generic rule such as free shipping > $10 (for the entire cart), and any merchants who have been using an app to achieve customised rules (for example free shipping > $10 but for a given postcode range only) can no longer achieve this. Furthermore as some merchants have unwittingly upgraded their stores to use this, they cannot go back and hence they lose the ability to control their rates.

2. the behaviour is inconsistent and causes confusion.

 

Anyone have any suggestions as to how we can get the original functionality back?

Replies 43 (43)

SBD_
Shopify Staff
1829 271 415

Hey @Parcel_Intellig,

 

There's a similar thread here: https://community.shopify.com/c/Shopify-APIs-SDKs/What-triggers-a-call-to-Custom-Carrier-Service/td-...

 

A new API is in the works to better handle this. No ETA yet.

I'll leave this thread open in case others have suggestions/work arounds.

Scott | Developer Advocate @ Shopify 

Parcel_Intellig
Shopify Partner
107 1 50

Hi Scotty,

 

Yes same root cause, but different use case here, if you guys are working on better APIs for carrier service, is there a way we can have a high level overview of what is proposed and a chance to provide some feedback?

 

Thanks

Min

SBD_
Shopify Staff
1829 271 415

Hey Min,

 

I'll shoot you a DM.

Scott | Developer Advocate @ Shopify 

Parcel_Intellig
Shopify Partner
107 1 50

Hi Scott,

 

Following up on this topic, is there a way we can detect if a store has multi-origin shipping enabled via the API?

 

I was looking at shop.json's multi_location_enabled parameter however this doesn't seem to be the correct parameter.

 

Thanks

Min

SBD_
Shopify Staff
1829 271 415

@Parcel_Intellig I would've suspected the same property - can you provide an example of it not working?

Scott | Developer Advocate @ Shopify 

Parcel_Intellig
Shopify Partner
107 1 50

Hi Scotty,

 

The parameter shows when multi-locations is enabled rather than multi-origin.

 

I'll PM you some details of a store with the parameter set to true but multi-origin not enabled.

 

Thanks

Min

Parcel_Intellig
Shopify Partner
107 1 50

Hi Scott,

 

looking at the graphql spec, perhaps the legacyMode is what i'm after? are you able to confirm what exactly this represents?

 

query {
  deliveryProfiles (first: 1) {
    edges {
      node {
    legacyMode
      }
    }
  }
}
 
testing on a store with multi-origin shipping enabled has this param set to false, and a store with multi-origin shipping disabled, has this param set to true. At face value it looks like it's the parameter i'm after.
SBD_
Shopify Staff
1829 271 415

Hey @Parcel_Intellig 

 

The parameter shows when multi-locations is enabled rather than multi-origin.

Ah yeah you're right.

 

perhaps the legacyMode is what i'm after?

That sounds like it: The legacyModeProfiles field returns true if the shop is still using single-origin shipping. More info.

Scott | Developer Advocate @ Shopify 

Timbersson
Shopify Partner
4 0 1

Hey @SBD_,

I was wondering if there was any update on this, especially with regard to news/ETA on the new API you mentioned?  

We would love to implement local delivery accross our stores, but we also have a custom shipping rate calculation that is based on the contents of the whole cart, and it's a deal-breaker for us if we can't keep this :(.

Thanks,

Tim

 

SBD_
Shopify Staff
1829 271 415

Hey @Timbersson 

Unfortunately there's no ETA available yet.

Scott | Developer Advocate @ Shopify 

Parcel_Intellig
Shopify Partner
107 1 50

Hi @SBD_ 

Are there any further updates to when this new API will be ready? We are finding increasing number of merchants finding that multi origin shipping is impacting their ability to set accurate shipping rates. It would be really good if either multi origin shipping can be turned off by default or more focus be made on this to make it easier for merchants to setup their rates.

Thanks

Min

Parcel_Intellig
Shopify Partner
107 1 50

Hi @SBD_ @Shopify 

I'm hearing reports from merchants that Shopify is no longer allowing them to disable multi-origin shipping.

I have a large number of merchants approach us still looking for a workaround to this as multi-origin shipping won't allow them to set a rate across the entire cart at all, this means it is impossible for them to offer say free shipping > $100 across their entire store and is a major deal breaker for new merchants, and also existing merchants who have inadvertently enabled multi-origin shipping without understanding the new behaviour.

Over the past few months, merchants have been able to successfully request via Shopify support for multi-origin shipping to be disabled, has a decision been made to stop this workaround? Without any updates to the Carrier Service API, there are currently no workarounds at all with or without an app for merchants. What is the recommended course of action here?

Thanks

Min

E2
Tourist
4 0 0

Bumping this once again.
We have multiple merchants with multiple locations, and this is getting old

Merchants needs to be able to set a single shipping price regardless of how many locations are involved in fulfilling the order.

Parcel_Intellig
Shopify Partner
107 1 50

@SBD_ Shopify 

Are there any updates to this?

There has been no updates since this was originally discussed, but things seem to be moving in the opposite direction:

- multi-origin shipping is being forced on everyone in the next few months

- new stores are already forced on multi-origin shipping

- shopify's own staff doesn't seem to know their own product and is telling merchants that shipping apps will solve the problem, however we all know this isn't possible

- 80% of new merchant queries to us is about turning off multi-origin shipping!!!

seems to be some unhappy campers in this thread too:

https://community.shopify.com/c/Shopify-APIs-SDKs/Multi-Origin-Shipping-How-do-you-fetch-only-one-sh...

Being able to set a single flat rate for the entire cart is something pretty trivial, it's a shame that Shopify is unable to support it.

Thanks

Min

SBD_
Shopify Staff
1829 271 415

Thanks for the frank feedback @Parcel_Intellig 

I just checked in with the shipping team. There's an internal ticket tracking these frustrations, I'll add your voice + the forum link to the thread.

 

Scott | Developer Advocate @ Shopify 

nathan-j
Shopify Partner
2 0 2

Any update on this ? I'm also hitting issues with Shopify splitting carrier service requests, making it impossible to do things such as
Free shipping based on total cart value
Conditional pricing based on total cart contents
Custom rate names and descriptions based on total cart content

HunkyBill
Shopify Partner
4851 60 555

@Parcel_IntelligI am curious as I review this thread, and you make numerous mentions of "Free Shipping > $100 in cart" as a thing.

That has not been accurately possible since forever as you get no information in the payload about pricing discounts. How have you handled that over the years? A customer spends $100 in the cart but uses a $30 off coupon, most merchants deny free shipping as it is bonus on top of a bonus. So you'd be handing out free shipping all the time, since most shops use discounts of some sort.

I am curious. Has that been a hell item for you, or you just inform your merchant clients to suck it up?

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
Parcel_Intellig
Shopify Partner
107 1 50

Yes discounts is a big problem, we have a way to look this up via abandoned carts/draft orders but its not very accurate, merchants are notified of this and they will need to put up with it because it's this or nothing. Some things we can workaround for PLUS merchants via scripts, but discounts is not one of them either.

HunkyBill
Shopify Partner
4851 60 555

When a customer is in checkout, it is not an abandoned cart, so that makes no sense? And when a customer is in checkout, that has zero to do with Draft Orders. So you lost me there.

I agree that Plus makes things much easier, but not for interfacing to an external rate generator for shipping. Again, I am not sure what you mean by workarounds there. 

One of the first things I learned about Plus and our Carrier Services, is that when you want to tell the customer WHY the rates they are seeing are stupid, you cannot. So they see stupid rates. Whereas if they corrected their checkout, due to our messaging telling them what they did wrong, they might get better rates. But alas, we can not go there.

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
Parcel_Intellig
Shopify Partner
107 1 50

actually not quite correct. as soon as a customer inputs their shipping address an abandoned cart is created or updated, except in some random situations where shopify doesn't do this (hence inaccurate) - most commonly if a customer has a long running session.

draft orders are for other apps which create orders via draft, but some wholesale apps don't do this properly either, i.e. they create a draft order but with no customer or shipping info and back populate this once the order is created, which again means we can't find any data.

i would say this method works 50% of the time, it really depends on customer behaviour and checkout workflows, some merchants have been using this feature for a number of years and is happy with it, others not so much, in which case they will need to exclude discounts in their rules.

with PLUS there is scope to display custom error messages, we do this for several PLUS merchants, however this requires a great deal of customisation on their checkout.

HunkyBill
Shopify Partner
4851 60 555

Ok. So an abandoned checkout exists, with or without good data. In your brief time trying to generate a rate and return it, you now subject yourself to making an API call, and fishing for an abandoned cart entry that matches the info you're given in the payload. Wow. OK... I guess address matching is all you get there. And the discounts applied to the cart? I would be amazed if they provided that, but OK.

As for Draft Orders, again, how is fishing that pool of data fast, efficient, useful or even accurate? And why would you even be there? Carrier Services are not even hooked up to that.

As for the SCOPES you mention in messaging, I will have to look into that. Clearly, that is not something I really care about, but if it improves messaging between external Apps and checkout details, it is neat. Usually, checkout is oblivious to what we do.

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
Parcel_Intellig
Shopify Partner
107 1 50

efficient? absolutely not! but the only way to pull any meaningful customer data from Shopify.

 

if a customer users a wholesale app then it will most likely be a draft order instead of abandoned cart, in which case we will need to go looking there for additional data, not really relevant for discounts but useful for other customer data.

HunkyBill
Shopify Partner
4851 60 555

@Parcel_IntelligJust a quick question. A CCS App in a full on multi-location, multi-origin store will get one request with all the items in an order, provided there is inventory stocking each item at each location. If there is an item not stocked at a location, you can expect a request for that location and that item where it is stocked. So you just return all your requests, and Shopify takes care to ensure the customer sees a correct CCS rate?

All this talk of people hashing and mapping, and storing N requests seems like "a garbage can fire, do not go"... but I am not sure? Is it really simple enough like the good old days? Get a request, figure it out, send a rate... done? 

 

 

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
Parcel_Intellig
Shopify Partner
107 1 50
nope, if one request has 2 items stocked at 2 locations then you get 2 separate requests, you never see the entire cart.

if 2 items are stocked at 2 locations and any 1 is in-stock at A and the other at A and B, shopify will pick A regardless of the actual distance.

if one is in stock at A and the other B then you get 2 requests.

if everything is in stock at multiple locations, shopify will pick a random location too, probably the first location you setup, rather than the closest as well.
HunkyBill
Shopify Partner
4851 60 555

Not sure I follow. For simplicity, a cart has 3 items, with the store having 3 locations. In this cart, 1 item in stock at each location. So you are saying you get three requests? And when you respond with a rate for each one, 3 responses, what does Shopify show? The sum of those 3?

That is what interests me I guess. This concept of what happens at the Shopify end of things.

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
Parcel_Intellig
Shopify Partner
107 1 50

If all 3 are in stock at all locations then it can be fulfilled from a single location and you get 1 request, the location is of shopify's choosing, may not be the closest. This is the first point merchants have issues with.

Furthermore, once the order is created, there is no way for us to find out which location this is via the API, but it does show on via the admin pages.

If item 1 is only at location A, item 2 only at location B and item 3 only at location C, then you will get 3 requests, each request only contains the items for that shipment so you never get a entire view of the cart. This is what prevents merchants from setting a rate for the whole cart and majority of merchants complain about this. I'm aware some developers have begun hashing/matching up these values, however there is no guarantee that requests will be received from the same server, in the right order or even close to each other, there's too much ambiguity here for us to build any type of robust solution as all this can change overnight. 

Next issue: If say all 3 items are available at all locations, but the app has been configured to NOT return rates for this particular order (i.e. say it contains restricted items and cannot be shipped), i've found Shopify will generate an exhaustive list of combinations for items and locations and run through these until it starts getting rates back, except it also seems to stop after the first 5-6 tries (upper bound), i can kind of see where this is coming from, i.e. a location might be out of stock, however given it gives up so quickly (which i can understand why too) it's not really going to give great results, whilst this may result in some rates being shown in some situations, it seems rather like a poorly thought out bandaid that's been hastly implemented after multi-origin shipping was put in and merchants started having major issues. It's also very poor efficiency, and probably why they needed to put in those faster timeouts.

And lastly, none of this behaviour is really documented anywhere, so i guess its bound to change, which makes it even harder for anyone to build any solutions to work around it.

 

 

 

HunkyBill
Shopify Partner
4851 60 555

Since you went to the effort of introducing CCS I am curious as to when you expect it to kick in? With your explanation as is, the customer has a $16 checkout and there free shipping. If you moved that into the CCS, then you have the issue that one request for $8 would return $10 and another for $8 would also return $10. But interestingly, since you are in fact shipping 2 cheaper items from 2 different places, collecting fees for shipping would make sense so you don't lose money. Can you glue the two requests together to form one response. Nope. So what can you do here.

I guess it boils down to not losing money on shipping, and not losing customers too. Shopify said they are doing this to usher in the era of multi-origin and multi-location orders, so clearly our payloads in CCS need more info too. Do you even see anything? In all my CCS Apps the payload looks same as it has for 8 years now... nothing new. Fun fun fun.

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
E2
Tourist
4 0 0
Hm, we have merchants who transfers stock internally before shipping together, stores with different stocks in the same building, merchants who refuses to let the end customer pay double shipping just because they happened to be sold out for one of the items on the location that carries the other x products in the order, and so on and so forth - we should be able to define whether to consolidate shipping cost or not (and a price based rate is a price based rate, not a price-per-location-based rate?). Am I missing something here, or are we, with multi origin shipping enabled, completely unable to serve the merchant a price based shipping rate through our app, if the merchant has more than one location?
Parcel_Intellig
Shopify Partner
107 1 50

From what i can tell, via CCS, what you're after isn't possible, which is also the issue i have with a large number of merchants that use my app, enabling multi-origin shipping is a non-backwards compatible change.

 

If you setup the rates within Shopify though and put everything as a single profile then you get the rate calculated over the entire order - but only if you use the in-built Shopify rates, it would be good if in the mean time this can be made compatible and consistent with CCS as well.

E2
Tourist
4 0 0

Definitely - and that is what we have to do as well, but then we cannot give the option of different pick up points, etc., that the carriers offer.
And in some carriers' case, that is violating the TOS.

So this is a major issue for our merchants..

HunkyBill
Shopify Partner
4851 60 555

It seems we all need patience. Shopify has admitted for a decade now that even though their commerce engine is advance, perhaps more so than any other, they lack the polish of the logistics domain. Heck, 5 years before Shopify was a dream I was tasked with building out an online (WWW) freight-forwarding logistics service to handle US transportation issues like LTL, lift-gate, modal, zone, express services +++, and yet, even today, there is nary a glimmer in the Shopify code that that kind of interface is coming to fruition any time soon. So you have one external industry with mature code, standards and operational understandings (Logistics), and another industry piggy-backed on top (commerce) that is only now attempting to plug-in, without causing too much trouble. And to boot, Shopify is now touting planet-wide 2-day delivery!

 

So yes, patience. We won't see CCS much longer as I imagine the in-house options will supersede that, instead we will likely be bridging external carriers to Shopify with a standard interface. Let the chips fall where they may,...

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
gnanakeethan
Tourist
6 0 3

Hi, 

 

I would like to say couple more words here. I just finished a simple integration process. 

 

It looks like multi-location shipping would be technically possible given that we have a hashing done on the carrier services requests. 

 

As per latest Shopify Carrier Services API, it gives 15 minutes of cache time. It also caches each time for any new change done in the request. Hence it would be possible that we can technically hash the destination part and create an index of origins given alongside. 

While returning consecutive responses for the same destination with different origins, it will be either possible to return a 0 price or price between origins & last destination sorted, which will then be combined by Shopify into a single price for the customer.  I am not sure how Shopify would merge each shipping lines, but technically it would be possible to put a check. 


I hope the same logic can be applied for each fulfillment connected to the choosen pricing to provide a unified price for the merchant from the shipping provider. 

 

I am going to try this logic with our app now. 

 

Best Regards,
Keethan. 

 

Parcel_Intellig
Shopify Partner
107 1 50
you need to be careful here, we have thought about this too, however Shopify will sometimes not refresh the rates at all, if you try and return a 0 rate for some requests but others you may end up in a situation where you unexpectedly show the customer free shipping. All this works well from what we observe doing unit tests, but none of this behaviour is guaranteed or documented which means it cannot be relied on. we run millions of CCS requests through our app and from what we observe there will always be a small percentage of cases where this will fail, due to timing, caching and what not. With such large numbers, even a small percentage is enough to cause significant problems for merchants.
gnanakeethan
Tourist
6 0 3

Yes. That is a highly probable case. There are lot of parameters for failure which we cannot calculate. 

 

I can confirm that Shopify doesn't allow if one of the requests fail while other succeeds. I have given alternator which makes the request fail for each alternate requests. 

 

You can see the functionality live here. 

https://shop.1courier.net/

 

There are few specific products. 

"Spiceland" - Main Warehouse

"Chequered Red Shirt" - Main warehouse
"Black Leather Bag" - Second warehouse

"Milliken"  -  Second Warehouse

which are from two different origin / warehouse locations. 

If you happen to have both of them, the requests will fail to give you a cart price. Where as if you have one of them, the request will be valid. 

 

@Parcel_Intellig Let me know of your thoughts on this.

 

Best Regards,
Keethan

 

 

Parcel_Intellig
Shopify Partner
107 1 50
I've found of one request has no rates, then Shopify automatically tries different locations, i think it assumes that those locations just have no stock, again none of this seems to be officially documented, but it also doesnt seem to try every combinations of locations and sometimes even retries the same location combinations too.

If your app runs on multiple servers, Shopify has also stated that they do not guarantee multiple shipment requests will arrive at the same server either, which further complicates matters.
Parcel_Intellig
Shopify Partner
107 1 50
I guess the other thing is, you are processing each one of these requests in a stateless fashion, well you can keep some state on your end, but these requests may take a second or two to make it across to you, you have a total of 10s to return a rate which doesnt give you much room to move, how are you going to tell of the request you receive includes items for the entire order? you will need to look up the cart via the cart token or abandoned cart or some other method right? these look ups will take time too, we have found that these lookup are highly inaccurate due to caching on Shopify's end, which means being able to tell which request to send real rates back, and which ones to zero out is not going to be 100% accurate either.
gnanakeethan
Tourist
6 0 3

Yes. I understand the complexity now. 

Thanks for your clarifications.

 

I will see if I can find a realistic solution in the space. 

Best Regards,
Keethan

gnanakeethan
Tourist
6 0 3

Here are my findings so far. 

1. Shopify does only allow 1 shipping provider /rate per order. 

2. Hence, it sends all the request to every CCS endpoint. 

3. Timeout for each CCS request is 10 seconds. Such that requests are sequential in no specific order.

4. Each request has a list of line items. 
5. Each request has the same signature for destination if properly hashed using a key of shop's domain name.

6. Such that each request cannot be found in a specific order. Which complicates the matter. It is sent in the order in which the cart is organized at the time of request.

7. However, given the complications it should be possible to arrive in a routable list of waypoints and destinations. 

8. OSRM / Google Map should be able to deduce the list of waypoints into a series which are pickups and a final delivery location.

9. Once we have the routed list, we map the line items into the waypoints. Hence a final route is produced with weight & packages. 

Last three steps are repeated, as long as there are new requests coming. The longest distance covered by a delivery service will always be a longest. Any location in between would adjust the distance length by a different margin, but shall not reduce it. Hence the price may become valid for certain services. At the end of the CCS requests, there will be a route of waypoints and final destinations. The cost can be calculated. 

 

The one which I configured in the server now, uses a simple local redis instance. It can be scaled to have a Cluster. In a cluster, max response time could not be more than 500ms. Single 1GB redis instance should be sufficient to do a minimum of 1000 carts per 15 mins. 

Which gives 8.5 seconds for the request to complete safe manner. 

 

At the end of order submission, when shopify marks the items fulfilled, it would be possible to capture each block and push them into a same order for the shipping provider if the provider was choosen. 

 

I do not think this can be much of a realistic solution. 

 

 

Best Regards,
Keethan

Parcel_Intellig
Shopify Partner
107 1 50

Hi Keethan,

 

Is the idea that you log all the CCS requests, hash by address, then on the last (nth) CCS request calculate the price for the entire route and return a rate?

 

How will you work out if said request is the last one? (i.e. find n)

 

I think for the responses to the other CCS requests (1 - n-1) you will also need to return the same rate names as what you intend to return on the nth request and set those rates to 0, this way Shopify will combine the rates correctly.

 

Thanks

Min

gnanakeethan
Tourist
6 0 3

Hi Min,

 

The idea is that each request is handled on full for each request until all requests are fulfilled. There is no need to worry about N number of requests. Each request is also cached in parts allowing to reconstruct the route each time. 

As long as request are sequential, the requests would either extend the route length / fall inside / extend by a smaller margin. We add the difference in margin to each sequential request as the rate. Shopify will finally combine. No need to worry about the next request until it comes. We only need to keep a list of previous requests and the line items up from that point. We build a route for the shipment company using that. The same code for shipment would be used. 

This is in the hope that pricing for a shipment would not go down. 

 

I have been exploring this problem in my mind space for couple of hours. I have one more question, given that Store having multiple inventories, Shopify will send multiple requests for the same item to be delivered from different locations? If this is a no, Then the flow control logic I have been thinking should be valid to process for an order.

 

Thanks,
Keethan

Parcel_Intellig
Shopify Partner
107 1 50

Hi Keethan,

 

Ah i see, your use case is a bit different to mine, as you want the rates to be added anyway, this doesn't really pose a problem, whereas in my use case, the customer wants to apply a rate over the entire cart, i.e. free shipping if entire cart is > $x, in which case we will need wait til the nth request so we know what the entire cart is, or use a cart token to look it up before hand (however all the unknowns makes this method inaccurate).

 

For multiple locations, if multi-origin shipping is enabled (and this is where the inconsistencies lie anyway), Shopify will try and find a single location which can fulfil all items in the cart, if it can't find a single location, it will work its way down the list of locations stored in the Shopify admin and try and fulfil as many as possible at each location until all locations are fulfilled. From what i can tell, once it's split the order into these 'shipments' it should send a single request for each location and those items, unless that request receives no rates in response, in which case it may try another location.

Raymon_Luo
Shopify Partner
1 0 0

It's 2022 now, any updates/solution for this? 

SMDHH
Shopify Partner
4 1 8

It's 2024 now, any updates/solution for this? 😣