With shopify plus, would guess so, yes. I can see the app entry in my setup of shipping rates, other "manual" shipping rates are removed.

Hey folks.... I am currently testing around with carrierService requests to a callback_url to fetch shipping rates for the checkout but unfortunately i do not see any requests to my url and shopify sets the shipping rate to the value i set as fallbac...