Nick here from Shopify.
Really good question! The first thing I always recommend with tax questions is to visit your local tax office or a professional accountant. The reason being is a professional accountant would know the tax laws for your area/ surrounding areas better than anyone else. Because tax laws differ for every country and even specific regions within countries it's difficult to know them all.
I did do some digging for you on this and found a third party guide on when you should and should not charge taxes on shipping in California which might give you the answer you are looking for. You would not need to charge a shipping tax "if you pass on the actual cost of shipping to your customer and you list that charge on a separate line on your invoice", but you would if you "if you put the two together into a “shipping and handling” line on your invoice. Separating these two charges would enable your shipping charges to stay tax-free." You can see the guide with more information here.
Hope this helps.
All the best, Nick
For future persons finding this, here is official California Gov chart for it, Publication 100, Shipping and Delivery Charges. Shopify was not charging tax on our flat rate $15 local delivery fee, despite "charge tax on shipping rates" being checked, and I thought something was wrong. It may still be, but that's on our internal tracking and fee calculation, and probably not with Shopify. Like, we need to be able to justify the rate we're charging, but a plain "shipping" line-item isn't necessarily taxable.