Stephanie here from Shopify's guru team :)
Transactions fees will apply on any third-party/manual payment method when Shopify Payments is "not" enabled on the store.
This means that if you have Cash on Delivery but Shopify Payments is disabled on the store, then transaction fee's would be incurred on a per-order basis.
If your customers refuse to receive the shipment, you can refund and cancel the order. Once an order is cancelled and deleted, a credit for the transaction fee will be generated towards future transaction fees!
To cancel and delete an order, click on the 3 dots on the top right of the order page and select cancel. Once the order is cancelled, a 'delete' button will appear at the bottom of the order page.
Hope this helps!
Feel free to reach out to firstname.lastname@example.org if you have any other questions :)