Elias here with Shopify Support. Thank you for reaching out.
I understand you have two collections that aren't appearing when you try to set up the tax override. It may be difficult to determine the cause of the issue without being able to see your collection or tax settings. However, I'd recommend checking whether the collections are available for the online store sales channel first. Also, if you have a publishing date set for the collection, you'll want to ensure that there isn't an error with the dates. If the collections are already available for the online store, it may also be worth toggling them by deselecting all the sales channels in the collection's availability. Remember to hit save, then re-enable them for the each of the sales channels and save again.
If the issue persists, feel free to let me know your store URL and I'd be happy to take a closer look!