No, you are right.
THERE IS A BUG in Shopify and its been reported many times.
THE BUG
When you create a collection “first” (like any sane human) and then you make a rule like “auto add products of X type” and then later you begin to add products, that collection does not update correctly. Everything on the back end will show up, you will be pulling your hair out, and swearing at the Shopify logo. However, you are not broken, Shopify is
THE WAY
Do not create the collection first (like any reasonable human would), instead, first create all your products, then after that, go create the collection.
THE FIX
If you have a collection that is effected by “the bug” (you created a collection first and set a rule for auto adding products) then DELETE that collection and recreate it. Then MANUALLY ADD your products. However, if you have made all your products first, you could try the “auto create” feature. Important note: there is no way to change a collection from “auto” into “manual” you have to delete it and start over.
THE PATCH
Well… if we could get Shopify to come here, read this chain and take the issue seriously I bet they would fix it. But we would need like 1000 people to complain, otherwise they give the brush off. Which I understand, they are a large company after all. And as a developer, I tend to “not believe” most users there is a problem because 90% of the time the user is broken, lol, not the system. But in closing, this time, you are right and the system has a bug.