12 years later and this problem is still out there... Wow.
I do have a solution though, for anyone else that comes across this post. Arigato can easily scan your entire library of products and detect if the term "copy of" is in the URL. From there it can either document the issue in a spreadsheet so you can change it manually, or just update it for you. There's a library workflow available for it here: https://www.apps.bonify.io/arigato-automation-shopify/workflows/detect-copy-handles
There *IS* a quick and easy way to update it. I couldn't find this anywhere on the forums so I hope this helps some people.
Go to Products -> All Products
Select All, then click Select All 50+
Remove all fields except for url handle
Click the first entry in the url handle field, scroll down to the bottom of the first page of 50, hold shift and click the last cell.
Hit 'DEL' and then your url handle title will be updated to the current title of the listing.
Click Save, it will update them all, click next page and repeat the above 3 steps of selecting and deleting the old url handles in bulk.
I just finished updating about 100 manually when I found this out, so I hope it saves you guys some time too.