Steps to replicate:
Uninstalling the app doesn't seem to get rid of the issue so I assume the theme version is just cursed for now.
+1, having this same issue!
Installing an app with an App Embed Block into a Development Store seems to automatically enable the Embed Block, regardless of what the actual theme settings are; turning on the block in theme settings then loads a second copy of the embed block.
Haven't been able to find a resolution so far; any input from a Shopify dev would be appreciated!
The way to fix this currently is to manually edit config/settings_data.json from your theme and remove the additional embed block.