Hi there 
A setup like this can be achieved quite smoothly with Easify Custom Product Options 
Since each color in your store is created as a separate product, Shopify won’t automatically connect them together as color variants. However, you can create a swatch-based navigation experience that allows customers to easily switch between different color products directly from the product page.
I’ve created a similar demo to help you better visualize how it works:
First, you can keep your current setup where each color is created as a separate product in Shopify. This allows every color to have its own images, URL, and dedicated product gallery while still maintaining size variants within each product.
Then inside the app, you can create a Color Swatches or Image Swatches option to represent the available colors. For the size selection, simply create another option using Buttons or Dropdowns and add your available sizes such as S, M, L, XL, etc.
Next, use the Add/Edit Cross-product Links feature within the Color Swatches option. There, you’ll simply paste the corresponding product URL for each color. For example, the Black swatch links to the Black product page, the White swatch links to the White product page, and so on.
Once configured, when customers click a color swatch, they’ll be automatically redirected to the corresponding product page for that color. This gives shoppers a seamless way to browse between colors while allowing each color to maintain its own product gallery, images, and inventory setup.
This approach works particularly well for stores that prefer managing each color as a separate product but still want the familiar color-swatch experience customers expect. Everything can be configured directly inside the app without requiring custom code or complex theme modifications 