Product handles with international characters are transliterated for some languages

[Maris]

Hello!

Merchants started to complain about how Product handles are working with international characters. Don't know how long has it been working this way, but it was definitely different some time ago.

If you create a product with non-ascii symbols in the handle, then for some characters Shopify accepts them fine, but for the others, they are being transliterated (replaced with other English characters).

For example, handles with values like "ёжик" or "一辆车" work just fine (handle is assigned as such, and the product URL also has this handle). While "mašīna" will be changed to "masina" (e.g. characters of Nordic and Baltic countries).

This is causing serious issues for our Merchants, so we would like to know if there is anything we can do to save handles with any valid UTF-8 symbols.

An additional issue is that we also need to search products by those handles and if they have been changed, we just can't find them. I know, that the API returns the changed handle in the response, but in our particular case it doesn't help much.

If this new logic of transliterating some language characters to English is permanent, then please describe the logic - what characters are being transliterated to what, and what are kept as the original. So that we can replicate the same logic on our end, when searching for Products by the Handle.

Ideally, of course, if Shopify could have this same transliteration applied when searching by the Handle - then, if we search for handle "mašīna", then it would search by the same logic as it creates the Handle, hence, "masina".

Here's how it looks in the REST API call:

D, [2021-11-20T15:32:54.017654 #13612 8460] DEBUG -- : [httplog] Connecting: excelify-andris-3.myshopify.com:443
D, [2021-11-20T15:32:54.880480 #13612 8460] DEBUG -- : [httplog] Sending: POST http://excelify-andris-3.myshopify.com:443/admin/api/2021-10/products.json
D, [2021-11-20T15:32:54.880624 #13612 8460] DEBUG -- : [httplog] Data: {"product":{"title":"Product 123","handle":"mašīna"}}
D, [2021-11-20T15:32:54.880686 #13612 8460] DEBUG -- : [httplog] Status: 201
D, [2021-11-20T15:32:54.880742 #13612 8460] DEBUG -- : [httplog] Benchmark: 0.83782 seconds
D, [2021-11-20T15:32:54.880823 #13612 8460] DEBUG -- : [httplog] Response:
{"product":{"id":6599629537332,"title":"Product 123","body_html":null,"vendor":"Excelify Andris 3","product_type":"","created_at":"2021-11-20T15:32:54+02:00","handle":"masina","updated_at":"2021-11-20T15:32:54+02:00","published_at":"2021-11-20T15:32:54+02:00","template_suffix":null,"status":"active","published_scope":"global","tags":"","admin_graphql_api_id":"gid:\/\/shopify\/Product\/6599629537332","variants":[{"id":39492014112820,"product_id":6599629537332,"title":"Default Title","price":"0.00","sku":"","position":1,"inventory_policy":"deny","compare_at_price":null,"fulfillment_service":"manual","inventory_management":null,"option1":"Default Title","option2":null,"option3":null,"created_at":"2021-11-20T15:32:54+02:00","updated_at":"2021-11-20T15:32:54+02:00","taxable":true,"barcode":null,"grams":0,"image_id":null,"weight":0.0,"weight_unit":"kg","inventory_item_id":41586303860788,"inventory_quantity":0,"old_inventory_quantity":0,"requires_shipping":true,"admin_graphql_api_id":"gid:\/\/shopify\/ProductVariant\/39492014112820"}],"options":[{"id":8488440528948,"product_id":6599629537332,"name":"Title","position":1,"values":["Default Title"]}],"images":[],"image":null}}

You can see that we have sent a "mašīna" handle, and REST API returned the handle "masina".

This causes a related issues (even disasters) if we update an existing Product that has international characters in the handle, with ShopifyAPI ruby gem - if someone just saves the product back (changing for example just the tags attribute value), then it will also change the Handle to English transliteration.

That is super important issue and will cause many Merchants not only frustrations, but messing up their store. So this needs to be addressed seriously, and fast.

Thank you so much!

Maris