I have raised this issue with Shopify support, they have done some testing and managed to replicate my results.
They agree that this is an issue and have put it on the development list.
Make sure you reach out to Shopify and let them know if this affects you and how you would like the speed test to work for you in the future.