Shopify report "Online store speed" and with tools such as Google PageSpeed Insight, GTMetrics, Pingdom Tools or crawlers like Screaming Frog and Ahrefs.com you can also analyze site speed.
But why are the numbers (often) not the same? Are some of the tools wrong?
No, all the above tools are actually good. But the data they report on is just not the same and in order to truly understand your sites performance you need to understand the difference.
The biggest and most fundamental difference is "lab-data" and "field-data"
Most of the tools above is based on lab-tests. It's a single test in a controlled environment. That is true for the Shopify speed report too.
Lab-tests are good if you want to test specific pages under specific conditions - such as mobile users with a 3G connection, no cookies set and connecting from USA. Just kep in mind that the data may be totally different for another user with different conditions.
Field-tests, on the other hand, is based on how well your website actually perform for the users that visit your website. Shopify do not report that.
The largest data-set for field-data is found in Googles Chrome User Experience Report (CrUX). It is what Google use in the upper part of the PageSpeed Insight report.
Here is an example with partakefoods.com (no affiliation)
At first sight the data looks OK so why is the (mobile) score only 21?
To understand this we have to look at the lab-data as that is what Google use for the speed score. The lab-data is listed below.
As you can see the lab-data is far from as good as the field-data. So what does this mean and how do you deal with it?
There can be many reasons for the differences. It can be because the site have a lot of repeat users that have already cached some of the largest resources in the browser. So for them the site is pretty fast - but for new users it may not be. That IS a problem if you want to attract new customers!
So the lab-test can help you zoom in on issues like this - how do the average user experience your site versus new users or users outside your primary regions. The better it can perform for all users the better for both SEO and sales.
This was just a short explanation. There are off course many more important details to all this site speed stuff. But for now I just wanted to explain the basics of that important difference between lab- and field-data as it confuses so many people.