I am only getting a speed of 17.
"A speed score of 17 means your store is slower than similar stores on Shopify."
I can't seem to speed the site up, and I am thinking of switching themes in order to push the number up.
The theme is just one of the reasons for slow speed. Before switching theme, I'd recommend you optimize your Shopify speed then if it not works, you should consider for a faster theme.
Here is a guide for Shopify speed optimization: 14 Factors for 99/100 Page Speed Scores.
Hope it helps!