Hey, I’ve noticed more and more often that Sidekick tends to hallucinate quite a bit. So I wanted to ask you: What do you actually use Sidekick for actively, and where do you find it genuinely useful?
The only use case where I’ve personally found Sidekick helpful is searching for specific pieces of code. In almost every other situation I’ve encountered, Sidekick hasn’t been particularly useful.
Here are a few examples:
Sidekick makes up reference variables. When trying to assist with low-code tasks, it frequently references random variables that simply don’t exist.
It also seems to struggle significantly with the logic of Shopify Flow. In our experience, it very often hallucinates incorrect IDs or generates workflows that result in endless loops. This issue doesn’t seem to be limited to us either — quite a few people I’ve spoken to have experienced similar problems.
So I’m genuinely interested in hearing: What are the use cases where Sidekick actually works well for you? Are there specific tasks where you’ve found it to be reliable and genuinely helpful?
One thing I’d like to add before this becomes part of the discussion: We only use AI as a tool for smaller technical tasks. Our actual content is still created by humans.
In my experience Sidekick sucks in areas you’d think it’d excel, most of its weakness is in Admin itself and Shopify in general. It has literally told me Shopify Payments is not active, that there is an error, not just once but 3 times. It just makes sht up lol.
What I can verify that Sidekick is exceptional at is writing code. I have done extensive coding work with Sidekick. But you do have to set the preference of not going to the ai code generator, that thing is just a waste of time. Sidekick itself has a deep knowledge base of liquid, css, Javascript, typescript, even to the extent of over-engineering.
It’s got a huge problem with not caching its conversation, and it doesn’t take that much to make the browser crash.
Yes, it’s true that Sidekick understands the logic and general syntax of code. I didn’t mean to question that. However, it does happen quite often that it hallucinates reference variables, which can make the whole process quite annoying and impractical.
By the way, we’re using a Shopify theme, not a third-party theme. Otherwise, that could have been used as an argument.
I haven’t used it too much to be honest and when I did that was for small technical tasks pretty similar to your experience. I have found it a bit useful if not perfect. The most important limitation is sidekick doesn’t actually edit the code in the file editor, nor makes a section when you are trying to make one, in short it doesn’t touch the theme code at all.
My most recent experience was in changing the compare at prices for a few of the products and even though it could suggest the correct products it wasnt able to edit the products in bulk, what it did do was generated the links for all products so I could just click and edit them but i believe this is something very basic and it should have been able to do that.
So there are limitations yet. Since I haven’t used it too much because of its limitations for longer code work so I don’t have much experience of it hallucinating.
Sidekick is useful for me mostly for quick admin work, to see where a setting lives, to explain Shopify features, or to get a jumping off point for small technical questions. I would be more wary of Flow logic, or really anything with complex conditions even though they aren’t really complex and references. I consider its suggestions as a sketch and double check the real variables, IDs and behaviour of the workflow before making any change. Used like that, it can save some time without becoming a source of unanticipated automation problems.
I’ve found Sidekick most useful when it is treated as a map, not an authority. Ask it to provide the exact Admin path, the setting name, and what will change; also tell it to say when it is uncertain. Then verify the screen yourself before saving anything.
For higher-risk work- billing, Markets, shipping, customer data, or theme code - I’d require a second source such as Shopify Help Center documentation or a reversible test. The warning sign is an answer that sounds precise but cannot identify where the setting lives.
Tim here, from Sonder Sites in Melbourne. We write the code ourselves.
Sidekick is useful as a map, not as the source of truth. For Flow, keep anything it generates inactive. Check every ID and variable against a real order in your store. Then run one pass and one fail test event before you turn it on. Invented reference variables are the usual failure, not a one-off.
If a workflow already looped or wrote bad data, stop it in the activity log first, then rebuild the conditions by hand. Happy to look at a screenshot of the Flow if you want a second pair of eyes.