One thing first: We autoamte everything. Deployment, Changelogs, Coverage checks, unit, functional and even integration testing is done automatically.
Currently shopify has circa one line of code in test cases for every one line of code in production. The actual numbers fluctuate a bit because we keep improving our test case methods and replace many lines of test case code with single asset statemens hence shrinking the LOC.
The regression today happend because we tie to the latest SVN version of rails using a facility called svn:externals. When we deploy our code the rails source is automatically updated every time. This is because I’m also one of the rails developers and frequently fix bugs in rails directly which i need for shopify.
However today it bit us in the behind: I ran unit tests before deployment without running an update on the rails code. Everything was fine locally but deployment on our production server farm caused even newer rails code to be fetched which caused urls with points in them to be handled badly.
Thank you for your quick response. You guys have built an awesome product and based on your comments, are also building a very tested tool. Kristy has been raving about shopify since she began working with her storefront and I have enjoyed enhancing it for her as well.
I didn’t see that you responded to the question about defects found by account holders. Do you intend on setting up a channel for collecting defects from the accoutn holders, or would you rather a post to the forum of someone’s findings?
Ah right, There is a feedback link at the bottom of the admin interface. This is a pretty direct channel to me.
I also get paged when there are exceptions in shopify so I’m usually amongst the first to know. Unfortunatly the routing errors were thrown so far up the rails stack ( sorry for the tech talk ) that they weren’t cought by my exceptions loggers.
Might have to take a different route for logging exceptions in the future.