One of the blessings and curses of Rails development is the ability to use Rails console for debugging issues and inspecting data. The console is oftentimes used in a production environment, as it is the quickest method to glean information about any problems. With great power comes great responsibility: A command that attempts to reset a local developer environment by deleting all records of a model could easily be input into a production console. In addition, with potentially unknown clipboard data (thanks "copy-on-select"), any valid Ruby code with line breaks will automatically be executed if pasted. Any user who is deleting models, queuing up events, and updating records accidentally or intentionally should be made aware of the implications. At Salsify, we've solved this problem using a combination of open-source and home-brewed improvements and rolled them into a handy gem. Read on to learn more!
A quick search suggests that the solution to all of our woes is to "get in the habit of always running in sandbox mode when using a console in the production environment". Sandbox mode operates in a simple fashion - wrapping the console session in a database transaction and rolling back upon exit which therefore reverts any changes that would have been persisted. This could work, but would require a behavior change across the organization. Launching console sessions in sandbox mode is very prone to human error as sandbox mode is opt-in.
Defaulting to Sandbox
As one can probably guess, we changed it so that sandbox mode is the default mode in production - and having a writable console is opt-in. This means that we wanted a different behavior for each environment, as developer playgrounds (local machine/staging environments) should still default to unsafe Rails console. Fortunately, this was fairly simple as we could simply detect the environment and automatically enable sandbox mode in production.
Now that we have split default behaviors, Rails only comes with a --sandbox flag to enable sandbox mode which left us no way to disable it for the case where we do want dangerous access to production, so we need additional flags to disable sandbox mode:
Great!
Better Transactions Handling
Remember how I mentioned Rails sandbox wraps the entire console session in a transaction, which is rolled back upon exit therefore isolating the console session? Fantastic! Off to investigate an issue with a user...
Oops I misheard what the name column was, lets try that again.
Wait what? F**king Postgres!! Transactions in Postgres are very strict, and break on failed existence checks or syntax errors within a transaction. I suppose I could relaunch the console session. Or I could send a command to rollback the transaction. That would effectively break out of the sandbox though, so can we do better? Of course we can! Transactions in sandbox mode should be automatically rolled back and restarted when they break, eliminating this headache. One additional thought to note here is that, by default, transactions can take out locks as they can queue up data-altering statements. A quick fix for this is to simply inject a 'SET TRANSACTION READ ONLY' when creating new database connections for sandboxed console usage.
Making It Pretty
Here's something hopefully we can all relate to: the 10x average engineer's terminal
I do, and they were a mix of local development, staging, and production consoles for the many microservices here at Salsify. I'd be lying if I said I never typed or pasted the wrong things into the wrong windows. The simple solution is to just sprinkle a little color into the terminal prompt. Color-coding has done miracles in helping toddlers engineers distinguish between toys applications.
Before:
After:
Conclusion
As aforementioned, we've compiled the ideas detailed here and more into a gem.
This is not the only solution, as proper set of admin tools could offer more safety and functionality at the expense of additional development. Strictly only altering production data through data migrations would be an even safer option for mutations. However, the improvements to Rails console has allowed us to keep console usage in our toolkit and help us move fast.
Here at Salsify, we've utilized safer_rails_console successfully to help protect our rapidly evolving applications. Just remember to communicate the change, as we had to deal with a sudden influx of "I made a change from console to fix an issue, but it's not fixed and nothing gets saved. HELP!!"
Thanks to https://travisofthenorth.com/blog/2016/4/8/rails-jail for inspiration for this process and project.