Have you ever wanted to do pre-production checks before rolling out a full deployment? Sometimes with increasing team size you may find that your branching model leaves potential for your master branch to not contain the up to date code that you tried to integrate via a merge, pull-request or interactive rebase. If you have set up a deployment pipeline that allows you to deploy a certain branch e.g. with Capistrano you can set the deploy branch then you can test deployments from certain branches. For instance you can spin up a UAT environment and deploy the master branch for regression testing before deploying live. This is taken to the extreme when any user can reproduce the deployment environment on demand on another system. With increasing complexity in terms of Service Oriented Architecture and Micro-Services this becomes harder to achieve. Sometimes just sharing a running server shared over a local network could be enough to achieve your testing goals. There are new tools like ngrok that allow us to do this.
There is always one major consideration to be taken into account and that is whilst trying to work with legacy code and features, this becomes apparent when trying to manage the state of the database schema. First of all imagine a situation where a new feature with database requirements adds a few tables to a database, it’s tested on staging but not quite ready for production. However the master branch needs to be regression tested on the staging. In this situation it will be more complex but deploying master to staging will still work. The staging database will have the new tables and data but it will not have the feature in the code base. Following this the reverse situation where code is ready to be removed from the master code base which in turn removes tables from the staging environments. Then master is ready to be regression tested on staging. The code base will break and the regression testing will fail because of the missing data/tables in the staging environments database.
There are however some interesting solutions to this problem. The first is mentioned above that a dedicated UAT environment with a new DB is used for regression testing, then because there is a separate database new features or refactoring to the schema should never be applied to the database. This is probably the simplest solution, especially in conjunction with being able to redeploy fresh environments.
If you can’t get fresh deployment environments for cost or complexity reasons then there are some other novel solutions. The first is quite simple, in the Rails world you can set your staging database.yml to point to the master database, so that the schema matches the database. This comes with certain considerations such as having your live environment and a new environment having concurrent access to the same database. Another solution is to manage the migrations manually in the staging environment, for example, ssh into the staging environment and role back the migrations first before deploying the master branch so that the data schema matches the code. If a mistake is made in this situation it can be harder to recover from if you have a staging environment that cannot easily be reset, i.e. with migrations to be run and seed data that cannot easily be generated.
Other solutions to be taken into account involve not doing any regression testing and relying on a quick role back via a deploy script, force pushing git (a very bad idea) or only doing regression testing on localhost with accurate data (not very accurate). Scripts can be produced which pull a sanitised staging/production database to a local machine to help this last case.
Depending on your git branching model you may not actually get these problems as you may not allow new feature developments on staging to not be in a point to release all together. It depends on your release schedule and your propensity to use continuous deployment. To get a flavour of if there may be some problems ahead do a git diff with master and staging.