Releases

Conceptually, this difference in the operation of the last part of the de- ployment pipeline is the major difference between continuous delivery and continuous deployment although there has been some ambiguity in the use of the terms and their meaning has been mixed over the years (Fitzgerald and Stol, 2014, 2017).

An automated deployment pipeline can make the task of releasing new software versions to the end users easier, especially so if the pipeline extends all the way to the production environment as it should in continuous de- ployment. Releases and releasing changes, however, can take many forms.

Strictly speaking, if a deployed software change or feature that is part of the release cannot be used by the end users or is not otherwise available, the change has not been released yet. Forms and modes of release exist where this is exactly the case as new code and other changes can be silently de- ployed to production but associated features remain disabled for end users with the help of feature flags or then the whole feature is built incremen- tally step-by-step with dark launches (Parnin et al., 2017). Other options include releasing changes only to some users but not to all of them with canary releasing (Humble and Farley, 2010; Adams and McIntosh, 2016;

Parnin et al., 2017).

Beyond disabling individual features and releasing versions only to a small subset of users lies the most experimental release and deployment methodologies where end users are exposed todifferent versions based on a certain logic in dividing users into various groups. Experimental methods like A/B testing are part of this continuum in which implicit user feed- back and telemetry from the production environment guides development efforts (Adams and McIntosh, 2016). In continuous software engineering, continuous innovation and continuous experimentation are linked to work- ing methods that combine frequent releases of continuous deployment with user centered experimentation (Fitzgerald and Stol, 2017). Such methods form the basis of experiment systems that are at the top of the conceptual user centered agile software development staircase (Olsson et al., 2012) – methods that require the capability to deploy and release often to work in a sensible manner. The next section covers some of the release models that can be used in conjunction with continuous deployment.

Production

Developers see and feel Users

Rapid change merge Invisible to users

Deployment –dark launch release

🎭Staging 0 %

(a)Dark launching

Production

Developers gather feedback Users Visible to some users

Deployment –canary release

🎭Staging 10 %

(b) Canary releasing

Production

Developers gather feedback Users Visible to all users

Deployment –full release Feature flags configurable

🎭Staging 100 %

(c) Full release

Figure 2.3: Software changes deployed to the production environment can be hidden from all or some users with dark launching and canary releasing models to prepare for a full release (Humble and Farley, 2010; Adams and McIntosh, 2016;

Parnin et al., 2017).

Every change that is ready to be deployed or that is alternatively deployed directly to the production environment, however, does not form a mean- ingful feature that could readily be taken into use by the users (Humble and Farley, 2010). Dark launching andfeature toggles support such partial deployment of changes and provide the minimal exposure or no exposure at all to users (Adams and McIntosh, 2016; Parnin et al., 2017). Gradual rollouts withcanary releasing are useful to making the changes available to some, but not all, users to determine whether the changes work well enough and are received favorably by the users (Humble and Farley, 2010; Adams and McIntosh, 2016; Parnin et al., 2017). A change that is made available to all users is a release that has possibly gone through the various release phases but developer and release teams can also skip all of the intermedi- ate phases by deploying the change to the production environment without restrictions on user visibility. Figure 2.3 illustrates the main features of the various release models starting from dark launching (Figure 2.3a) and canary releasing (Figure 2.3b) that are useful in preparing a full release (Figure 2.3c) later on.

Dark launching and feature flags defy logic at first glance. Why would anyone want to make a release that has no evident effect on how the system works from the users’ perspective? One reason lies with the hardships in- volved in integrating work from version control system branches that have not been integrated to the main line of development in a while (Mårtensson et al., 2017). Developers practicing continuous integration saw benefits in integrating often to the main development branch with small batches of changes, and the same applies in continuous deployment with dark launch- ing. A smaller set of changes is easier to merge and large architectural

changes in software architecture less painful to piece together by deploying the changes often by the principle of dark launching (Parnin et al., 2017).

Likewise, feature toggles or feature flags allow the same flexibility as dark launching, making it possible for developers to try out and run code in the production environment by including the necessary software services in the set of changes deployed to production but hiding components from the users if certain Boolean expressions or other conditions in code evaluate to true or false (Adams and McIntosh, 2016; Parnin et al., 2017). Feature flags can be used either at compile time or at run time. Compile time flags that are disabled by setting the condition to false might not end up in the compiled code at all depending on the compiler. Run time feature flags are evaluated when the code is executed so the value for the feature flag – whether to show or hide a feature – can be read from a dynamic configuration source like a configuration server. What is helpful in development is also helpful when a released feature is not working as expected since the configuration source can be used to disable the feature without making a new release to fix the possibly faulty feature.

Release strategies where the changes are deployed and released to a share of the users while withholding the changes from others derive from the principle of canary releasing (Humble and Farley, 2010; Parnin et al., 2017). The decision which users get to interact with the released feature can be based e.g. on the geographical region where the user is from (Adams and McIntosh, 2016). Canary releasing functions much like a dynamically configured feature flag: if the result of the evaluation is true, the feature is shown to the user and if it is false, the feature is hidden. The configuration and evaluation of a release canary feature might just involve more logic and the value of the configuration is not globally the same to all of the users.

Monitoring the production environment closely is a necessity with release canaries. Tracking feature usage through telemetry is needed to see how the users interact with the features and to see that the new features are not causing any system failures (Adams and McIntosh, 2016; Parnin et al., 2017). The general objective of canary releasing is to gather implicit or explicit feedback from the users (Humble and Farley, 2010).

Eventual full releases to all of the users follow the intermediate release phases if everything seems to be in working order and the quality metrics from the production environment look acceptable. Telemetry data is still useful when a release has been done to catch the occasional program ex- ception or two and to generally get a feeling of how new features are being used after a release (Parnin et al., 2017).

Continuous deployment as a practice does not strictly dictate whether to silently push changes to production with dark launching, or to use re- lease canaries to try out new features with a limited audience, or to deploy and release all changes without prior experimentation on users. The gen- eral expectation for continuous deployment is to have changes deployed and released quite often (Fitz, 2009). Since the release frequency is not locked as such, it has been suggested that the concept ofcontinuous releaseshould be added to the family of continuous software engineering practices to dif- ferentiate between deployment and actual release of changes (Ståhl et al., 2017). Continuous deployment still encourages developer teams to release more often, turning months between releases into weeks or even days (Chen, 2017).