<img alt="" src="https://secure.leadforensics.com/150446.png " style="display:none;">

Magento 2 Pipeline Deployment with minimum downtime

Pravin Patil Sep 25, 2018

Magento 2.0 Pipeline Deployment

magento 2-pipeline-deployment
Magento 2.2.x has introduced a new way of deployment process called "Pipeline Deployment". Magento has recommended this deployment process for large sites that prefer minimum downtime during deployment. This process also helps us to maintain consistent configurations for different environments.

Using pipeline deployment approach we can minimize the downtime by deploying static assets and compile code on a machine separate from the production system. Production system requires limited downtime to move files from the built system to production system.

Advantages of Pipeline Deployment:

  • Build step no longer needs database access
  • Faster compilation and asset generation
  • Compact web assets reduce bundle and extract time
To implement this process we will be using the below systems:

Development System

The machine on which developer can write code, customize third-party extensions and push code into the repository.

Build System

System on which we can pull the latest code from the repository, deploy static assets and compile the code. As we build an asset on this system, it helps us to minimize production downtime.

Staging System (Optional)

It is an optional system, we can use this system for UAT and this is identical to a production system.

Production System

This is the live system on which we can pull the latest code from the repository and also import all configuration changes.

Configuration Management

Magento now stores all configuration details in .config files, which helps us to make configuration consistent across all above systems. Magento supports three different types of configuration:

  • Shared Configuration
These are all settings that we want to be consistent in both development and production environments. Magento has provided utility to dump all configuration settings in app/etc/config.php which can be pushed into the code repository and shared between different systems.

  • System Specific Configuration
These are all settings which cannot be pushed into the repository and should be system specific e.g. Configuration of third-party systems.

  • Sensitive configuration
All sensitive settings go into app/etc/env.php and which cannot be included in the code repository. The information like DB credentials, API keys and personally-identifiable information which is environment specific and different environment have different configuration settings.

Recommended pipeline deployment workflow:

The following diagram shows how to deploy on a different system with managed configuration.

pipeline-deployment-workflow
Note: Once we run PHP bin/Magento config: dump command, it will lock all the config settings in Magento back office. If you want to change the specific setting then you have to run a command which will update settings in app/etc/config.php which pushes it into the repository and then you can import the setting production using PHP bin/Magento config: import.

Below is the step by step guide to build a different system:

As I have created two different branches - master and develop, the reason behind this is to maintain different .gitignore settings on different systems.

Development System

Step 1

Create new folder in your document root /var/www/HTML/magedev and change directory to magedev

Step2

Pull latest Magento code from Magento repository
composer create-project --repository-url=https://repo.magento.com/ Magento/project-community-edition.

It will prompt you to enter public and private key.

Step3

Run composer install

Step4

Create new database eg. magedev

Step5

Run Magento setup and provide required details and make sure that it runs on development system http://localhost/magedev

Step6

Push your code into develop branch and make sure that below are included in .gitignore and app/etc/config should be the part of code repository.

vendor directory (and subdirectories)
generated directory (and subdirectories)
pub/static directory (and subdirectories)
app/etc/env.php file

Step7
Do the required settings in Magento back office and run below command to export all configuration settings and dump them into app/etc/config.php
php bin/magento app:config:dump

Step8

Write/Modify the code and run below command to generate assets and make sure that changes are working as expected.

php bin/magento setup:upgrade
php bin/magento setup:static-content:deploy -f
php bin/magento cache:flush

Step9

Push the modified changes including config.php into the code repository.

Build System

The Build system does not need to have DB and Magento software installed.

Step1

Pull the latest code from the developing branch and merge it into the master branch.

Step2

Modify .gitignore and make sure that below are included in the code repository.

app/etc/config.php
generated directory (and subdirectories)
pub/media directory
pub/media/wysiwyg directory (and subdirectories)
pub/static directory (and subdirectories)

Step3

Run composer install and make sure vendor/ folder is up to date

Step4

Generate static asset and compile code using below command

php bin/magento setup:di:compile
php bin/magento setup:static-content:deploy -f

Step5

Push all modified files from generated/ and pub/ folder into the master branch

Now we are ready for production deployment.

Production System

Write a simple shell script to execute below commands in sequence.

php bin/magento maintenance:enable
git pull origin master
php bin/magento app:config:import
php bin/magento setup:upgrade --keep-generated
php bin/magento cache:flush
php bin/magento maintenance:disable

Note: Once we execute the above commands then do not forget to restart web server and varnish server (if it is configured)

The assumption from Magento regarding Pipeline deployment:

  • System integrator should have an excellent knowledge of Magento configuration options.
  • The integrator should have some experience in managing large Magento sites and has the know-how of how to keep production site downtime to a minimum.
  • He or she should have a deep knowledge of PHP programming.
  • He or she is experienced with source control systems like Git.

e-Zest is a leading digital innovation partner for enterprises and technology companies that utilizes emerging technologies for creating engaging customers experiences. Being a customer-focused and technology-driven company, it always helps clients in crafting holistic business value for their software development efforts. It offers software development and consulting services for cloud computing, enterprise mobility, big data and analytics, user experience and digital commerce.