Development Operations (DevOps)
- Kevin Lim (Unlicensed)
- Diego Cesa de Meira (Unlicensed)
Introduction
This section documents how the eGIS team develops, tests, builds, and deploys features for eGIS. In this context, feature means any eGIS configuration item that the eGIS team develops, and is not directly correlated with the Feature work item type in Azure DevOps. This section is limited to the treatment of features for creating and configuring the eGIS infrastructure, and does not cover eGIS content development.
Configuration Items
Features comprise one or more configuration items (CIs), which are artifacts that are maintained under version control. We've identified the following types of CIs for eGIS:
- Azure Resource Manager (ARM) templates
- PowerShell scripts
- Desired State Configurations (DSC)
- Python scripts
We'll expand this definition to include additional types (e.g., web apps) as appropriate as we onboard clients to eGIS.
Environments
Nominally, there are 3 eGIS environments:
- Production (PRD): the deployed eGIS environment in which TC users work.
- Staging (STG): an environment that, as closely as possible, mirrors the PRD environment; here, features that ready to deploy to PRD are tested for final approval.
- Development (DEV): features under development are deployed here.
Features are deployed first to DEV, then to STG, and finally, to PRD.
The eGIS team used Azure DevOps.
DevOps Process
The following diagram is a high-level illustration of the eGIS DevOps process. The original document can be found here .
Create Feature Branch
The owner [lead developer?] of the feature creates a new branch for the feature in the EGIS Git repository, according to the guidelines provided in Git branching strategy.
Develop Item
The developer(s) develop the feature in the feature branch. Development includes developing the source code for the feature, along with automated tests, manual test scripts, and documentation.
As the developer(s) develop the feature, they deploy the CIs to the DEV environment, and run automated tests on the DEV environment.
Once the developer(s) complete development on the feature, and all tests pass in the DEV environment, all of the source code is checked into the feature branch.
DEV Build Pipeline
When the feature branch is pused to the server, an Azure DevOps build pipeline is triggered. This pipeline builds the feature branch, deploys the feature* to the DEV environment, and runs the automated tests for the feature.
*if downstream components are dependent on the feature, then those features must also be deployed and tested on the DEV environment; for example, if the Python scripts that configure Portal for ArcGIS in the eGIS environment are updated in the feature branch, then all other features that depend on a properly configured Portal for ArcGIS must also be configured; this likely necessitates different build pipelines for different features
Create Pull Request
When the developer(s) have completed the feature and all tests run successfully on the DEV environment, they create a pull request to intiate the process of merging the feature branch with the master branch. In order to complete the pull request, the code in the feature branch must be reviewed by # other eGIS team members. If the feature passes code review, then it is merged into the master branch; if not, then the developer(s) make the necessary changes in the feature branch. When all issues have been resolved, the developer(s) complete the pull request. Completing the pull request merges the feature branch into the master branch.
STG Build Pipeline
When the feature branch is merged into the master branch, it triggers an Azure DevOps pipeline that builds the master branch, deploys the affected CIs to STG, and runs the suite of automated tests against STG. The pipeline can also include definitions of manual tests that must be performed in STG. These manual tests might include a sanity check to guard against regression and to ensure that major functionality is still available.
If additional re-work is required, then the developer(s) complete the required work on the feature branch and initiate another pull request.
Open Question: Is User Acceptance Testing (UAT) conducted in the STG environment? Should it be conducted earlier (i.e., in the DEV environment)?
PRD Build Pipeline
When all tests pass in the STG environment, then additional steps in the Azure DevOps pipeline are triggered to deploy the feature to PRD.
Open Question: Are individual changes pushed to DEV immediately when they're ready; or, are changes packaged up and released periodically?
Open Question: Who are the groups who have a stake in moving eGIS to Production, and what checks must be included in this step for their satisfaction?