“Companies that embrace the principles of Software Factory (DevOps) experience 60 times fewer failures and can deploy 30 times faster than organisations that don’t follow these principles.”
State of DevOps Report
The Safe & Secure Software Factory (SSSF) merges the principles of DevOps, Safety Criticality, Cyber Security and Industrial Manufacturing. SSSF enables you to create a completely automated process for developing & integrating software to the highest standards and compliance in your organization.
Earlier detection of issues
One of the biggest challenges with software development today is the unintended propagation of defects or issues through the system development cycle. If we consider the simple analogy of our tooth cavities, it is well understood that prevention is better than correction. This holds true for software where experts have proven it is 2000% more cost-effective to prevent mistakes than later detect and then correct them.
These issues can often be identified very early in the development cycle. But they are missed because software changes are merged without the adequate verification and validation in place. Resulting in a huge impact on both cost and schedule.
To address this quality issue, there are various discussions on the topic of ‘Shift Left’ (i.e. test earlier in the process). The introduction of Test-Driven Development, Agile, etc. attempt to facilitate the introduction of testing new functionality at the time of creation, versus being delayed further in the cycle. However, these topics today still focus on enabling the developer, but do not address the integration of the system components, or the updating of features and the side effects they may produce on existing software in the system.
A growing problem
Today every aspect of software is growing rapidly; codebase, complexity, infrastructure and the teams. This growth is much like Moore’s law: exponential. Improved integration is the secret to managing this growth. A traditional “develop then test” approach is no longer feasible in today’s competitive market.
As the famous quote says, “A computer lets you make more mistakes faster than any invention in human history-with the possible exception of tequila.”
Toyota Product System
Fortunately, we can take inspiration from lean manufacturing principles and apply them to our Safe & Secure Software Factory. We can look at the Toyota Production System (TPS). Specifically how it tightly integrates quality control principles to its manufacturing process through the principle of Jidoka (自働化).
Jidoka is a term commonly used in lean manufacturing. It is widely considered one of the pillars of the Toyota Production System, the other being Just in Time (JIT). This quality control process applies the following four principles:
- Detect the abnormality.
- Stop.
- Fix or correct the immediate condition.
- Investigate the root cause and install a countermeasure.
TPS aims to prevent the production of defective products, eliminate overproduction and focus attention on understanding the problems and ensuring that they do not reoccur.
Quality and Security Assurance Gateways
In our journey to improve our mature levels for our Safe & Secure Software Factory, we can apply the concept of Jidoka to our workflow. We introduce quality and security assurance (QSA)-Gateways earlier into the software development life cycle (SDLC). Thus we ensure every code delta added into the software repository has acceptable quality in its isolation prior to merging into the main release branch.
The Safe & Secure Software Factory, much like a modern-day factory, decomposes the construction of software into independent components. These components are developed incrementally with several quality gate milestones. For each component to be integrated into the final system, it MUST pass all quality gates. Maximize the value of this approach: The smaller the development increment (e.g. a developer commit), the easier it is to identify the root cause of a failure at a quality gate. And hence the faster it is to remediate and fix.
In a typical development environment, there might be 30-40 commits per day by the development team. Without a highly automated process in place, it would be infeasible to gate every commit. The Safe & Secure Software Factory introduces a highly automated pipeline. This allows the complete automation for analyzing every commit incrementally and using automated software quality gates to accept or revert every commit to be integrated into the mainline.
As an example, given that developer A creates a branch for their feature (isolates their change). The developer then makes this branch available on the server repository. ( E.g. available to the team and thus our Continuous Integration server). We can now leverage the full power of automation to verify developer A’s changes before they are allowed to merge them into shared branches. Such as the release branch, where developer B will inherit code from. This checkpoint in our workflow is managed by an automated QSA-Gateway.
Building the Safe & Secure Software Factory
Before we start any journey, it is important to understand how well these elements of our software development life cycle are working together through a maturity perspective of the development workflow today. We will consider 5 levels of maturity to capture this:
| Maturity Level | Defining Practices | Outcomes |
| 1 | No DevOps | Outages, war-rooms, blame, unplanned work, delays and defects |
| 2 | Beginning DevOps | Thinking about cultural change, starting to write scripts, looking at test automation |
| 3 | Fundamentals Achieved | Automated build, cross-functional teams, product-focused, cultural change happening |
| 4 | Monitored DevOps | Happy people with integrated toolchain to pre-empt failure, automate test and deployment – Continuous Delivery |
| 5 | Fine Tune | DevOps DONE – fine tuning and tied tightly to business goals |
As we consider the outcomes at each of the maturity levels, the first and most critical step is to get a QSA gate in place as part of the pipeline. Initially, the QSA gate may check very little, but as an organization progresses up the maturity levels, more and more checks can be introduced into the QSA gate.
This incremental approach will allow you to create a Safe & Secure Software Factory in your organization, for high-quality development and maintenance of your software portfolio.
