DevSecOps: Integrating Security into DevOps

With data shaping the business landscape today more than ever before, security issues are at the forefront of everything a business does. Being ignorant about the risks of system vulnerability is detrimental no matter which industry you’re working in, but especially when you’re dealing with large amounts of consumer data. Financial, healthcare, and many other organizations are required to undergo obligatory security certification processes to prove they are compliant with all the necessary industry standards and regulations. 

However, security checks are often considered a bottleneck to deployment because they typically happen at the end of the delivery lifecycle or even after release. These checks are often manual; detecting issues means unplanned work for dev, test, and ops teams, causing delays and frustration.

Fortunately, there’s a way to make security cheaper and, at the same time, avoid time-consuming processes and hindering system development. The solution is DevSecOps. It aims to achieve a secure SDLC and a CI/CD pipeline all the way through from start to finish by shifting security “left” to the earliest stages of the project so that the reliability of your system is no longer an area of concern.

DevSecOps vs DevOps: a fresh look at the security problem or the same thing with another name?

There are two opinions on the term DevSecOps and its place in DevOps.The first one is that to include security in the software development lifecycle from the very beginning, we need an explicit call to action. Many people take the “DevOps” label too literally and think that it encompasses only development and operations. Hence, creating “DevSecOps” looks like a good opportunity to highlight the importance of the security role.  

Good symbols, labels, and stories change the world. The pithiness of “DevOps” drove mass adoption and actual improvement far more than the “Agile System Administration” movement that preceded it. DevSecOps is fine.

— Nigel Kersten, Field CTO, Puppet  

The second view is that DevSecOps shouldn’t exist as a separate label because security is an integral part of DevOps already. 

If we keep putting every responsibility people should do in the name, we’ll run out of room for the hashtag. “DevSecOps” is dumb. #DevSecITSMTestAutomation­­MonitoringObservability­­­PeopleFinanceMarketingQAOps.

— Michael Stahnke, Director of Engineering, Puppet

Sometimes the idea of shifting security to the left may go as far as contradicting SecDevOps vs. DevSecOps. Perhaps you’re thinking: “What?! Are you kidding me?” No, we aren’t actually. Anyway, let’s not juggle the words and just agree with Bill, not Gates, but Shakespeare, “…that which we call a rose by any other name would smell as sweet.” The real issue to solve here is how to deal with the silos between security and DevOps teams? Because perhaps, only in a parallel universe could engineers and developers be okay with waiting for 48 hours while the security team runs their tests. So, then what are the middle ground solutions that DevSecOps practices can offer? 

Fighting against the deadly waterfall: why DevSecOps is a savior

With a traditional development method, such as the waterfall model, you usually can’t go back to the previous steps to modify a project. Security testing is tucked at the end of the SDLC. 

But, what are the consequences of such an approach? Significant security problems are detected only at the last stage of software development. Fixing them is painful for the team, and costly for the business owners as it results in delayed delivery. To deal with this problem, the agile methodology was invented. It allows businesses to minimize risk when adding new functionalities. And with an iterative method, it’s easier to be aware of security during the whole development process because you can go back to the previous stage and quickly fix a bug, monitor cost overruns, or change requirements earlier. With such an approach, you minimize the risk of a small mistake turning into a snowball that cripples the whole project, as it happened with SolarWinds. The company reported that up to 18,000 of its clients installed insecure updates and became vulnerable to hackers. Considering SolarWinds has many high-profile customers, such as agencies in the US government and Fortune 500 companies, the situation was quite critical for the organization and incredibly beneficial for its competitors. 

Outcomes of integrating security into DevOps in the long term:

• Accelerating deployment frequency. Even if initially it doesn’t seem like that, the more you learn how to interact with security throughout the entire SDLC, the more frequent your deployments to production become. The case of NIAID proves that DevSecOps practices such as IaC (infrastructure-as-code) and automated testing are helpful in shortening the lead time to deliver software and patch critical defects. But as usual, when you’re changing how you work, things get worse before they get better. Early stages of integration are troublesome as security practices are introduced into stages where they weren’t before. Delivery speed takes a hit, too, and that’s frustrating for all involved. After all, who is happy about deployment time being increased by a third? These problems eventually go away as teams collaborate more smoothly to embed security in the delivery cycle, refine their processes, and see the positive outputs of their work. 

• Decreasing time to remediate critical vulnerabilities thanks to DevSecOps automated security testing. Meanwhile, without DevOps or DevSecOps integrated into an organization’s development lifecycle, error fixing is manual or, at most, only semi-automated.
• Easier risk mitigation and flaw prevention. You are more likely to stop a known-vulnerable code being pushed to production by giving this responsibility not to a centralized security team but to a delivery team. Thus, you make the process faster by removing a bureaucratic constituent and improve decision-making by relying on people from the delivery team who use their knowledge of both the technology and the business to do what is best for the company and the customer. When responsibility for security is shared across delivery teams, rather than siloed within one team, security issues are caught earlier — there are more eyes looking for potential security threats. It costs much more to fix a bug found during regular maintenance than to fix one identified during the design.

Building a DevSecOps pipeline within a SDLC: theory and reality 

Efficient security implementation into the DevOps pipeline is a tricky task. According to the GitLab global survey results, 72% of 4,300 respondents described their security level as good or strong. Simultaneously, in almost a third of organizations (30.73%), only the security team is in charge of security. So, organizational silos are still a relevant problem.

There are two options for creating a DevOps security pipeline:

• Using a traditional DevOps pipeline with security checking tools implemented at every stage: Plan – Code&Build – Test – Release – Deploy – Operate&Monitor.
• Building a DevSecOps pipeline: Threat modeling – Scan – Analyze – Remediate – Monitor. 

In theory, it’s easier to create a pipeline with integrated security when you are only starting the project rather than implementing security checks into the existing DevOps pipeline as security becomes a matter of routine from the beginning. But let’s face the reality, the thing is that barely anyone truly cares about security before the preproduction stage.

According to the Sonatype survey, 48% of developers know security is important but don’t have enough time to spend on it. It doesn’t mean that it’s deemed unnecessary, but a lot of other issues with a high business priority and value are waiting to be resolved. So, then how does the process work?

When you start building a pipeline, you only have an idea of the final product. So you have to code and build something as fast as possible. It means, first of all, a business owner invests money in development, operational, and sales teams. DevOps security, at this stage, is only a rainbow unicorn perspective. Security implementation from the start is too costly for businesses. It requires specific tools and specialists to set them up. It’s hard to find additional thousands of dollars just for security when you don’t know if the product will be successful. 

The desire to turn a blind eye to security checks when you are caught in the crossfire of deadlines and frustrated employees is totally understandable. You may sleep well for many years with your software functioning just fine until one day you awaken to mind-boggling downtime instead of peace and quiet.

Basic actions you can take for DevOps pipeline security in any case 

Underlining the obvious importance of integrating security into DevOps is a kind of “thanks, Captain Obvious” advice. It’s easy to say and hard to master. That’s why our goal is to show how to integrate security into the DevOps pipeline seamlessly without creating a drag in release times and hold up the deployment cycle, and, of course, without spending a huge part of the project’s budget on it. 

Do threat modeling and risk assessment

This practice will help you deepen the understanding of the weak points in your DevOps security, the types and sensitivities of your assets, and how to protect them. You can do threat modeling even before you shift to DevSecOps. It’ll provide you with:

• Inventory of sensitive data
• List of vulnerabilities with possible migration options
• Summary of potential attack scenarios

With threat modeling, you kill two birds with one stone: eliminate vulnerabilities in the DevSecOps pipeline and improve the security knowledge within the development and operational teams. At first sight, threat modeling may seem quite a time-consuming process that affects the speed of deployment. But it won’t be an obstacle if you analyze which types of attacks are more likely to happen beforehand and choose the appropriate security checking tools. 

Tools to check how secure your SDLC and CI/CD pipeline

Automated security testing is a key component of the successful implementation of DevSecOps. With that, the speed of deployment will be affected minimally. Specially designed tools can provide you with static, dynamic, and interactive analysis of CI/CD pipeline’s security.

What are these tools specifically?

1. SAST (static analysis security testing) software is used for white-box security testing (the “developer approach”) to check the security of the DevOps pipeline from the inside out during the building phase. You have access to the underlying framework, design, and implementation of the software. 
2. DAST (dynamic analysis security testing) tools are needed for black-box security testing (the “hacker approach”) to prove the system’s security from external attacks outside its environment during the testing phase. In this case, you don’t have access to the underlying framework, design, and implementation of the software.
3. IAST (interactive analysis security testing) works inside the product and analyzes code for security vulnerabilities in real-time during the QA or testing phase. It may seem like a win-win situation as far as you check security and don’t add extra time to your CI/CD pipeline. But remember that IAST tests aren’t always suitable for testing a codebase or an entire application. They only check whatever is exercised by the functional test, so you can select the activity that is a part of continuous integration, and check how secure it is. The best use for IAST tools is in combination with QA tests. 

Define why your company needs continuous security monitoring for DevOps because security for security’s sake is a trap and a waste of time and money. First of all, specify your security priorities, choose testing tools accordingly, and decide on the phases of the DevOps pipeline where you’d like to implement them.

DevSecOps automated security testing is a heavy hitter in any sphere but there are three industries where security plays a crucial part: Finance, Healthcare, and Politics. The worst thing that can happen to a bank or a medical lab isn’t downtime. It’s data leakage. That’s why bank staff may not even have permission to install unrequired programs on computers. And if a database of a medical laboratory is attacked, executives are likely to shut down the whole infrastructure until the breach is found. It means that customers won’t get the results of their analyses or be able to book an appointment for 1-2 days minimum. But the risk of a hacker publishing customers’ data or using it against them is much worse than negative reviews.

Finding the middle ground between security level and deployment speed 

By putting speed-to-market on a pedestal while ignoring other DevSecOps objectives, you risk leaving a lot of value on the table and, more importantly, you imperil your entire business by jeopardizing customers’ data. Without security incorporated into your SDLC, users will suffer from repercussions caused by the unreliability of your system. That’s why prioritizing security is the key to better outcomes overall.

In theory, the ways of seamless integration security into the DevOps pipeline are clear and understandable. But once you start putting them into practice on your own, reality might kick in. If you have fallen into the trap of security implementation challenges, *instinctools security experts are ready to help.