News
DevSecOps Expert: Use 'Stages, Not Gates' to Secure Fast-Moving Pipelines
Modern software delivery pipelines can move code from a developer's workstation toward production quickly, while pulling in third-party libraries, container images, infrastructure code and cloud services along the way. That speed creates a scaling problem for security practices built around manual review near the end of the process.
At today's online Best Practices for Modern DevSecOps Summit hosted by Virtualization & Cloud Review, Carlos Rivas, an AWS-Certified DevOps Professional, cloud consultant, software developer and AWS instructor, presented an alternative: make security part of the normal delivery pipeline instead of a checkpoint that developers encounter just before release.
His session, "Shift Left, Secure Right: Embedding Security from Code to Cloud," followed software through four broad phases -- code, build, deploy and run -- with different security controls applied throughout. Rivas used Amazon Web Services (AWS) and GitHub examples from his own work, while noting that the overall approach also applies to other cloud platforms and traditional datacenter environments.
"So everything is a lot cheaper to fix as early as possible in your deployment pipeline."
Carlos Rivas, AWS-Certified DevOps Professional
'Stages, Not Gates'
Rivas framed the problem around continuous integration/continuous delivery (CI/CD) systems that can push changes rapidly while a security team still depends on manual review. For organizations deploying frequently, he said, that approach can become a bottleneck. "The proposed solution that I have for you guys is to do things in stages, not in gates," he said.
[Click on image for larger view.] Security Checks Across Pipeline Stages (source: Carlos Rivas).
The idea is to distribute checks across the delivery process and automate them where possible. At the code stage, for example, Rivas discussed triggering checks from a developer commit rather than requiring the developer to leave the normal workflow and run separate security tools. He used GitHub Dependabot as an example of checking third-party dependencies after a commit and surfacing known vulnerabilities without adding a separate manual step.
He applied the same model farther down the pipeline. Commit-time checks can look for credentials and insecure patterns; build-time controls can scan dependencies; test stages can check infrastructure as code and application tests; artifact stages can scan and verify container images; deployment can evaluate policies and approvals; and production systems can continue monitoring network and API activity.
Rivas also argued that security findings should appear where developers already work. He contrasted findings deposited in a separate security portal -- where they can sit in another backlog -- with findings attached to a pull request while the code is still open and the developer is already working on the change.
'Most of What You Ship, You Did Not Write'
Rivas devoted particular attention to the build stage and software supply chain, describing third-party components as a major source of security exposure. Applications can include libraries, web servers, packages and multiple layers of container images that the development team did not write but still ships and operates.
[Click on image for larger view.] Reducing Software Supply Chain Risk (source: Carlos Rivas).
For container workloads, he recommended starting with a minimal base image rather than a feature-heavy image loaded with tools. The same utilities that make troubleshooting convenient, he noted, can also become available to an attacker who compromises the container. His approach is to minimize the base, scan images as they are pushed into a registry and keep rebuilding against patched versions as vulnerabilities are disclosed.
Rivas said his own AWS workflow uses Amazon Inspector to scan images pushed to Amazon Elastic Container Registry. He also recommended a regular rebuild rhythm, suggesting roughly every three to four months as a practical cadence for keeping base images and packages current.
The supply chain discussion extended to the pipeline itself. Rivas described old CI/CD setups in which API keys can remain unchanged for years. He said he now uses Open ID Connect (OIDC) federation between services such as GitHub Actions and AWS, avoiding long-lived credentials stored in pipeline secrets and scoping access to the work the pipeline needs to perform.
He also recommended maintaining a software bill of materials (SBOM), essentially an inventory of libraries, container bases and other components used in a build. The value becomes apparent when a new vulnerability is disclosed: instead of manually determining where a component might be running, a team can query its inventory and identify affected applications.
The 'Bypass Problem'
Automation does not mean that every security finding should stop a release, according to Rivas. He differentiated warnings from issues serious enough to block deployment and recommended considering severity, exploitability and exposure when deciding where to stop the pipeline.
[Click on image for larger view.] Avoiding Security Control Bypasses (source: Carlos Rivas).
The danger of making controls too restrictive is that delivery teams eventually find another route. Rivas said he had encountered that pattern himself earlier in his career: when security controls block too much, developers under pressure to restore service or ship an urgent change look for exceptions or manual paths around the official pipeline.
His recommendation is to keep the compliant route usable and establish an explicit exception process rather than force work into unofficial channels. "So when you implement all this stuff, don't set it up too strict in the beginning," he said.
He made a related point about alert volume. Security products enabled with default settings can initially report large numbers of findings that do not reflect the context of a particular environment. Rivas gave examples such as a public Amazon S3 bucket that intentionally serves a public website or an Amazon EC2 instance that appears open to Secure Shell access but sits behind a private network requiring a virtual private network connection.
If teams are flooded with findings they know do not require action, he said, they will eventually start ignoring the output. His recommendation was to tune controls for the environment, suppress known-good conditions and preserve the significance of the remaining high-priority findings.
Start Narrow and Build Out
Rivas closed the prepared portion of the session by emphasizing incremental adoption. Rather than applying a full DevSecOps program across every application and account at once, he recommended first measuring the scope, auditing CI/CD credentials, assigning ownership and selecting one team and pipeline as a pilot.
[Click on image for larger view.] Three DevSecOps Steps to Start (source: Carlos Rivas).
That pilot gives the organization a place to tune scanners, establish thresholds, determine ownership and measure costs before expanding. Rivas described the broader rollout as a months-long process in which teams add metrics, alarms, dashboards, runbooks and automation over time. "But again, this is a very very slow process, so don't think that you can just flip a switch and you'll be done with this, right?" he said.
The incremental approach also came up during the audience Q&A. Asked whether adding DevSecOps controls would slow delivery, Rivas answered, "Initially, yes," while noting that some automated checks can be introduced without immediately becoming blockers. Asked about cost, he again recommended starting small because AWS security services can be consumption-based, making a pilot useful for estimating what broader deployment will cost.
The Rest of the Session
Rivas also covered organizational guardrails, policy as code, encryption defaults, network segmentation, durable logging, audit evidence, production vulnerability scanning, incident-response runbooks and metrics. In the Q&A, he discussed applying the same concepts to on-premises environments and protecting legacy applications that no longer have a modern pipeline by using infrastructure controls such as firewalls and network segmentation.
AI also surfaced near the end of the presentation and during the Q&A. Rivas said he uses AI as an assistant for tasks such as analyzing logs and identifying potential problems, but does not currently allow an AI agent to make changes directly in his AWS environments. "I'm still not comfortable just giving the full keys to the kingdom to an AI agent," he said.
The event is being made available for on-demand replay thanks to Wiz.
And More
While replays are convenient and informative -- especially up-to-date sessions that just concluded -- attending live events offers advantages, including the ability to ask specific implementation questions and receive guidance in real time (not to mention the chance to win great prizes, in this case a $300 Amazon gift certificate, thanks to sponsor Wiz, which also presented). With that in mind, here are some upcoming online webcasts from Virtualization & Cloud Review
:
Also upcoming are these online virtual summits from Redmond
: