Skip to main content
banner image
venafi logo

The Dangers of Using Development Certificates in Production

The Dangers of Using Development Certificates in Production

DevOps Automation
August 17, 2020 | Anastasios Arampatzis

The Disconnect between DevOps and Security

Modern application development teams need to move fast to develop products that are customer-centric and in line with market needs. The problem is that the DevOps teams’ speed is often far greater than the security teams’ operational tempo. As such, developers may view Information Security as a barrier rather than an enabler. Plus, they have a vested interest in staying within their “siloed” toolchain.



Secure software development requires following the following policies during application development and testing:

  • Production data is not to be used in a test environment, mostly due to lack of security and integrity controls
  • Developers are prohibited from accessing production systems unless their role explicitly requires it (this may vary based on team and organization size)

The above policies indicate that development and testing environments should be separated from production. When it comes to using different certificates based on the environment, there are excellent reasons for doing this. Unfortunately, existing security processes often do not enable developers to get the certificates they need at the speed they require it.

Security processes for SSL/TLS certificates are often antiquated and require manual steps such as submitting a ticket. As a result, developers look for an easier way, often using certificates without the security team’s stamp of approval. Instead, they create their own security infrastructure, issuing certificates using a combination of OpenSSL, secrets management tools, DevOps platforms and scripts. They can then issue certificates as fast as they need them, but that comes with security risks.

The hard truth is that developers are not famous for following corporate security policy. As the book titled “The New Kingmakers, How Developers Conquered the World,” developers now have access to open source and cloud computing which enables them to be largely self-sufficient.

In addition, development teams are focused on rapid SLAs, continuous delivery and compressed timelines. By leveraging scripts or readily available options within their DevOps platform, their certificate management processes often are not compliant with the enterprise-wide security policy. By doing this, developers may inadvertently create a false sense of trust in unauthorized certificates, whether they are issuing self-signed certificates or spinning up their own internal certificate authorities (CAs) that are outside of corporate security policy.

The current state of certificate provisioning within modern application environments also poses a variety of challenges:

  • Certificate provisioning is not embedded into software-defined infrastructure
  • Development teams are unable to easily comply with policy
  • Information security, risk and governance teams have little visibility into certificates
  • Certificate-related outages continue to plague organizations

The Dangers of Using “Test” Certificates in Production

Visibility, accountability and ownership are the topmost challenges when it comes to managing certificates across the full software development life cycle. Most organizations underestimate the number of active certificates in use on their systems. For example, one major retail organization estimated it had 5,000 active certificates, but in reality, it had 20,000! Even more alarming, more than 5,000 certificates had no owners, and no one knew what they did, what they allowed, and who was responsible for them. This lack of insight can easily occur when organizations use self-signed certificates issued outside security-controlled processes.

Remember the Heartbleed vulnerability from 2014? That flaw showed exactly how organizations blindly trust keys and certificates and how easily cybercriminals can exploit a trust-related vulnerability.

The underlying assumption is that “test” certificates are to be used only in development environments and that they should be replaced by corporate-compliant certificates when the application moves to production. But this does not always happen according to plan. As a result, organizations are exposed to a veritable avalanche of certificates that they are not aware of and they cannot discover.

According to a Venafi study, 62% of mature DevOps organizations consistently replace development and test certificates with production certificates when code rolls into production. In organizations that are just adopting DevOps practices, only a 36% of the respondents follow this critical best practice. Without changing certificates, there is no way to distinguish between the identities of trusted machines that are safe to place in production and untested machines that should remain in development environments.

Unauthorized CAs raise another issue. Every time developers set up an internal CA, they are distributing that CA as a root of trust within the organization. Accordingly, they create a set of applications that are configured to “trust” this CA and the certificates that it issues. Without the proper visibility into your application trust stores, it’s very difficult to remediate this issue. Even if you re-issue these certificates under corporate policy, you’re still left with the challenge of eradicating the unauthorized CAs from your trust store.

How Do You Solve this Problem?

Here’s the dilemma. Do you force your application development teams to follow security policies for certificates and jeopardize their SLA? Or do you allow them to create self-signed or unauthorized subordinate CA certificates to move as fast as possible and boost your population rogue certificates? The answer is neither. Ultimately, you need to provide development teams a safer process that gives you visibility into all certificates, while also providing developers easy access to certificates from within their native DevOps toolchain.

By “shifting security left” into the continuous integration/continuous delivery (CI/CD) pipeline, your security team can prevent vulnerabilities early in the development process. In “shifting security right,” you can enable your certificate PKI team to provide as many machine identities as the development team needs, when and where they need them, across every environment, from development to production. This security shift can be achieved effectively through automation using a platform that integrates with the tools in use. Proactive InfoSec and development teams know that the security policy should be treated the same as any application code—it should be defined as code, versioned as code and tested as code.

Taking a programmatic approach allows for “baking in” compliant and trusted certificates into CI/CD pipelines. This is the gold standard and should be the blueprint to managing and protecting machine identities in modern applications. If this is done, developers can continue to use the tools they love, while security can ensure compliance and audit against it.

How much do you know about the certificates used in your containers, microservices, and other infrastructure-as-code?

Related posts


Like this blog? We think you will love this.
Featured Blog

Applying Identity to DevSecOps Processes

Identity Means Secrets

Read More
Subscribe to our Weekly Blog Updates!

Join thousands of other security professionals

Get top blogs delivered to your inbox every week

See Popular Tags

You might also like

TLS Machine Identity Management for Dummies

TLS Machine Identity Management for Dummies

Certificate-Related Outages Continue to Plague Organizations
White Paper

CIO Study: Certificate-Related Outages Continue to Plague Organizations

About the author

Anastasios Arampatzis
Anastasios Arampatzis

Anastasios Arampatzis is a retired Hellenic Air Force officer with over 20 years of experience in evaluating cybersecurity and managing IT projects. He works as an informatics instructor at AKMI Educational Institute, while his interests include exploring the human side of cybersecurity.

Read Posts by Author
get-started-overlay close-overlay cross icon
get-started-overlay close-overlay cross icon
Venafi Risk assessment Form Image

Sign up for Venafi Cloud

Venafi Cloud manages and protects certificates

* Please fill in this field Please enter valid email address
* Please fill in this field Password must be
At least 8 characters long
At least one digit
At last one lowercase letter
At least one uppercase letter
At least one special character
* Please fill in this field
* Please fill in this field
* Please fill in this field

End User License Agreement needs to be viewed and accepted

Already have an account? Login Here

get-started-overlay close-overlay cross icon

How can we help you?

Thank you!

Venafi will reach out to you within 24 hours. If you need an immediate answer please use our chat to get a live person.

In the meantime, please explore more of our solutions

Explore Solutions

learn more

Email Us a Question

learn more

Chat With Us

learn more