Internet outages are a serious concern for organizations. Whether the outage affects an entire company, or it touches just one area of the company, the results can impact businesses with nearly the same consequences as when a full breach occurs.
When an outage occurs, the first response is a very general “what happened”? As the problem becomes isolated, the field of inquiry narrows to which areas are affected, was the cause due to malicious activity or human error, and most importantly, how are our customers impacted? If your team has a strong awareness and a clear visibility of the environment, they can usually pinpoint the problem with enough accuracy to recover fairly quickly. However, if the cause is due to a certificate expiration, the outage can be a bit trickier to uncover.
As seen from inside a network, an expired certificate can give an entirely different impression of what is occurring. To the outside world, this expired machine identity can indicate that a site is untrusted, and in the case of more restrictive TLS implementations, such as HTTP Strict Transport Security, it could result in the appearance that a site is entirely offline. When a person reports that a site is offline, many network engineers will check to make sure that the network is working correctly. This is deceptive, as all the technical checkpoints will appear to be functioning normally.
A machine with an expired certificate will respond to a ping, it will display that it is up and running on all the monitoring consoles, and it will even allow a full administrative login. It is only the TLS mechanism that will be broken, and that is not readily apparent from a machine perspective. In some cases, attempting to get to the root of an outage will result in going so far as spinning up a new instance of a machine image. In a drastic overreaction, one may be inclined to reach for a gold image to rule out all other possibilities. However, that will also exhibit the same outage problem.
The average person may think that an SSL/TLS certificate is used just to verify the authenticity of a domain. At a very basic level, this is true. However, with all the newer implementations of web-based interfaces, these machine identities are everywhere. In a recent report, it was found that certificate usage was increasing, and expected to continue to grow over the next few years. According to the report:
Every trusted connection will have a certificate that governs that trust. It is no longer just a public-facing web site that may use a certificate. Something as seemingly insignificant as an internal corporate message site may use a certificate. Software manufacturers use certificates to verify the authenticity and origin of software.
Certificate-based outages are also on the rise. This is due not only to what may be described as certificate sprawl, but because of shorter lifespan settings for newer certificate initiatives. Certificate outages can impact business while the problem is triaged and remedied. It can also cause internal staff to lose confidence in the organization. Both of these can be costly.
One may think that the only positive phenomenon of this type of outage is that it does not require reporting to a regulatory authority, as it is not a breach event. However, that is not entirely true, as it can also compromise customer information. The problem of what can happen if a malicious attacker gains control of an expired certificate was demonstrated as far back as 2013. Also, a U.S. Government Accountability Office report about the Equifax breach revealed that an expired certificate on a TLS inspection device allowed attackers to exfiltrate data without detection—for 10 months!
There are other examples of larger corporate certificate problems. A compromised code-signing certificate can be used to spread malware under the guise of a legitimate company. The rise of ransomware has caused these types of attacks to fall out of the headlines, but it makes them no less important, as they are still a viable attack vector.
Like all data, certificate expirations have grown to a level that is no longer manageable with a spreadsheet. Along with that, certificate oversight has also escaped some organizations. Some teams may register a certificate without the knowledge of the security team, making it impossible for that certificate to be properly managed. This is another often overlooked aspect of Shadow IT.
Some of the best practices for protecting your certificates include using a trusted certificate authority for certificate issuance, safe storage of cryptographic keys, and establishing an organized plan for the entire certificate lifecycle, from creation, to renewal, to revocation.
Since these tasks are becoming more expansive within an organization, one way to keep better track is to use a trusted partner who can help you take an objective look at your environment to discover and manage all of the machine identities in your organization. Venafi offers expertise to help you achieve these goals, so that you can control your certificate inventory.