mTLS everywhere in Kubernetes: not just ingress and egress
Thu, 04/07/2022 – 16:18
TLS everywhere is good practice
To prevent security incidents from happening in the first place, Kubernetes Documentation mentions that “TLS must be enabled for each component that supports it to prevent traffic sniffing, verify server identity, and (for mutual TLS) verify client identity”.
In the meantime, it’s important to understand that TLS should be used to encrypt all communication in the cluster between services, not just ingress. However, this security check is often overlooked since the cluster is secure and there is no need to provide encryption in transit within the cluster. (There are also many cases where mTLS is used, but the application platform team misconfigures mTLS or configures it in a way that does not comply with the organization’s security policy. But that’s a topic for another blog.)
OWASP Kubernetes Security Cheat Sheet mentions that “Kubernetes expects all API communications in the cluster to be encrypted by default with TLS”. However, there is a downside with TLS: TLS guarantees authenticity but by default this only happens one way: the client authenticates the server, but the server does not authenticate the client.
This is where mTLS comes in handy – mTLS makes authenticity symmetric.
What is Mutual TLS Authentication?
Mutual authentication, also called two-way authentication, is a security process in which entities authenticate each other before any actual communication occurs. In a network environment, this requires both client and server to provide digital certificates to prove their identity. In a mutual authentication process, a connection can only occur if the client and server exchange, verify, and trust each other’s certificates.
The default option for TLS is to only authenticate in one direction because the identity of the client is often irrelevant, this is the case with websites, for example. Of course, not validating the client’s identity makes sense for serving web pages, but there are many types of communication where the client’s identity is important. API calls are an example of this: if you call a service, it needs to know who you are for a variety of reasons, from providing personalized information to sending the invoice. You cannot make an API call without providing some sort of client identity.
Many services use an authentication token to validate the identity of clients. Instead of using this token, mutual authentication via mTLS provides some features that are particularly beneficial for services like Kubernetes.
mTLS authentication is performed entirely outside of the application without requiring any application-level functionality for creating, registering, or managing identities. With mTLS, a new customer can authenticate instantly. Also, the app doesn’t need to know anything about authentication or provide endpoints to handle it.
These characteristics make mTLS so ideal for securing all communications within Kubernetes.
mTLS everywhere is even better practice
mTLS is a great way to secure all communications between microservices in a Kubernetes environment, for all the reasons listed above.
First, you secure all communications. When we implement our applications as multiple services, sensitive customer data is sent over the network between these services. Without proper encryption, anyone with access to the network can potentially read this sensitive data and tamper with requests.
Second is verification and authorization. When calls to services come from authenticated entities, you can audit and record the appropriate metrics. Auditing gives you insight into what the customer is doing and how they are doing it, and you can also spot anomalous activity and prevent breaches. This is also important for authorization. Is the authenticated call authorized to access the specific data? Authentication and authorization work together to ensure the privacy and integrity of your communications.
Third, it takes care of authentication. Because authentication takes place outside of the application, your developers don’t have to create flows and procedures to handle identity verification. Instead, they can focus on building and securing business logic and delivering a well-functioning application.
Last but not least, it’s about the user experience. Traditional token authentication can become cumbersome for repeat users. Instead, mTLS authentication takes place once, at the platform level, and the user is not required to validate their identity when switching between services.
TLS certificate management is important
mTLS is ideal for securing communication between microservices. But there is a catch. Machine identity management faces the challenge of creating and distributing all these certificates. This certificate distribution challenge is compounded by the fact that in environments like Kubernetes, a “service” is actually an ephemeral instance that can be created or destroyed on the fly, each needing its own set of certificates.
This challenge is further compounded by certificate rotation in response to certificate compromise. As the lifecycle of certificates gets shorter each year, you must reissue certificates before they expire. This means that we have to repeat the entire CSR (certificate signing request) and certificate flow, for each ephemeral service, from time to time.
This is where cert-manager comes in handy. cert-manager, developed by Venafi’s Jetstack team, makes CAs and certificates first-class resource types in the Kubernetes API, allowing developers to easily request machine identities for applications, while platform and security can maintain control and visibility. Learn more about the success of cert-manager here.