Connaisseur v2.2 - Improving Usability of Container Signature Validation in Kubernetes
Connaisseur is a Kubernetes admission controller for container image signature verification. The latest release v2.2 improves usability and compatibility.
Connaisseur allows you to verify the signatures of container images before deployment to a Kubernetes cluster. To learn more about Connaisseur, checkout the docs or read our previous articles:
- Connaisseur v2.0 release
- Validation of Cosign-based image signatures in Kubernetes
- Validation of Notary-based image signatures in Kubernetes
If you want to get started using Connaisseur, there is - as always - a quick demo below ...
So what's new?
While the detailed changes in version 2.2 can be found in the release log, the overarching scope is usability and compatibility:
- More native Helm integration
- Simplified and more stable administration (deploy, update, delete)
- Better compatibility with different flavors of Kubernetes (e.g. OpenShift)
- Better compatibility with different versions of Kubernetes (v1.16+)
- Improved KMS support for Cosign
We are pleased to announce that Connaisseur Helm charts are now published in a Connaisseur Artifact Hub repository. As a consequence, Connaisseur can now be administered directly via Helm without the need to clone the whole repository, greatly facilitating integration into the CI/CD pipeline.
Furthermore, we reworked the way Connaisseur is deployed, updated and deleted in order to make administration faster and considerably more stable. Specifically, changing the Connaisseur configuration at runtime via
helm upgrade was improved and now works with zero downtime. The default configuration has been adjusted to allow compatibility with different flavors and versions of Kubernetes. Tests for Kubernetes v1.16 and higher were implemented to ensure future compatibility. Finally, the key management service (KMS) support of Cosign for a range of Providers (e.g. AWS, Azure, GCP, HashiCorp, Kubernetes) has been experimentally integrated into Connaisseur.
So let's take a look at some of those improvements ...
Demo Time 🚀
Within the demo, we will install and customize Connaisseur to use our own KMS via the public Helm repository. Here is a screencast of the demo to get a feeling for Connaisseur in action:
The video shows a screencast of adding Connaisseur’s Helm repository and installing the admission controller in default configuration. Next, the configuration is adjusted to use Kubernetes secret as a KMS for container image signature verification. The configurations are tested by creating pods with signed/unsigned images that are admitted/denied.
We start by adding the Connaisseur Helm repository and installing it in the
$ helm repo add connaisseur https://sse-secure-systems.github.io/connaisseur/charts "connaisseur" has been added to your repositories $ helm install connaisseur connaisseur/connaisseur --atomic --create-namespace --namespace connaisseurNAME: connaisseur LAST DEPLOYED: Sun Oct 17 14:39:58 2021 NAMESPACE: connaisseur STATUS: deployed REVISION: 1 TEST SUITE: NoneCopy to clipboard
You can verify the installation by checking the deployed resources via
kubectl get all -n connaisseur or simply deploying some test images:
# unsigned image is denied $ kubectl run demo1 --image=docker.io/securesystemsengineering/testimage:unsigned # is denied Error from server: admission webhook "connaisseur-svc.connaisseur.svc" denied the request: Unable to find signed digest for image docker.io/securesystemsengineering/testimage:unsigned. # signed image is admitted $ kubectl run demo2 --image=docker.io/securesystemsengineering/testimage:signed # is accepted pod/demo2 createdCopy to clipboard
Connaisseur is now successfully enabled and you verified your first image! 🚀
Let's customize the configuration to verify a container image signed by ourselves, but let's do this as a KMS instead of directly configuring the public key. For simplicity, we will use a Kubernetes secret as a "KMS" here, since no external service or vendor is required. The secret can be created directly using Cosign:
$ cosign generate-key-pair k8s://connaisseur/cosign-keys Enter password for private key: Enter password for private key again: Successfully created secret cosign-keys in namespace connaisseur Public key written to cosign.pub # check if keys were created $ kubectl get secrets -n connaisseur cosign-keys NAME TYPE DATA AGE cosign-keys Opaque 3 92sCopy to clipboard
Connaisseur cannot, by default, access Kubernetes secrets, so we will have to manually grant access via a role and role binding:
$ cat << EOF | kubectl apply -f - apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: connaisseur-kms-role namespace: connaisseur labels: app.kubernetes.io/name: connaisseur rules: - apiGroups: ["*"] resources: ["secrets"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: connaisseur-kms-rolebinding namespace: connaisseur labels: app.kubernetes.io/name: connaisseur subjects: - kind: ServiceAccount name: connaisseur-serviceaccount namespace: connaisseur roleRef: kind: Role name: connaisseur-kms-role apiGroup: rbac.authorization.k8s.io EOFrole.rbac.authorization.k8s.io/connaisseur-kms-role created rolebinding.rbac.authorization.k8s.io/connaisseur-kms-rolebinding createdCopy to clipboard
Next, we create the actual custom configuration for Connaisser in a
validators: # our new custom validator - name: my-validator type: cosign trust_roots: - name: my-key key: k8s://connaisseur/cosign-keys # the preconfigured Connaisseur public key is required! - name: dockerhub-basics type: notaryv1 host: notary.docker.io trust_roots: - name: securesystemsengineering-official key: | -----BEGIN PUBLIC KEY----- MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEsx28WV7BsQfnHF1kZmpdCTTLJaWe d0CA+JOi8H4REuBaWSZ5zPDe468WuOJ6f71E7WFg3CVEVYHuoZt2UYbN/Q== -----END PUBLIC KEY-----policy: # using the custom validator to validate all non-Connaisseur images - pattern: "*:*" validator: my-validator with: trust_root: my-key - pattern: "docker.io/securesystemsengineering/*:*" validator: dockerhub-basics with: trust_root: securesystemsengineering-officialCopy to clipboard
Note that in case you use a registry that requires authentication, you also need to add an auth configuration to
Let's update Connaisseur to use our new custom configuration:
$ helm upgrade connaisseur connaisseur/connaisseur -n connaisseur --wait -f values.yaml Release "connaisseur" has been upgraded. Happy Helming! NAME: connaisseur LAST DEPLOYED: Sun Oct 17 15:06:31 2021 NAMESPACE: connaisseur STATUS: deployed REVISION: 2 TEST SUITE: NoneCopy to clipboard
To test our changes, we need a suitable image in a registry. Either use your own, or simply re-tag a public image:
# give it a name via IMG environment variable $ export IMG=<registry>/<your-repo>/demo:v1 $ docker pull docker.io/library/hello-world $ docker tag docker.io/library/hello-world $IMG $ docker push $IMGCopy to clipboard
However, we have not signed this image so far, and if we try deploying it, it will be denied:
$ kubectl run my-img --image=$IMG Error from server: admission webhook "connaisseur-svc.connaisseur.svc" denied the request: No trust data for image "<registry>/<your-repo>/demo:v1".Copy to clipboard
So let`s sign it and try again:
$ cosign sign -key k8s://connaisseur/cosign-keys $IMG $ kubectl run my-img --image=$IMGpod/my-img createdCopy to clipboard
Congratulations! You have rolled out your own custom configuration and verified your first container images.🎉
To clean up, delete the test containers and uninstall Connaisseur via:
$ helm uninstall connaisseur -n connaisseur release "connaisseur" uninstalledCopy to clipboard
Why the Community is so important to us
What is the common denominator of the changes in version 2.2? Right: they were all inspired, requested or even directly contributed by the community, and that has been immensely helpful in steering the development of Connaisseur. While we do our best and run a significant number of linters, integration and unit tests on all changes, it is sheer impossible to check the behavior in all different environments and use-cases. Furthermore, feedback helps us to identify the critical features for future releases.
Besides the direct feedback on GitHub, we have also seen a lot of mentions and reviews in newsletters, blogs and podcasts which has been a great inspiration.
In essence: when developing an open source project, it`s difficult to understand what people think and what the community really needs. In that respect, GitHub stars give you motivation while issues give you focus.