Kubernetes Controller Security

The Kubernetes controller manages Helm releases through the Kubernetes API, so it has the access of its service account.

The controller opens every connection to Distr itself, over HTTPS, so the cluster needs no inbound ports.

The manifest that installs the controller creates these resources in the namespace it is applied to:

Resource (default)	Purpose	Optional	Changed by
distr-controller ServiceAccount, bound to cluster-admin with cluster scope or to the namespace-admin Role with namespace scope	Install the resources of Helm charts and read their status, logs and metrics	No	Deployment target editor, when creating the deployment target
distr-controller Deployment, one replica as non-root user with 256Mi memory and a CPU limit of 500m	Run the controller	No	Deployment target editor, with custom resource requirements
distr-controller-auth Secret	Hold the target secret	No	Deployment target editor, by reconnecting
distr-controller-env ConfigMap	Hold the Distr endpoints and the controller version	No	Not configurable
distr-controller-dockerconfig Secret	Provide the registry credentials the Distr instance configures for all controllers	Yes, only with CONTROLLER_DOCKER_CONFIG set on a self-hosted Distr	Distr administrator
emptyDir cache volume	Keep the pulled Helm charts	No	Not configurable

A deployment target editor is a user with the read_write or admin role in the vendor organization or in the customer organization the deployment target belongs to. The scope cannot be changed after the deployment target is created. The namespace-admin Role grants every verb on every resource of the namespace, including Secrets, Roles and RoleBindings. cluster-admin grants the same on every resource of the cluster, which metrics need to read the nodes.

Distr renders the manifest when the controller connects and again for every self-update. A change to the custom resource requirements reaches the cluster with the next reconnect or controller update.

When	What the controller does
A deployment is created or updated	Writes the image pull secret sh.distr.agent.v1.<release>.pull, adds it to every imagePullSecrets and pullSecrets list in the values and runs helm install or helm upgrade in its namespace. By default Helm waits for the resources to become ready and rolls back on failure, unless the Helm options of the deployment say otherwise
A deployment is force restarted	After the upgrade, patches the workloads of the release the way kubectl rollout restart does
A deployment is undeployed	helm uninstall, then deletes the tracking secret sh.distr.agent.v1.<release>
Continuously	Reads the tracking secrets and Helm releases of its namespace and the status of the release resources. With log collection enabled it reads the logs of their pods, with metrics enabled the pods and pod metrics of its namespace and the nodes and node metrics of the cluster
The controller version changes	Fetches the new manifest from Distr and creates or updates its resources, including its own role binding

Anything the chart of a deployment defines runs as well, such as Helm hooks. Releases are installed in the controller’s namespace, but with cluster scope a chart can create resources anywhere in the cluster.

• The distr-controller-auth Secret holds the target secret. Authentication explains how to replace it.
• The image pull secret of every deployment holds its registry credentials. Registry credentials are also kept in the temporary directory of the controller container.
• Helm stores every revision of a release, including its values with their secrets filled in, in Secrets named sh.helm.release.v1.<release>.v<revision>.

Anyone who can read Secrets in the controller’s namespace can read all of these.