Add support for users with namespace-scoped RBAC permissions

[amorey]

Overview

We would like to make modifications to Kubetail to support users with namespace-scoped RBAC permissions. In order to do this we need to make several changes to the app itself, the development environment and also to the production helm charts. Our overall strategy is to register the Cluster API as a Kubernetes API extension (APIService) and offload authentication to the kube-apiserver which will make it easier to support all of the Kubernetes authentication methods.

Motivation

Granting namespace-scoped RBAC permissions to users is a common way to implement multi-tenancy in a Kubernetes cluster. Under this configuration, users have cluster-scope permissions to get/list/watch nodes and namespaces and namespace-scoped permissions to get/list/watch other resources (e.g. pods, deployments). Currently, users with namespace-scoped permissions will encounter permission errors when running Kubetail, making it unusable. After we have implemented the required changes, Kubetail will "just work" for those users.

Goals

• Enable access to Kubetail for users with namespace-scoped RBAC permissions on Desktop
• Verify that support for namespace-scoped RBAC permissions works for auth-mode: token on Cluster
• Ensure that internal authentication methods have unit tests

Design Details

When you register a Service as an APIService, it becomes available as a "Kubernetes API extension". Requests to Kubernetes API extensions go through the Kubernetes "proxy" system and are routed internally through the kube-apiserver which performs authentication and forwards information about the originating client to the target service in HTTP headers. Kubernetes requires that communication between kube-apiserver and the target service be authenticated with mTLS which is why we are implementing SSL as part of this project.

In the development environment we will need to make these changes:

• Split shared ConfigMap into separate ConfigMaps's one each for component (dashboard, cluster-api, cluster-agent)
• Add hard-coded SSL certs to dashboard and cluster-api

In the production environment (helm templates) we will need to make these changes:

• Add self-signed SSL certs to helm generated at install-time

In the app and other related systems we will need to make these changes:

• Register cluster-api as Kubernetes API extension using APIService
• Modify cluster-api to utilize authentication headers from kube-apiserver
• Modify cluster-api and cluster-agent to utilize new auth headers to perform RBAC permission checks
• Eliminate use of kubetail-cli ServiceAccount
• Replace "namespace" de-referencer with a permissions-aware version
• Modify dashboard to connect to cluster-api using APIService

In addition, it's possible we may run into problems with the current CSRF protection mechanism in which case we should re-examine our approach to CSRF protection and possibly:

• Replace CSRF protection with check for Sec-Fetch-Site: same-origin

Also note that adding SSL to the dashboard isn't strictly necessary for this project but it can be done at the same time as the cluster-api with very little added effort.

Alternatives Considered

We briefly looked into implementing full Kubernetes authentication ourselves at the application layer but it didn't look like this would be easy to do and would also introduce a maintenance burden. Also, long-term registering the cluster-api as a Kubernetes api extensions seems like a good way to expose Kubetail to other ecosystem tools.

Open Questions

• We have to look into how to verify requests from kube-apiserver so we can trust the request auth headers
• APIService should work with upgrade requests but we don't have any actual experience with this