K8s-kube-apiserver(授权)

基于1.25

kube-apiserver的授权

客户端请求通过认证之后,进入授权阶段,校验对应用户是否具有对应数据读写的权限

  • 支持多种授权机制,并且支持开启多个授权功能
  • 如果开启多个授权,按照授权顺序执行
  • 只要有一个授权器通过,授权就成功

目前支持六种授权模式:

  • AlwaysAllow:允许所有请求
  • AlwaysDeny:阻止所有请求
  • ABAC:基于属性的访问控制
  • Webhook:基于Webhook的一种HTTP回调,远程授权管理
  • RBAC:基于角色的访问控制
  • Node:节点模式,专门授权kubelet发出的API请求
  • 集群默认开启Node和RBAC模式

在kube-apiserver中,Authorization授权有三个概念,分别是Decision决策状态、授权器接口和Rule Resolver规则解析器

  1. Decision决策状态

    Decision决策状态类似于身份认证中的true和false,用来表示授权是否成功

  2. 授权器接口

    每种是授权器都要实现Aurhorizer接口

  3. RuleResovler规则解析器

    • Ref:https://github.com/kubernetes/apiserver/blob/ba592e4ccd41a320ceb91bab90eebee3bb4a4f33/pkg/authorization/authorizer/interfaces.go#L81

      // RuleResolver provides a mechanism for resolving the list of rules that apply to a given user within a namespace.
      type RuleResolver interface {
      // RulesFor get the list of cluster wide rules, the list of rules in the specific namespace, incomplete status and errors.
      // 解析出规则列表
      // ResourceRuleInfo:资源类型的规则列表,如/api/v1/pods资源接口
      // NonResourceRuleInfo:非资源类型的规则列表,如/api或/health资源接口
      RulesFor(user user.Info, namespace string) ([]ResourceRuleInfo, []NonResourceRuleInfo, bool, error)
      }
  4. Ref:https://github.com/kubernetes/apiserver/blob/ba592e4ccd41a320ceb91bab90eebee3bb4a4f33/pkg/endpoints/filters/authorization.go#L45

    // WithAuthorizationCheck passes all authorized requests on to handler, and returns a forbidden error otherwise.
    // 依次进行加载授权器
    func WithAuthorization(handler http.Handler, a authorizer.Authorizer, s runtime.NegotiatedSerializer) http.Handler {
    if a == nil {
    klog.Warning("Authorization is disabled")
    return handler
    }
    return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
    ctx := req.Context()

    attributes, err := GetAuthorizerAttributes(ctx)
    if err != nil {
    responsewriters.InternalError(w, req, err)
    return
    }
    authorized, reason, err := a.Authorize(ctx, attributes)
    // an authorizer like RBAC could encounter evaluation errors and still allow the request, so authorizer decision is checked before error here.
    if authorized == authorizer.DecisionAllow {
    audit.AddAuditAnnotations(ctx,
    decisionAnnotationKey, decisionAllow,
    reasonAnnotationKey, reason)
    handler.ServeHTTP(w, req)
    return
    }
    if err != nil {
    audit.AddAuditAnnotation(ctx, reasonAnnotationKey, reasonError)
    responsewriters.InternalError(w, req, err)
    return
    }

    klog.V(4).InfoS("Forbidden", "URI", req.RequestURI, "Reason", reason)
    audit.AddAuditAnnotations(ctx,
    decisionAnnotationKey, decisionForbid,
    reasonAnnotationKey, reason)
    responsewriters.Forbidden(ctx, attributes, w, req, reason, s)
    })
    }

AlwaysAllow授权

AlwaysAllow授权器允许所有请求,如果未配置--aurhorization-mode,则默认使用此授权模式

启用AlwaysAllow授权

kube-apiserver通过制定--authorization-mode=AlwaysAllow参数(或者不配置)启用授权

AlwaysAllow授权实现模式

AlwaysDeny授权

AlwaysDeny授权拒绝所有请求,很少单独使用

启用AlwaysDeny授权

kube-apiserver通过制定--authorization-mode=AlwaysDeny参数启用AlwaysDeny授权

AlwaysDeny授权实现管理

ABAC授权

ABAC授权是一种基于属性的访问控制模式,能够根据属性配置信息为用户授予访问权限

启用ABAC授权

kube-apiserver通过指定以下参数启用ABAC授权

  • --authorization-mode=ABAC:启用ABAC授权
  • --aurhorization-policy-file:指定策略模式,该文件使用JSON格式,每一行都是一个策略对象

ABAC授权实现原理

  • Ref:https://github.com/kubernetes/kubernetes/blob/88e994f6bf8fc88114c5b733e09afea339bea66d/pkg/auth/authorizer/abac/abac.go#L229


    // Authorize implements authorizer.Authorize
    func (pl PolicyList) Authorize(ctx context.Context, a authorizer.Attributes) (authorizer.Decision, string, error) {
    for _, p := range pl {
    if matches(*p, a) {
    return authorizer.DecisionAllow, "", nil
    }
    }
    return authorizer.DecisionNoOpinion, "No policy matched.", nil
    // TODO: Benchmark how much time policy matching takes with a medium size
    // policy file, compared to other steps such as encoding/decoding.
    // Then, add Caching only if needed.
    }

    // RulesFor returns rules for the given user and namespace.
    func (pl PolicyList) RulesFor(user user.Info, namespace string) ([]authorizer.ResourceRuleInfo, []authorizer.NonResourceRuleInfo, bool, error) {
    var (
    resourceRules []authorizer.ResourceRuleInfo
    nonResourceRules []authorizer.NonResourceRuleInfo
    )

    for _, p := range pl {
    if subjectMatches(*p, user) {
    if p.Spec.Namespace == "*" || p.Spec.Namespace == namespace {
    if len(p.Spec.Resource) > 0 {
    r := authorizer.DefaultResourceRuleInfo{
    Verbs: getVerbs(p.Spec.Readonly),
    APIGroups: []string{p.Spec.APIGroup},
    Resources: []string{p.Spec.Resource},
    }
    var resourceRule authorizer.ResourceRuleInfo = &r
    resourceRules = append(resourceRules, resourceRule)
    }
    if len(p.Spec.NonResourcePath) > 0 {
    r := authorizer.DefaultNonResourceRuleInfo{
    Verbs: getVerbs(p.Spec.Readonly),
    NonResourceURLs: []string{p.Spec.NonResourcePath},
    }
    var nonResourceRule authorizer.NonResourceRuleInfo = &r
    nonResourceRules = append(nonResourceRules, nonResourceRule)
    }
    }
    }
    }
    return resourceRules, nonResourceRules, false, nil
    }

Webhook授权

Webhook授权是一种基于HTTP回调的机制,当用户需要授权,kube-apiserver通过查询外部的Webhook服务器获得授权结果

启用Webhook授权

kube-apiserver通过制定以下参数启用Webhook授权

  • --authorization-mode=Webhook:启用Webhook授权器
  • --authorization-webhook-config-file:kubeconfig格式的Webhook配置文件,描述如何访问远程Webhook服务器

Example:

# Kubernetes API 版本
apiVersion: v1
# API 对象种类
kind: Config
# clusters 代表远程服务
clusters:
- name: name-of-remote-authz-service
cluster:
# 对远程服务进行身份认证的 CA
certificate-authority: /path/to/ca.pem
# 远程服务的查询 URL。必须使用 'https'。不可以包含参数。
server: https://authz.example.com/authorize

# users 代表 API 服务器的 webhook 配置
users:
- name: name-of-api-server
user:
client-certificate: /path/to/cert.pem # 要使用的 webhook 插件的证书
client-key: /path/to/key.pem # 与证书匹配的密钥

# kubeconfig 文件必须有 context。需要提供一个给 API 服务器。
current-context: webhook
contexts:
- context:
cluster: name-of-remote-authz-service
user: name-of-api-server
name: webhook

Webhook实现原理

    1. 首选尝试 w.responseCache.Get(string(key));缓存中查找
  • Ref:https://github.com/kubernetes/apiserver/blob/ba592e4ccd41a320ceb91bab90eebee3bb4a4f33/plugin/pkg/authorizer/webhook/webhook.go#L166


    // Authorize makes a REST request to the remote service describing the attempted action as a JSON
    // serialized api.authorization.v1beta1.SubjectAccessReview object. An example request body is
    // provided below.
    //
    // {
    // "apiVersion": "authorization.k8s.io/v1beta1",
    // "kind": "SubjectAccessReview",
    // "spec": {
    // "resourceAttributes": {
    // "namespace": "kittensandponies",
    // "verb": "GET",
    // "group": "group3",
    // "resource": "pods"
    // },
    // "user": "jane",
    // "group": [
    // "group1",
    // "group2"
    // ]
    // }
    // }
    //
    // The remote service is expected to fill the SubjectAccessReviewStatus field to either allow or
    // disallow access. A permissive response would return:
    //
    // {
    // "apiVersion": "authorization.k8s.io/v1beta1",
    // "kind": "SubjectAccessReview",
    // "status": {
    // "allowed": true
    // }
    // }
    //
    // To disallow access, the remote service would return:
    //
    // {
    // "apiVersion": "authorization.k8s.io/v1beta1",
    // "kind": "SubjectAccessReview",
    // "status": {
    // "allowed": false,
    // "reason": "user does not have read access to the namespace"
    // }
    // }
    //
    // TODO(mikedanese): We should eventually support failing closed when we
    // encounter an error. We are failing open now to preserve backwards compatible
    // behavior.
    func (w *WebhookAuthorizer) Authorize(ctx context.Context, attr authorizer.Attributes) (decision authorizer.Decision, reason string, err error) {
    r := &authorizationv1.SubjectAccessReview{}
    if user := attr.GetUser(); user != nil {
    r.Spec = authorizationv1.SubjectAccessReviewSpec{
    User: user.GetName(),
    UID: user.GetUID(),
    Groups: user.GetGroups(),
    Extra: convertToSARExtra(user.GetExtra()),
    }
    }

    if attr.IsResourceRequest() {
    r.Spec.ResourceAttributes = &authorizationv1.ResourceAttributes{
    Namespace: attr.GetNamespace(),
    Verb: attr.GetVerb(),
    Group: attr.GetAPIGroup(),
    Version: attr.GetAPIVersion(),
    Resource: attr.GetResource(),
    Subresource: attr.GetSubresource(),
    Name: attr.GetName(),
    }
    } else {
    r.Spec.NonResourceAttributes = &authorizationv1.NonResourceAttributes{
    Path: attr.GetPath(),
    Verb: attr.GetVerb(),
    }
    }
    key, err := json.Marshal(r.Spec)
    if err != nil {
    return w.decisionOnError, "", err
    }
    if entry, ok := w.responseCache.Get(string(key)); ok {
    r.Status = entry.(authorizationv1.SubjectAccessReviewStatus)
    } else {
    var result *authorizationv1.SubjectAccessReview
    // WithExponentialBackoff will return SAR create error (sarErr) if any.
    if err := webhook.WithExponentialBackoff(ctx, w.retryBackoff, func() error {
    var sarErr error
    var statusCode int

    start := time.Now()
    result, statusCode, sarErr = w.subjectAccessReview.Create(ctx, r, metav1.CreateOptions{})
    latency := time.Since(start)

    if statusCode != 0 {
    w.metrics.RecordRequestTotal(ctx, strconv.Itoa(statusCode))
    w.metrics.RecordRequestLatency(ctx, strconv.Itoa(statusCode), latency.Seconds())
    return sarErr
    }

    if sarErr != nil {
    w.metrics.RecordRequestTotal(ctx, "<error>")
    w.metrics.RecordRequestLatency(ctx, "<error>", latency.Seconds())
    }

    return sarErr
    }, webhook.DefaultShouldRetry); err != nil {
    klog.Errorf("Failed to make webhook authorizer request: %v", err)
    return w.decisionOnError, "", err
    }

    r.Status = result.Status
    if shouldCache(attr) {
    if r.Status.Allowed {
    w.responseCache.Add(string(key), r.Status, w.authorizedTTL)
    } else {
    w.responseCache.Add(string(key), r.Status, w.unauthorizedTTL)
    }
    }
    }
    switch {
    case r.Status.Denied && r.Status.Allowed:
    return authorizer.DecisionDeny, r.Status.Reason, fmt.Errorf("webhook subject access review returned both allow and deny response")
    case r.Status.Denied:
    return authorizer.DecisionDeny, r.Status.Reason, nil
    case r.Status.Allowed:
    return authorizer.DecisionAllow, r.Status.Reason, nil
    default:
    return authorizer.DecisionNoOpinion, r.Status.Reason, nil
    }

    }

RBAC授权

RBAC是基于角色访问控制

RBAC核心数据结构

在kube-apiserver中设计RBAC 添加咯角色和集群绑定的概念

  • kube-apiserver,可以通过Role、ClusterRole、RoleBinding、ClusterRoleBinding四种表示角色授权

Rol和ClusterRole:

  • Role:角色是一组用户的集合,与规则关联,角色只能赋予一个namespace的权限
  • ClusterRole:能够赋予集群范围内的权限,比如节点、非资源类型的服务端点
  • PolicyRule:操作权限,定义了何种资源何种权限

RoleBinding和ClusterRoleBinding:

  • Subject:主体可以是Group、User和ServiceAccount
  • RoleBinding:将角色关联的权限赋予一个或一组用户,只能授予一个namespace权限
  • ClusterRoleBinding:将角色关联的权限赋予一个或一组用户,赋予集群内权限
  • Role Ref:被授予权限的引用信息

启用RBAC授权

kube-apiserver通过指定--aurhorization-mode=RBAC启用

RBAC授权实现原理

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: role-grantor
rules:
- apiGroups: ["rbac.authorization.k8s.io"]
resources: ["rolebindings"]
verbs: ["create"]
- apiGroups: ["rbac.authorization.k8s.io"]
resources: ["clusterroles"]
verbs: ["bind"]
# 忽略 resourceNames 意味着允许绑定任何 ClusterRole
resourceNames: ["admin","edit","view"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: role-grantor-binding
namespace: user-1-namespace
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: role-grantor
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: user-1

创建内置集群角色

kube-apiserver在启动时会通过rbac/bootstrap-roles PostStartHook初始化内置的角色

Node授权

Node授权是一种特殊用途的授权模式,专门为kubelet发出的API请求进行授权

授权基于RBAC授权方式,对kubelet进行基于systen:node内置角色的权限控制

  • Ref:https://github.com/kubernetes/kubernetes/blob/88e994f6bf8fc88114c5b733e09afea339bea66d/plugin/pkg/auth/authorizer/rbac/bootstrappolicy/policy.go#L105

    / NodeRules returns node policy rules, it is slice of rbacv1.PolicyRule.
    func NodeRules() []rbacv1.PolicyRule {
    nodePolicyRules := []rbacv1.PolicyRule{
    // Needed to check API access. These creates are non-mutating
    rbacv1helpers.NewRule("create").Groups(authenticationGroup).Resources("tokenreviews").RuleOrDie(),
    rbacv1helpers.NewRule("create").Groups(authorizationGroup).Resources("subjectaccessreviews", "localsubjectaccessreviews").RuleOrDie(),

    // Needed to build serviceLister, to populate env vars for services
    rbacv1helpers.NewRule(Read...).Groups(legacyGroup).Resources("services").RuleOrDie(),

    // Nodes can register Node API objects and report status.
    // Use the NodeRestriction admission plugin to limit a node to creating/updating its own API object.
    rbacv1helpers.NewRule("create", "get", "list", "watch").Groups(legacyGroup).Resources("nodes").RuleOrDie(),
    rbacv1helpers.NewRule("update", "patch").Groups(legacyGroup).Resources("nodes/status").RuleOrDie(),
    rbacv1helpers.NewRule("update", "patch").Groups(legacyGroup).Resources("nodes").RuleOrDie(),

    // TODO: restrict to the bound node as creator in the NodeRestrictions admission plugin
    rbacv1helpers.NewRule("create", "update", "patch").Groups(legacyGroup).Resources("events").RuleOrDie(),

    // TODO: restrict to pods scheduled on the bound node once field selectors are supported by list/watch authorization
    rbacv1helpers.NewRule(Read...).Groups(legacyGroup).Resources("pods").RuleOrDie(),

    // Needed for the node to create/delete mirror pods.
    // Use the NodeRestriction admission plugin to limit a node to creating/deleting mirror pods bound to itself.
    rbacv1helpers.NewRule("create", "delete").Groups(legacyGroup).Resources("pods").RuleOrDie(),
    // Needed for the node to report status of pods it is running.
    // Use the NodeRestriction admission plugin to limit a node to updating status of pods bound to itself.
    rbacv1helpers.NewRule("update", "patch").Groups(legacyGroup).Resources("pods/status").RuleOrDie(),
    // Needed for the node to create pod evictions.
    // Use the NodeRestriction admission plugin to limit a node to creating evictions for pods bound to itself.
    rbacv1helpers.NewRule("create").Groups(legacyGroup).Resources("pods/eviction").RuleOrDie(),

    // Needed for imagepullsecrets, rbd/ceph and secret volumes, and secrets in envs
    // Needed for configmap volume and envs
    // Use the Node authorization mode to limit a node to get secrets/configmaps referenced by pods bound to itself.
    rbacv1helpers.NewRule("get", "list", "watch").Groups(legacyGroup).Resources("secrets", "configmaps").RuleOrDie(),
    // Needed for persistent volumes
    // Use the Node authorization mode to limit a node to get pv/pvc objects referenced by pods bound to itself.
    rbacv1helpers.NewRule("get").Groups(legacyGroup).Resources("persistentvolumeclaims", "persistentvolumes").RuleOrDie(),

    // TODO: add to the Node authorizer and restrict to endpoints referenced by pods or PVs bound to the node
    // Needed for glusterfs volumes
    rbacv1helpers.NewRule("get").Groups(legacyGroup).Resources("endpoints").RuleOrDie(),
    // Used to create a certificatesigningrequest for a node-specific client certificate, and watch
    // for it to be signed. This allows the kubelet to rotate it's own certificate.
    rbacv1helpers.NewRule("create", "get", "list", "watch").Groups(certificatesGroup).Resources("certificatesigningrequests").RuleOrDie(),

    // Leases
    rbacv1helpers.NewRule("get", "create", "update", "patch", "delete").Groups("coordination.k8s.io").Resources("leases").RuleOrDie(),

    // CSI
    rbacv1helpers.NewRule("get").Groups(storageGroup).Resources("volumeattachments").RuleOrDie(),

    // Use the Node authorization to limit a node to create tokens for service accounts running on that node
    // Use the NodeRestriction admission plugin to limit a node to create tokens bound to pods on that node
    rbacv1helpers.NewRule("create").Groups(legacyGroup).Resources("serviceaccounts/token").RuleOrDie(),
    }

    // Use the Node authorization mode to limit a node to update status of pvc objects referenced by pods bound to itself.
    // Use the NodeRestriction admission plugin to limit a node to just update the status stanza.
    pvcStatusPolicyRule := rbacv1helpers.NewRule("get", "update", "patch").Groups(legacyGroup).Resources("persistentvolumeclaims/status").RuleOrDie()
    nodePolicyRules = append(nodePolicyRules, pvcStatusPolicyRule)

    // CSI
    csiDriverRule := rbacv1helpers.NewRule("get", "watch", "list").Groups("storage.k8s.io").Resources("csidrivers").RuleOrDie()
    nodePolicyRules = append(nodePolicyRules, csiDriverRule)
    csiNodeInfoRule := rbacv1helpers.NewRule("get", "create", "update", "patch", "delete").Groups("storage.k8s.io").Resources("csinodes").RuleOrDie()
    nodePolicyRules = append(nodePolicyRules, csiNodeInfoRule)

    // RuntimeClass
    nodePolicyRules = append(nodePolicyRules, rbacv1helpers.NewRule("get", "list", "watch").Groups("node.k8s.io").Resources("runtimeclasses").RuleOrDie())
    return nodePolicyRules
    }

启用Node授权

kube-apiserver通过指定--authorization-mode=Node,RBAC参数启用Node授权器与RBAC授权器

Node授权实现原理

  • Ref:https://github.com/kubernetes/kubernetes/blob/88e994f6bf8fc88114c5b733e09afea339bea66d/plugin/pkg/auth/authorizer/node/node_authorizer.go#L93

    func (r *NodeAuthorizer) Authorize(ctx context.Context, attrs authorizer.Attributes) (authorizer.Decision, string, error) {
    nodeName, isNode := r.identifier.NodeIdentity(attrs.GetUser())
    if !isNode {
    // reject requests from non-nodes
    return authorizer.DecisionNoOpinion, "", nil
    }
    if len(nodeName) == 0 {
    // reject requests from unidentifiable nodes
    klog.V(2).Infof("NODE DENY: unknown node for user %q", attrs.GetUser().GetName())
    return authorizer.DecisionNoOpinion, fmt.Sprintf("unknown node for user %q", attrs.GetUser().GetName()), nil
    }

    // subdivide access to specific resources
    if attrs.IsResourceRequest() {
    requestResource := schema.GroupResource{Group: attrs.GetAPIGroup(), Resource: attrs.GetResource()}
    switch requestResource {
    case secretResource:
    return r.authorizeReadNamespacedObject(nodeName, secretVertexType, attrs)
    case configMapResource:
    return r.authorizeReadNamespacedObject(nodeName, configMapVertexType, attrs)
    case pvcResource:
    if attrs.GetSubresource() == "status" {
    return r.authorizeStatusUpdate(nodeName, pvcVertexType, attrs)
    }
    return r.authorizeGet(nodeName, pvcVertexType, attrs)
    case pvResource:
    return r.authorizeGet(nodeName, pvVertexType, attrs)
    case vaResource:
    return r.authorizeGet(nodeName, vaVertexType, attrs)
    case svcAcctResource:
    return r.authorizeCreateToken(nodeName, serviceAccountVertexType, attrs)
    case leaseResource:
    return r.authorizeLease(nodeName, attrs)
    case csiNodeResource:
    return r.authorizeCSINode(nodeName, attrs)
    }

    }

    // Access to other resources is not subdivided, so just evaluate against the statically defined node rules
    if rbac.RulesAllow(attrs, r.nodeRules...) {
    return authorizer.DecisionAllow, "", nil
    }
    return authorizer.DecisionNoOpinion, "", nil
    }