K8s-kube-apiserver(授权)
K8s-kube-apiserver(授权)
基于1.25
kube-apiserver的授权
客户端请求通过认证之后,进入授权阶段,校验对应用户是否具有对应数据读写的权限
- 支持多种授权机制,并且支持开启多个授权功能
- 如果开启多个授权,按照授权顺序执行
- 只要有一个授权器通过,授权就成功
目前支持六种授权模式:
- AlwaysAllow:允许所有请求
- AlwaysDeny:阻止所有请求
- ABAC:基于属性的访问控制
- Webhook:基于Webhook的一种HTTP回调,远程授权管理
- RBAC:基于角色的访问控制
- Node:节点模式,专门授权kubelet发出的API请求
- 集群默认开启Node和RBAC模式
在kube-apiserver中,Authorization授权有三个概念,分别是Decision决策状态、授权器接口和Rule Resolver规则解析器
Decision决策状态
Decision决策状态类似于身份认证中的true和false,用来表示授权是否成功
-
1
2
3
4
5
6
7
8
9
10
11
12const (
// DecisionDeny means that an authorizer decided to deny the action.
// 拒绝该操作
DecisionDeny Decision = iota
// DecisionAllow means that an authorizer decided to allow the action.
// 允许该操作
DecisionAllow
// DecisionNoOpionion means that an authorizer has no opinion on whether
// to allow or deny an action.
// 允许或者拒绝拒绝没有意见
DecisionNoOpinion
)
-
授权器接口
每种是授权器都要实现Aurhorizer接口
-
1
2
3
4
5
6// Authorizer makes an authorization decision based on information gained by making
// zero or more calls to methods of the Attributes interface. It returns nil when an action is
// authorized, otherwise it returns an error.
type Authorizer interface {
Authorize(ctx context.Context, a Attributes) (authorized Decision, reason string, err error)
}
-
RuleResovler规则解析器
-
1
2
3
4
5
6
7
8// RuleResolver provides a mechanism for resolving the list of rules that apply to a given user within a namespace.
type RuleResolver interface {
// RulesFor get the list of cluster wide rules, the list of rules in the specific namespace, incomplete status and errors.
// 解析出规则列表
// ResourceRuleInfo:资源类型的规则列表,如/api/v1/pods资源接口
// NonResourceRuleInfo:非资源类型的规则列表,如/api或/health资源接口
RulesFor(user user.Info, namespace string) ([]ResourceRuleInfo, []NonResourceRuleInfo, bool, error)
}
-
-
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37// WithAuthorizationCheck passes all authorized requests on to handler, and returns a forbidden error otherwise.
// 依次进行加载授权器
func WithAuthorization(handler http.Handler, a authorizer.Authorizer, s runtime.NegotiatedSerializer) http.Handler {
if a == nil {
klog.Warning("Authorization is disabled")
return handler
}
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
ctx := req.Context()
attributes, err := GetAuthorizerAttributes(ctx)
if err != nil {
responsewriters.InternalError(w, req, err)
return
}
authorized, reason, err := a.Authorize(ctx, attributes)
// an authorizer like RBAC could encounter evaluation errors and still allow the request, so authorizer decision is checked before error here.
if authorized == authorizer.DecisionAllow {
audit.AddAuditAnnotations(ctx,
decisionAnnotationKey, decisionAllow,
reasonAnnotationKey, reason)
handler.ServeHTTP(w, req)
return
}
if err != nil {
audit.AddAuditAnnotation(ctx, reasonAnnotationKey, reasonError)
responsewriters.InternalError(w, req, err)
return
}
klog.V(4).InfoS("Forbidden", "URI", req.RequestURI, "Reason", reason)
audit.AddAuditAnnotations(ctx,
decisionAnnotationKey, decisionForbid,
reasonAnnotationKey, reason)
responsewriters.Forbidden(ctx, attributes, w, req, reason, s)
})
}
AlwaysAllow授权
AlwaysAllow授权器允许所有请求,如果未配置--aurhorization-mode
,则默认使用此授权模式
启用AlwaysAllow授权
kube-apiserver通过制定--authorization-mode=AlwaysAllow
参数(或者不配置)启用授权
AlwaysAllow授权实现模式
-
1
2
3
4
5
6
7
8
9
10
11
12
13
14func (alwaysAllowAuthorizer) RulesFor(user user.Info, namespace string) ([]authorizer.ResourceRuleInfo, []authorizer.NonResourceRuleInfo, bool, error) {
return []authorizer.ResourceRuleInfo{
&authorizer.DefaultResourceRuleInfo{
Verbs: []string{"*"},
APIGroups: []string{"*"},
Resources: []string{"*"},
},
}, []authorizer.NonResourceRuleInfo{
&authorizer.DefaultNonResourceRuleInfo{
Verbs: []string{"*"},
NonResourceURLs: []string{"*"},
},
}, false, nil
}
AlwaysDeny授权
AlwaysDeny授权拒绝所有请求,很少单独使用
启用AlwaysDeny授权
kube-apiserver通过制定--authorization-mode=AlwaysDeny
参数启用AlwaysDeny授权
AlwaysDeny授权实现管理
直接返回DecisionNoOpinion,继续存在执行下一个授权器,如果不存在下一个,则拒绝所有请求
原理是把资源组设置为空
-
1
2
3func (alwaysDenyAuthorizer) Authorize(ctx context.Context, a authorizer.Attributes) (decision authorizer.Decision, reason string, err error) {
return authorizer.DecisionNoOpinion, "Everything is forbidden.", nil
}
ABAC授权
ABAC授权是一种基于属性的访问控制模式,能够根据属性配置信息为用户授予访问权限
启用ABAC授权
kube-apiserver通过指定以下参数启用ABAC授权
--authorization-mode=ABAC
:启用ABAC授权--aurhorization-policy-file
:指定策略模式,该文件使用JSON格式,每一行都是一个策略对象
ABAC授权实现原理
-
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
// Authorize implements authorizer.Authorize
func (pl PolicyList) Authorize(ctx context.Context, a authorizer.Attributes) (authorizer.Decision, string, error) {
for _, p := range pl {
if matches(*p, a) {
return authorizer.DecisionAllow, "", nil
}
}
return authorizer.DecisionNoOpinion, "No policy matched.", nil
// TODO: Benchmark how much time policy matching takes with a medium size
// policy file, compared to other steps such as encoding/decoding.
// Then, add Caching only if needed.
}
// RulesFor returns rules for the given user and namespace.
func (pl PolicyList) RulesFor(user user.Info, namespace string) ([]authorizer.ResourceRuleInfo, []authorizer.NonResourceRuleInfo, bool, error) {
var (
resourceRules []authorizer.ResourceRuleInfo
nonResourceRules []authorizer.NonResourceRuleInfo
)
for _, p := range pl {
if subjectMatches(*p, user) {
if p.Spec.Namespace == "*" || p.Spec.Namespace == namespace {
if len(p.Spec.Resource) > 0 {
r := authorizer.DefaultResourceRuleInfo{
Verbs: getVerbs(p.Spec.Readonly),
APIGroups: []string{p.Spec.APIGroup},
Resources: []string{p.Spec.Resource},
}
var resourceRule authorizer.ResourceRuleInfo = &r
resourceRules = append(resourceRules, resourceRule)
}
if len(p.Spec.NonResourcePath) > 0 {
r := authorizer.DefaultNonResourceRuleInfo{
Verbs: getVerbs(p.Spec.Readonly),
NonResourceURLs: []string{p.Spec.NonResourcePath},
}
var nonResourceRule authorizer.NonResourceRuleInfo = &r
nonResourceRules = append(nonResourceRules, nonResourceRule)
}
}
}
}
return resourceRules, nonResourceRules, false, nil
}
Webhook授权
Webhook授权是一种基于HTTP回调的机制,当用户需要授权,kube-apiserver通过查询外部的Webhook服务器获得授权结果
启用Webhook授权
kube-apiserver通过制定以下参数启用Webhook授权
--authorization-mode=Webhook
:启用Webhook授权器--authorization-webhook-config-file
:kubeconfig格式的Webhook配置文件,描述如何访问远程Webhook服务器
Example:
1 | # Kubernetes API 版本 |
Webhook实现原理
- 首选尝试 w.responseCache.Get(string(key));缓存中查找
-
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
// Authorize makes a REST request to the remote service describing the attempted action as a JSON
// serialized api.authorization.v1beta1.SubjectAccessReview object. An example request body is
// provided below.
//
// {
// "apiVersion": "authorization.k8s.io/v1beta1",
// "kind": "SubjectAccessReview",
// "spec": {
// "resourceAttributes": {
// "namespace": "kittensandponies",
// "verb": "GET",
// "group": "group3",
// "resource": "pods"
// },
// "user": "jane",
// "group": [
// "group1",
// "group2"
// ]
// }
// }
//
// The remote service is expected to fill the SubjectAccessReviewStatus field to either allow or
// disallow access. A permissive response would return:
//
// {
// "apiVersion": "authorization.k8s.io/v1beta1",
// "kind": "SubjectAccessReview",
// "status": {
// "allowed": true
// }
// }
//
// To disallow access, the remote service would return:
//
// {
// "apiVersion": "authorization.k8s.io/v1beta1",
// "kind": "SubjectAccessReview",
// "status": {
// "allowed": false,
// "reason": "user does not have read access to the namespace"
// }
// }
//
// TODO(mikedanese): We should eventually support failing closed when we
// encounter an error. We are failing open now to preserve backwards compatible
// behavior.
func (w *WebhookAuthorizer) Authorize(ctx context.Context, attr authorizer.Attributes) (decision authorizer.Decision, reason string, err error) {
r := &authorizationv1.SubjectAccessReview{}
if user := attr.GetUser(); user != nil {
r.Spec = authorizationv1.SubjectAccessReviewSpec{
User: user.GetName(),
UID: user.GetUID(),
Groups: user.GetGroups(),
Extra: convertToSARExtra(user.GetExtra()),
}
}
if attr.IsResourceRequest() {
r.Spec.ResourceAttributes = &authorizationv1.ResourceAttributes{
Namespace: attr.GetNamespace(),
Verb: attr.GetVerb(),
Group: attr.GetAPIGroup(),
Version: attr.GetAPIVersion(),
Resource: attr.GetResource(),
Subresource: attr.GetSubresource(),
Name: attr.GetName(),
}
} else {
r.Spec.NonResourceAttributes = &authorizationv1.NonResourceAttributes{
Path: attr.GetPath(),
Verb: attr.GetVerb(),
}
}
key, err := json.Marshal(r.Spec)
if err != nil {
return w.decisionOnError, "", err
}
if entry, ok := w.responseCache.Get(string(key)); ok {
r.Status = entry.(authorizationv1.SubjectAccessReviewStatus)
} else {
var result *authorizationv1.SubjectAccessReview
// WithExponentialBackoff will return SAR create error (sarErr) if any.
if err := webhook.WithExponentialBackoff(ctx, w.retryBackoff, func() error {
var sarErr error
var statusCode int
start := time.Now()
result, statusCode, sarErr = w.subjectAccessReview.Create(ctx, r, metav1.CreateOptions{})
latency := time.Since(start)
if statusCode != 0 {
w.metrics.RecordRequestTotal(ctx, strconv.Itoa(statusCode))
w.metrics.RecordRequestLatency(ctx, strconv.Itoa(statusCode), latency.Seconds())
return sarErr
}
if sarErr != nil {
w.metrics.RecordRequestTotal(ctx, "<error>")
w.metrics.RecordRequestLatency(ctx, "<error>", latency.Seconds())
}
return sarErr
}, webhook.DefaultShouldRetry); err != nil {
klog.Errorf("Failed to make webhook authorizer request: %v", err)
return w.decisionOnError, "", err
}
r.Status = result.Status
if shouldCache(attr) {
if r.Status.Allowed {
w.responseCache.Add(string(key), r.Status, w.authorizedTTL)
} else {
w.responseCache.Add(string(key), r.Status, w.unauthorizedTTL)
}
}
}
switch {
case r.Status.Denied && r.Status.Allowed:
return authorizer.DecisionDeny, r.Status.Reason, fmt.Errorf("webhook subject access review returned both allow and deny response")
case r.Status.Denied:
return authorizer.DecisionDeny, r.Status.Reason, nil
case r.Status.Allowed:
return authorizer.DecisionAllow, r.Status.Reason, nil
default:
return authorizer.DecisionNoOpinion, r.Status.Reason, nil
}
}
RBAC授权
RBAC是基于角色访问控制
RBAC核心数据结构
在kube-apiserver中设计RBAC 添加咯角色和集群绑定的概念
- kube-apiserver,可以通过Role、ClusterRole、RoleBinding、ClusterRoleBinding四种表示角色授权
Rol和ClusterRole:
- Role:角色是一组用户的集合,与规则关联,角色只能赋予一个namespace的权限
- ClusterRole:能够赋予集群范围内的权限,比如节点、非资源类型的服务端点
- PolicyRule:操作权限,定义了何种资源何种权限
RoleBinding和ClusterRoleBinding:
- Subject:主体可以是Group、User和ServiceAccount
- RoleBinding:将角色关联的权限赋予一个或一组用户,只能授予一个namespace权限
- ClusterRoleBinding:将角色关联的权限赋予一个或一组用户,赋予集群内权限
- Role Ref:被授予权限的引用信息
启用RBAC授权
kube-apiserver通过指定--aurhorization-mode=RBAC
启用
RBAC授权实现原理
1 | apiVersion: rbac.authorization.k8s.io/v1 |
创建内置集群角色
kube-apiserver在启动时会通过rbac/bootstrap-roles PostStartHook初始化内置的角色
cluster-admin拥有对K8s的最高权限
-
1
2
3
4
5
6
7
8
9
10
11
12// ClusterRoles returns the cluster roles to bootstrap an API server with
func ClusterRoles() []rbacv1.ClusterRole {
roles := []rbacv1.ClusterRole{
{
// a "root" role which can do absolutely anything
ObjectMeta: metav1.ObjectMeta{Name: "cluster-admin"},
Rules: []rbacv1.PolicyRule{
rbacv1helpers.NewRule("*").Groups("*").Resources("*").RuleOrDie(),
rbacv1helpers.NewRule("*").URLs("*").RuleOrDie(),
},
},
...
Node授权
Node授权是一种特殊用途的授权模式,专门为kubelet发出的API请求进行授权
授权基于RBAC授权方式,对kubelet进行基于systen:node内置角色的权限控制
-
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73/ NodeRules returns node policy rules, it is slice of rbacv1.PolicyRule.
func NodeRules() []rbacv1.PolicyRule {
nodePolicyRules := []rbacv1.PolicyRule{
// Needed to check API access. These creates are non-mutating
rbacv1helpers.NewRule("create").Groups(authenticationGroup).Resources("tokenreviews").RuleOrDie(),
rbacv1helpers.NewRule("create").Groups(authorizationGroup).Resources("subjectaccessreviews", "localsubjectaccessreviews").RuleOrDie(),
// Needed to build serviceLister, to populate env vars for services
rbacv1helpers.NewRule(Read...).Groups(legacyGroup).Resources("services").RuleOrDie(),
// Nodes can register Node API objects and report status.
// Use the NodeRestriction admission plugin to limit a node to creating/updating its own API object.
rbacv1helpers.NewRule("create", "get", "list", "watch").Groups(legacyGroup).Resources("nodes").RuleOrDie(),
rbacv1helpers.NewRule("update", "patch").Groups(legacyGroup).Resources("nodes/status").RuleOrDie(),
rbacv1helpers.NewRule("update", "patch").Groups(legacyGroup).Resources("nodes").RuleOrDie(),
// TODO: restrict to the bound node as creator in the NodeRestrictions admission plugin
rbacv1helpers.NewRule("create", "update", "patch").Groups(legacyGroup).Resources("events").RuleOrDie(),
// TODO: restrict to pods scheduled on the bound node once field selectors are supported by list/watch authorization
rbacv1helpers.NewRule(Read...).Groups(legacyGroup).Resources("pods").RuleOrDie(),
// Needed for the node to create/delete mirror pods.
// Use the NodeRestriction admission plugin to limit a node to creating/deleting mirror pods bound to itself.
rbacv1helpers.NewRule("create", "delete").Groups(legacyGroup).Resources("pods").RuleOrDie(),
// Needed for the node to report status of pods it is running.
// Use the NodeRestriction admission plugin to limit a node to updating status of pods bound to itself.
rbacv1helpers.NewRule("update", "patch").Groups(legacyGroup).Resources("pods/status").RuleOrDie(),
// Needed for the node to create pod evictions.
// Use the NodeRestriction admission plugin to limit a node to creating evictions for pods bound to itself.
rbacv1helpers.NewRule("create").Groups(legacyGroup).Resources("pods/eviction").RuleOrDie(),
// Needed for imagepullsecrets, rbd/ceph and secret volumes, and secrets in envs
// Needed for configmap volume and envs
// Use the Node authorization mode to limit a node to get secrets/configmaps referenced by pods bound to itself.
rbacv1helpers.NewRule("get", "list", "watch").Groups(legacyGroup).Resources("secrets", "configmaps").RuleOrDie(),
// Needed for persistent volumes
// Use the Node authorization mode to limit a node to get pv/pvc objects referenced by pods bound to itself.
rbacv1helpers.NewRule("get").Groups(legacyGroup).Resources("persistentvolumeclaims", "persistentvolumes").RuleOrDie(),
// TODO: add to the Node authorizer and restrict to endpoints referenced by pods or PVs bound to the node
// Needed for glusterfs volumes
rbacv1helpers.NewRule("get").Groups(legacyGroup).Resources("endpoints").RuleOrDie(),
// Used to create a certificatesigningrequest for a node-specific client certificate, and watch
// for it to be signed. This allows the kubelet to rotate it's own certificate.
rbacv1helpers.NewRule("create", "get", "list", "watch").Groups(certificatesGroup).Resources("certificatesigningrequests").RuleOrDie(),
// Leases
rbacv1helpers.NewRule("get", "create", "update", "patch", "delete").Groups("coordination.k8s.io").Resources("leases").RuleOrDie(),
// CSI
rbacv1helpers.NewRule("get").Groups(storageGroup).Resources("volumeattachments").RuleOrDie(),
// Use the Node authorization to limit a node to create tokens for service accounts running on that node
// Use the NodeRestriction admission plugin to limit a node to create tokens bound to pods on that node
rbacv1helpers.NewRule("create").Groups(legacyGroup).Resources("serviceaccounts/token").RuleOrDie(),
}
// Use the Node authorization mode to limit a node to update status of pvc objects referenced by pods bound to itself.
// Use the NodeRestriction admission plugin to limit a node to just update the status stanza.
pvcStatusPolicyRule := rbacv1helpers.NewRule("get", "update", "patch").Groups(legacyGroup).Resources("persistentvolumeclaims/status").RuleOrDie()
nodePolicyRules = append(nodePolicyRules, pvcStatusPolicyRule)
// CSI
csiDriverRule := rbacv1helpers.NewRule("get", "watch", "list").Groups("storage.k8s.io").Resources("csidrivers").RuleOrDie()
nodePolicyRules = append(nodePolicyRules, csiDriverRule)
csiNodeInfoRule := rbacv1helpers.NewRule("get", "create", "update", "patch", "delete").Groups("storage.k8s.io").Resources("csinodes").RuleOrDie()
nodePolicyRules = append(nodePolicyRules, csiNodeInfoRule)
// RuntimeClass
nodePolicyRules = append(nodePolicyRules, rbacv1helpers.NewRule("get", "list", "watch").Groups("node.k8s.io").Resources("runtimeclasses").RuleOrDie())
return nodePolicyRules
}
启用Node授权
kube-apiserver通过指定--authorization-mode=Node,RBAC
参数启用Node授权器与RBAC授权器
Node授权实现原理
-
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45func (r *NodeAuthorizer) Authorize(ctx context.Context, attrs authorizer.Attributes) (authorizer.Decision, string, error) {
nodeName, isNode := r.identifier.NodeIdentity(attrs.GetUser())
if !isNode {
// reject requests from non-nodes
return authorizer.DecisionNoOpinion, "", nil
}
if len(nodeName) == 0 {
// reject requests from unidentifiable nodes
klog.V(2).Infof("NODE DENY: unknown node for user %q", attrs.GetUser().GetName())
return authorizer.DecisionNoOpinion, fmt.Sprintf("unknown node for user %q", attrs.GetUser().GetName()), nil
}
// subdivide access to specific resources
if attrs.IsResourceRequest() {
requestResource := schema.GroupResource{Group: attrs.GetAPIGroup(), Resource: attrs.GetResource()}
switch requestResource {
case secretResource:
return r.authorizeReadNamespacedObject(nodeName, secretVertexType, attrs)
case configMapResource:
return r.authorizeReadNamespacedObject(nodeName, configMapVertexType, attrs)
case pvcResource:
if attrs.GetSubresource() == "status" {
return r.authorizeStatusUpdate(nodeName, pvcVertexType, attrs)
}
return r.authorizeGet(nodeName, pvcVertexType, attrs)
case pvResource:
return r.authorizeGet(nodeName, pvVertexType, attrs)
case vaResource:
return r.authorizeGet(nodeName, vaVertexType, attrs)
case svcAcctResource:
return r.authorizeCreateToken(nodeName, serviceAccountVertexType, attrs)
case leaseResource:
return r.authorizeLease(nodeName, attrs)
case csiNodeResource:
return r.authorizeCSINode(nodeName, attrs)
}
}
// Access to other resources is not subdivided, so just evaluate against the statically defined node rules
if rbac.RulesAllow(attrs, r.nodeRules...) {
return authorizer.DecisionAllow, "", nil
}
return authorizer.DecisionNoOpinion, "", nil
}