audit審計的總結
- Kubernetes 審計（Auditing） 功能提供了與安全相關的、按時間順序排列的記錄集，記錄每個用戶、使用 Kubernetes API 的應用以及控制面自身引發的活動
- 審計功能使得集羣管理員能夠回答以下問題：
- 發生了什麼？
- 什麼時候發生的？
- 誰觸發的？
- 活動發生在哪個（些）對象上？
- 在哪觀察到的？
- 它從哪觸發的？
- 活動的後續處理行爲是什麼？
- 審計策略
- None - 符合這條規則的日誌將不會記錄。
- Metadata - 記錄請求的元數據（請求的用戶、時間戳、資源、動詞等等）， 但是不記錄請求或者響應的消息體。
- Request - 記錄事件的元數據和請求的消息體，但是不記錄響應的消息體。 這不適用於非資源類型的請求。
- RequestResponse - 記錄事件的元數據，請求和響應的消息體。這不適用於非資源類型的請求。
審計功能介紹
audit源碼閱讀
- 入口位置 D:\go_path\src\github.com\kubernetes\kubernetes\cmd\kube-apiserver\app\server.go
- buildGenericConfig
lastErr = s.Audit.ApplyTo(genericConfig)
if lastErr != nil {
return
}
1. 從配置的 --audit-policy-file加載audit策略
- 你可以使用 --audit-policy-file 標誌將包含策略的文件傳遞給 kube-apiserver
- 如果不設置該標誌，則不記錄事件
- rules 字段 必須 在審計策略文件中提供。沒有（0）規則的策略將被視爲非法配置。
// 1. Build policy evaluator
evaluator, err := o.newPolicyRuleEvaluator()
if err != nil {
return err
}
2. 從配置的 --audit-log-path設置 logBackend
// 2. Build log backend
var logBackend audit.Backend
w, err := o.LogOptions.getWriter()
if err != nil {
return err
}
- 如果後端 --audit-log-path="-" 代表記錄到標準輸出
func (o *AuditLogOptions) getWriter() (io.Writer, error) {
if !o.enabled() {
return nil, nil
}
if o.Path == "-" {
return os.Stdout, nil
}
if err := o.ensureLogFile(); err != nil {
return nil, fmt.Errorf("ensureLogFile: %w", err)
}
return &lumberjack.Logger{
Filename: o.Path,
MaxAge: o.MaxAge,
MaxBackups: o.MaxBackups,
MaxSize: o.MaxSize,
Compress: o.Compress,
}, nil
}
獲取到日誌writer對象後校驗下有沒有evaluator
- 如果沒有evaluator，打印條提示日誌
- 如果將backend設置爲w
if w != nil {
if evaluator == nil {
klog.V(2).Info("No audit policy file provided, no events will be recorded for log backend")
} else {
logBackend = o.LogOptions.newBackend(w)
}
}
3 根據配置構建webhook的 後端
// 3. Build webhook backend
var webhookBackend audit.Backend
if o.WebhookOptions.enabled() {
if evaluator == nil {
klog.V(2).Info("No audit policy file provided, no events will be recorded for webhook backend")
} else {
if c.EgressSelector != nil {
var egressDialer utilnet.DialFunc
egressDialer, err = c.EgressSelector.Lookup(egressselector.ControlPlane.AsNetworkContext())
if err != nil {
return err
}
webhookBackend, err = o.WebhookOptions.newUntruncatedBackend(egressDialer)
} else {
webhookBackend, err = o.WebhookOptions.newUntruncatedBackend(nil)
}
if err != nil {
return err
}
}
}
4 如果有webhook就把它封裝爲dynamicBackend
// 4. Apply dynamic options.
var dynamicBackend audit.Backend
if webhookBackend != nil {
// if only webhook is enabled wrap it in the truncate options
dynamicBackend = o.WebhookOptions.TruncateOptions.wrapBackend(webhookBackend, groupVersion)
}
5. 設置審計的策略計算對象 evaluator
// 5. Set the policy rule evaluator
c.AuditPolicyRuleEvaluator = evaluator
6 把logBackend和 dynamicBackend 做union
// 6. Join the log backend with the webhooks
c.AuditBackend = appendBackend(logBackend, dynamicBackend)
func appendBackend(existing, newBackend audit.Backend) audit.Backend {
if existing == nil {
return newBackend
}
if newBackend == nil {
return existing
}
return audit.Union(existing, newBackend)
}
7.最終的運行方法
- backend接口方法
- D:\go_path\src\github.com\kubernetes\kubernetes\staging\src\k8s.io\apiserver\pkg\audit\types.go
type Sink interface {
// ProcessEvents handles events. Per audit ID it might be that ProcessEvents is called up to three times.
// Errors might be logged by the sink itself. If an error should be fatal, leading to an internal
// error, ProcessEvents is supposed to panic. The event must not be mutated and is reused by the caller
// after the call returns, i.e. the sink has to make a deepcopy to keep a copy around if necessary.
// Returns true on success, may return false on error.
ProcessEvents(events ...*auditinternal.Event) bool
}
type Backend interface {
Sink
// Run will initialize the backend. It must not block, but may run go routines in the background. If
// stopCh is closed, it is supposed to stop them. Run will be called before the first call to ProcessEvents.
Run(stopCh <-chan struct{}) error
// Shutdown will synchronously shut down the backend while making sure that all pending
// events are delivered. It can be assumed that this method is called after
// the stopCh channel passed to the Run method has been closed.
Shutdown()
// Returns the backend PluginName.
String() string
}
- 最終調用audit的ProcessEvents方法，以log舉例，位置 D:\go_path\src\github.com\kubernetes\kubernetes\staging\src\k8s.io\apiserver\plugin\pkg\audit\log\backend.go
- 流程分析，根據format類型做最後的line生成，然後調用 fmt.Fprint(b.out, line)，向b.out中寫入line即可
func (b *backend) ProcessEvents(events ...*auditinternal.Event) bool {
success := true
for _, ev := range events {
success = b.logEvent(ev) && success
}
return success
}
func (b *backend) logEvent(ev *auditinternal.Event) bool {
line := ""
switch b.format {
case FormatLegacy:
line = audit.EventString(ev) + "\n"
case FormatJson:
bs, err := runtime.Encode(b.encoder, ev)
if err != nil {
audit.HandlePluginError(PluginName, err, ev)
return false
}
line = string(bs[:])
default:
audit.HandlePluginError(PluginName, fmt.Errorf("log format %q is not in list of known formats (%s)",
b.format, strings.Join(AllowedFormats, ",")), ev)
return false
}
if _, err := fmt.Fprint(b.out, line); err != nil {
audit.HandlePluginError(PluginName, err, ev)
return false
}
return true
}
8. http側調用的handler
- D:\go_path\src\github.com\kubernetes\kubernetes\staging\src\k8s.io\apiserver\pkg\endpoints\filters\audit.go
// WithAudit decorates a http.Handler with audit logging information for all the
// requests coming to the server. Audit level is decided according to requests'
// attributes and audit policy. Logs are emitted to the audit sink to
// process events. If sink or audit policy is nil, no decoration takes place.
func WithAudit(handler http.Handler, sink audit.Sink, policy audit.PolicyRuleEvaluator, longRunningCheck request.LongRunningRequestCheck) http.Handler {
if sink == nil || policy == nil {
return handler
}
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
req, ev, omitStages, err := createAuditEventAndAttachToContext(req, policy)
if err != nil {
utilruntime.HandleError(fmt.Errorf("failed to create audit event: %v", err))
responsewriters.InternalError(w, req, errors.New("failed to create audit event"))
return
}
ctx := req.Context()
if ev == nil || ctx == nil {
handler.ServeHTTP(w, req)
return
}
ev.Stage = auditinternal.StageRequestReceived
if processed := processAuditEvent(ctx, sink, ev, omitStages); !processed {
audit.ApiserverAuditDroppedCounter.WithContext(ctx).Inc()
responsewriters.InternalError(w, req, errors.New("failed to store audit event"))
return
}
// intercept the status code
var longRunningSink audit.Sink
if longRunningCheck != nil {
ri, _ := request.RequestInfoFrom(ctx)
if longRunningCheck(req, ri) {
longRunningSink = sink
}
}
respWriter := decorateResponseWriter(ctx, w, ev, longRunningSink, omitStages)
// send audit event when we leave this func, either via a panic or cleanly. In the case of long
// running requests, this will be the second audit event.
defer func() {
if r := recover(); r != nil {
defer panic(r)
ev.Stage = auditinternal.StagePanic
ev.ResponseStatus = &metav1.Status{
Code: http.StatusInternalServerError,
Status: metav1.StatusFailure,
Reason: metav1.StatusReasonInternalError,
Message: fmt.Sprintf("APIServer panic'd: %v", r),
}
processAuditEvent(ctx, sink, ev, omitStages)
return
}
// if no StageResponseStarted event was sent b/c neither a status code nor a body was sent, fake it here
// But Audit-Id http header will only be sent when http.ResponseWriter.WriteHeader is called.
fakedSuccessStatus := &metav1.Status{
Code: http.StatusOK,
Status: metav1.StatusSuccess,
Message: "Connection closed early",
}
if ev.ResponseStatus == nil && longRunningSink != nil {
ev.ResponseStatus = fakedSuccessStatus
ev.Stage = auditinternal.StageResponseStarted
processAuditEvent(ctx, longRunningSink, ev, omitStages)
}
ev.Stage = auditinternal.StageResponseComplete
if ev.ResponseStatus == nil {
ev.ResponseStatus = fakedSuccessStatus
}
processAuditEvent(ctx, sink, ev, omitStages)
}()
handler.ServeHTTP(respWriter, req)
})
}
audit審計的總結
- Kubernetes 審計（Auditing） 功能提供了與安全相關的、按時間順序排列的記錄集，記錄每個用戶、使用 Kubernetes API 的應用以及控制面自身引發的活動
- 審計功能使得集羣管理員能夠回答以下問題：
- 發生了什麼？
- 什麼時候發生的？
- 誰觸發的？
- 活動發生在哪個（些）對象上？
- 在哪觀察到的？
- 它從哪觸發的？
- 活動的後續處理行爲是什麼？
- 審計策略
- None - 符合這條規則的日誌將不會記錄。
- Metadata - 記錄請求的元數據（請求的用戶、時間戳、資源、動詞等等）， 但是不記錄請求或者響應的消息體。
- Request - 記錄事件的元數據和請求的消息體，但是不記錄響應的消息體。 這不適用於非資源類型的請求。
- RequestResponse - 記錄事件的元數據，請求和響應的消息體。這不適用於非資源類型的請求。
