Continental Innovates with Rancher and Kubernetes
You can enable the API audit log to record the sequence of system events initiated by individual users. You can know what happened, when it happened, who initiated it, and what cluster it affected. When you enable this feature, all requests to the Rancher API and all responses from it are written to a log.
You can enable API Auditing during Rancher installation or upgrade.
The Audit Log is enabled and configured by passing environment variables to the Rancher server container. See the following to enable on your installation.
Docker Install
Kubernetes Install
The usage below defines rules about what the audit log should record and what data it should include:
AUDIT_LEVEL
0
1
2
3
auditID
AUDIT_LOG_PATH
/var/log/auditlog/rancher-api-audit.log
AUDIT_LOG_PATH=/my/custom/path/
AUDIT_LOG_MAXAGE
AUDIT_LOG_MAXBACKUP
AUDIT_LOG_MAXSIZE
The following table displays what parts of API transactions are logged for each AUDIT_LEVEL setting.
Share the AUDIT_LOG_PATH directory (Default: /var/log/auditlog) with the host system. The log can be parsed by standard CLI tools or forwarded on to a log collection tool like Fluentd, Filebeat, Logstash, etc.
/var/log/auditlog
Enabling the API Audit Log with the Helm chart install will create a rancher-audit-log sidecar container in the Rancher pod. This container will stream the log to standard output (stdout). You can view the log as you would any container log.
rancher-audit-log
The rancher-audit-log container is part of the rancher pod in the cattle-system namespace.
rancher
cattle-system
kubectl -n cattle-system logs -f rancher-84d886bdbb-s4s69 rancher-audit-log
You can enable Rancher’s built in log collection and shipping for the cluster to ship the audit and other services logs to a supported collection endpoint. See Rancher Tools - Logging for details.
After you enable auditing, each API request or response is logged by Rancher in the form of JSON. Each of the following code samples provide examples of how to identify each API transaction.
If you set your AUDIT_LEVEL to 1, Rancher logs the metadata header for every API request, but not the body. The header provides basic information about the API transaction, such as the transaction’s ID, who initiated the transaction, the time it occurred, etc.
{ "auditID": "30022177-9e2e-43d1-b0d0-06ef9d3db183", "requestURI": "/v3/schemas", "sourceIPs": ["::1"], "user": { "name": "user-f4tt2", "group": ["system:authenticated"] }, "verb": "GET", "stage": "RequestReceived", "stageTimestamp": "2018-07-20 10:22:43 +0800" }
If you set your AUDIT_LEVEL to 2, Rancher logs the metadata header and body for every API request.
The code sample below depicts an API request, with both its metadata header and body.
{ "auditID": "ef1d249e-bfac-4fd0-a61f-cbdcad53b9bb", "requestURI": "/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx", "sourceIPs": ["::1"], "user": { "name": "user-f4tt2", "group": ["system:authenticated"] }, "verb": "PUT", "stage": "RequestReceived", "stageTimestamp": "2018-07-20 10:28:08 +0800", "requestBody": { "hostIPC": false, "hostNetwork": false, "hostPID": false, "paused": false, "annotations": {}, "baseType": "workload", "containers": [ { "allowPrivilegeEscalation": false, "image": "nginx", "imagePullPolicy": "Always", "initContainer": false, "name": "nginx", "ports": [ { "containerPort": 80, "dnsName": "nginx-nodeport", "kind": "NodePort", "name": "80tcp01", "protocol": "TCP", "sourcePort": 0, "type": "/v3/project/schemas/containerPort" } ], "privileged": false, "readOnly": false, "resources": { "type": "/v3/project/schemas/resourceRequirements", "requests": {}, "limits": {} }, "restartCount": 0, "runAsNonRoot": false, "stdin": true, "stdinOnce": false, "terminationMessagePath": "/dev/termination-log", "terminationMessagePolicy": "File", "tty": true, "type": "/v3/project/schemas/container", "environmentFrom": [], "capAdd": [], "capDrop": [], "livenessProbe": null, "volumeMounts": [] } ], "created": "2018-07-18T07:34:16Z", "createdTS": 1531899256000, "creatorId": null, "deploymentConfig": { "maxSurge": 1, "maxUnavailable": 0, "minReadySeconds": 0, "progressDeadlineSeconds": 600, "revisionHistoryLimit": 10, "strategy": "RollingUpdate" }, "deploymentStatus": { "availableReplicas": 1, "conditions": [ { "lastTransitionTime": "2018-07-18T07:34:38Z", "lastTransitionTimeTS": 1531899278000, "lastUpdateTime": "2018-07-18T07:34:38Z", "lastUpdateTimeTS": 1531899278000, "message": "Deployment has minimum availability.", "reason": "MinimumReplicasAvailable", "status": "True", "type": "Available" }, { "lastTransitionTime": "2018-07-18T07:34:16Z", "lastTransitionTimeTS": 1531899256000, "lastUpdateTime": "2018-07-18T07:34:38Z", "lastUpdateTimeTS": 1531899278000, "message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.", "reason": "NewReplicaSetAvailable", "status": "True", "type": "Progressing" } ], "observedGeneration": 2, "readyReplicas": 1, "replicas": 1, "type": "/v3/project/schemas/deploymentStatus", "unavailableReplicas": 0, "updatedReplicas": 1 }, "dnsPolicy": "ClusterFirst", "id": "deployment:default:nginx", "labels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" }, "name": "nginx", "namespaceId": "default", "projectId": "c-bcz5t:p-fdr4s", "publicEndpoints": [ { "addresses": ["10.64.3.58"], "allNodes": true, "ingressId": null, "nodeId": null, "podId": null, "port": 30917, "protocol": "TCP", "serviceId": "default:nginx-nodeport", "type": "publicEndpoint" } ], "restartPolicy": "Always", "scale": 1, "schedulerName": "default-scheduler", "selector": { "matchLabels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" }, "type": "/v3/project/schemas/labelSelector" }, "state": "active", "terminationGracePeriodSeconds": 30, "transitioning": "no", "transitioningMessage": "", "type": "deployment", "uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd", "workloadAnnotations": { "deployment.kubernetes.io/revision": "1", "field.cattle.io/creatorId": "user-f4tt2" }, "workloadLabels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" }, "scheduling": { "node": {} }, "description": "my description", "volumes": [] } }
If you set your AUDIT_LEVEL to 3, Rancher logs:
{ "auditID": "a886fd9f-5d6b-4ae3-9a10-5bff8f3d68af", "requestURI": "/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx", "sourceIPs": ["::1"], "user": { "name": "user-f4tt2", "group": ["system:authenticated"] }, "verb": "PUT", "stage": "RequestReceived", "stageTimestamp": "2018-07-20 10:33:06 +0800", "requestBody": { "hostIPC": false, "hostNetwork": false, "hostPID": false, "paused": false, "annotations": {}, "baseType": "workload", "containers": [ { "allowPrivilegeEscalation": false, "image": "nginx", "imagePullPolicy": "Always", "initContainer": false, "name": "nginx", "ports": [ { "containerPort": 80, "dnsName": "nginx-nodeport", "kind": "NodePort", "name": "80tcp01", "protocol": "TCP", "sourcePort": 0, "type": "/v3/project/schemas/containerPort" } ], "privileged": false, "readOnly": false, "resources": { "type": "/v3/project/schemas/resourceRequirements", "requests": {}, "limits": {} }, "restartCount": 0, "runAsNonRoot": false, "stdin": true, "stdinOnce": false, "terminationMessagePath": "/dev/termination-log", "terminationMessagePolicy": "File", "tty": true, "type": "/v3/project/schemas/container", "environmentFrom": [], "capAdd": [], "capDrop": [], "livenessProbe": null, "volumeMounts": [] } ], "created": "2018-07-18T07:34:16Z", "createdTS": 1531899256000, "creatorId": null, "deploymentConfig": { "maxSurge": 1, "maxUnavailable": 0, "minReadySeconds": 0, "progressDeadlineSeconds": 600, "revisionHistoryLimit": 10, "strategy": "RollingUpdate" }, "deploymentStatus": { "availableReplicas": 1, "conditions": [ { "lastTransitionTime": "2018-07-18T07:34:38Z", "lastTransitionTimeTS": 1531899278000, "lastUpdateTime": "2018-07-18T07:34:38Z", "lastUpdateTimeTS": 1531899278000, "message": "Deployment has minimum availability.", "reason": "MinimumReplicasAvailable", "status": "True", "type": "Available" }, { "lastTransitionTime": "2018-07-18T07:34:16Z", "lastTransitionTimeTS": 1531899256000, "lastUpdateTime": "2018-07-18T07:34:38Z", "lastUpdateTimeTS": 1531899278000, "message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.", "reason": "NewReplicaSetAvailable", "status": "True", "type": "Progressing" } ], "observedGeneration": 2, "readyReplicas": 1, "replicas": 1, "type": "/v3/project/schemas/deploymentStatus", "unavailableReplicas": 0, "updatedReplicas": 1 }, "dnsPolicy": "ClusterFirst", "id": "deployment:default:nginx", "labels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" }, "name": "nginx", "namespaceId": "default", "projectId": "c-bcz5t:p-fdr4s", "publicEndpoints": [ { "addresses": ["10.64.3.58"], "allNodes": true, "ingressId": null, "nodeId": null, "podId": null, "port": 30917, "protocol": "TCP", "serviceId": "default:nginx-nodeport", "type": "publicEndpoint" } ], "restartPolicy": "Always", "scale": 1, "schedulerName": "default-scheduler", "selector": { "matchLabels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" }, "type": "/v3/project/schemas/labelSelector" }, "state": "active", "terminationGracePeriodSeconds": 30, "transitioning": "no", "transitioningMessage": "", "type": "deployment", "uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd", "workloadAnnotations": { "deployment.kubernetes.io/revision": "1", "field.cattle.io/creatorId": "user-f4tt2" }, "workloadLabels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" }, "scheduling": { "node": {} }, "description": "my decript", "volumes": [] } }
The code sample below depicts an API response, with both its metadata header and body.
{ "auditID": "a886fd9f-5d6b-4ae3-9a10-5bff8f3d68af", "responseStatus": "200", "stage": "ResponseComplete", "stageTimestamp": "2018-07-20 10:33:06 +0800", "responseBody": { "actionLinks": { "pause": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=pause", "resume": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=resume", "rollback": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=rollback" }, "annotations": {}, "baseType": "workload", "containers": [ { "allowPrivilegeEscalation": false, "image": "nginx", "imagePullPolicy": "Always", "initContainer": false, "name": "nginx", "ports": [ { "containerPort": 80, "dnsName": "nginx-nodeport", "kind": "NodePort", "name": "80tcp01", "protocol": "TCP", "sourcePort": 0, "type": "/v3/project/schemas/containerPort" } ], "privileged": false, "readOnly": false, "resources": { "type": "/v3/project/schemas/resourceRequirements" }, "restartCount": 0, "runAsNonRoot": false, "stdin": true, "stdinOnce": false, "terminationMessagePath": "/dev/termination-log", "terminationMessagePolicy": "File", "tty": true, "type": "/v3/project/schemas/container" } ], "created": "2018-07-18T07:34:16Z", "createdTS": 1531899256000, "creatorId": null, "deploymentConfig": { "maxSurge": 1, "maxUnavailable": 0, "minReadySeconds": 0, "progressDeadlineSeconds": 600, "revisionHistoryLimit": 10, "strategy": "RollingUpdate" }, "deploymentStatus": { "availableReplicas": 1, "conditions": [ { "lastTransitionTime": "2018-07-18T07:34:38Z", "lastTransitionTimeTS": 1531899278000, "lastUpdateTime": "2018-07-18T07:34:38Z", "lastUpdateTimeTS": 1531899278000, "message": "Deployment has minimum availability.", "reason": "MinimumReplicasAvailable", "status": "True", "type": "Available" }, { "lastTransitionTime": "2018-07-18T07:34:16Z", "lastTransitionTimeTS": 1531899256000, "lastUpdateTime": "2018-07-18T07:34:38Z", "lastUpdateTimeTS": 1531899278000, "message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.", "reason": "NewReplicaSetAvailable", "status": "True", "type": "Progressing" } ], "observedGeneration": 2, "readyReplicas": 1, "replicas": 1, "type": "/v3/project/schemas/deploymentStatus", "unavailableReplicas": 0, "updatedReplicas": 1 }, "dnsPolicy": "ClusterFirst", "hostIPC": false, "hostNetwork": false, "hostPID": false, "id": "deployment:default:nginx", "labels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" }, "links": { "remove": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx", "revisions": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx/revisions", "self": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx", "update": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx", "yaml": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx/yaml" }, "name": "nginx", "namespaceId": "default", "paused": false, "projectId": "c-bcz5t:p-fdr4s", "publicEndpoints": [ { "addresses": ["10.64.3.58"], "allNodes": true, "ingressId": null, "nodeId": null, "podId": null, "port": 30917, "protocol": "TCP", "serviceId": "default:nginx-nodeport" } ], "restartPolicy": "Always", "scale": 1, "schedulerName": "default-scheduler", "selector": { "matchLabels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" }, "type": "/v3/project/schemas/labelSelector" }, "state": "active", "terminationGracePeriodSeconds": 30, "transitioning": "no", "transitioningMessage": "", "type": "deployment", "uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd", "workloadAnnotations": { "deployment.kubernetes.io/revision": "1", "field.cattle.io/creatorId": "user-f4tt2" }, "workloadLabels": { "workload.user.cattle.io/workloadselector": "deployment-default-nginx" } } }