Skip to content
Cloud Provider Benchmarks

Methodology

Cloud security and audit logging methodology

How cloud audit-logging and security capabilities are assessed from documentation using qualitative values, feature definitions and the conceptual audit event model.

Version 1.0 Updated:

Scope

  • Control-plane audit logging, identity attribution and request metadata
  • Data-plane event logging, search, retention, export, integrity and pricing model
  • Security capabilities such as security groups, private networks and certifications
  • Mapping of audit fields to a common conceptual event model

Qualitative values only

Security and audit capabilities are not scored numerically. Each feature receives one of five values:

Value Meaning
Yes Documented by the provider and available generally
Partial Available with documented limitations (for example only some services or regions)
Planned Announced by the provider but not generally available
No Documented as unavailable, or clearly absent from the provider’s service catalogue
Unverified Not enough public documentation to decide

“Unverified” is never read as “no”. The value changes when documentation appears.

Feature definitions

Nine features are assessed: control-plane scope, identity attribution, request metadata, data-plane events, event search, retention, export, integrity protection and pricing model. Each definition is shown next to the values on the cloud auditing benchmark page and applies identically to every provider.

Sources

Only official documentation, product pages and provider-supplied statements are used. Provider-supplied information is labelled as such. Third-party blog posts are not used to establish a capability.

Audit event model

Fields of PortvMind’s vTrail event model are mapped to conceptual equivalents in AWS CloudTrail, Google Cloud Audit Logs and DMTF CADF. Mappings are conceptual — two fields with similar meaning may differ in format or semantics.

Conceptual cloud audit event data model For research documentation; does not represent production audit records.
Field Type Description AWS CloudTrail Google Cloud Audit Logs OpenStack CADF
event_time string (RFC 3339) When the operation was received, in UTC. eventTime timestamp eventTime
request_id string Unique request identifier used to correlate gateway, service and resource logs. requestID insertId / operation.id id / requestId (attachment)
http_method string HTTP verb of the API call (GET, POST, PUT, PATCH, DELETE). — httpRequest.requestMethod action (mapped)
full_path string Requested API path, identifying the target resource. eventSource + eventName protoPayload.methodName / resourceName target.id / requestPath
http_status integer Response status code — the outcome of the operation. errorCode (on failure) protoPayload.status.code outcome / reason.reasonCode
client_ip string (IP) Source IP address of the caller. sourceIPAddress protoPayload.requestMetadata.callerIp initiator.host.address
user_agent string Client software (console, CLI, SDK, Terraform). userAgent protoPayload.requestMetadata.callerSuppliedUserAgent initiator.host.agent
auth_user_id string Stable identifier of the authenticated user. userIdentity.principalId protoPayload.authenticationInfo.principalSubject initiator.id
auth_user_name string Human-readable user name at the time of the request. userIdentity.userName / arn protoPayload.authenticationInfo.principalEmail initiator.name
auth_project_id string Project (tenant) the token was scoped to. recipientAccountId resource.labels.project_id initiator.project_id
auth_scope string Authorisation scope or role context of the token. userIdentity.sessionContext protoPayload.authorizationInfo initiator.credential
service_id string Backend cloud service that handled the request (compute, network, storage). eventSource protoPayload.serviceName observer.id / target.typeURI
route_id string API gateway route that matched the request. — — —
request_time number (ms) End-to-end request processing time. — httpRequest.latency —
Empty template (JSON)
{
  "event_time": "",
  "request_id": "",
  "http_method": "",
  "full_path": "",
  "http_status": null,
  "client_ip": "",
  "user_agent": "",
  "auth_user_id": "",
  "auth_user_name": "",
  "auth_project_id": "",
  "auth_scope": "",
  "service_id": "",
  "route_id": "",
  "request_time": null
}

No performance testing

Logging pipelines are not load-tested and delivery latency is not measured. Doing so would require provider cooperation and could not be applied identically to every provider.

Limitations

  • Documentation can lag behind product changes in either direction.
  • Certifications are listed as claimed by providers; certificates are not independently audited by this site.

Benchmarks using this methodology

Other methodologies