Skip to main content

ce Dataset

The ce dataset is the primary repository for all data generated by the Compliance Engine. It provides a comprehensive and historical view of your organization's compliance posture, tracking every policy evaluation from its execution down to the per-resource result. This dataset is essential for auditing, reporting, and analyzing compliance trends over time.

Common Use Cases​

  • Generating detailed compliance reports for specific frameworks or resource types.
  • Tracking the compliance history of a critical resource to understand when its status changed.
  • Analyzing the performance and cost of policy evaluations by inspecting the underlying BigQuery jobs.
  • Joining compliance data with the sobjects dataset to correlate policy violations with specific resource configurations.

Tables​

The ce dataset is composed of several tables that provide different levels of detail about the compliance evaluation process.

Core Evaluation Results​

  • PolicyOutput: The most critical table, containing the detailed, per-resource results of every policy evaluation. It records the compliance status (COMPLIANT, INCOMPLIANT, etc.), status change dates, and messages for each resource.
  • PolicyOutputRelatedLists: Stores the evaluation results for related objects that are checked as part of a parent object's policy logic.

Execution Metadata​

  • PolicyRun: A high-level log of each time a policy was executed. It includes the policy's identity, run time, and summary statistics (e.g., the number of compliant, incompliant, and inapplicable resources).
  • PolicyRunJob: Contains detailed metadata about the underlying BigQuery jobs that executed the policies, including bytes processed and billed. This is invaluable for performance tuning and cost analysis.

Framework Mappings​

  • Sections: Defines the hierarchical structure of compliance frameworks, such as CIS or NIST, including their individual sections and controls.
  • SectionPolicies: Maps Compliance Engine policies to their corresponding sections within the various compliance frameworks.

Views​

The dataset also includes pre-built views to simplify common queries and data analysis.

  • FrameworksWithMappedCePolicies
  • HistoricalPolicyOutputWithPolicyRun
  • PolicyOutputWithPolicyRun