🛡️ Azure Databricks Diagnostic Log Delivery is not configured🟢

• Contextual name: 🛡️ Databricks Diagnostic Log Delivery is not configured🟢
• ID: /ce/ca/azure/databricks/diagnostic-log-delivery
• Tags:
  • 🟢 Policy with categories
  • 🟢 Policy with type
  • 🟢 Production policy
• Policy Type: COMPLIANCE_POLICY
• Policy Categories: RELIABILITY

Logic

• 🧠 prod.logic.yaml🟢
  • 📗 Azure Databricks Workspace
  • 🔌 Azure Diagnostic Setting - object.extracts.yaml
  • 🧪 test-data.json

Description

Open File

Description

This policy identifies Azure Databricks Workspaces that are not configured to deliver diagnostic logging for the following categories: accounts, clusters, notebook, jobs, workspace.

Azure Databricks Diagnostic Logging provides visibility into system operations, user activities, and security events within a Databricks workspace. Enabling diagnostic logs allows organizations to:

• Detect security threats by capturing access events, job executions, and cluster activities.
• Maintain compliance with regulatory frameworks such as SOC 2, HIPAA, and GDPR.
• Monitor operational performance and proactively troubleshoot issues.

Rationale

Diagnostic logging delivers critical visibility into security and operational activities within Databricks workspaces. It ensures auditability for forensic investigations and supports compliance requirements that mandate continuous logging and monitoring of cloud resources.

Impact

• Logs consume storage and may require additional monitoring solutions, potentially increasing operational overhead and costs.

... see more

Remediation

Open File

Remediation

Enable Diagnostic Logging

From Azure Portal

1. Navigate to your Azure Databricks workspace.

2. Select Monitoring > Diagnostic settings from the left-hand menu.

3. Click + Add diagnostic setting.

4. Under Category details, select the log categories you want to capture, such as:

   • AuditLogs
   • Clusters
   • Notebooks
   • Jobs
   • Workspace

5. Choose a destination for the logs:

   • Log Analytics workspace - for advanced querying and monitoring.
   • Storage account : for long-term retention.
   • Event Hub : for integration with third-party systems.

6. Provide a Name for the diagnostic setting and click Save.

From Azure CLI

az monitor diagnostic-settings create \
    --name "DatabricksLogging" \
    --resource {{databricks-resource-id}} \
    --logs '[{"category": "AuditLogs", "enabled": true}, {"category": "Clusters", "enabled": true}, {"category": "Notebooks", "enabled": true}, {"category": "Jobs", "enabled": true}, {"category": "Workspace", "enabled": true}]' \

... [see more](remediation.md)

policy.yaml

Open File

Linked Framework Sections

Section	Sub Sections	Internal Rules	Policies	Flags	Compliance
💼 CIS Azure v5.0.0 → 💼 2.1.7 Ensure that diagnostic log delivery is configured for Azure Databricks (Automated)			1		no data
💼 Cloudaware Framework → 💼 System Configuration			61		no data