📝 Azure Databricks Diagnostic Log Delivery is not configured 🟢

• Contextual name: 📝 Databricks Diagnostic Log Delivery is not configured 🟢
• ID: /ce/ca/azure/databricks/diagnostic-log-delivery
• Located in: 📁 Azure Databricks

Flags

• 🟢 Impossible policy
• 🟢 Policy with categories
• 🟢 Policy with type

Our Metadata

• Policy Type: COMPLIANCE_POLICY
• Policy Category:
  • RELIABILITY

Description

Open File

Description

Azure Databricks Diagnostic Logging provides insights into system operations, user activities, and security events within a Databricks workspace. Enabling diagnostic logs helps organizations:

• Detect security threats by logging access, job executions, and cluster activities.
• Ensure compliance with industry regulations such as SOC 2, HIPAA, and GDPR.
• Monitor operational performance and troubleshoot issues proactively.

Rationale

Diagnostic logging provides visibility into security and operational activities within Databricks workspaces while maintaining an audit trail for forensic investigations, and it supports compliance with regulatory standards that require logging and monitoring.

Impact

Logs consume storage and may require additional monitoring tools, leading to increased operational overhead and costs. Incomplete log configurations may result in missing critical events, reducing monitoring effectiveness.

Audit

From Azure Portal

Check if diagnostic logging is enabled for the Databricks workspace:

... see more

Remediation

Open File

Remediation

From Azure Portal

Enable diagnostic logging for Azure Databricks

1. Navigate to your Azure Databricks workspace.

2. In the left-hand menu, select Monitoring > Diagnostic settings.

3. Click + Add diagnostic setting.

4. Under Category details, select the log categories you wish to capture, such as AuditLogs, Clusters, Notebooks, and Jobs.

5. Choose a destination for the logs:

   • Log Analytics workspace: For advanced querying and monitoring.
   • Storage account: For long-term retention.
   • Event Hub: For integration with third-party systems.

6. Provide a Name for the diagnostic setting.

7. Click Save.

Implement log retention policies

1. Navigate to your Log Analytics workspace.
2. Under General, select Usage and estimated costs.
3. Click Data Retention.
4. Adjust the retention period slider to the desired number of days (up to 730 days).
5. Click OK.

Monitor logs for anomalies

1. Navigate to Azure Monitor.
2. Select Alerts > + New alert rule.
3. Under Scope, specify the Databricks resource.

... see more

policy.yaml

Open File

Linked Framework Sections

Section	Sub Sections	Internal Rules	Policies	Flags
💼 CIS Azure v4.0.0 → 💼 3.1.7 Ensure that diagnostic log delivery is configured for Azure Databricks (Manual)			1
💼 Cloudaware Framework → 💼 System Configuration			30