Infra Module: Databricks #1229
Description
Description
Our goal is to create a Databricks platform infrastructure deployment module, that takes the best practices of all our existing databricks samples and simplifies the infra deployment. We want to take into account:
- Optionality on permissions (Entra Global Admin vs Azure Contributor)
- Storage Accounts, Security Groups if necessary
- Service Principal vs Managed Identity permissioning
- Needs to be documented (see parking sensors sample for examples on each)
Our audience/persona are software and data engineers hoping to easily deploy the necessary compute and storage to set up a platform Databricks deployment.
Acceptance Criteria
- Must use a devcontainer
- Spike on AZD CLI
- Spike on Deployment Stacks
- IaC in Bicep or Terraform (consider using AVMs if they exist)
- Testing approach documented