[IDEA] Add 'connect_storage' and 'create_catalog' modules to Azure Terraform code

[gergelj]

Pull Request

Description

Brief description of what this PR does and why.

Category

• core-platform
• data-engineering
• data-governance
• data-warehousing
• genai-ml
• launch-accelerator
• workspace-setup

Type of Change

• New project
• Bug fix
• Enhancement
• Documentation

Project Details

Project Name:
Purpose:
Technologies Used:

Testing

• Code runs without errors
• Documentation is complete
• Used only synthetic data

Security Compliance ✅

• No customer data, PII, or proprietary information
• No credentials or access tokens
• Only synthetic data used
• Third-party licenses acknowledged

---

By submitting this PR, I confirm I have followed the CONTRIBUTING.md guidelines and security requirements.

[gergelj]

This is currently at the idea stage; one of my customers requested this feature, and I thought it would be a nice addition. What do you think?

[haleyyyblue]

This is currently at the idea stage; one of my customers requested this feature, and I thought it would be a nice addition. What do you think?

Hi @gergelj :)
I took a quick look at the code, and it seems like it’s mostly Terraform around Catalogs and storage accounts. Given that the go/uc-quickstart project already provides and maintains automation for UC, could you clarify which specific gap or use case you’re trying to cover here?

If this is a use case that’s not currently covered by go/uc-quickstart and we think it’s worth adding, wouldn’t it make more sense to add it under data governance rather than workspace-setup?
Curious to hear what you think as well, @andresgarciaf

[gergelj]

I was just thinking, due to convenience, that an initial storage and catalog could be provisioned with this setup. I understand it might fall under Unity Catalog setup and the data governance topic. But on the other hand, if we don't add storage and a catalog, can we say the workspace is finalized?

I'm happy to start this discussion, and if we decide to move this under a different directory, like data governance, I'd be happy to help there as well.

[haleyyyblue]

I was just thinking, due to convenience, that an initial storage and catalog could be provisioned with this setup. I understand it might fall under Unity Catalog setup and the data governance topic. But on the other hand, if we don't add storage and a catalog, can we say the workspace is finalized?

I'm happy to start this discussion, and if we decide to move this under a different directory, like data governance, I'd be happy to help there as well.

I agree with the point that customers often want to set up storage and Unity Catalog together during the workspace setup phase.

That said, I’m a bit concerned that the current project folder (azure-vnet-injection) already has a lot of options and parameters, and adding storage and UC layers on top could make it increasingly heavy and harder to maintain. What if we try a slightly different approach?

For example, we could create a new folder like azure-vnet-injection-uc, keep the parameters much more opinionated and minimal (e.g., assuming a new VNet and a new metastore, focusing on the most common scenarios), and then layer in storage and UC setup there. This way, we avoid overloading the existing project while still addressing the “end-to-end” setup customers are asking for.

In that case, it would also be good to clearly position how this project differs from uc-quickstart. How do we want to explain when customers should use one versus the other?

Curious to hear your thoughts.