Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,7 @@ Private endpoints ensure secure, internal-only connectivity. Private endpoints a
```text
modules-network-secured/
├── add-project-capability-host.bicep # Configuring the project's capability host
├── agent-subnet-udr-association.bicep # Associates UDR with agent subnet (GSA proxy)
├── ai-account-identity.bicep # Azure AI Foundry deployment and configuration
├── ai-project-identity.bicep # Foundry project deployment and connection configuration
├── ai-search-role-assignments.bicep # AI Search RBAC configuration
Expand All @@ -331,6 +332,7 @@ modules-network-secured/
├── cosmosdb-account-role-assignment.bicep # CosmosDB Account RBAC configuration
├── existing-vnet.bicep # Bring your existing virtual network to template deployment
├── format-project-workspace-id.bicep # Formatting the project workspace ID
├── gsa-proxy.bicep # GSA AI Connector proxy VM, NIC, NSG, UDR
├── network-agent-vnet.bicep # Logic for routing virtual network set-up if existing virtual network is selected
├── private-endpoint-and-dns.bicep # Creating virtual networks and DNS zones.
├── standard-dependent-resources.bicep # Deploying CosmosDB, Storage, and Search
Expand Down Expand Up @@ -527,3 +529,256 @@ Each new project deployment creates:
- [Private Endpoint Documentation](https://learn.microsoft.com/en-us/azure/private-link/)
- [RBAC Documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/)
- [Network Security Best Practices](https://learn.microsoft.com/en-us/azure/security/fundamentals/network-best-practices)

---
---

# (Optional) Deploying with GSA Proxy for Egress Traffic Control

This guide explains how to deploy a **GSA (Global Secure Access) AI Connector proxy** alongside your network-secured agent setup. The GSA proxy acts as a transparent egress proxy that intercepts outbound traffic from the agent subnet, adds managed identity authentication, and forwards requests to Azure services.

## Overview

The GSA proxy deployment is an **optional add-on** that runs after your base `main.bicep` deployment. It creates:

- ✅ **Dedicated GSA proxy subnet** with its own NSG
- ✅ **GSA AI Connector VM** from the Azure Marketplace with managed identity and IP forwarding
- ✅ **User-Defined Route (UDR)** on the agent subnet routing all egress (`0.0.0.0/0`) through the proxy
- ✅ **Service tag exceptions** allowing critical Azure traffic to bypass the proxy

### Architecture

```text
┌─────────────────────────────────────────────────────────────┐
│ Virtual Network │
│ │
│ ┌──────────────────┐ ┌──────────────────────────────┐ │
│ │ Agent Subnet │ │ GSA Proxy Subnet │ │
│ │ 192.168.0.0/24 │ │ 192.168.2.0/24 │ │
│ │ │ │ │ │
│ │ ┌─────────────┐ │ │ ┌──────────────────────────┐ │ │
│ │ │ Agent Pods │ │────▶│ │ GSA AI Connector VM │ │ │
│ │ │ (Container │ │ UDR │ │ (Envoy Proxy) │ │ │
│ │ │ Apps) │ │ │ │ - IP Forwarding: ON │ │ │
│ │ └─────────────┘ │ │ │ - Managed Identity: ON │ │ │
│ │ │ │ │ - Adds auth headers │ │ │
│ └──────────────────┘ │ └──────────────────────────┘ │ │
│ └──────────────────────────────┘ │
│ ┌──────────────────┐ │
│ │ PE Subnet │ │
│ │ 192.168.1.0/24 │ │
│ │ (Private │ │
│ │ Endpoints) │ │
│ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────┐
│ Azure Services │
│ (MCR, ACA, etc) │
└─────────────────┘
```

### UDR Service Tag Exceptions

The following Azure service tags bypass the proxy and route directly to the Internet, ensuring core Azure platform services remain accessible:

| Service Tag | Purpose |
|-------------|---------|
| `AzureActiveDirectory` | Entra ID authentication |
| `AzureResourceManager` | ARM management plane |
| `AzureMonitor` | Logging and monitoring |
| `GuestAndHybridManagement` | VM agent and extensions |
| `AzureContainerRegistry` | Container image pulls |
| `AzureKeyVault` | Key/secret/certificate access |
| `Storage` | Azure Storage service |

## Files

| File | Purpose |
|------|---------|
| `deploy-gsa-proxy.bicep` | Orchestrator template for GSA proxy deployment |
| `deploy-gsa-proxy.bicepparam` | Parameter file with defaults |
| `modules-network-secured/gsa-proxy.bicep` | Core module: subnet, NSG, NIC, VM, UDR |
| `modules-network-secured/agent-subnet-udr-association.bicep` | Associates UDR with agent subnet |

## Prerequisites

1. ✅ **Base deployment completed** — `main.bicep` must be deployed first
2. ✅ **Azure CLI** installed and logged in
3. ✅ **SSH key pair** for VM authentication
4. ✅ **Marketplace terms accepted** — run the following command once per subscription:

```bash
az vm image terms accept \
--publisher microsoftcorporation1687208452115 \
--offer gsaaiconnector1-preview \
--plan gsaaiconnectorplan1
```

5. ✅ **Available subnet address range** that doesn't overlap with existing subnets (default: `192.168.2.0/24`)

## Step-by-Step Deployment

### Step 1: Deploy Base Infrastructure

If not already done, deploy the base network-secured agent setup:

```bash
# Create resource group
az group create --name <your-resource-group> --location <region>

# Deploy base infrastructure
az deployment group create \
--resource-group <your-resource-group> \
--template-file main.bicep \
--parameters main.bicepparam
```

### Step 2: Generate SSH Key (if needed)

```bash
ssh-keygen -t rsa -b 2048 -f ~/.ssh/gsa-proxy-key -N ""
```

### Step 3: Deploy the GSA Proxy

```bash
az deployment group create \
--resource-group <your-resource-group> \
--template-file deploy-gsa-proxy.bicep \
--parameters \
name=gsa-proxy \
vnetName=<your-vnet-name> \
gsaProxySubnetPrefix=192.168.2.0/24 \
agentSubnetName=agent-subnet \
vmSize=Standard_D2s_v3 \
adminUsername=azureuser \
sshPublicKey="$(cat ~/.ssh/gsa-proxy-key.pub)"
```

> **Note:** Adjust `gsaProxySubnetPrefix` to a CIDR range that does not overlap with your agent subnet or PE subnet.

### Step 4: Verify Deployment

```bash
# Check all GSA proxy resources
az resource list --resource-group <your-resource-group> \
--query "[?contains(name,'gsa-proxy')].{name:name, type:type}" -o table

# Verify UDR routes
az network route-table show \
--name <name>-agent-udr \
--resource-group <your-resource-group> \
--query "routes[].{name:name, prefix:addressPrefix, nextHop:nextHopIpAddress, nextHopType:nextHopType}" \
-o table

# Verify IP forwarding on NIC
az network nic show \
--name <name>-gsa-proxy-nic \
--resource-group <your-resource-group> \
--query "{ipForwarding:enableIPForwarding}" -o json

# Verify VM managed identity
az vm show \
--name <name>-gsa-proxy-vm \
--resource-group <your-resource-group> \
--query "{identity:identity.type}" -o json
```

### Step 5: Check Proxy Logs

Use `az vm run-command` to check the Envoy proxy logs without needing SSH access:

```bash
# View recent Envoy proxy logs
az vm run-command invoke \
--resource-group <your-resource-group> \
--name <name>-gsa-proxy-vm \
--command-id RunShellScript \
--scripts "journalctl -u envoy --no-pager -n 50"

# Check Envoy admin stats
az vm run-command invoke \
--resource-group <your-resource-group> \
--name <name>-gsa-proxy-vm \
--command-id RunShellScript \
--scripts "curl -s http://127.0.0.1:9901/stats | grep -E 'cx_active|cx_total|rq_total'"
```

## Post-Deployment: Enable Portal Access

By default, the network-secured deployment blocks public access to the AI account. To access your project in the [Azure AI Foundry portal](https://ai.azure.com), follow these steps:

### 1. Enable Public Network Access

```bash
# Enable public network access on the AI account
az resource update \
--ids /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<account-name> \
--set properties.publicNetworkAccess=Enabled properties.networkAcls.defaultAction=Allow
```

### 2. Assign the Azure AI User Role

The deploying user (and any other users who need portal access) must have the **Azure AI User** role:

```bash
# Assign to a specific user
az role assignment create \
--assignee <user-principal-id-or-email> \
--role "Azure AI User" \
--scope /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<account-name>

# Or assign at subscription level for broader access
az role assignment create \
--assignee <user-principal-id-or-email> \
--role "Azure AI User" \
--scope /subscriptions/<subscription-id>
```

> **Note:** Role assignments may take up to 5 minutes to propagate.

### 3. Re-secure After Testing (Optional)

To re-lock the account after portal browsing:

```bash
az resource update \
--ids /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<account-name> \
--set properties.publicNetworkAccess=Disabled properties.networkAcls.defaultAction=Deny
```

## Configuration Reference

### Parameters

| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `name` | Yes | — | Name prefix for all GSA proxy resources |
| `vnetName` | Yes | — | Name of the existing VNet from `main.bicep` |
| `gsaProxySubnetPrefix` | No | `10.0.4.0/24` | CIDR for the GSA proxy subnet |
| `agentSubnetName` | No | `agent-subnet` | Name of the agent subnet to attach UDR to |
| `vmSize` | No | `Standard_D2s_v3` | VM SKU for the proxy |
| `adminUsername` | No | `azureuser` | SSH admin username |
| `sshPublicKey` | Yes | — | SSH public key for VM authentication |
| `tags` | No | `{}` | Tags to apply to all resources |
| `location` | No | Resource group location | Azure region |

### Resources Created

| Resource | Type | Purpose |
|----------|------|---------|
| GSA Proxy Subnet | `Microsoft.Network/virtualNetworks/subnets` | Hosts the proxy VM |
| NSG | `Microsoft.Network/networkSecurityGroups` | Allows VirtualNetwork traffic |
| NIC | `Microsoft.Network/networkInterfaces` | IP forwarding enabled |
| VM | `Microsoft.Compute/virtualMachines` | GSA AI Connector (Envoy proxy) |
| UDR | `Microsoft.Network/routeTables` | Routes agent egress through proxy |

## Troubleshooting

1. **Marketplace terms not accepted** — Run `az vm image terms accept` as shown in Prerequisites
2. **VM SKU not available** — Check SKU availability: `az vm list-skus --location <region> --size Standard_D2s_v3 -o table`
3. **Subnet overlap** — Ensure `gsaProxySubnetPrefix` doesn't overlap with existing subnets
4. **Proxy not forwarding traffic** — Check Envoy service: `az vm run-command invoke ... --scripts "systemctl status envoy"`
5. **Cannot see project in portal** — Follow the "Post-Deployment: Enable Portal Access" steps above
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
// ---- GSA Proxy Deployment ----
// This is the top-level deployment template for adding a GSA AI Connector proxy
// to an existing private-network agent setup. It creates:
// 1. A dedicated subnet for the GSA proxy
// 2. An NSG allowing VNet traffic
// 3. A VM from the GSA AI Connector marketplace image with managed identity & IP forwarding
// 4. A UDR on the agent subnet routing 0/0 through the proxy (with Azure service tag exceptions)
//
// Prerequisites:
// - An existing VNet deployed by main.bicep (or an existing VNet)
// - An existing agent subnet in that VNet
// - The marketplace terms for microsoftcorporation1687208452115:gsaaiconnector1-preview:gsaaiconnectorplan1
// must be accepted before deployment:
// az vm image terms accept --publisher microsoftcorporation1687208452115 \
// --offer gsaaiconnector1-preview --plan gsaaiconnectorplan1

targetScope = 'resourceGroup'

@description('Azure region for deployment')
param location string = resourceGroup().location

@description('Name prefix for all GSA proxy resources')
param name string

@description('Name of the existing virtual network')
param vnetName string

@description('Address prefix for the GSA proxy subnet (must not overlap with existing subnets)')
param gsaProxySubnetPrefix string = '10.0.4.0/24'

@description('Name of the existing agent subnet to apply the UDR to')
param agentSubnetName string

@description('VM size for the GSA proxy VM')
param vmSize string = 'Standard_D2s_v3'

@description('Admin username for the proxy VM')
param adminUsername string = 'azureuser'

@description('SSH public key for VM authentication')
@secure()
param sshPublicKey string

@description('Tags to apply to all resources')
param tags object = {}

// ---- Get existing VNet resource ID ----
resource existingVnet 'Microsoft.Network/virtualNetworks@2024-01-01' existing = {
name: vnetName
}

// ---- Deploy GSA Proxy ----
module gsaProxy 'modules-network-secured/gsa-proxy.bicep' = {
name: 'gsa-proxy-deployment'
params: {
location: location
name: name
vnetId: existingVnet.id
vnetName: vnetName
gsaProxySubnetPrefix: gsaProxySubnetPrefix
agentSubnetName: agentSubnetName
vmSize: vmSize
adminUsername: adminUsername
sshPublicKey: sshPublicKey
tags: tags
}
}

// ---- Outputs ----
output gsaProxySubnetId string = gsaProxy.outputs.gsaProxySubnetId
output gsaProxyPrivateIp string = gsaProxy.outputs.gsaProxyPrivateIp
output gsaProxyVmId string = gsaProxy.outputs.gsaProxyVmId
output gsaProxyManagedIdentityPrincipalId string = gsaProxy.outputs.gsaProxyVmPrincipalId
output routeTableId string = gsaProxy.outputs.routeTableId
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
using 'deploy-gsa-proxy.bicep'

param name = 'agent-network'
param vnetName = 'agent-network-vnet'
param gsaProxySubnetPrefix = '10.0.4.0/24'
param agentSubnetName = 'agent-network-subnet'
param vmSize = 'Standard_D2s_v3'
param adminUsername = 'azureuser'
param sshPublicKey = readEnvironmentVariable('SSH_PUBLIC_KEY')
param tags = {}
Loading