Private Endpoints + DNS baseline: stop outages before they happen

Why Private Endpoints break at 02:00

Private Endpoints without governance are outage factories. The moment someone links a zone to the wrong vNet, deletes the record, or forgets reverse lookups, apps stop resolving. A baseline keeps ownership, automation, and rollback in the platform team instead of relying on every workload engineer remembering the DNS handshake.

What good looks like

• Zones in one place – all Azure Private DNS zones owned by platform, tagged, versioned.
• Hub-based resolver – Azure DNS Private Resolver (inbound/outbound) deployed to the hub, workloads linked centrally.
• Automated linking – subscription vending calls Terraform/Bicep modules to link spokes and register records.
• Policy guardrails – Azure Policy ensures Private Endpoints only land in approved subnets and zones are linked.
• Monitoring + rollback – scripts compare expected vs actual links, alerts fire on deletion, playbooks exist per service.

Baseline decision points

1. Zone ownership – Platform subscription is the default; avoids sprawl and simplifies approvals.
2. Linking pattern – Hub-only links + resolver vs direct workload links. Hub pattern reduces churn and RBAC bloat.
3. Per-service vs shared zones – Usually one zone per Azure service (privatelink.<service>.windows.net); easier for lifecycle and rollback.
4. Automation tooling – Terraform/Bicep modules invoked by vending or change pipeline. No portal.
5. Resolver strategy – Azure DNS Private Resolver with inbound/outbound endpoints in hub VNets; no custom DNS VMs.
6. Subscription vending hook – Decide when the linking script runs (during networking module or Private Endpoint request).
7. Monitoring + drift – How you detect missing links, stale records, or unauthorised zones. Use KQL and automation.

Signal vs noise

• Enable now: Zones, resolver, linking automation, policies for subnet restrictions, monitoring scripts, request workflow.
• Enable at Day 30: Reverse lookup zones and advanced alerting once the baseline is stable.
• Probably never: Letting workloads own their own zones, manual linking (“just this once”), or building custom DNS servers because “it feels familiar.”

Phased rollout

• Day 0 baseline – Deploy zones/resolver, build Terraform module for linking, document the request workflow, integrate with vending.
• Day 30 hardening – Enable policy guardrails, add reverse lookup zones, wire drift detection into alerting, start logging.
• Quarterly review – Compare expected vs actual links, rotate resolver credentials, test rollback playbooks, update service catalogue.

Runbook: Private Endpoints + DNS baseline

Policy guardrails

Use Bicep to prevent rogue Private Endpoints and ensure zone linking happens. Replace policy IDs with your own definitions.

targetScope = 'managementGroup'

@description('Region for the assignment identity. Not where policy applies.')
param location string = 'uksouth'

// Resolve real definition IDs first, e.g.:
//   az policy definition list --query "[?contains(displayName,'<search term>')].{name:displayName,id:id}" -o table
resource initiative 'Microsoft.Authorization/policySetDefinitions@2023-04-01' = {
  name: 'private-link-guardrails'
  properties: {
    displayName: 'Private Link Guardrails'
    description: 'Keep private endpoints in approved subnets and auto-link zones.'
    policyType: 'Custom'
    policyDefinitions: [
      {
        policyDefinitionReferenceId: 'allowed-subnets'
        policyDefinitionId: '/providers/Microsoft.Authorization/policyDefinitions/<allowed-subnets-id>'
      }
      {
        policyDefinitionReferenceId: 'deploy-zone-link'
        policyDefinitionId: '/providers/Microsoft.Authorization/policyDefinitions/<deploy-zone-link-id>'
      }
    ]
  }
}

resource assignment 'Microsoft.Authorization/policyAssignments@2023-04-01' = {
  name: 'private-link-guardrails'
  // Required: the initiative contains a deployIfNotExists effect, which cannot
  // remediate without an identity that holds rights at the assigned scope.
  location: location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    displayName: 'Private Link Guardrails'
    policyDefinitionId: initiative.id
  }
}

resource remediationRights 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(managementGroup().id, assignment.id, 'Contributor')
  properties: {
    // Contributor — narrow this to the least-privileged role your definitions need.
    roleDefinitionId: subscriptionResourceId(
      'Microsoft.Authorization/roleDefinitions',
      'b24988ac-6180-42a0-ab88-20f7382dd24c'
    )
    principalId: assignment.identity.principalId
    principalType: 'ServicePrincipal'
  }
}

Validation checks

Drift detection here is a state question, not a log question: a zone that was never linked to a vNet generates no events at all, so no amount of log querying will surface it. Use Azure Resource Graph to read the actual links, then diff against what your IaC declares.

resources
| where type =~ 'microsoft.network/privatednszones/virtualnetworklinks'
| project zone = tostring(split(id, '/')[8]),
          link = name,
          vnet = tostring(properties.virtualNetwork.id),
          registrationEnabled = tobool(properties.registrationEnabled),
          subscriptionId
| order by zone asc, vnet asc

To spot the opposite problem — zones with no links at all, usually left behind by a retired subscription:

resources
| where type =~ 'microsoft.network/privatednszones'
| extend linkCount = toint(properties.numberOfVirtualNetworkLinks)
| where linkCount == 0
| project subscriptionId, resourceGroup, zone = name

Logs still earn their place for the destructive events. This alerts on links being removed or zones deleted, which is the failure mode that pages you at 02:00:

AzureActivity
| where TimeGenerated > ago(1d)
| where OperationNameValue has "PRIVATEDNSZONES"
| where OperationNameValue has "DELETE"
| where ActivityStatusValue =~ "Success"
| project TimeGenerated, Caller, OperationNameValue, _ResourceId
| order by TimeGenerated desc

Common pitfalls

Rollback / back-out plan

• To remove a Private Endpoint: delete the endpoint, clear the DNS A record, and run the automation that restores public connectivity if needed. Expect short outages.
• To revert resolver changes: disable forwarding rules, but note that cached results may linger until TTLs expire. Communicate clearly with workloads before flipping.
• To undo zone links: remove the link and flush DNS on affected hosts. Document why the link was removed and ensure alternatives (public endpoints or other regions) exist.

Rollback is never “instant” because DNS caching and dependency on Private Link may require maintenance windows. Keep playbooks per service.