Safe Rollouts: Slots, Dependencies & Hotfix Paths

Why Safe Rollouts Require Planning

Dependency Deployment Ordering

When your application has multiple tiers (database, API, frontend, background workers), deployment order matters. The golden rule: deploy bottom-up — infrastructure and data layers first, presentation layers last.

The Deployment Order

1. Database schema changes (expand phase)
2. Background services / workers
3. Backend APIs
4. API gateways / BFF layers
5. Frontend applications
6. Database cleanup (contract phase — after old code is fully retired)

The Expand-Contract Pattern

The expand-contract pattern (also called parallel change) ensures backward compatibility during multi-service deployments:

Expand phase:

• Add the new database column (nullable or with default value)
• Deploy new API version that writes to BOTH old and new columns
• Old and new API versions coexist safely

Contract phase (after all consumers updated):

• Migrate remaining data from old column to new column
• Remove old column
• Remove backward-compatibility code

This eliminates the “deploy database and API at the exact same millisecond” problem. Both versions work throughout the transition.

stages:
  - stage: Database
    jobs:
      - job: MigrateSchema
  - stage: NotificationService
    dependsOn: Database
  - stage: ClaimsAPI
    dependsOn: Database
  - stage: Portal
    dependsOn:
      - ClaimsAPI
      - NotificationService

Minimising Downtime

Load Balancing Strategies

Health Checks and Readiness Probes

Health checks ensure traffic only routes to healthy instances:

• Liveness probe — is the process alive? Restart if not.
• Readiness probe — can the instance serve traffic? Remove from LB if not.
• Startup probe — is the app still starting up? Do not check liveness until startup completes.

In Azure App Service, configure the Health Check feature at /health — the platform automatically removes unhealthy instances from the load balancer rotation.

Hotfix Path Planning

A hotfix path is a pre-planned, expedited route from code fix to production that bypasses the normal release cadence. Every team needs one BEFORE the first emergency.

Standard Flow vs Hotfix Flow

Hotfix Branching Approaches

Git Flow hotfix: Create hotfix/critical-fix from the main (or release) branch. Fix, test, deploy. Merge back into BOTH main and develop to prevent regression.

Trunk-based hotfix: Cherry-pick the fix commit from a feature branch (or commit directly to main if CI is fast enough). Deploy from main. The fix is already in the trunk.

Release branch hotfix: If you maintain release branches (release/2.4), apply the fix to the release branch, deploy, then cherry-pick to main for the next release.

Resiliency Strategies for Deployment

Resiliency is not just about the application — your deployment pipeline itself must be resilient.

Application Resiliency Patterns

Pipeline Resiliency Patterns

• Automatic rollback — if post-deployment health checks fail, automatically redeploy the previous version
• Deployment gates — automated quality gates between stages (Azure Monitor alerts, SonarQube quality gate, custom API checks)
• Approval timeouts — approvals expire after a window to prevent stale deployments sitting in the pipeline
• Retry on transient failure — configure pipeline tasks to retry on infrastructure errors (network timeout, agent unavailable)

Automatic Rollback Configuration

In Azure Pipelines, configure automatic rollback using the on: failure hook:

stages:
  - stage: Production
    jobs:
      - deployment: Deploy
        strategy:
          runOnce:
            deploy:
              steps:
                - task: AzureWebApp@1
                  inputs:
                    appName: 'claims-api'
            on:
              failure:
                steps:
                  - task: AzureAppServiceManage@0
                    inputs:
                      Action: 'Swap Slots'
                      WebAppName: 'claims-api'
                      SourceSlot: 'production'
                      TargetSlot: 'staging'

In GitHub Actions, use a separate rollback job that runs if: failure() and references the previous stable deployment.

Next up: Deployment Implementations: Containers, Scripts and Databases