Skip to content

Deployment Guide

Overview

This comprehensive guide covers deploying Flaggy across different environments, from development to production. It includes detailed configuration examples, best practices, troubleshooting tips, and deployment checklists for each environment.

Table of Contents

Environment Overview

Environment Strategy

Flaggy supports a multi-environment deployment strategy:

Environment Purpose Provider Caching Dashboard Access
Development Local development InMemory Memory Open (no auth)
Testing Automated tests InMemory Memory Not deployed
Staging Pre-production validation MySQL/PostgreSQL Redis Restricted
Production Live application MySQL/PostgreSQL/MS SQL Redis Highly restricted

Configuration Files Structure

appsettings.json                 # Default/Development
appsettings.Development.json     # Development overrides
appsettings.Staging.json         # Staging configuration
appsettings.Production.json      # Production configuration

Development Environment

Development environment setup focuses on rapid iteration and easy debugging.

Configuration

appsettings.Development.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Debug",
      "Flaggy": "Debug"
    }
  },
  "Flaggy": {
    "Provider": "InMemory",
    "FilePath": "./data/flags-dev.json",
    "CacheExpiration": "00:00:30",
    "AutoMigrate": true
  },
  "Dashboard": {
    "Enabled": true,
    "RoutePrefix": "/flaggy",
    "RequireAuthorization": false
  }
}

Program.cs Setup

using Flaggy.Extensions;
using Flaggy.Providers;

var builder = WebApplication.CreateBuilder(args);

if (builder.Environment.IsDevelopment())
{
    // Use InMemory provider for fast development
    var filePath = builder.Configuration["Flaggy:FilePath"] ?? "./data/flags-dev.json";
    builder.Services.AddFlaggy(new InMemoryFeatureFlagProvider(filePath));

    // Seed development flags
    var app = builder.Build();

    await app.SeedFeatureFlagsAsync(
        new FeatureFlag { Key = "dev-mode", IsEnabled = true, Description = "Development mode enabled" },
        new FeatureFlag { Key = "debug-logging", IsEnabled = true, Description = "Enable debug logs" },
        new FeatureFlag { Key = "hot-reload", IsEnabled = true, Description = "Hot reload features" }
    );

    // Enable dashboard without authentication
    app.UseFlaggyUI(options =>
    {
        options.RoutePrefix = "/flaggy";
        options.RequireAuthorization = false;
    });

    app.Run();
}

Development Workflow

# 1. Clone repository
git clone https://github.com/your-org/your-app.git
cd your-app

# 2. Restore dependencies
dotnet restore

# 3. Run application
dotnet run

# 4. Access dashboard
# Navigate to: https://localhost:5001/flaggy

Development Best Practices

  1. Use InMemory Provider: Fast, no database setup required
  2. Short Cache TTL: 30 seconds for quick flag updates
  3. Seed Development Flags: Pre-populate common flags
  4. Open Dashboard Access: No authentication needed
  5. Verbose Logging: Enable detailed logs for debugging

Development Checklist

  • Install Flaggy packages
  • Configure InMemory provider
  • Set up development flags file
  • Seed initial flags
  • Enable dashboard without auth
  • Test flag evaluation
  • Verify cache behavior
  • Check logs for errors

Staging Environment

Staging mirrors production as closely as possible for final validation.

Configuration

appsettings.Staging.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Flaggy": "Information"
    }
  },
  "ConnectionStrings": {
    "DefaultConnection": "Server=staging-db.company.com;Database=myapp_staging;User=app_user;Password=${DB_PASSWORD};",
    "Redis": "staging-redis.company.com:6379,password=${REDIS_PASSWORD}"
  },
  "Flaggy": {
    "Provider": "MySQL",
    "TableName": "feature_flags",
    "UserTableName": "flaggy_users",
    "CacheExpiration": "00:05:00",
    "AutoMigrate": true
  },
  "Dashboard": {
    "Enabled": true,
    "RoutePrefix": "/admin/flags",
    "RequireAuthorization": true,
    "AllowedRoles": ["Admin", "Developer"]
  }
}

Program.cs Setup

using Flaggy.Extensions;
using Flaggy.Enums;

var builder = WebApplication.CreateBuilder(args);

if (builder.Environment.IsStaging())
{
    // Configure MySQL provider
    var connectionString = builder.Configuration.GetConnectionString("DefaultConnection");
    builder.Services.AddFlaggy(options =>
    {
        options.UseMySQL(
            connectionString: connectionString,
            tableName: "feature_flags",
            autoMigrate: true
        );
        options.UseMemoryCache(TimeSpan.FromMinutes(5));
    });

    // Configure Redis caching
    var redisConnectionString = builder.Configuration.GetConnectionString("Redis");
    builder.Services.AddFlaggy(
        provider: sp => sp.GetRequiredService<IFeatureFlagProvider>(),
        cachingProvider: CachingProvider.Redis,
        redisConnectionString: redisConnectionString,
        cacheExpiration: TimeSpan.FromMinutes(5)
    );

    var app = builder.Build();

    // Create staging users
    await app.SeedDashboardUsersAsync(
        new User { Username = "staging-admin", Password = "StagingPass123!" },
        new User { Username = "qa-tester", Password = "QAPass123!" }
    );

    // Enable dashboard with authentication
    app.UseFlaggyUI(options =>
    {
        options.RoutePrefix = "/admin/flags";
        options.RequireAuthorization = true;
        options.AuthorizationFilter = context =>
        {
            return context.User.IsInRole("Admin") || context.User.IsInRole("Developer");
        };
    });

    app.Run();
}

Environment Variables

# Staging environment variables
export ASPNETCORE_ENVIRONMENT=Staging
export DB_PASSWORD="staging-db-password-secure"
export REDIS_PASSWORD="staging-redis-password-secure"
export FLAGGY_DASHBOARD_ENABLED=true

Staging Deployment Process

Step 1: Prepare Infrastructure

# Ensure database server is running
mysql -h staging-db.company.com -u app_user -p

# Verify Redis is accessible
redis-cli -h staging-redis.company.com -p 6379 -a $REDIS_PASSWORD ping

Step 2: Deploy Application

# Build the application
dotnet publish -c Release -o ./publish --runtime linux-x64

# Copy to staging server
scp -r ./publish user@staging-server.company.com:/var/www/myapp

# SSH to staging server
ssh user@staging-server.company.com

# Set environment variables
export ASPNETCORE_ENVIRONMENT=Staging
export DB_PASSWORD="staging-db-password"
export REDIS_PASSWORD="staging-redis-password"

# Run application
cd /var/www/myapp
dotnet MyApp.dll

Step 3: Verify Deployment

# Check application health
curl https://staging.company.com/health

# Check Flaggy health
curl https://staging.company.com/health/flaggy

# Access dashboard
# Navigate to: https://staging.company.com/admin/flags

Staging Best Practices

  1. Mirror Production: Use same database provider and caching strategy
  2. Use Real Database: MySQL/PostgreSQL, not InMemory
  3. Enable Redis: Test distributed caching
  4. Require Authentication: Dashboard access controlled
  5. Moderate Logging: Information level, not Debug
  6. Auto-Migration On: Let Flaggy manage schema
  7. Test Rollouts: Validate flag changes before production

Staging Checklist

  • Database server provisioned and accessible
  • Redis cache server running
  • Connection strings configured with secrets
  • MySQL/PostgreSQL provider configured
  • Redis caching enabled
  • Dashboard authentication enabled
  • Users created for QA team
  • Health checks passing
  • Flag operations tested
  • Cache invalidation verified
  • Performance acceptable
  • Logs reviewed for errors

Production Environment

Production deployment requires careful attention to security, performance, and reliability.

Configuration

appsettings.Production.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning",
      "Flaggy": "Information"
    }
  },
  "ConnectionStrings": {
    "DefaultConnection": "Server=prod-db.company.com;Database=myapp_prod;User=app_user;Password=${DB_PASSWORD};SslMode=Required;Pooling=true;MinimumPoolSize=10;MaximumPoolSize=100;",
    "Redis": "prod-redis.company.com:6379,password=${REDIS_PASSWORD},ssl=true"
  },
  "Flaggy": {
    "Provider": "MySQL",
    "TableName": "feature_flags",
    "UserTableName": "flaggy_users",
    "CacheExpiration": "00:10:00",
    "AutoMigrate": false
  },
  "Dashboard": {
    "Enabled": true,
    "RoutePrefix": "/admin/feature-flags",
    "RequireAuthorization": true,
    "AllowedRoles": ["Admin"],
    "IPWhitelist": ["10.0.0.0/8", "172.16.0.0/12"]
  }
}

Program.cs Setup

using Flaggy.Extensions;
using Microsoft.AspNetCore.HttpOverrides;

var builder = WebApplication.CreateBuilder(args);

if (builder.Environment.IsProduction())
{
    // Configure MySQL provider with production settings and Redis caching
    var connectionString = builder.Configuration.GetConnectionString("DefaultConnection");
    var redisConnectionString = builder.Configuration.GetConnectionString("Redis");

    builder.Services.AddFlaggy(options =>
    {
        options.UseMySQL(
            connectionString: connectionString,
            tableName: "feature_flags",
            userTableName: "flaggy_users",
            autoMigrate: false // Run migrations manually in production
        );
        options.UseRedisCache(redisConnectionString, TimeSpan.FromMinutes(10));
    });

    // Configure health checks
    builder.Services.AddHealthChecks()
        .AddCheck<FeatureFlagHealthCheck>("feature_flags")
        .AddMySql(connectionString, name: "database")
        .AddRedis(redisConnectionString, name: "redis_cache");

    var app = builder.Build();

    // Enable forwarded headers for load balancer
    app.UseForwardedHeaders(new ForwardedHeadersOptions
    {
        ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
    });

    // Use HTTPS redirection
    app.UseHttpsRedirection();

    // Enable authentication
    app.UseAuthentication();
    app.UseAuthorization();

    // Enable dashboard with strict security
    app.UseFlaggyUI(options =>
    {
        options.RoutePrefix = "/admin/feature-flags";
        options.RequireAuthorization = true;
        options.AuthorizationFilter = context =>
        {
            // Only admins can access
            if (!context.User.IsInRole("Admin"))
                return false;

            // IP whitelist check
            var ipAddress = context.Connection.RemoteIpAddress?.ToString();
            var allowedIPs = builder.Configuration.GetSection("Dashboard:IPWhitelist").Get<string[]>();

            return allowedIPs?.Any(ip => IsIpInRange(ipAddress, ip)) ?? false;
        };
    });

    // Health check endpoints
    app.MapHealthChecks("/health");
    app.MapHealthChecks("/health/ready", new HealthCheckOptions
    {
        Predicate = check => check.Tags.Contains("ready")
    });

    app.Run();
}

static bool IsIpInRange(string ipAddress, string cidrRange)
{
    // Implementation of IP range checking
    // This is a simplified example
    return true;
}

Production Infrastructure

Database Configuration

-- MySQL Production Settings

-- Create dedicated database
CREATE DATABASE myapp_prod CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

-- Create dedicated user with limited privileges
CREATE USER 'app_user'@'%' IDENTIFIED BY 'secure-password';
GRANT SELECT, INSERT, UPDATE, DELETE ON myapp_prod.* TO 'app_user'@'%';
FLUSH PRIVILEGES;

-- Configure connection pooling
SET GLOBAL max_connections = 200;
SET GLOBAL wait_timeout = 300;
SET GLOBAL interactive_timeout = 300;

-- Enable slow query log
SET GLOBAL slow_query_log = 'ON';
SET GLOBAL long_query_time = 2;

Redis Configuration

# redis.conf for production

# Network
bind 0.0.0.0
port 6379
protected-mode yes

# Security
requirepass "secure-redis-password"

# SSL/TLS
tls-port 6380
tls-cert-file /path/to/redis.crt
tls-key-file /path/to/redis.key
tls-ca-cert-file /path/to/ca.crt

# Persistence
save 900 1
save 300 10
save 60 10000

# Memory management
maxmemory 2gb
maxmemory-policy allkeys-lru

# Performance
tcp-backlog 511
timeout 300
tcp-keepalive 300

Production Deployment Process

Step 1: Pre-Deployment Validation

# Run in staging environment
# 1. Verify all tests pass
dotnet test

# 2. Load test critical paths
# Use tools like k6, Apache JMeter, or Artillery

# 3. Verify flag behavior
# Test all flags in staging

# 4. Check database migrations
# Review migration scripts before applying

Step 2: Database Migration

# SSH to production database server
ssh db-admin@prod-db.company.com

# Backup current database
mysqldump -u root -p myapp_prod > backup_$(date +%Y%m%d_%H%M%S).sql

# Apply migrations manually (since autoMigrate is false)
mysql -u root -p myapp_prod < migrations/001_initial_schema.sql
mysql -u root -p myapp_prod < migrations/002_add_users_table.sql

# Verify schema
mysql -u root -p myapp_prod -e "SHOW TABLES;"
mysql -u root -p myapp_prod -e "DESCRIBE feature_flags;"

Step 3: Blue-Green Deployment

# Deploy to green environment (new version)
# Keep blue environment (current version) running

# 1. Deploy application to green servers
ansible-playbook deploy-green.yml

# 2. Warm up caches
curl https://green.company.com/health/warm-cache

# 3. Run smoke tests
./scripts/smoke-tests.sh https://green.company.com

# 4. Switch load balancer to green
# Configure load balancer to route traffic to green

# 5. Monitor for 15 minutes
# Watch metrics, logs, and error rates

# 6. If successful, decommission blue
# If issues arise, rollback by switching load balancer back to blue

Step 4: Zero-Downtime Deployment (Rolling Update)

# For Kubernetes deployment

# 1. Update deployment with new image
kubectl set image deployment/myapp myapp=myapp:v2.0.0 -n production

# 2. Watch rollout status
kubectl rollout status deployment/myapp -n production

# 3. Verify new pods are healthy
kubectl get pods -n production -l app=myapp

# 4. If issues occur, rollback
kubectl rollout undo deployment/myapp -n production

Production Secrets Management

Using Azure Key Vault

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

var builder = WebApplication.CreateBuilder(args);

if (builder.Environment.IsProduction())
{
    // Configure Azure Key Vault
    var keyVaultUrl = builder.Configuration["KeyVault:Url"];
    var client = new SecretClient(new Uri(keyVaultUrl), new DefaultAzureCredential());

    // Retrieve secrets
    var dbPassword = (await client.GetSecretAsync("DbPassword")).Value.Value;
    var redisPassword = (await client.GetSecretAsync("RedisPassword")).Value.Value;

    // Build connection strings with secrets
    var connectionString = builder.Configuration.GetConnectionString("DefaultConnection")
        .Replace("${DB_PASSWORD}", dbPassword);

    var redisConnectionString = builder.Configuration.GetConnectionString("Redis")
        .Replace("${REDIS_PASSWORD}", redisPassword);

    // Configure Flaggy
    builder.Services.AddFlaggy(options =>
    {
        options.UseMySQL(connectionString: connectionString);
        options.UseRedisCache(redisConnectionString, TimeSpan.FromMinutes(10));
    });
}

Using AWS Secrets Manager

using Amazon.SecretsManager;
using Amazon.SecretsManager.Model;

var builder = WebApplication.CreateBuilder(args);

if (builder.Environment.IsProduction())
{
    // Configure AWS Secrets Manager
    var client = new AmazonSecretsManagerClient(Amazon.RegionEndpoint.USEast1);

    // Retrieve secrets
    var dbSecretRequest = new GetSecretValueRequest { SecretId = "prod/myapp/database" };
    var dbSecretResponse = await client.GetSecretValueAsync(dbSecretRequest);
    var dbSecrets = JsonSerializer.Deserialize<DatabaseSecrets>(dbSecretResponse.SecretString);

    var redisSecretRequest = new GetSecretValueRequest { SecretId = "prod/myapp/redis" };
    var redisSecretResponse = await client.GetSecretValueAsync(redisSecretRequest);
    var redisSecrets = JsonSerializer.Deserialize<RedisSecrets>(redisSecretResponse.SecretString);

    // Build connection strings
    var connectionString = $"Server={dbSecrets.Host};Database={dbSecrets.Database};User={dbSecrets.Username};Password={dbSecrets.Password};";
    var redisConnectionString = $"{redisSecrets.Host}:{redisSecrets.Port},password={redisSecrets.Password}";

    // Configure Flaggy
    builder.Services.AddFlaggy(options =>
    {
        options.UseMySQL(connectionString: connectionString);
    });
}

Production Best Practices

  1. Disable Auto-Migration: Apply migrations manually with review process
  2. Use SSL/TLS: Encrypt all connections (database, Redis, HTTP)
  3. Restrict Dashboard: IP whitelist + role-based access
  4. Long Cache TTL: 10-15 minutes to reduce database load
  5. Connection Pooling: Configure appropriate pool sizes
  6. Health Checks: Monitor application and dependencies
  7. Secrets Management: Use vault services (Azure Key Vault, AWS Secrets Manager)
  8. Audit Logging: Log all flag changes
  9. Backup Strategy: Regular database backups
  10. Monitoring: Track metrics and set up alerts

Production Checklist

  • Database server hardened and secured
  • Redis server configured with SSL/TLS
  • Connection strings use secrets management
  • Auto-migration disabled
  • Database migrations tested and reviewed
  • Connection pooling configured
  • Health checks implemented
  • Dashboard access restricted (IP + roles)
  • SSL/TLS enabled for all connections
  • Load balancer configured
  • Backup strategy in place
  • Monitoring and alerting configured
  • Disaster recovery plan documented
  • Rollback procedure tested
  • Performance benchmarks established
  • Security scan passed
  • Penetration testing completed

Container Deployment

Docker Configuration

Dockerfile

# Multi-stage build for optimized image size
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src

# Copy project files
COPY ["MyApp.csproj", "./"]
RUN dotnet restore "MyApp.csproj"

# Copy source code
COPY . .

# Build application
RUN dotnet build "MyApp.csproj" -c Release -o /app/build

# Publish application
FROM build AS publish
RUN dotnet publish "MyApp.csproj" -c Release -o /app/publish

# Runtime image
FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS runtime
WORKDIR /app

# Install MySQL client for health checks
RUN apt-get update && apt-get install -y default-mysql-client redis-tools && rm -rf /var/lib/apt/lists/*

# Copy published application
COPY --from=publish /app/publish .

# Set environment variables
ENV ASPNETCORE_ENVIRONMENT=Production
ENV ASPNETCORE_URLS=http://+:8080

# Expose port
EXPOSE 8080

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1

# Run application
ENTRYPOINT ["dotnet", "MyApp.dll"]

docker-compose.yml (Local Development)

version: '3.8'

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "5000:8080"
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ConnectionStrings__DefaultConnection=Server=db;Database=myapp;User=root;Password=dev_password;
      - ConnectionStrings__Redis=redis:6379
    depends_on:
      - db
      - redis
    networks:
      - app-network

  db:
    image: mysql:8.0
    environment:
      - MYSQL_ROOT_PASSWORD=dev_password
      - MYSQL_DATABASE=myapp
    ports:
      - "3306:3306"
    volumes:
      - mysql-data:/var/lib/mysql
    networks:
      - app-network

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data
    networks:
      - app-network

networks:
  app-network:
    driver: bridge

volumes:
  mysql-data:
  redis-data:

Kubernetes Deployment

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: production
  labels:
    app: myapp
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: myregistry.azurecr.io/myapp:latest
        ports:
        - containerPort: 8080
          name: http
        env:
        - name: ASPNETCORE_ENVIRONMENT
          value: "Production"
        - name: ConnectionStrings__DefaultConnection
          valueFrom:
            secretKeyRef:
              name: myapp-secrets
              key: db-connection-string
        - name: ConnectionStrings__Redis
          valueFrom:
            secretKeyRef:
              name: myapp-secrets
              key: redis-connection-string
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "1Gi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 3
          failureThreshold: 3
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
          timeoutSeconds: 3
          failureThreshold: 3
---
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
  namespace: production
spec:
  type: LoadBalancer
  selector:
    app: myapp
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
---
apiVersion: v1
kind: Secret
metadata:
  name: myapp-secrets
  namespace: production
type: Opaque
stringData:
  db-connection-string: "Server=prod-db.company.com;Database=myapp_prod;User=app_user;Password=secure-password;SslMode=Required;"
  redis-connection-string: "prod-redis.company.com:6379,password=secure-redis-password,ssl=true"

Container Deployment Checklist

  • Dockerfile optimized with multi-stage build
  • Health checks configured
  • Environment variables externalized
  • Secrets stored securely
  • Resource limits defined
  • Logging configured
  • Networking configured
  • Persistent volumes for data
  • Readiness and liveness probes
  • Rolling update strategy

Cloud Platform Deployment

Azure App Service

# Login to Azure
az login

# Create resource group
az group create --name myapp-rg --location eastus

# Create App Service plan
az appservice plan create \
  --name myapp-plan \
  --resource-group myapp-rg \
  --sku P1V2 \
  --is-linux

# Create web app
az webapp create \
  --name myapp \
  --resource-group myapp-rg \
  --plan myapp-plan \
  --runtime "DOTNET|8.0"

# Configure app settings
az webapp config appsettings set \
  --name myapp \
  --resource-group myapp-rg \
  --settings \
    ASPNETCORE_ENVIRONMENT=Production \
    ConnectionStrings__DefaultConnection="@Microsoft.KeyVault(SecretUri=https://mykeyvault.vault.azure.net/secrets/DbConnectionString/)" \
    ConnectionStrings__Redis="@Microsoft.KeyVault(SecretUri=https://mykeyvault.vault.azure.net/secrets/RedisConnectionString/)"

# Deploy application
az webapp deployment source config-zip \
  --name myapp \
  --resource-group myapp-rg \
  --src ./publish.zip

AWS Elastic Beanstalk

# Install EB CLI
pip install awsebcli

# Initialize EB application
eb init -p "64bit Amazon Linux 2 v2.6.1 running .NET Core" myapp --region us-east-1

# Create environment
eb create production-env \
  --instance-type t3.medium \
  --scale 3 \
  --envvars \
    ASPNETCORE_ENVIRONMENT=Production,\
    ConnectionStrings__DefaultConnection="{{resolve:secretsmanager:prod/db:SecretString:connectionString}}",\
    ConnectionStrings__Redis="{{resolve:secretsmanager:prod/redis:SecretString:connectionString}}"

# Deploy application
eb deploy production-env

Google Cloud Run

# Build container image
gcloud builds submit --tag gcr.io/my-project/myapp

# Deploy to Cloud Run
gcloud run deploy myapp \
  --image gcr.io/my-project/myapp \
  --platform managed \
  --region us-central1 \
  --allow-unauthenticated \
  --set-env-vars ASPNETCORE_ENVIRONMENT=Production \
  --set-secrets ConnectionStrings__DefaultConnection=db-connection-string:latest,\
                 ConnectionStrings__Redis=redis-connection-string:latest \
  --min-instances 1 \
  --max-instances 10 \
  --memory 512Mi \
  --cpu 1

Monitoring and Health Checks

Health Check Implementation

using Microsoft.Extensions.Diagnostics.HealthChecks;

public class FeatureFlagHealthCheck : IHealthCheck
{
    private readonly IFeatureFlagService _flagService;
    private readonly ILogger<FeatureFlagHealthCheck> _logger;

    public FeatureFlagHealthCheck(
        IFeatureFlagService flagService,
        ILogger<FeatureFlagHealthCheck> logger)
    {
        _flagService = flagService;
        _logger = logger;
    }

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        try
        {
            var sw = Stopwatch.StartNew();

            // Test flag retrieval
            var flags = await _flagService.GetAllFlagsAsync(cancellationToken);
            var flagCount = flags.Count();

            sw.Stop();

            var responseTime = sw.ElapsedMilliseconds;

            if (responseTime > 1000)
            {
                _logger.LogWarning("Flaggy health check slow: {ResponseTime}ms", responseTime);
                return HealthCheckResult.Degraded(
                    $"Feature flag service is slow ({responseTime}ms)",
                    data: new Dictionary<string, object>
                    {
                        { "responseTime", responseTime },
                        { "flagCount", flagCount }
                    }
                );
            }

            _logger.LogDebug("Flaggy health check passed: {ResponseTime}ms, {FlagCount} flags", responseTime, flagCount);
            return HealthCheckResult.Healthy(
                $"Feature flag service is healthy ({responseTime}ms, {flagCount} flags)",
                data: new Dictionary<string, object>
                {
                    { "responseTime", responseTime },
                    { "flagCount", flagCount }
                }
            );
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Flaggy health check failed");
            return HealthCheckResult.Unhealthy(
                "Feature flag service is unavailable",
                ex,
                data: new Dictionary<string, object>
                {
                    { "error", ex.Message }
                }
            );
        }
    }
}

Application Insights Integration (Azure)

using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;

var builder = WebApplication.CreateBuilder(args);

// Configure Application Insights
builder.Services.AddApplicationInsightsTelemetry(options =>
{
    options.ConnectionString = builder.Configuration["ApplicationInsights:ConnectionString"];
});

// Instrumented Flaggy service
public class InstrumentedFeatureFlagService : IFeatureFlagService
{
    private readonly IFeatureFlagService _inner;
    private readonly TelemetryClient _telemetry;

    public async Task<bool> IsEnabledAsync(string key, CancellationToken ct = default)
    {
        var sw = Stopwatch.StartNew();
        try
        {
            var result = await _inner.IsEnabledAsync(key, ct);

            _telemetry.TrackEvent("FeatureFlagEvaluated", new Dictionary<string, string>
            {
                { "flagKey", key },
                { "result", result.ToString() },
                { "duration", sw.ElapsedMilliseconds.ToString() }
            });

            _telemetry.TrackMetric("FeatureFlagEvaluationTime", sw.ElapsedMilliseconds);

            return result;
        }
        catch (Exception ex)
        {
            _telemetry.TrackException(ex, new Dictionary<string, string>
            {
                { "operation", "IsEnabledAsync" },
                { "flagKey", key }
            });
            throw;
        }
    }
}

Prometheus Metrics

using Prometheus;

public class MetricsFeatureFlagService : IFeatureFlagService
{
    private readonly IFeatureFlagService _inner;

    private static readonly Counter FlagEvaluations = Metrics.CreateCounter(
        "flaggy_evaluations_total",
        "Total number of flag evaluations",
        new CounterConfiguration { LabelNames = new[] { "flag_key", "result" } }
    );

    private static readonly Histogram FlagEvaluationDuration = Metrics.CreateHistogram(
        "flaggy_evaluation_duration_seconds",
        "Duration of flag evaluations",
        new HistogramConfiguration { LabelNames = new[] { "flag_key" } }
    );

    public async Task<bool> IsEnabledAsync(string key, CancellationToken ct = default)
    {
        using (FlagEvaluationDuration.WithLabels(key).NewTimer())
        {
            var result = await _inner.IsEnabledAsync(key, ct);
            FlagEvaluations.WithLabels(key, result.ToString()).Inc();
            return result;
        }
    }
}

Troubleshooting

Common Deployment Issues

Issue 1: Database Connection Failures

Symptoms: Application fails to start, logs show database connection errors

Solutions:

# Verify database connectivity
mysql -h prod-db.company.com -u app_user -p

# Check firewall rules
telnet prod-db.company.com 3306

# Verify connection string
echo $ConnectionStrings__DefaultConnection

# Test from application server
dotnet run --urls="http://localhost:5000" --environment=Production

# Check database user permissions
SHOW GRANTS FOR 'app_user'@'%';

Issue 2: Redis Cache Not Working

Symptoms: Slow performance, high database load

Solutions:

# Test Redis connectivity
redis-cli -h prod-redis.company.com -p 6379 -a $REDIS_PASSWORD ping

# Check Redis keys
redis-cli -h prod-redis.company.com -p 6379 -a $REDIS_PASSWORD
> KEYS flaggy:*
> GET flaggy:feature-key

# Monitor Redis
redis-cli -h prod-redis.company.com -p 6379 -a $REDIS_PASSWORD --latency
redis-cli -h prod-redis.company.com -p 6379 -a $REDIS_PASSWORD INFO stats

# Clear Redis cache if stale
redis-cli -h prod-redis.company.com -p 6379 -a $REDIS_PASSWORD FLUSHDB

Issue 3: Dashboard Inaccessible

Symptoms: 404 or 403 errors when accessing dashboard

Solutions:

// Verify dashboard is enabled
Console.WriteLine($"Dashboard Enabled: {builder.Configuration["Dashboard:Enabled"]}");

// Check route prefix
app.UseFlaggyUI(options =>
{
    options.RoutePrefix = "/admin/feature-flags"; // Ensure correct route
    Console.WriteLine($"Dashboard Route: /{options.RoutePrefix}");
});

// Temporarily disable authorization for testing
app.UseFlaggyUI(options =>
{
    options.RequireAuthorization = false; // ONLY FOR DEBUGGING
});

// Check IP whitelist
var clientIp = context.Connection.RemoteIpAddress?.ToString();
Console.WriteLine($"Client IP: {clientIp}");

Issue 4: Slow Flag Evaluation

Symptoms: Application response times increased

Solutions:

// Increase cache expiration
builder.Services.AddFlaggy(options =>
{
    options.UseMySQL(connectionString: connectionString);
    options.UseMemoryCache(TimeSpan.FromMinutes(15)); // Increase from 10 to 15
});

// Check cache hit rate
public class CacheMetrics
{
    public long CacheHits { get; set; }
    public long CacheMisses { get; set; }
    public double HitRate => (double)CacheHits / (CacheHits + CacheMisses) * 100;
}

// Optimize flag checks
// Bad: Check flag on every request
var isEnabled = await _flagService.IsEnabledAsync("my-flag");

// Good: Cache result in memory for short period
private static bool? _cachedFlag;
private static DateTime _lastCheck = DateTime.MinValue;

if (_cachedFlag == null || DateTime.UtcNow - _lastCheck > TimeSpan.FromSeconds(30))
{
    _cachedFlag = await _flagService.IsEnabledAsync("my-flag");
    _lastCheck = DateTime.UtcNow;
}

Rollback Procedures

Immediate Rollback (Critical Issues)

# Load balancer rollback (immediate)
# Switch load balancer back to previous version

# Kubernetes rollback
kubectl rollout undo deployment/myapp -n production

# Docker Swarm rollback
docker service update --rollback myapp

# Azure App Service rollback
az webapp deployment slot swap \
  --name myapp \
  --resource-group myapp-rg \
  --slot staging \
  --target-slot production

# AWS Elastic Beanstalk rollback
eb abort production-env
eb deploy production-env --version previous-version

Database Rollback

# Restore from backup
mysql -u root -p myapp_prod < backup_20250118_120000.sql

# Verify data integrity
mysql -u root -p myapp_prod -e "SELECT COUNT(*) FROM feature_flags;"

Next Steps

After successful deployment: