WASM Orchestration with SpinKube and Fermyon: Production-Ready WebAssembly Platform

WASM Orchestration with SpinKube and Fermyon: Production-Ready WebAssembly Platform

The WebAssembly ecosystem has evolved beyond individual modules to comprehensive orchestration platforms. SpinKube and Fermyon represent the cutting-edge of WASM application deployment, providing Kubernetes-native solutions for managing WebAssembly workloads at scale. This guide covers everything from basic setup to advanced production patterns for WASM orchestration.

Table of Contents

1. Platform Architecture Overview
2. SpinKube Installation and Setup
3. Fermyon Platform Configuration
4. Building Spin Applications
5. Deployment Patterns and Strategies
6. Service Mesh Integration
7. Scaling and Resource Management
8. Monitoring and Observability
9. CI/CD Pipeline Integration
10. Advanced Configuration and Tuning
11. Production Operations
12. Migration Strategies

Platform Architecture Overview

SpinKube Architecture

SpinKube Platform Stack:
┌─────────────────────────────────────────────────┐
│                kubectl/UI                       │
├─────────────────────────────────────────────────┤
│           Kubernetes API Server                 │
├─────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────────────┐│
│  │ Spin Operator   │  │  SpinApp Controller     ││
│  │                 │  │                         ││
│  │ • Lifecycle     │  │ • Custom Resources      ││
│  │ • Scheduling    │  │ • WASM Runtime Mgmt     ││
│  │ • Scaling       │  │ • Service Discovery     ││
│  └─────────────────┘  └─────────────────────────┘│
├─────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────────────┐│
│  │ containerd-shim │  │    WASM Runtime         ││
│  │ -spin-v2        │  │                         ││
│  │                 │  │ • wasmtime              ││
│  │ • Process Mgmt  │  │ • Spin Framework        ││
│  │ • Runtime API   │  │ • WASI Support          ││
│  │ • OCI Support   │  │ • Component Model       ││
│  └─────────────────┘  └─────────────────────────┘│
├─────────────────────────────────────────────────┤
│              Node Operating System              │
└─────────────────────────────────────────────────┘

Fermyon Cloud Architecture

Fermyon Cloud Platform:
┌─────────────────────────────────────────────────┐
│              Fermyon Cloud UI/CLI               │
├─────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────────────┐│
│  │  Application    │  │    Edge Network         ││
│  │  Gateway        │  │                         ││
│  │                 │  │ • Global CDN            ││
│  │ • Routing       │  │ • Edge Caching          ││
│  │ • Load Balancing│  │ • TLS Termination       ││
│  │ • Rate Limiting │  │ • DDoS Protection       ││
│  └─────────────────┘  └─────────────────────────┘│
├─────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────────────┐│
│  │ Orchestrator    │  │  Runtime Cluster        ││
│  │                 │  │                         ││
│  │ • Auto-scaling  │  │ • Multi-region          ││
│  │ • Health Checks │  │ • Cold Start Opt        ││
│  │ • Blue/Green    │  │ • Resource Isolation    ││
│  │ • Canary Deploy │  │ • Security Sandboxing   ││
│  └─────────────────┘  └─────────────────────────┘│
├─────────────────────────────────────────────────┤
│            Managed Infrastructure               │
└─────────────────────────────────────────────────┘

Component Interaction Flow

# Component interaction patterns
interaction_patterns:
  spin_application:
    - name: "HTTP Trigger"
      type: "inbound"
      protocol: "HTTP/HTTPS"
      
    - name: "Database Access"
      type: "outbound"
      protocol: "PostgreSQL/MySQL"
      
    - name: "Key-Value Store"
      type: "outbound" 
      protocol: "Redis/DynamoDB"
      
    - name: "External API"
      type: "outbound"
      protocol: "HTTP/REST"
      
  platform_services:
    - name: "Service Discovery"
      provider: "Kubernetes DNS"
      
    - name: "Load Balancing"
      provider: "Ingress Controller"
      
    - name: "Monitoring"
      provider: "Prometheus/Grafana"
      
    - name: "Logging"
      provider: "Fluentd/ELK Stack"

SpinKube Installation and Setup

Prerequisites and Dependencies

# Install required tools
# kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl && sudo mv kubectl /usr/local/bin/

# Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# spin CLI
curl -fsSL https://developer.fermyon.com/downloads/install.sh | bash
sudo mv spin /usr/local/bin/

# Verify installations
kubectl version --client
helm version
spin --version

SpinKube Operator Installation

# Add SpinKube Helm repository
helm repo add spinkube https://spinkube.dev/charts
helm repo update

# Install cert-manager (required dependency)
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml

# Wait for cert-manager to be ready
kubectl wait --for=condition=ready pod --all -n cert-manager --timeout=300s

# Install SpinKube operator
helm install spinkube-operator spinkube/spin-operator \
  --namespace spinkube \
  --create-namespace \
  --wait

# Install runtime class executor
kubectl apply -f https://github.com/spinkube/spin-operator/releases/latest/download/spin-operator.runtime-class.yaml

# Verify installation
kubectl get pods -n spinkube
kubectl get runtimeclass

Runtime Configuration

# spinkube-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: spinkube-config
  namespace: spinkube
data:
  config.yaml: |
    runtime:
      executor: containerd-shim-spin
      default_executor_image: ghcr.io/spinkube/containerd-shim-spin:v0.13.0
      
    scaling:
      min_replicas: 1
      max_replicas: 100
      target_cpu_utilization: 70
      
    networking:
      service_type: ClusterIP
      ingress_class: nginx
      
    security:
      run_as_non_root: true
      read_only_root_filesystem: true
      capabilities:
        drop:
          - ALL
          
    observability:
      metrics_enabled: true
      tracing_enabled: true
      logging_level: info
      
    storage:
      key_value_store: redis
      database_connections: postgresql
      
    features:
      wasi_preview2: true
      component_model: true
      experimental_features: false

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: spinkube-operator
  namespace: spinkube
spec:
  template:
    spec:
      containers:
      - name: manager
        env:
        - name: RUNTIME_CONFIG
          valueFrom:
            configMapKeyRef:
              name: spinkube-config
              key: config.yaml

Node Configuration

# node-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: containerd-config
  namespace: kube-system
data:
  config.toml: |
    version = 2
    
    [plugins]
      [plugins."io.containerd.grpc.v1.cri"]
        [plugins."io.containerd.grpc.v1.cri".containerd]
          default_runtime_name = "runc"
          
          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
            [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
              runtime_type = "io.containerd.runc.v2"
              
            [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.spin]
              runtime_type = "io.containerd.spin.v2"
              
            [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmtime-spin]
              runtime_type = "io.containerd.wasmtime-spin.v1"

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: containerd-config-updater
  namespace: kube-system
spec:
  selector:
    matchLabels:
      name: containerd-config-updater
  template:
    metadata:
      labels:
        name: containerd-config-updater
    spec:
      hostPID: true
      containers:
      - name: updater
        image: alpine:latest
        command:
        - /bin/sh
        - -c
        - |
          cp /config/config.toml /host/etc/containerd/config.toml
          kill -USR1 $(pgrep containerd)
          sleep infinity
        volumeMounts:
        - name: config
          mountPath: /config
        - name: containerd-config
          mountPath: /host/etc/containerd
        securityContext:
          privileged: true
      volumes:
      - name: config
        configMap:
          name: containerd-config
      - name: containerd-config
        hostPath:
          path: /etc/containerd

Fermyon Platform Configuration

Fermyon Cloud Setup

# Install Fermyon CLI
curl -fsSL https://developer.fermyon.com/downloads/install.sh | bash

# Login to Fermyon Cloud
spin login

# Create a new Fermyon application
spin new http-rust my-spin-app
cd my-spin-app

# Deploy to Fermyon Cloud
spin deploy

# Check deployment status
spin apps list

Self-Hosted Fermyon Platform

# fermyon-platform.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: fermyon-platform

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fermyon-platform
  namespace: fermyon-platform
spec:
  replicas: 3
  selector:
    matchLabels:
      app: fermyon-platform
  template:
    metadata:
      labels:
        app: fermyon-platform
    spec:
      containers:
      - name: platform
        image: ghcr.io/fermyon/fermyon-platform:v2.0.0
        ports:
        - containerPort: 3000
        - containerPort: 9090
        env:
        - name: FERMYON_PLATFORM_MODE
          value: "kubernetes"
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: platform-secrets
              key: database-url
        - name: REGISTRY_URL
          value: "https://registry.example.com"
        - name: SPIN_RUNTIME_IMAGE
          value: "ghcr.io/fermyon/spin:v2.0.0"
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        volumeMounts:
        - name: platform-config
          mountPath: /etc/fermyon
        - name: tls-certs
          mountPath: /etc/ssl/certs
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: platform-config
        configMap:
          name: fermyon-config
      - name: tls-certs
        secret:
          secretName: fermyon-tls

---
apiVersion: v1
kind: Service
metadata:
  name: fermyon-platform-service
  namespace: fermyon-platform
spec:
  selector:
    app: fermyon-platform
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 3000
    - name: metrics
      protocol: TCP
      port: 9090
      targetPort: 9090
  type: LoadBalancer

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: fermyon-config
  namespace: fermyon-platform
data:
  platform.toml: |
    [platform]
    name = "Self-Hosted Fermyon"
    version = "2.0.0"
    
    [server]
    host = "0.0.0.0"
    port = 3000
    
    [database]
    max_connections = 100
    connection_timeout = 30
    
    [runtime]
    spin_version = "2.0.0"
    wasmtime_version = "16.0.0"
    default_memory_limit = "128MB"
    default_timeout = "30s"
    
    [scaling]
    min_instances = 0
    max_instances = 1000
    scale_down_delay = "5m"
    
    [security]
    enable_tls = true
    require_authentication = true
    
    [observability]
    enable_metrics = true
    enable_tracing = true
    log_level = "info"

Building Spin Applications

HTTP Service Application

// src/lib.rs - Rust HTTP service
use spin_sdk::{
    http::{Request, Response, Method},
    http_component,
    key_value::Store,
    sqlite::{Connection, Value},
};
use serde::{Deserialize, Serialize};
use anyhow::Result;

#[derive(Serialize, Deserialize)]
struct User {
    id: u32,
    name: String,
    email: String,
    created_at: String,
}

#[derive(Serialize, Deserialize)]
struct CreateUserRequest {
    name: String,
    email: String,
}

#[http_component]
fn handle_request(req: Request) -> Result<Response> {
    let method = req.method();
    let path = req.uri().path();
    
    match (method, path) {
        (Method::GET, "/users") => get_users(),
        (Method::POST, "/users") => create_user(req),
        (Method::GET, path) if path.starts_with("/users/") => {
            let id = path.strip_prefix("/users/").unwrap();
            get_user(id.parse().unwrap_or(0))
        },
        (Method::PUT, path) if path.starts_with("/users/") => {
            let id = path.strip_prefix("/users/").unwrap();
            update_user(id.parse().unwrap_or(0), req)
        },
        (Method::DELETE, path) if path.starts_with("/users/") => {
            let id = path.strip_prefix("/users/").unwrap();
            delete_user(id.parse().unwrap_or(0))
        },
        (Method::GET, "/health") => Ok(Response::builder()
            .status(200)
            .header("content-type", "application/json")
            .body(r#"{"status": "healthy"}"#)?),
        _ => Ok(Response::builder()
            .status(404)
            .body("Not Found")?),
    }
}

fn get_users() -> Result<Response> {
    let connection = Connection::open_default()?;
    
    let users = connection.execute(
        "SELECT id, name, email, created_at FROM users ORDER BY created_at DESC",
        &[]
    )?;
    
    let mut user_list = Vec::new();
    
    for row in users.rows() {
        let user = User {
            id: row.get::<u32>("id")?,
            name: row.get::<&str>("name")?.to_string(),
            email: row.get::<&str>("email")?.to_string(),
            created_at: row.get::<&str>("created_at")?.to_string(),
        };
        user_list.push(user);
    }
    
    let json = serde_json::to_string(&user_list)?;
    
    Ok(Response::builder()
        .status(200)
        .header("content-type", "application/json")
        .body(json)?)
}

fn create_user(req: Request) -> Result<Response> {
    let body = req.body();
    let create_request: CreateUserRequest = serde_json::from_slice(body)?;
    
    // Validate input
    if create_request.name.is_empty() || create_request.email.is_empty() {
        return Ok(Response::builder()
            .status(400)
            .body("Name and email are required")?);
    }
    
    if !create_request.email.contains('@') {
        return Ok(Response::builder()
            .status(400)
            .body("Invalid email format")?);
    }
    
    let connection = Connection::open_default()?;
    
    // Check if email already exists
    let existing = connection.execute(
        "SELECT COUNT(*) as count FROM users WHERE email = ?",
        &[Value::Text(create_request.email.clone())]
    )?;
    
    if let Some(row) = existing.rows().next() {
        let count: u32 = row.get("count")?;
        if count > 0 {
            return Ok(Response::builder()
                .status(409)
                .body("Email already exists")?);
        }
    }
    
    // Insert new user
    connection.execute(
        "INSERT INTO users (name, email, created_at) VALUES (?, ?, datetime('now'))",
        &[
            Value::Text(create_request.name.clone()),
            Value::Text(create_request.email.clone()),
        ]
    )?;
    
    // Get the created user
    let result = connection.execute(
        "SELECT id, name, email, created_at FROM users WHERE email = ?",
        &[Value::Text(create_request.email)]
    )?;
    
    if let Some(row) = result.rows().next() {
        let user = User {
            id: row.get::<u32>("id")?,
            name: row.get::<&str>("name")?.to_string(),
            email: row.get::<&str>("email")?.to_string(),
            created_at: row.get::<&str>("created_at")?.to_string(),
        };
        
        let json = serde_json::to_string(&user)?;
        
        Ok(Response::builder()
            .status(201)
            .header("content-type", "application/json")
            .body(json)?)
    } else {
        Ok(Response::builder()
            .status(500)
            .body("Failed to create user")?)
    }
}

fn get_user(id: u32) -> Result<Response> {
    let connection = Connection::open_default()?;
    
    let result = connection.execute(
        "SELECT id, name, email, created_at FROM users WHERE id = ?",
        &[Value::Integer(id as i64)]
    )?;
    
    if let Some(row) = result.rows().next() {
        let user = User {
            id: row.get::<u32>("id")?,
            name: row.get::<&str>("name")?.to_string(),
            email: row.get::<&str>("email")?.to_string(),
            created_at: row.get::<&str>("created_at")?.to_string(),
        };
        
        let json = serde_json::to_string(&user)?;
        
        Ok(Response::builder()
            .status(200)
            .header("content-type", "application/json")
            .body(json)?)
    } else {
        Ok(Response::builder()
            .status(404)
            .body("User not found")?)
    }
}

fn update_user(id: u32, req: Request) -> Result<Response> {
    let body = req.body();
    let update_request: CreateUserRequest = serde_json::from_slice(body)?;
    
    let connection = Connection::open_default()?;
    
    // Check if user exists
    let existing = connection.execute(
        "SELECT COUNT(*) as count FROM users WHERE id = ?",
        &[Value::Integer(id as i64)]
    )?;
    
    if let Some(row) = existing.rows().next() {
        let count: u32 = row.get("count")?;
        if count == 0 {
            return Ok(Response::builder()
                .status(404)
                .body("User not found")?);
        }
    }
    
    // Update user
    connection.execute(
        "UPDATE users SET name = ?, email = ? WHERE id = ?",
        &[
            Value::Text(update_request.name),
            Value::Text(update_request.email),
            Value::Integer(id as i64),
        ]
    )?;
    
    // Return updated user
    get_user(id)
}

fn delete_user(id: u32) -> Result<Response> {
    let connection = Connection::open_default()?;
    
    let result = connection.execute(
        "DELETE FROM users WHERE id = ?",
        &[Value::Integer(id as i64)]
    )?;
    
    if result.rows_affected() > 0 {
        Ok(Response::builder()
            .status(204)
            .body("")?)
    } else {
        Ok(Response::builder()
            .status(404)
            .body("User not found")?)
    }
}

Application Configuration

# spin.toml
spin_manifest_version = 2

[application]
name = "user-management-api"
version = "1.0.0"
description = "User management REST API built with Spin"
authors = ["Fenil Sonani <[email protected]>"]

[application.trigger.http]
base = "/"

[[trigger.http]]
id = "user-api"
component = "user-api"
route = "/..."

[component.user-api]
source = "target/wasm32-wasi/release/user_api.wasm"
allowed_outbound_hosts = []
key_value_stores = ["default"]
sqlite_databases = ["default"]
environment = { LOG_LEVEL = "info" }

[component.user-api.build]
command = "cargo build --target wasm32-wasi --release"
workdir = "."

# Database schema initialization
[component.user-api.sqlite_databases.default]
type = "spin"
default = true

# Key-value store for caching
[component.user-api.key_value_stores.default]
type = "spin"
default = true

# Environment-specific configurations
[application.variables]
database_url = { required = false, default = "sqlite://default.db" }
cache_ttl = { required = false, default = "300" }
rate_limit = { required = false, default = "100" }

Build and Deployment Scripts

#!/bin/bash
# build.sh

set -e

echo "Building Spin application..."

# Install dependencies
rustup target add wasm32-wasi

# Build the application
cargo build --target wasm32-wasi --release

# Initialize database schema
cat << 'EOF' > schema.sql
CREATE TABLE IF NOT EXISTS users (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,
    email TEXT UNIQUE NOT NULL,
    created_at TEXT NOT NULL
);

CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
CREATE INDEX IF NOT EXISTS idx_users_created_at ON users(created_at);

-- Insert sample data
INSERT OR IGNORE INTO users (name, email, created_at) VALUES 
('John Doe', '[email protected]', datetime('now')),
('Jane Smith', '[email protected]', datetime('now'));
EOF

# Test the application locally
echo "Testing application locally..."
spin build
spin up --detach

# Wait for the application to start
sleep 5

# Run basic tests
echo "Running API tests..."
curl -s http://localhost:3000/health | grep -q "healthy" && echo "✓ Health check passed"
curl -s http://localhost:3000/users | jq length | grep -q "2" && echo "✓ Users endpoint working"

# Stop local instance
spin down

echo "Build completed successfully!"

Deployment Patterns and Strategies

SpinKube Deployment

# spinapp-deployment.yaml
apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: user-management-api
  namespace: production
spec:
  image: "registry.example.com/user-api:v1.0.0"
  replicas: 5
  
  # Resource configuration
  resources:
    limits:
      cpu: 500m
      memory: 128Mi
    requests:
      cpu: 100m
      memory: 64Mi
  
  # Scaling configuration
  autoscaler:
    minReplicas: 2
    maxReplicas: 50
    targetCPUUtilizationPercentage: 70
    targetMemoryUtilizationPercentage: 80
    
  # Environment variables
  variables:
    - name: LOG_LEVEL
      value: "info"
    - name: DATABASE_URL
      valueFrom:
        secretKeyRef:
          name: database-secrets
          key: url
    - name: CACHE_TTL
      value: "300"
    - name: RATE_LIMIT
      value: "1000"
  
  # Service configuration
  executor: containerd-shim-spin
  
  # Health checks
  livenessProbe:
    httpGet:
      path: /health
      port: 80
    initialDelaySeconds: 10
    periodSeconds: 30
    
  readinessProbe:
    httpGet:
      path: /health
      port: 80
    initialDelaySeconds: 5
    periodSeconds: 10
  
  # Security context
  securityContext:
    runAsNonRoot: true
    readOnlyRootFilesystem: true

---
apiVersion: v1
kind: Service
metadata:
  name: user-api-service
  namespace: production
spec:
  selector:
    app: user-management-api
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 3000
  type: ClusterIP

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: user-api-ingress
  namespace: production
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/rate-limit: "100"
    nginx.ingress.kubernetes.io/rate-limit-window: "1m"
spec:
  tls:
  - hosts:
    - api.example.com
    secretName: api-tls
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: user-api-service
            port:
              number: 80

Multi-Environment Deployment

# environments/staging.yaml
apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: user-management-api
  namespace: staging
spec:  
  image: "registry.example.com/user-api:staging"
  replicas: 2
  
  resources:
    limits:
      cpu: 200m
      memory: 64Mi
    requests:
      cpu: 50m
      memory: 32Mi
  
  variables:
    - name: LOG_LEVEL
      value: "debug"
    - name: DATABASE_URL
      value: "sqlite://staging.db"
    - name: RATE_LIMIT
      value: "50"

---
# environments/production.yaml
apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: user-management-api
  namespace: production
spec:
  image: "registry.example.com/user-api:v1.0.0"
  replicas: 10
  
  resources:
    limits:
      cpu: 1000m
      memory: 256Mi
    requests:
      cpu: 200m
      memory: 128Mi
  
  autoscaler:
    minReplicas: 5
    maxReplicas: 100
    targetCPUUtilizationPercentage: 60
    
  variables:
    - name: LOG_LEVEL
      value: "warn"
    - name: DATABASE_URL
      valueFrom:
        secretKeyRef:
          name: production-db-secrets
          key: url
    - name: RATE_LIMIT
      value: "2000"
      
  # Blue-Green deployment support
  strategy:
    type: BlueGreen
    blueGreen:
      autoPromotionEnabled: false
      scaleDownDelaySeconds: 300

Canary Deployment Pattern

# canary-deployment.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: user-api-rollout
  namespace: production
spec:
  replicas: 10
  strategy:
    canary:
      steps:
      - setWeight: 10
      - pause: {duration: 2m}
      - setWeight: 25
      - pause: {duration: 5m}
      - setWeight: 50
      - pause: {duration: 10m}
      - setWeight: 75
      - pause: {duration: 10m}
      canaryService: user-api-canary
      stableService: user-api-stable
      trafficRouting:
        nginx:
          stableIngress: user-api-ingress
          annotationPrefix: nginx.ingress.kubernetes.io
      analysis:
        templates:
        - templateName: success-rate
        args:
        - name: service-name
          value: user-api-canary
  selector:
    matchLabels:
      app: user-management-api
  template:
    metadata:
      labels:
        app: user-management-api
    spec:
      runtimeClassName: spin
      containers:
      - name: user-api
        image: registry.example.com/user-api:v1.1.0
        ports:
        - containerPort: 3000

---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  args:
  - name: service-name
  metrics:
  - name: success-rate
    interval: 60s
    count: 5
    successCondition: result[0] >= 0.95
    provider:
      prometheus:
        address: http://prometheus.monitoring.svc.cluster.local:9090
        query: |
          sum(rate(http_requests_total{service="{{args.service-name}}", status!~"5.."}[5m])) /
          sum(rate(http_requests_total{service="{{args.service-name}}"}[5m]))

Service Mesh Integration

Istio Integration

# istio-spinapp.yaml
apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: user-management-api
  namespace: production
  annotations:
    sidecar.istio.io/inject: "true"
spec:
  image: "registry.example.com/user-api:v1.0.0"
  replicas: 5
  
  # Istio-specific annotations
  template:
    metadata:
      annotations:
        sidecar.istio.io/proxyCPU: "10m"
        sidecar.istio.io/proxyMemory: "64Mi"
        traffic.sidecar.istio.io/includeInboundPorts: "3000"

---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: user-api-vs
  namespace: production
spec:
  hosts:
  - api.example.com
  gateways:
  - user-api-gateway
  http:
  - match:
    - uri:
        prefix: "/v2/"
    rewrite:
      uri: "/"
    route:
    - destination:
        host: user-api-service
        port:
          number: 80
      weight: 90
    - destination:
        host: user-api-canary-service
        port:
          number: 80
      weight: 10
    fault:
      delay:
        percentage:
          value: 0.1
        fixedDelay: 100ms
    retries:
      attempts: 3
      perTryTimeout: 30s

---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: user-api-dr
  namespace: production
spec:
  host: user-api-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http1MaxPendingRequests: 50
        http2MaxRequests: 100
        maxRequestsPerConnection: 10
        maxRetries: 3
    loadBalancer:
      simple: LEAST_CONN
    outlierDetection:
      consecutiveErrors: 3
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: user-api-authz
  namespace: production
spec:
  selector:
    matchLabels:
      app: user-management-api
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/frontend/sa/frontend-service"]
  - to:
    - operation:
        methods: ["GET", "POST", "PUT", "DELETE"]
        paths: ["/users/*", "/health"]
  - when:
    - key: request.headers[authorization]
      values: ["Bearer *"]

Linkerd Integration

# linkerd-spinapp.yaml
apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: user-management-api
  namespace: production
  annotations:
    linkerd.io/inject: enabled
    config.linkerd.io/proxy-cpu-request: "10m"
    config.linkerd.io/proxy-memory-request: "32Mi"
spec:
  image: "registry.example.com/user-api:v1.0.0"
  replicas: 5

---
apiVersion: policy.linkerd.io/v1beta1
kind: Server
metadata:
  name: user-api-server
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: user-management-api
  port: 3000
  proxyProtocol: HTTP/2

---
apiVersion: policy.linkerd.io/v1beta1
kind: HTTPRoute
metadata:
  name: user-api-route
  namespace: production
spec:
  parentRefs:
  - name: user-api-server
    kind: Server
    group: policy.linkerd.io
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: "/users"
    backendRefs:
    - name: user-api-service
      port: 80
  - matches:
    - path:
        type: Exact
        value: "/health"
    backendRefs:
    - name: user-api-service
      port: 80

---
apiVersion: policy.linkerd.io/v1beta1
kind: ServerAuthorization
metadata:
  name: user-api-authz
  namespace: production
spec:
  server:
    name: user-api-server
  requiredRoutes:
  - pathRegex: "/users.*"
    methods: ["GET", "POST", "PUT", "DELETE"]
  - pathRegex: "/health"
    methods: ["GET"]
  client:
    meshTLS:
      serviceAccounts:
      - name: frontend-service
        namespace: frontend
      - name: admin-service
        namespace: admin

Scaling and Resource Management

Horizontal Pod Autoscaling

# hpa-configuration.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: user-api-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: core.spinkube.dev/v1alpha1
    kind: SpinApp
    name: user-management-api
  minReplicas: 3
  maxReplicas: 100
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "100"
  - type: External
    external:
      metric:
        name: sqs_queue_depth
        selector:
          matchLabels:
            queue: user-processing
      target:
        type: AverageValue
        averageValue: "10"
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
      - type: Pods
        value: 4
        periodSeconds: 15
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60
      selectPolicy: Min

---
apiVersion: v1
kind: ServiceMonitor
metadata:
  name: user-api-metrics
  namespace: production
spec:
  selector:
    matchLabels:
      app: user-management-api
  endpoints:
  - port: metrics
    interval: 30s
    path: /metrics

Vertical Pod Autoscaling

# vpa-configuration.yaml
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: user-api-vpa
  namespace: production
spec:
  targetRef:
    apiVersion: core.spinkube.dev/v1alpha1
    kind: SpinApp
    name: user-management-api
  updatePolicy:
    updateMode: "Auto"
  resourcePolicy:
    containerPolicies:
    - containerName: user-api
      minAllowed:
        cpu: 50m
        memory: 32Mi
      maxAllowed:
        cpu: 2000m
        memory: 512Mi
      controlledResources: ["cpu", "memory"]
      controlledValues: RequestsAndLimits

Custom Resource Scaling

# custom-scaler.yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: user-api-scaler
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: core.spinkube.dev/v1alpha1
    kind: SpinApp
    name: user-management-api
  minReplicaCount: 2
  maxReplicaCount: 50
  pollingInterval: 30
  cooldownPeriod: 300
  idleReplicaCount: 1
  
  triggers:
  # HTTP requests trigger
  - type: prometheus
    metadata:
      serverAddress: http://prometheus.monitoring.svc.cluster.local:9090
      metricName: http_requests_per_second
      threshold: '100'
      query: sum(rate(http_requests_total{service="user-api"}[1m]))
  
  # Queue depth trigger
  - type: aws-sqs-queue
    metadata:
      queueURL: https://sqs.us-east-1.amazonaws.com/123456789/user-processing
      queueLength: '10'
      awsRegion: us-east-1
    authenticationRef:
      name: aws-credentials
  
  # Database connection trigger  
  - type: postgresql
    metadata:
      connection: postgresql://user:[email protected]:5432/userdb
      query: "SELECT COUNT(*) FROM pg_stat_activity WHERE state = 'active'"
      targetQueryValue: '5'
    authenticationRef:
      name: postgres-credentials

---
apiVersion: v1
kind: Secret
metadata:
  name: aws-credentials
  namespace: production
type: Opaque
data:
  AWS_ACCESS_KEY_ID: <base64-encoded-key>
  AWS_SECRET_ACCESS_KEY: <base64-encoded-secret>

---
apiVersion: v1
kind: Secret
metadata:
  name: postgres-credentials
  namespace: production
type: Opaque
data:
  connection: <base64-encoded-connection-string>

Monitoring and Observability

Prometheus Metrics Collection

// src/metrics.rs
use prometheus::{Counter, Histogram, Gauge, Registry, Encoder, TextEncoder};
use std::sync::Arc;
use std::time::Instant;

pub struct Metrics {
    pub registry: Registry,
    pub http_requests_total: Counter,
    pub http_request_duration: Histogram,
    pub active_connections: Gauge,
    pub database_queries_total: Counter,
    pub cache_hits_total: Counter,
    pub cache_misses_total: Counter,
}

impl Metrics {
    pub fn new() -> Self {
        let registry = Registry::new();
        
        let http_requests_total = Counter::new(
            "http_requests_total",
            "Total number of HTTP requests"
        ).unwrap();
        
        let http_request_duration = Histogram::with_opts(
            prometheus::HistogramOpts::new(
                "http_request_duration_seconds",
                "HTTP request duration in seconds"
            ).buckets(vec![0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0])
        ).unwrap();
        
        let active_connections = Gauge::new(
            "active_connections",
            "Number of active connections"
        ).unwrap();
        
        let database_queries_total = Counter::new(
            "database_queries_total",
            "Total number of database queries"
        ).unwrap();
        
        let cache_hits_total = Counter::new(
            "cache_hits_total", 
            "Total number of cache hits"
        ).unwrap();
        
        let cache_misses_total = Counter::new(
            "cache_misses_total",
            "Total number of cache misses"
        ).unwrap();
        
        // Register metrics
        registry.register(Box::new(http_requests_total.clone())).unwrap();
        registry.register(Box::new(http_request_duration.clone())).unwrap();
        registry.register(Box::new(active_connections.clone())).unwrap();
        registry.register(Box::new(database_queries_total.clone())).unwrap();
        registry.register(Box::new(cache_hits_total.clone())).unwrap();
        registry.register(Box::new(cache_misses_total.clone())).unwrap();
        
        Self {
            registry,
            http_requests_total,
            http_request_duration,
            active_connections,
            database_queries_total,
            cache_hits_total,
            cache_misses_total,
        }
    }
    
    pub fn render(&self) -> String {
        let encoder = TextEncoder::new();
        let metric_families = self.registry.gather();
        encoder.encode_to_string(&metric_families).unwrap()
    }
}

pub struct RequestTimer {
    start: Instant,
    histogram: Histogram,
}

impl RequestTimer {
    pub fn new(histogram: Histogram) -> Self {
        Self {
            start: Instant::now(),
            histogram,
        }
    }
}

impl Drop for RequestTimer {
    fn drop(&mut self) {
        let duration = self.start.elapsed().as_secs_f64();
        self.histogram.observe(duration);
    }
}

// Global metrics instance
static METRICS: once_cell::sync::Lazy<Arc<Metrics>> = 
    once_cell::sync::Lazy::new(|| Arc::new(Metrics::new()));

pub fn get_metrics() -> Arc<Metrics> {
    Arc::clone(&METRICS)
}

// Middleware for request metrics
pub fn record_request(path: &str, method: &str, status: u16) {
    let metrics = get_metrics();
    metrics.http_requests_total
        .with_label_values(&[path, method, &status.to_string()])
        .inc();
}

pub fn start_request_timer() -> RequestTimer {
    let metrics = get_metrics();
    RequestTimer::new(metrics.http_request_duration.clone())
}

// Integration with Spin
#[http_component]
fn handle_metrics(_req: Request) -> Result<Response> {
    let metrics = get_metrics();
    let metrics_output = metrics.render();
    
    Ok(Response::builder()
        .status(200)
        .header("content-type", "text/plain; version=0.0.4; charset=utf-8")
        .body(metrics_output)?)
}

Grafana Dashboard Configuration

{
  "dashboard": {
    "id": null,
    "title": "SpinKube WASM Applications",
    "tags": ["spinkube", "wasm", "kubernetes"],
    "style": "dark",
    "timezone": "UTC",
    "panels": [
      {
        "id": 1,
        "title": "Request Rate",
        "type": "stat",
        "targets": [
          {
            "expr": "sum(rate(http_requests_total{job=\"user-api\"}[5m]))",
            "legendFormat": "Requests/sec"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "thresholds"
            },
            "thresholds": {
              "steps": [
                {"color": "green", "value": null},
                {"color": "yellow", "value": 100},
                {"color": "red", "value": 1000}
              ]
            }
          }
        }
      },
      {
        "id": 2,
        "title": "Response Time",
        "type": "graph",
        "targets": [
          {
            "expr": "histogram_quantile(0.50, sum(rate(http_request_duration_seconds_bucket{job=\"user-api\"}[5m])) by (le))",
            "legendFormat": "50th percentile"
          },
          {
            "expr": "histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{job=\"user-api\"}[5m])) by (le))",
            "legendFormat": "95th percentile"
          },
          {
            "expr": "histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket{job=\"user-api\"}[5m])) by (le))",
            "legendFormat": "99th percentile"
          }
        ],
        "yAxes": [
          {
            "unit": "s",
            "min": 0
          }
        ]
      },
      {
        "id": 3,
        "title": "Error Rate",
        "type": "stat",
        "targets": [
          {
            "expr": "sum(rate(http_requests_total{job=\"user-api\", status=~\"5..\"}[5m])) / sum(rate(http_requests_total{job=\"user-api\"}[5m])) * 100",
            "legendFormat": "Error Rate %"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "unit": "percent",
            "thresholds": {
              "steps": [
                {"color": "green", "value": null},
                {"color": "yellow", "value": 1},
                {"color": "red", "value": 5}
              ]
            }
          }
        }
      },
      {
        "id": 4,
        "title": "Pod Scaling",
        "type": "graph",
        "targets": [
          {
            "expr": "kube_deployment_status_replicas{deployment=\"user-management-api\"}",
            "legendFormat": "Current Replicas"
          },
          {
            "expr": "kube_deployment_spec_replicas{deployment=\"user-management-api\"}",
            "legendFormat": "Desired Replicas"
          }
        ]
      },
      {
        "id": 5,
        "title": "Resource Usage",
        "type": "graph",
        "targets": [
          {
            "expr": "sum(container_cpu_usage_seconds_total{pod=~\"user-management-api-.*\"})",
            "legendFormat": "CPU Usage"
          },
          {
            "expr": "sum(container_memory_usage_bytes{pod=~\"user-management-api-.*\"})/1024/1024",
            "legendFormat": "Memory Usage (MB)"
          }
        ]
      },
      {
        "id": 6,
        "title": "Cache Performance",
        "type": "graph",
        "targets": [
          {
            "expr": "sum(rate(cache_hits_total{job=\"user-api\"}[5m]))",
            "legendFormat": "Cache Hits/sec"
          },
          {
            "expr": "sum(rate(cache_misses_total{job=\"user-api\"}[5m]))",
            "legendFormat": "Cache Misses/sec"
          }
        ]
      }
    ],
    "time": {
      "from": "now-1h",
      "to": "now"
    },
    "refresh": "30s"
  }
}

Distributed Tracing with Jaeger

// src/tracing.rs
use opentelemetry::{global, KeyValue, trace::TraceResult};
use opentelemetry_jaeger::JaegerPropagator;
use tracing::{info, span, Level};
use tracing_opentelemetry::OpenTelemetrySpanExt;
use tracing_subscriber::{layer::SubscriberExt, Registry};

pub fn init_tracing(service_name: &str) -> TraceResult<()> {
    global::set_text_map_propagator(JaegerPropagator::new());
    
    let tracer = opentelemetry_jaeger::new_agent_pipeline()
        .with_service_name(service_name)
        .with_endpoint("http://jaeger-collector.monitoring.svc.cluster.local:14268/api/traces")
        .install_batch(opentelemetry::runtime::Tokio)?;

    let telemetry = tracing_opentelemetry::layer().with_tracer(tracer);
    
    let subscriber = Registry::default()
        .with(tracing_subscriber::fmt::layer())
        .with(telemetry);
    
    tracing::subscriber::set_global_default(subscriber)
        .expect("Failed to set subscriber");
    
    Ok(())
}

// Modified request handler with tracing
#[http_component]
fn handle_request(req: Request) -> Result<Response> {
    let span = span!(Level::INFO, "http_request", 
        method = %req.method(),
        path = %req.uri().path(),
        user_agent = %req.headers().get("user-agent").unwrap_or(&HeaderValue::from_static("")).to_str().unwrap_or("")
    );
    
    let _enter = span.enter();
    
    // Extract trace context from headers
    let parent_context = global::get_text_map_propagator(|propagator| {
        propagator.extract(&HeaderExtractor::new(req.headers()))
    });
    span.set_parent(parent_context);
    
    let _timer = start_request_timer();
    
    let method = req.method();
    let path = req.uri().path();
    
    info!("Processing request");
    
    let result = match (method, path) {
        (Method::GET, "/users") => {
            let db_span = span!(Level::INFO, "database_query", query = "SELECT * FROM users");
            let _db_enter = db_span.enter();
            get_users()
        },
        (Method::POST, "/users") => {
            let validation_span = span!(Level::INFO, "input_validation");
            let _validation_enter = validation_span.enter();
            create_user(req)
        },
        _ => Ok(Response::builder().status(404).body("Not Found")?),
    };
    
    match &result {
        Ok(response) => {
            record_request(path, method.as_str(), response.status().as_u16());
            info!(status = %response.status(), "Request completed successfully");
        },
        Err(e) => {
            record_request(path, method.as_str(), 500);
            tracing::error!(error = %e, "Request failed");
        }
    }
    
    result
}

struct HeaderExtractor<'a> {
    headers: &'a HeaderMap,
}

impl<'a> HeaderExtractor<'a> {
    fn new(headers: &'a HeaderMap) -> Self {
        Self { headers }
    }
}

impl<'a> opentelemetry::propagation::Extractor for HeaderExtractor<'a> {
    fn get(&self, key: &str) -> Option<&str> {
        self.headers.get(key)?.to_str().ok()
    }

    fn keys(&self) -> Vec<&str> {
        self.headers.keys().map(|k| k.as_str()).collect()
    }
}

CI/CD Pipeline Integration

GitHub Actions Pipeline

# .github/workflows/spinkube-deploy.yml
name: SpinKube Deployment Pipeline

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}/user-api

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    
    - name: Setup Rust
      uses: actions-rs/toolchain@v1
      with:
        toolchain: stable
        target: wasm32-wasi
        override: true
        components: rustfmt, clippy
    
    - name: Cache cargo registry
      uses: actions/cache@v3
      with:
        path: ~/.cargo/registry
        key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
    
    - name: Cache cargo index
      uses: actions/cache@v3
      with:
        path: ~/.cargo/git
        key: ${{ runner.os }}-cargo-index-${{ hashFiles('**/Cargo.lock') }}
    
    - name: Install Spin CLI
      run: |
        curl -fsSL https://developer.fermyon.com/downloads/install.sh | bash
        sudo mv spin /usr/local/bin/
    
    - name: Format check
      run: cargo fmt -- --check
    
    - name: Lint check
      run: cargo clippy -- -D warnings
    
    - name: Unit tests
      run: cargo test
    
    - name: Build WASM module
      run: |
        cargo build --target wasm32-wasi --release
        spin build
    
    - name: Integration tests
      run: |
        spin up --detach &
        sleep 10
        
        # API tests
        curl -f http://localhost:3000/health
        curl -f http://localhost:3000/users
        
        # Load test
        ab -n 1000 -c 10 http://localhost:3000/users
        
        spin down

  security-scan:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    
    - name: Security audit
      run: |
        cargo install cargo-audit
        cargo audit
    
    - name: WASM security scan
      run: |
        # Install custom WASM scanner
        wget https://github.com/example/wasm-security-scanner/releases/latest/download/wasm-scanner
        chmod +x wasm-scanner
        ./wasm-scanner target/wasm32-wasi/release/*.wasm

  build-and-push:
    needs: [test, security-scan]
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    outputs:
      image-digest: ${{ steps.build.outputs.digest }}
    steps:
    - uses: actions/checkout@v4
    
    - name: Setup Rust
      uses: actions-rs/toolchain@v1
      with:
        toolchain: stable
        target: wasm32-wasi
    
    - name: Install Spin CLI
      run: |
        curl -fsSL https://developer.fermyon.com/downloads/install.sh | bash
        sudo mv spin /usr/local/bin/
    
    - name: Log in to Container Registry
      uses: docker/login-action@v3
      with:
        registry: ${{ env.REGISTRY }}
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}
    
    - name: Build and package
      id: build
      run: |
        # Build WASM module
        cargo build --target wasm32-wasi --release
        spin build
        
        # Create OCI artifact
        IMAGE_TAG="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}"
        
        # Package as OCI artifact using spin registry push
        spin registry push "$IMAGE_TAG"
        
        # Output image digest for later use
        echo "digest=$(spin registry info $IMAGE_TAG --format json | jq -r '.digest')" >> $GITHUB_OUTPUT

  deploy-staging:
    needs: build-and-push
    if: github.ref == 'refs/heads/develop'
    runs-on: ubuntu-latest
    environment: staging
    steps:
    - uses: actions/checkout@v4
    
    - name: Setup kubectl
      uses: azure/setup-kubectl@v3
      
    - name: Configure kubeconfig
      run: |
        echo "${{ secrets.KUBE_CONFIG_STAGING }}" | base64 -d > kubeconfig
        export KUBECONFIG=kubeconfig
    
    - name: Deploy to staging
      run: |
        # Update image in SpinApp manifest
        IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}"
        sed -i "s|image:.*|image: \"$IMAGE\"|" k8s/staging/spinapp.yaml
        
        kubectl apply -f k8s/staging/
        kubectl rollout status spinapp/user-management-api -n staging --timeout=300s
    
    - name: Staging smoke tests
      run: |
        # Wait for service to be ready
        kubectl wait --for=condition=ready pod -l app=user-management-api -n staging --timeout=300s
        
        # Get service endpoint
        ENDPOINT=$(kubectl get ingress user-api-ingress -n staging -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
        
        # Run smoke tests
        curl -f "https://$ENDPOINT/health"
        curl -f "https://$ENDPOINT/users"

  deploy-production:
    needs: [build-and-push, deploy-staging]
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production
    steps:
    - uses: actions/checkout@v4
    
    - name: Setup kubectl
      uses: azure/setup-kubectl@v3
      
    - name: Configure kubeconfig
      run: |
        echo "${{ secrets.KUBE_CONFIG_PRODUCTION }}" | base64 -d > kubeconfig
        export KUBECONFIG=kubeconfig
    
    - name: Blue-Green Deployment
      run: |
        IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}"
        
        # Create green deployment
        sed -i "s|image:.*|image: \"$IMAGE\"|" k8s/production/spinapp-green.yaml
        kubectl apply -f k8s/production/spinapp-green.yaml
        
        # Wait for green deployment
        kubectl rollout status spinapp/user-management-api-green -n production --timeout=600s
        
        # Run health checks on green deployment
        GREEN_ENDPOINT=$(kubectl get service user-api-green-service -n production -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
        
        for i in {1..10}; do
          if curl -f "https://$GREEN_ENDPOINT/health"; then
            echo "Health check $i passed"
          else
            echo "Health check $i failed"
            exit 1
          fi
          sleep 10
        done
        
        # Switch traffic to green
        kubectl patch service user-api-service -n production -p '{"spec":{"selector":{"version":"green"}}}'
        
        # Wait and monitor
        sleep 300
        
        # Clean up blue deployment
        kubectl delete spinapp user-management-api-blue -n production || true

  performance-test:
    needs: deploy-production
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    
    - name: Performance testing
      run: |
        # Install k6
        wget https://github.com/grafana/k6/releases/latest/download/k6-linux-amd64.tar.gz
        tar -xzf k6-linux-amd64.tar.gz
        sudo mv k6*/k6 /usr/local/bin/
        
        # Get production endpoint
        ENDPOINT="https://api.example.com"
        
        # Run performance tests
        k6 run --env ENDPOINT="$ENDPOINT" tests/performance/load-test.js
    
    - name: Performance regression check
      run: |
        # Compare against baseline metrics
        python scripts/performance-regression-check.py \
          --current-results k6-results.json \
          --baseline-results baseline-performance.json \
          --threshold 0.1

  notify:
    needs: [deploy-production, performance-test]
    if: always()
    runs-on: ubuntu-latest
    steps:
    - name: Slack notification
      uses: 8398a7/action-slack@v3
      with:
        status: ${{ job.status }}
        channel: '#deployments'
        webhook_url: ${{ secrets.SLACK_WEBHOOK }}
        fields: repo,message,commit,author,action,eventName,ref,workflow

ArgoCD GitOps Configuration

# argocd/application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: user-management-api
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: spinkube-apps
  source:
    repoURL: https://github.com/example/user-api-config
    targetRevision: HEAD
    path: k8s/overlays/production
    kustomize:
      images:
      - ghcr.io/example/user-api:v1.0.0
  destination:  
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
    syncOptions:
    - CreateNamespace=true
    - PrunePropagationPolicy=foreground
    - PruneLast=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m
  revisionHistoryLimit: 10

---
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: spinkube-apps
  namespace: argocd
spec:
  description: SpinKube WASM Applications
  sourceRepos:
  - 'https://github.com/example/*'
  destinations:
  - namespace: 'production'
    server: https://kubernetes.default.svc
  - namespace: 'staging'
    server: https://kubernetes.default.svc
  clusterResourceWhitelist:
  - group: ''
    kind: Namespace
  - group: 'core.spinkube.dev'
    kind: SpinApp
  namespaceResourceWhitelist:
  - group: ''
    kind: Service
  - group: ''
    kind: ConfigMap
  - group: ''
    kind: Secret
  - group: 'apps'
    kind: Deployment
  - group: 'networking.k8s.io'
    kind: Ingress
  - group: 'autoscaling'
    kind: HorizontalPodAutoscaler
  roles:
  - name: readonly
    policies:
    - p, proj:spinkube-apps:readonly, applications, get, spinkube-apps/*, allow
    groups:
    - example:developers
  - name: admin
    policies:
    - p, proj:spinkube-apps:admin, applications, *, spinkube-apps/*, allow
    groups:
    - example:platform-team

Advanced Configuration and Tuning

Performance Optimization

# performance-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: spin-performance-config
  namespace: production
data:
  wasmtime.toml: |
    [cranelift]
    # Enable optimizations
    opt_level = "speed"
    enable_nan_canonicalization = false
    enable_pinned_reg = true
    enable_simd = true
    
    [memory]
    # Memory configuration  
    static_memory_maximum_size = "2GiB"
    static_memory_guard_size = "64KiB"
    dynamic_memory_guard_size = "64KiB"
    
    [runtime]
    # Runtime settings
    async_stack_size = "2MiB"
    wasm_bulk_memory = true
    wasm_reference_types = true
    wasm_multi_value = true
    wasm_simd = true
    
    [cache]
    # Compilation caching
    enabled = true
    directory = "/tmp/wasmtime-cache"
    
  spin.toml: |
    [runtime]
    # Spin-specific settings
    max_concurrent_requests = 1000
    request_timeout = "30s"
    
    [database]
    # Database connection pooling
    max_connections = 100
    min_connections = 10
    connection_timeout = "5s"
    idle_timeout = "300s"
    
    [cache]
    # Redis caching
    max_connections = 50
    default_ttl = "300s"
    
    [logging]
    level = "info"
    format = "json"

---
apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: user-management-api
  namespace: production
spec:
  image: "registry.example.com/user-api:v1.0.0"
  replicas: 10
  
  # Performance-tuned resource allocation
  resources:
    limits:
      cpu: "2000m"
      memory: "512Mi"
    requests:
      cpu: "500m"
      memory: "256Mi"
  
  # Node affinity for performance
  nodeSelector:
    node-type: "compute-optimized"
  
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: app
              operator: In
              values: ["user-management-api"]
          topologyKey: kubernetes.io/hostname
  
  # Volume mounts for performance data
  volumeMounts:
  - name: performance-config
    mountPath: /etc/spin
  - name: cache-volume
    mountPath: /tmp/wasmtime-cache
  
  volumes:
  - name: performance-config
    configMap:
      name: spin-performance-config
  - name: cache-volume
    emptyDir:
      sizeLimit: "1Gi"
  
  # Startup optimization
  startupProbe:
    httpGet:
      path: /health
      port: 3000
    initialDelaySeconds: 5
    periodSeconds: 2
    failureThreshold: 30

Resource Quotas and Limits

# resource-management.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: spinkube-quota
  namespace: production
spec:
  hard:
    requests.cpu: "50"
    requests.memory: "100Gi"
    limits.cpu: "100" 
    limits.memory: "200Gi"
    pods: "100"
    persistentvolumeclaims: "20"
    services: "50"
    secrets: "100"
    configmaps: "100"
    count/spinapps.core.spinkube.dev: "50"

---
apiVersion: v1
kind: LimitRange
metadata:
  name: spinkube-limits
  namespace: production
spec:
  limits:
  - type: Container
    default:
      cpu: "500m"
      memory: "256Mi"
    defaultRequest:
      cpu: "100m"
      memory: "128Mi"
    max:
      cpu: "4000m"
      memory: "2Gi"
    min:
      cpu: "50m"
      memory: "64Mi"
  - type: Pod
    max:
      cpu: "8000m"
      memory: "4Gi"

---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: user-api-pdb
  namespace: production
spec:
  minAvailable: 3
  selector:
    matchLabels:
      app: user-management-api

Production Operations

Backup and Disaster Recovery

# backup-strategy.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: spinkube-backup
  namespace: production
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  successfulJobsHistoryLimit: 7
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: velero/velero:v1.12.0
            command:
            - /bin/sh
            - -c
            - |
              # Backup SpinApps
              velero backup create spinkube-backup-$(date +%Y%m%d-%H%M%S) \
                --include-namespaces production \
                --include-resources spinapps,configmaps,secrets,services,ingresses \
                --storage-location aws-backup \
                --ttl 720h
              
              # Backup application data
              kubectl exec -n production deployment/database -- \
                pg_dump -U postgres userdb > /backups/userdb-$(date +%Y%m%d).sql
              
              # Upload to S3
              aws s3 cp /backups/ s3://spinkube-backups/$(date +%Y%m%d)/ --recursive
            env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: aws-credentials
                  key: access-key-id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: aws-credentials
                  key: secret-access-key
            volumeMounts:
            - name: backup-storage
              mountPath: /backups
          volumes:
          - name: backup-storage
            emptyDir:
              sizeLimit: "10Gi"
          restartPolicy: OnFailure

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: disaster-recovery-runbook
  namespace: production
data:
  runbook.md: |
    # SpinKube Disaster Recovery Runbook
    
    ## Scenario 1: Complete Cluster Failure
    
    1. **Provision new cluster**:
       ```bash
       # Create new cluster with SpinKube
       ./scripts/provision-cluster.sh --environment production

2. Restore from backup:

   # Install Velero
   velero install --provider aws --bucket spinkube-backups
   
   # List available backups
   velero backup get
   
   # Restore latest backup
   velero restore create --from-backup spinkube-backup-20240124-020000
   

3. Verify restoration:

   kubectl get spinapps -n production
   kubectl get pods -n production
   curl -f https://api.example.com/health
   

Scenario 2: Application Corruption

1. Rollback to previous version:

   kubectl rollout undo spinapp/user-management-api -n production
   

2. If rollback fails, restore from backup:

   velero restore create --from-backup spinkube-backup-latest \
     --include-resources spinapps \
     --namespace-mappings production:production-restore
   

Scenario 3: Data Loss

1. Stop application traffic:

   kubectl scale spinapp user-management-api --replicas=0 -n production
   

2. Restore database:

   # Download latest backup
   aws s3 cp s3://spinkube-backups/latest/userdb.sql /tmp/
   
   # Restore database
   kubectl exec -n production deployment/database -- \
     psql -U postgres -d userdb < /tmp/userdb.sql
   

3. Resume application:

   kubectl scale spinapp user-management-api --replicas=10 -n production
   

### Monitoring and Alerting Rules

```yaml
# alerting-rules.yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: spinkube-alerts
  namespace: production
spec:
  groups:
  - name: spinkube.rules
    rules:
    # High error rate
    - alert: SpinAppHighErrorRate
      expr: |
        (
          sum(rate(http_requests_total{job="user-api", status=~"5.."}[5m])) /
          sum(rate(http_requests_total{job="user-api"}[5m]))
        ) * 100 > 5
      for: 2m
      labels:
        severity: critical
        service: user-api
      annotations:
        summary: "High error rate detected"
        description: "Error rate is {{ $value }}% for the last 5 minutes"
    
    # High response time
    - alert: SpinAppHighLatency
      expr: |
        histogram_quantile(0.95, 
          sum(rate(http_request_duration_seconds_bucket{job="user-api"}[5m])) by (le)
        ) > 1.0
      for: 5m
      labels:
        severity: warning
        service: user-api
      annotations:
        summary: "High response time detected"
        description: "95th percentile latency is {{ $value }}s"
    
    # Pod crash looping
    - alert: SpinAppCrashLooping
      expr: |
        rate(kube_pod_container_status_restarts_total{container="user-api"}[15m]) > 0
      for: 5m
      labels:
        severity: critical
        service: user-api
      annotations:
        summary: "Pod is crash looping"
        description: "Pod {{ $labels.pod }} is restarting frequently"
    
    # Low replica count
    - alert: SpinAppLowReplicas
      expr: |
        kube_deployment_status_replicas{deployment="user-management-api"} < 3
      for: 10m
      labels:
        severity: warning
        service: user-api
      annotations:
        summary: "Low replica count"
        description: "Only {{ $value }} replicas are running"
    
    # High memory usage
    - alert: SpinAppHighMemoryUsage
      expr: |
        (
          sum(container_memory_usage_bytes{pod=~"user-management-api-.*"}) /
          sum(container_spec_memory_limit_bytes{pod=~"user-management-api-.*"})
        ) * 100 > 80
      for: 10m
      labels:
        severity: warning
        service: user-api
      annotations:
        summary: "High memory usage"
        description: "Memory usage is {{ $value }}% of the limit"

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: alertmanager-config
  namespace: monitoring
data:
  alertmanager.yml: |
    global:
      smtp_smarthost: 'localhost:587'
      smtp_from: '[email protected]'
    
    route:
      group_by: ['alertname', 'service']
      group_wait: 10s
      group_interval: 10s
      repeat_interval: 1h
      receiver: 'web.hook'
      routes:
      - match:
          severity: critical
        receiver: 'critical-alerts'
      - match:
          service: user-api
        receiver: 'api-team'
    
    receivers:
    - name: 'web.hook'
      webhook_configs:
      - url: 'http://webhook.example.com/alerts'
    
    - name: 'critical-alerts'
      email_configs:
      - to: '[email protected]'
        subject: 'CRITICAL: {{ .GroupLabels.alertname }}'
        body: |
          {{ range .Alerts }}
          Alert: {{ .Annotations.summary }}
          Description: {{ .Annotations.description }}
          {{ end }}
      slack_configs:
      - api_url: 'https://hooks.slack.com/services/...'
        channel: '#critical-alerts'
        title: 'CRITICAL Alert'
        text: '{{ range .Alerts }}{{ .Annotations.summary }}{{ end }}'
    
    - name: 'api-team'
      email_configs:
      - to: '[email protected]'
        subject: 'API Alert: {{ .GroupLabels.alertname }}'
      slack_configs:
      - api_url: 'https://hooks.slack.com/services/...'
        channel: '#api-alerts'

Migration Strategies

Container to SpinKube Migration

#!/bin/bash
# migrate-to-spinkube.sh

set -e

NAMESPACE=${1:-default}
APP_NAME=${2:-my-app}
CONTAINER_IMAGE=${3}

echo "Starting migration from container to SpinKube for $APP_NAME in $NAMESPACE"

# Step 1: Analyze existing deployment
echo "Analyzing existing deployment..."
kubectl get deployment $APP_NAME -n $NAMESPACE -o yaml > original-deployment.yaml

# Extract key information
REPLICAS=$(kubectl get deployment $APP_NAME -n $NAMESPACE -o jsonpath='{.spec.replicas}')
CPU_REQUEST=$(kubectl get deployment $APP_NAME -n $NAMESPACE -o jsonpath='{.spec.template.spec.containers[0].resources.requests.cpu}')
MEMORY_REQUEST=$(kubectl get deployment $APP_NAME -n $NAMESPACE -o jsonpath='{.spec.template.spec.containers[0].resources.requests.memory}')

echo "Current configuration:"
echo "  Replicas: $REPLICAS"
echo "  CPU Request: $CPU_REQUEST"
echo "  Memory Request: $MEMORY_REQUEST"

# Step 2: Create SpinApp manifest
echo "Creating SpinApp manifest..."
cat > spinapp-migration.yaml << EOF
apiVersion: core.spinkube.dev/v1alpha1
kind: SpinApp
metadata:
  name: $APP_NAME
  namespace: $NAMESPACE
  labels:
    migrated-from: container
    migration-date: $(date +%Y%m%d)
spec:
  image: $CONTAINER_IMAGE
  replicas: $REPLICAS
  
  resources:
    requests:
      cpu: $CPU_REQUEST
      memory: $MEMORY_REQUEST
    limits:
      cpu: $(echo $CPU_REQUEST | sed 's/m/*2m/' | bc)
      memory: $(echo $MEMORY_REQUEST | sed 's/Mi/*2Mi/' | bc)
  
  # Copy environment variables
  variables:
EOF

# Extract and add environment variables
kubectl get deployment $APP_NAME -n $NAMESPACE -o jsonpath='{.spec.template.spec.containers[0].env[*]}' \
  | jq -r '.[] | "  - name: " + .name + "\n    value: \"" + .value + "\""' >> spinapp-migration.yaml

# Step 3: Perform blue-green migration
echo "Starting blue-green migration..."

# Scale down original deployment to 1 replica
kubectl scale deployment $APP_NAME --replicas=1 -n $NAMESPACE

# Deploy SpinApp (green)
kubectl apply -f spinapp-migration.yaml

# Wait for SpinApp to be ready
echo "Waiting for SpinApp to be ready..."
kubectl wait --for=condition=ready spinapp/$APP_NAME -n $NAMESPACE --timeout=300s

# Step 4: Traffic switch preparation
echo "Preparing traffic switch..."

# Get service configuration
kubectl get service $APP_NAME -n $NAMESPACE -o yaml > original-service.yaml

# Update service selector to point to SpinApp
kubectl patch service $APP_NAME -n $NAMESPACE -p '{"spec":{"selector":{"app":"'$APP_NAME'"}}}'

# Step 5: Validation tests
echo "Running validation tests..."
SERVICE_IP=$(kubectl get service $APP_NAME -n $NAMESPACE -o jsonpath='{.spec.clusterIP}')

# Health check
if curl -f "http://$SERVICE_IP/health" > /dev/null 2>&1; then
    echo "✓ Health check passed"
else
    echo "✗ Health check failed"
    exit 1
fi

# Load test
echo "Running load test..."
kubectl run load-test --rm -i --tty --image=alpine/curl -- \
  sh -c "for i in \$(seq 1 100); do curl -s http://$SERVICE_IP/ > /dev/null; done"

echo "✓ Load test completed"

# Step 6: Monitor and finalize
echo "Monitoring new deployment for 5 minutes..."
sleep 300

# Check for errors
ERROR_COUNT=$(kubectl logs -l app=$APP_NAME -n $NAMESPACE --since=5m | grep -i error | wc -l)
if [ "$ERROR_COUNT" -gt 0 ]; then
    echo "⚠ Found $ERROR_COUNT errors in logs"
    echo "Rolling back..."
    kubectl apply -f original-deployment.yaml
    kubectl apply -f original-service.yaml
    exit 1
fi

# Step 7: Cleanup
echo "Migration successful! Cleaning up..."
kubectl delete deployment $APP_NAME -n $NAMESPACE

# Create migration report
cat > migration-report.md << EOF
# Migration Report: $APP_NAME

## Migration Details
- **Date**: $(date)
- **Source**: Container Deployment
- **Target**: SpinKube SpinApp
- **Namespace**: $NAMESPACE

## Configuration Changes
- **Runtime**: Container → WASM
- **Image**: $CONTAINER_IMAGE
- **Replicas**: $REPLICAS (unchanged)
- **Resources**: 
  - CPU: $CPU_REQUEST (request) 
  - Memory: $MEMORY_REQUEST (request)

## Performance Impact
- **Startup Time**: Improved (estimated 10x faster)
- **Memory Usage**: Reduced (estimated 50% less)
- **CPU Overhead**: Reduced (estimated 20% less)

## Post-Migration Tasks
- [ ] Update monitoring dashboards
- [ ] Update alerting rules
- [ ] Update documentation
- [ ] Schedule performance review

## Rollback Procedure
If rollback is needed:
\`\`\`bash
kubectl apply -f original-deployment.yaml
kubectl apply -f original-service.yaml
kubectl delete spinapp $APP_NAME -n $NAMESPACE
\`\`\`
EOF

echo "Migration completed successfully!"
echo "Report saved to: migration-report.md"

Conclusion

SpinKube and Fermyon represent the maturation of WebAssembly as a first-class container orchestration platform. By providing Kubernetes-native tooling for WASM applications, these platforms enable organizations to leverage the performance, security, and efficiency benefits of WebAssembly while maintaining familiar operational patterns.

Key Benefits Achieved

• ✅ Kubernetes Integration: Native CRDs and operators for seamless management
• ✅ Developer Experience: Familiar tooling with kubectl and standard manifests
• ✅ Production Ready: Comprehensive monitoring, scaling, and operational features
• ✅ Multi-Cloud Support: Deploy across any Kubernetes-compatible environment
• ✅ Security: Built-in sandboxing and capability-based security model
• ✅ Performance: Sub-millisecond startup times and efficient resource usage

Platform Comparison

Feature	SpinKube	Fermyon Cloud	Traditional K8s
WASM Native	✅	✅	❌
Cold Start	<1ms	<1ms	1-5s
Memory Overhead	<10MB	<10MB	50MB+
Kubernetes Integration	✅	Limited	✅
Multi-tenancy	✅	✅	Manual
Edge Deployment	✅	✅	Complex

Best Practices Recap

1. Start Small: Begin with stateless HTTP services
2. Monitor Everything: Implement comprehensive observability from day one
3. Plan for Scale: Design with auto-scaling and resource limits
4. Security First: Leverage WASM's capability-based security model
5. GitOps Integration: Use ArgoCD or Flux for deployment automation
6. Test Thoroughly: Include performance and security testing in CI/CD

Future Roadmap

The SpinKube and Fermyon ecosystems continue evolving:

• Enhanced component model support for complex applications
• Multi-language runtime improvements and better tooling
• Advanced networking features for service mesh integration
• Database integrations with native WASM drivers
• AI/ML workload optimization for edge deployment

Next Steps

Ready to explore serverless WASM functions? Our next article covers building and deploying serverless WebAssembly functions in Kubernetes environments!

Resources

• SpinKube Documentation
• Fermyon Platform Guide
• Spin Framework Tutorials
• WASM Component Model

The future of container orchestration is here, and it's powered by WebAssembly! 🚀