# CAIN-42 Sovereign Extension Containment: Deployment & Rollback Runbook

**Document Reference:** `CAIN42-OPS-SOVEREIGN-EXT-v1`  
**Classification:** Production Operations Runbook  
**Target Services:** `cain-plugin-sentinel.service`, `cain-sovereign-sync.service`, `cain-containment-pod.yaml`  
**Status:** VALIDATED  

---

## 1. Reproducible Build Instructions

### 1.1 Prerequisites
Ensure the host environment satisfies the minimum requirements:
```bash
# Check host tools
which bwrap clang bpftool python3 wasmedge

# Verify cgroup v2 and BPF filesystem mounts
mount | grep -E "cgroup2|bpf"
# Expected:
# cgroup2 on /sys/fs/cgroup type cgroup2
# bpf on /sys/fs/bpf type bpf
```

### 1.2 Compile Kernel eBPF Program
Compile `cain_plugin_sentinel.bpf.c` targeting BPF:
```bash
clang -target bpf -O2 -g \
  -I/usr/include \
  -I/usr/include/x86_64-linux-gnu \
  -c cain/bpf/cain_plugin_sentinel.bpf.c \
  -o cain/bpf/cain_plugin_sentinel.bpf.o

# Or using the built-in loader:
python3 -m cain.cain_plugin_sentinel --build
```

---

## 2. Production Deployment Steps

### 2.1 Storage & Permission Provisioning
Create required system directories:
```bash
mkdir -p /var/lib/cain_sovereign
mkdir -p /sys/fs/cgroup/cain_sovereign_ext
chmod 700 /var/lib/cain_sovereign
```

### 2.2 Systemd Service Installation
Copy unit files into `/etc/systemd/system/`:
```bash
cp deploy/systemd/cain-plugin-sentinel.service /etc/systemd/system/
cp deploy/systemd/cain-sovereign-sync.service /etc/systemd/system/
systemctl daemon-reload
```

### 2.3 Starting and Enabling Services
Start the kernel sentinel first, then start the sovereign sync daemon:
```bash
# Start Sentinel
systemctl start cain-plugin-sentinel.service
systemctl enable cain-plugin-sentinel.service

# Start Sync Daemon
systemctl start cain-sovereign-sync.service
systemctl enable cain-sovereign-sync.service
```

### 2.4 Verifying Service Health
```bash
# Check status of sentinel
python3 -m cain.cain_plugin_sentinel --status
python3 -m cain.cain_plugin_sentinel --metrics

# Check status of sync daemon
python3 -m cain.sovereign_sync_daemon --status
python3 -m cain.sovereign_sync_daemon --verify-audit
```

---

## 3. Kubernetes / Container Deployment

For sovereign workloads operating in Kubernetes environments:
```bash
# Apply sovereign namespace and containment pod
kubectl apply -f deploy/k8s/cain-containment-pod.yaml

# Verify pod status and network policy isolation
kubectl get pods -n cain-system -l cain.security/containment-boundary=true
kubectl describe networkpolicy cain-containment-egress-lock -n cain-system
```

---

## 4. Rollback & Emergency Teardown Runbook

If a critical fault or host instability occurs, execute the following emergency rollback sequence:

### 4.1 Step 1: Stop Services
```bash
systemctl stop cain-sovereign-sync.service
systemctl stop cain-plugin-sentinel.service
```

### 4.2 Step 2: Detach and Unpin Kernel eBPF Sentinel
Unload the sentinel and remove all pinned BPF programs and maps:
```bash
python3 -m cain.cain_plugin_sentinel --unload

# Verify BPF pins are cleared
ls -la /sys/fs/bpf/cain_sentinel_* 2>/dev/null || echo "Sentinel BPF pins cleanly removed"
```

### 4.3 Step 3: Cleanup Cgroups and Process Trees
Kill any remaining contained cgroups:
```bash
for cg in /sys/fs/cgroup/cain_sovereign_ext/*; do
    if [ -d "$cg" ]; then
        echo "1" > "$cg/cgroup.kill" 2>/dev/null || true
        rmdir "$cg" 2>/dev/null || true
    fi
done
```

### 4.4 Step 4: Revert Policy (if needed)
Restore previous policy or genesis policy:
```bash
cp /var/lib/cain_sovereign/current_policy.json.bak /var/lib/cain_sovereign/current_policy.json
systemctl restart cain-sovereign-sync.service
```
