# Customer Install Guide

**Audience:** Engineering, On-site Install
**Read time:** ~9 minutes
**Use this when:** SOW is signed and the customer's IT-OT team is preparing for the appliance arrival.

---

## What the customer downloads on purchase

At signature, the customer receives a per-tenant container bundle:

```
customer-image-{TENANT_ID}/
├── docker-compose.portal.yml      # Subset stack (Postgres + Civops app)
├── portal.env                      # Baked: TENANT_ID, BRAND_SLUG, ENGAGEMENT_ID, JOURNEY_STAGE
├── cloud-init/user-data            # Boot config for cloud-init enabled hosts
└── README.txt                      # Boot instructions
```

This is the **portal tier** — light, customer-self-host, cloud-AI for any inference. Lets the customer start exploring immediately while the production appliance ships.

For the production appliance:

```
appliance-image-{TENANT_ID}.qcow2   # Full appliance (Postgres + Timescale + Ollama + heavy-worker)
sealed-keys.bin                      # Encrypted vault keys (HSM-issued)
SBOM.spdx.json                       # Reproducible-build SBOM
checksum.sha256                      # Image integrity hash
```

---

## Pre-arrival checklist

The customer's IT-OT team confirms before the appliance lands:

| | Item | Owner |
|---|---|---|
| □ | DMZ at Purdue Level 3.5 with reachable L4 management interface | Customer Network |
| □ | One static IPv4 + DNS A-record for `civops.<customer-domain>` | Customer Network |
| □ | TLS cert (customer-provided OR allow Civops Let's Encrypt) | Customer Compliance |
| □ | Power + cooling spec confirmed for appliance form factor | Customer Facilities |
| □ | OT VLAN reachable on the second NIC, eth0/eth1 strictly separate | Customer Network |
| □ | Outbound to api.anthropic.com on L4 (pre-cutover only) | Customer Network |
| □ | Source-system credentials prepared in customer's vault | Customer App Owners |
| □ | Sponsor + Civops engineer + customer DBA on the kickoff call | Both |

---

## Day-of bootstrap

**Step 1 — Power up.** Appliance boots from `qcow2` → cloud-init reads `user-data` → systemd-units start `civops-stack.service` → Postgres + Ollama + Civops app come up on the `civops` internal network.

**Step 2 — First-boot wizard.** On-site engineer browses `https://civops.<customer-domain>/platform/setup`. The 10-step wizard walks:

1. Tenant confirmation
2. Network discovery (sweeps the OT VLAN — discovers PLCs, scanners, printers; classifies; binds)
3. PLC interrogation (legacy register sweep, classification, close-out)
4. Source-system connector configuration (per-system replacement ladder starts at passive_read)
5. Sheet Builder kickoff (first operator screens scoped)
6. Label setup (printer + scanner configuration)
7. Workforce roster import
8. User invitation (admin onboarding)
9. Compliance scan (DMZ posture validation, OT VLAN review, ISA naming check)
10. Cutover readiness check

**Step 3 — DMZ compliance scan.** `npm run appliance:compliance` runs automatically. Critical findings block the next step. Common criticals:

- OT VLAN bridged to IT VLAN (must separate)
- Outbound allowed to non-allowlisted hosts (must lockdown)
- Audit chain not initialized (must run init)
- Vault keys not sealed (must seal at HSM)

**Step 4 — Bridge from assessment.** If an assessment engagement preceded the install, click **"Bridge from assessment"** in step 4. The wizard pre-fills equipment, source systems, gap recommendations from the engagement record. **Operators don't re-enter what was already captured.**

---

## Cutover state machine

Civops doesn't ship in production-ready mode. Cutover is a disciplined sequence:

| State | What's happening |
|---|---|
| **offline** | Default at first boot. No agent calls run yet. |
| **preloading** | `npm run appliance:preload` pulls every vetted Ollama model and warms it (16-token warmup so first live request isn't a cold start). |
| **warming** | All models pulled. Heavy-worker is up. Inference is mid-vetting. |
| **vetting** | `npm run llm:vet` runs every wired agent against fixtures. Schema fidelity is required (Zod gate). |
| **ready** | All vets pass. Purpose-coverage complete. The "Go Live" button is now enabled in `/platform/admin/go-live`. |
| **live** | Cloud Anthropic stops being called. Every agent inference routes through Ollama. |

There is no implicit fallback after cutover. If a vet fails post-cutover, state regresses to `vetting` and admin re-clears before re-flipping to `live`.

---

## What can go wrong + the fix

| Symptom | Likely cause | Fix |
|---|---|---|
| Compliance scan flags "OT VLAN bridged" | eth0 + eth1 share a bridge | Reconfigure netplan to keep them separate |
| Wizard step 4 (connectors) hangs | Customer source-system credential not in vault | Run `scripts/vault-rotate.ts --apply` after credential is added |
| First operator screen renders with empty data | Connector still in passive_read with no data flowed yet | Wait for first data push (typically <5 min); if none, check the connector's read-permissions on source |
| Go Live button stays disabled | Some agent has no vetted local candidate | `/platform/admin/go-live` shows blocking issues per agent |
| First-boot wizard 404s | DNS A-record not propagated | `dig civops.<customer-domain>` to confirm; appliance is up but unreachable |

The full troubleshooting matrix lives in `docs/CUSTOMER_INSTALL_GUIDE.md` (technical) and the on-site engineer carries `docs/FIELD_SERVICE_PLAYBOOK.md` for in-field decisions.

---

## After cutover

- Daily-use loops unlock (Operations, HMI, Quality, Maintenance, Workforce in the sidebar).
- Lifecycle Value Ledger starts collecting baseline KPIs against the pilot success criteria.
- Period Close runs at month-end (CFO-class authority required).
- Connector promotion happens on the customer's timeline — not ours.

---

## Support model

- **Tier 1 — customer admin:** First-line response. Has full admin access, can promote connectors, manage users.
- **Tier 2 — partner support:** The reseller (or Civops directly) handles deeper config, agent vetting, vertical pack questions.
- **Tier 3 — Civops engineering:** Four-eyes (two engineers) policy for any production change. Outbound-only mTLS tunnel for remote support; customer approves session start; action allowlist enforced (no shell).

The Remote Support paved road is documented at `docs/REMOTE_SUPPORT.md`.

---

## Next step

After install, point the customer's operations team at the platform's persona-aware home (`/platform/start`). Their daily-use loops are the next thing they'll learn. Welcome to Civops.