chore: add AGENTS.md, move install.sh to root, rename frontend install.sh → setup.sh

AGENTS.md

@@ -0,0 +1,214 @@
+# AGENTS.md — VM Bench (VMware → Proxmox Image Converter)
+
+## Project Overview
+
+VM Bench is a two-tier web application for converting VMware disk images (VMDK, VHD,
+etc.) into Proxmox-compatible QCOW2 disks and provisioning VMs with automatic OS and
+boot type detection.
+
+- **Repository**: `https://git.lohmar.co.uk/cclohmar/vm-bench.git`
+- **Deployment root**: `/mnt/converter/` (bind-mounted between host and LXC)
+- **Frontend**: FastAPI + Jinja2 web UI in an LXC container (`vm-bench`) on port 5000
+- **Backend**: FastAPI REST API on the Proxmox host (`srv2`) on port 9000
+- **API spec**: `open-api.yaml` (single source of truth — derive models from this)
+- **Python version**: 3.13 (backend), 3.11 (frontend LXC)
+
+## Architecture
+
+```
+Browser            vm-bench LXC             Proxmox Host (srv2)
+                   (this container)
+──→ :5000 ──→  frontend/app.py  ──→  :9000  backend/app.py
+                                       │
+                                       ├─ qemu-img   (format, size)
+                                       ├─ guestfish  (OS, EFI detection)
+                                       ├─ 7z/unzip   (archive extraction)
+                                       └─ qm create  (VM provisioning)
+```
+
+- **Frontend** (`vm-bench` LXC) — user-facing web UI, manages sessions, proxies API calls
+- **Backend** (Proxmox host) — disk conversion, analysis, VM provisioning
+- **Shared storage** — `/mnt/converter/tmp/` (staging), `/mnt/converter/logs/` (logs)
+
+## Directory Layout
+
+```
+/mnt/converter/
+├── install.sh                    # Main deployment script (host-level)
+├── clean.sh                      # Clean tmp dirs + restart backend
+├── open-api.yaml                 # API specification (single source of truth)
+├── README.md
+├── AGENTS.md                     # This file
+├── backend/                      # Backend API (runs on Proxmox host)
+│   ├── app.py                    # FastAPI app, routes, logging
+│   ├── models.py                 # Pydantic request/response models
+│   ├── converter.py              # Archive extraction, disk probing, OS/EFI detection
+│   ├── provisioner.py            # VM provisioning (qm create/importdisk/convert)
+│   ├── requirements.txt          # fastapi, uvicorn, pydantic
+│   └── vm-bench-backend.service  # Systemd unit
+├── frontend/                     # Web UI (runs on vm-bench LXC)
+│   ├── app.py                    # FastAPI app, routes, download/SCP handling
+│   ├── api_client.py             # Typed REST client → backend
+│   ├── setup.sh                  # Frontend installer (inside LXC)
+│   ├── requirements.txt          # fastapi, uvicorn, jinja2, requests, python-multipart
+│   ├── vm-bench.service          # Systemd unit
+│   ├── templates/                # Jinja2 HTML templates
+│   │   ├── base.html             # Base layout with session GUID + header/footer
+│   │   ├── index.html            # New session form + download + progress
+│   │   ├── _analysis.html        # Analysis result + confirm form (HTML fragment)
+│   │   ├── polling.html          # Progress bar + job status + reuse section
+│   │   └── scp.html              # SCP pull page
+│   └── static/
+│       └── proxmox.css           # Proxmox VE-inspired CSS theme
+├── logs/                         # Shared log files (gitignored)
+├── tmp/                          # Staging files (gitignored)
+└── venv/                         # Backend Python venv (gitignored)
+```
+
+## Key Files by Responsibility
+
+### Backend (`backend/`)
+
+| File | Purpose |
+|------|---------|
+| `backend/app.py:1` | FastAPI app with routes: `/api/v1/health`, `/analyze`, `/jobs`, `/jobs/{id}`, `/jobs/{id}/cleanup`, `/clone` |
+| `backend/models.py:1` | Pydantic models: `JobSubmissionRequest`, `JobStatusResponse`, `AnalyzeRequest`, `AnalyzeResponse`, `DiskSpec`, `CleanupRequest`, `CloneRequest` |
+| `backend/converter.py:1` | Archive extraction (`extract_if_needed`), disk discovery (`discover_disk`), OS detection (`detect_os`), EFI detection (`detect_efi`) |
+| `backend/provisioner.py:1` | Background job processor: `submit_job`, `get_job_status`, `cleanup_staging`, `clone_vm`, `_process_job` (qemu-img convert → virt-resize → qm create → qm importdisk) |
+
+### Frontend (`frontend/`)
+
+| File | Purpose |
+|------|---------|
+| `frontend/app.py:1` | FastAPI app with routes: `/`, `/session/upload`, `/session/progress/{f}`, `/session/analyze`, `/session/confirm`, `/session/status/{id}`, `/session/cleanup/{id}`, `/session/clone`, `/scp`, `/scp/start`, `/scp/progress/{sid}/{f}` |
+| `frontend/api_client.py:1` | `ApiClient` class — typed HTTP client for backend API with `ApiError` exception |
+
+### Deployment (`install.sh`)
+
+| File | Purpose |
+|------|---------|
+| `install.sh:1` | Master deployment script (Proxmox host). Modes: `--deploy`, `--update`, `--remove`. Creates directory tree, clones repo, installs system packages (qemu-img, guestfish, archive tools), Python venv, backend systemd service, LXC container with bind mount. |
+| `frontend/setup.sh:1` | Frontend installer (run inside LXC). Installs system packages (python3, aria2, openssh-client, sshpass), Python deps, systemd service. |
+
+### Systemd Units
+
+| File | Port | Env Vars |
+|------|------|----------|
+| `backend/vm-bench-backend.service` | 9000 | None |
+| `frontend/vm-bench.service` | 5000 | `BACKEND_URL=http://10.2.0.2:9000`, `TMPDIR=/mnt/converter/tmp` |
+
+## Usage Flow
+
+1. User opens web UI → session form
+2. Upload or download a source image → lands in `/mnt/converter/tmp/{session_id}/in/`
+3. Backend analyses image → shows format, size, OS, EFI status
+4. Configure VM settings → name, CPU, RAM, storage, boot type
+5. Submit job → backend converts, shrinks (if needed), creates VM
+6. Poll progress → real-time status bar updates every 2 seconds via AJAX
+7. Reuse or cleanup → create another VM or delete staging files
+
+## Disk Sizing & Auto-Shrink Logic
+
+- If `target_disk_size_gb` is omitted and the source disk is larger than its
+  auto-shrink target (10% of virtual size, minimum 20 GB), the disk is shrunk
+  using `virt-resize --shrink --resize-force`.
+- If `virt-resize` fails, falls back to `qemu-img resize --shrink`.
+- If `target_disk_size_gb` is larger than current, disk is expanded.
+
+## Session Management
+
+- Frontend generates a UUID session ID stored in `localStorage` + cookie (`vm_bench_sid`)
+- Files are staged at `/mnt/converter/tmp/{session_id}/in/` and `out/`
+- Cleanup can delete the entire session directory or preserve for reuse
+
+## Source Image Ingestion Methods
+
+1. **Download URL** (`index.html`) — uses `aria2c` with 8 connections, kills download if ETA > 1 hour
+2. **SCP Pull** (`scp.html`) — uses `sshpass` + `scp` directly from remote servers
+3. File upload (legacy, referenced but not primary flow)
+
+## Backend Tools Required (on Proxmox host)
+
+- `qemu-img` — disk format and size detection, format conversion
+- `guestfish` / `virt-inspector` — OS type and EFI boot detection
+- `virt-resize` — safe disk shrinking
+- `7z`, `unzip`, `unrar`, `tar`, `gunzip`, `bunzip2`, `xz` — archive extraction
+- `qm` (Proxmox CLI) — VM creation, disk import, configuration
+
+## Frontend Tools Required (in LXC)
+
+- `python3`, `pip3`, `wget`, `curl`, `aria2c`, `openssh-client`, `sshpass`
+
+## Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `BACKEND_URL` | `http://10.2.0.2:9000` | Backend API base URL (set in systemd unit + env) |
+| `VM_ID_MIN` | `21000` | Minimum allowed Proxmox VM ID |
+| `VM_ID_MAX` | `21100` | Maximum allowed Proxmox VM ID |
+| `DOWNLOAD_TIMEOUT` | `14400` (4h) | Max download time (safety net) |
+| `SPEED_CHECK_AFTER` | `30` (seconds) | Wait before judging download speed |
+| `MAX_ETA_SECONDS` | `3600` (1h) | Kill download if ETA exceeds this |
+
+## Job States
+
+```
+queued → processing_conversion → importing_storage → completed
+                                                  → failed
+```
+
+## Logging
+
+Both services log to console (systemd journal) and rotating file handler:
+- Frontend: `/mnt/converter/logs/vm-bench.log`
+- Backend: `/mnt/converter/logs/vm-bench-backend.log`
+- Rotation: 10 MB max, 5 backup files
+
+## Development Commands
+
+### Backend (on Proxmox host)
+```bash
+# Install/update
+bash install.sh --deploy
+
+# Restart
+systemctl restart vm-bench-backend
+
+# View logs
+tail -f /mnt/converter/logs/vm-bench-backend.log
+journalctl -u vm-bench-backend -f
+
+# Health check
+curl http://127.0.0.1:9000/api/v1/health
+```
+
+### Frontend (inside LXC)
+```bash
+# Install/update
+bash /mnt/converter/frontend/setup.sh
+
+# Restart
+systemctl restart vm-bench
+
+# View logs
+tail -f /mnt/converter/logs/vm-bench.log
+journalctl -u vm-bench -f
+```
+
+### Cleanup script
+```bash
+bash /mnt/converter/clean.sh  # Clears tmp dirs + restarts backend
+```
+
+## Project Conventions
+
+- **No code comments** unless strictly necessary (follow existing patterns)
+- Models derived from `open-api.yaml` — update spec first, then code
+- `backend/provisioner.py` uses in-memory job store (`_jobs` dict) with `threading.Lock`
+- All paths use `pathlib.Path`
+- Loggers use module-level naming: `logging.getLogger("backend")`, `logging.getLogger("backend.converter")`, etc.
+- Frontend templates use `pve-` CSS prefix mimicking Proxmox VE design system
+- HTML fragments (e.g. `_analysis.html`) are injected via `innerHTML` — no `<script>` tags
+- Session-based isolation: each browser tab/session gets a UUID for file staging
+- VM names are sanitized to DNS-safe characters: `[^a-zA-Z0-9-]` replaced with `-`
+- VM ID range is validated client-side (21000–21100) and enforced in frontend routes

backend/install.sh

@@ -1,344 +0,0 @@
-#!/bin/bash
-set -e
-
-# ╔════════════════════════════════════════════════════════════════════╗
-# ║  VM Bench — Full Deployment Script (run on Proxmox host srv2)     ║
-# ║                                                                    ║
-# ║  Usage:  bash /mnt/converter/backend/install.sh                    ║
-# ║                                                                    ║
-# ║  Does everything:                                                  ║
-# ║    1. Creates /mnt/converter/ directory tree                       ║
-# ║    2. Clones (or pulls) the git repository                         ║
-# ║    3. Installs system packages + Python deps + systemd service     ║
-# ║    4. Prompts for container ID + network config                    ║
-# ║    5. Creates the vm-bench LXC container with bind mount           ║
-# ╚════════════════════════════════════════════════════════════════════╝
-
-# ───────────────────────────────────────────────────────────────────
-# Colours & paths
-# ───────────────────────────────────────────────────────────────────
-RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[0;33m'; CYAN='\033[0;36m'; NC='\033[0m'
-
-REPO_URL="https://git.lohmar.co.uk/cclohmar/vm-bench.git"
-CONVERTER_ROOT="/mnt/converter"
-SCRIPT_DIR="${CONVERTER_ROOT}/backend"
-SERVICE_NAME="vm-bench-backend"
-SERVICE_FILE="${SCRIPT_DIR}/${SERVICE_NAME}.service"
-SYSTEMD_DIR="/etc/systemd/system"
-
-# ───────────────────────────────────────────────────────────────────
-# Banner
-# ───────────────────────────────────────────────────────────────────
-echo ""
-echo "╔═════════════════════════════════════════════════════════╗"
-echo "║       VM Bench — Full Deployment Installer             ║"
-echo "║       Target: Proxmox host (srv2)                      ║"
-echo "╚═════════════════════════════════════════════════════════╝"
-echo ""
-
-# ───────────────────────────────────────────────────────────────────
-# Pre-flight: must run on a Proxmox host
-# ───────────────────────────────────────────────────────────────────
-echo "[preflight] Checking environment..."
-if ! command -v pct &>/dev/null; then
-    echo "  ${RED}✗${NC} 'pct' not found — this script must run on a Proxmox host."
-    exit 1
-fi
-if ! command -v git &>/dev/null; then
-    echo "  Installing git..."
-    apt-get update -qq && apt-get install -y -qq git
-fi
-if ! command -v qemu-img &>/dev/null; then
-    echo "  ${RED}✗${NC} 'qemu-img' not found — Proxmox ships this. Do NOT install qemu-utils (conflicts with Proxmox)."
-    exit 1
-fi
-echo "  ${GREEN}✓${NC} Running on Proxmox host."
-
-# ───────────────────────────────────────────────────────────────────
-# Step 1: Create directory tree
-# ───────────────────────────────────────────────────────────────────
-echo ""
-echo "[1/6] Creating directory structure..."
-
-for d in \
-    "$CONVERTER_ROOT" \
-    "$CONVERTER_ROOT/logs" \
-    "$CONVERTER_ROOT/tmp" \
-    "$CONVERTER_ROOT/backend" \
-    "$CONVERTER_ROOT/frontend" \
-    "$CONVERTER_ROOT/frontend/templates" \
-    "$CONVERTER_ROOT/frontend/static"; do
-    if [ ! -d "$d" ]; then
-        mkdir -p "$d"
-        echo "  Created: $d"
-    fi
-done
-echo "  ${GREEN}✓${NC} Directories ready."
-
-# ───────────────────────────────────────────────────────────────────
-# Step 2: Clone or update the git repository
-# ───────────────────────────────────────────────────────────────────
-echo ""
-echo "[2/6] Cloning / updating repository..."
-
-if [ -d "$CONVERTER_ROOT/.git" ]; then
-    echo "  Existing repo found, pulling latest..."
-    git -C "$CONVERTER_ROOT" pull --ff-only
-    echo "  ${GREEN}✓${NC} Repository updated."
-else
-    # Clone into a temp location, then move if /mnt/converter is not empty
-    if [ -z "$(ls -A "$CONVERTER_ROOT" 2>/dev/null)" ]; then
-        git clone "$REPO_URL" "$CONVERTER_ROOT"
-    else
-        echo "  ${YELLOW}!${NC} ${CONVERTER_ROOT} is not empty — cloning into temp and copying..."
-        TMP_CLONE=$(mktemp -d)
-        git clone "$REPO_URL" "$TMP_CLONE"
-        cp -a "$TMP_CLONE"/. "$CONVERTER_ROOT"/
-        rm -rf "$TMP_CLONE"
-    fi
-    echo "  ${GREEN}✓${NC} Repository cloned."
-fi
-
-# ───────────────────────────────────────────────────────────────────
-# Step 3: System packages
-# ───────────────────────────────────────────────────────────────────
-echo ""
-echo "[3/6] Checking system packages..."
-
-PACKAGES=()
-REQUIRED=(
-    python3            "python3"
-    python3-pip        "pip3"
-    libguestfs-tools   "guestfish"
-    p7zip-full         "7z"
-    unzip              "unzip"
-    unrar-free         "unrar"
-    tar                "tar"
-    gzip               "gunzip"
-    bzip2              "bunzip2"
-    xz-utils           "xz"
-)
-
-for ((i=0; i<${#REQUIRED[@]}; i+=2)); do
-    pkg="${REQUIRED[$i]}"
-    bin="${REQUIRED[$i+1]}"
-    if command -v "$bin" &>/dev/null; then
-        echo "  ${GREEN}✓${NC} $bin"
-    else
-        echo "  ${YELLOW}○${NC} $bin (will install ${pkg})"
-        PACKAGES+=("$pkg")
-    fi
-done
-
-if [ ${#PACKAGES[@]} -gt 0 ]; then
-    echo ""
-    echo "  Installing: ${PACKAGES[*]}"
-    apt-get update -qq
-    apt-get install -y -qq "${PACKAGES[@]}"
-    echo "  ${GREEN}✓${NC} System packages installed."
-else
-    echo "  ${GREEN}✓${NC} All system packages present."
-fi
-
-# ───────────────────────────────────────────────────────────────────
-# Step 4: Python dependencies + systemd service
-# ───────────────────────────────────────────────────────────────────
-echo ""
-echo "[4/6] Installing Python dependencies..."
-
-cd "$SCRIPT_DIR"
-python3 -m pip install --break-system-packages -r requirements.txt -q
-echo "  ${GREEN}✓${NC} Python dependencies installed."
-
-echo ""
-echo "[5/6] Installing systemd service..."
-
-if [ ! -f "$SERVICE_FILE" ]; then
-    echo "  ${RED}✗${NC} Service file not found: $SERVICE_FILE"
-    echo "  Make sure the repository was cloned correctly."
-    exit 1
-fi
-
-cp "$SERVICE_FILE" "$SYSTEMD_DIR/${SERVICE_NAME}.service"
-systemctl daemon-reload
-systemctl enable "$SERVICE_NAME"
-systemctl restart "$SERVICE_NAME"
-sleep 2
-
-if systemctl is-active --quiet "$SERVICE_NAME"; then
-    echo "  ${GREEN}✓${NC} Backend is RUNNING on port 9000"
-    echo "  Verify:  curl http://127.0.0.1:9000/api/v1/health"
-else
-    echo "  ${RED}✗${NC} Backend failed to start. Check logs:"
-    echo "    journalctl -u ${SERVICE_NAME} -n 30"
-    exit 1
-fi
-
-# ───────────────────────────────────────────────────────────────────
-# Step 6: Create the vm-bench LXC container
-# ───────────────────────────────────────────────────────────────────
-echo ""
-echo "╔═════════════════════════════════════════════════════════╗"
-echo "║  LXC Container Setup (vm-bench frontend)               ║"
-echo "╚═════════════════════════════════════════════════════════╝"
-echo ""
-
-# ── Prompt for container ID ──────────────────────────────────────
-read -p "  Container ID (e.g. 20020): " CTID
-if [ -z "$CTID" ]; then
-    echo "  ${RED}✗${NC} Container ID is required."
-    exit 1
-fi
-
-# Check if ID already exists
-if pct status "$CTID" &>/dev/null; then
-    echo "  ${YELLOW}!${NC} Container $CTID already exists."
-    read -p "  Overwrite? [y/N]: " OVERWRITE
-    if [ "$OVERWRITE" != "y" ] && [ "$OVERWRITE" != "Y" ]; then
-        echo "  Aborted."
-        exit 0
-    fi
-    echo "  Destroying existing container $CTID..."
-    pct stop "$CTID" --skiplock 2>/dev/null || true
-    pct destroy "$CTID" --purge 2>/dev/null || true
-fi
-
-# ── Network settings ─────────────────────────────────────────────
-echo ""
-echo "  ${CYAN}Network configuration:${NC}"
-read -p "    IP address/CIDR  [10.2.0.20/16]: " CT_IP
-read -p "    Gateway          [10.2.0.2]:     " CT_GW
-read -p "    Bridge           [vmbr0]:        " CT_BRIDGE
-read -p "    MAC address      [BC:24:11:80:5A:B1]: " CT_MAC
-
-CT_IP="${CT_IP:-10.2.0.20/16}"
-CT_GW="${CT_GW:-10.2.0.2}"
-CT_BRIDGE="${CT_BRIDGE:-vmbr0}"
-CT_MAC="${CT_MAC:-BC:24:11:80:5A:B1}"
-
-NET_CFG="name=eth0,bridge=${CT_BRIDGE},firewall=1,gw=${CT_GW},hwaddr=${CT_MAC},ip=${CT_IP},type=veth"
-ROOTFS="local-lvm:vm-${CTID}-disk-0,size=24G"
-
-# ── Storage path ──────────────────────────────────────────────────
-echo ""
-echo "  ${CYAN}Storage mount:${NC}"
-echo "    Path on the Proxmox host where VM Bench stores its data."
-echo "    The LXC will see this as /mnt/converter internally."
-read -p "    Host storage path [/mnt/pve/hosted-thin]: " STORAGE_PATH
-STORAGE_PATH="${STORAGE_PATH:-/mnt/pve/hosted-thin}"
-
-# Remove trailing slash
-STORAGE_PATH="${STORAGE_PATH%/}"
-
-# Create the directory on the host if it doesn't exist
-if [ ! -d "$STORAGE_PATH" ]; then
-    echo "  Creating: $STORAGE_PATH"
-    mkdir -p "$STORAGE_PATH"
-fi
-
-# ── Template selection ────────────────────────────────────────────
-echo ""
-echo "  ${CYAN}Available Debian templates:${NC}"
-pveam available 2>/dev/null | grep debian-12 || echo "  (none found locally, will attempt to download)"
-
-TEMPLATE=$(pveam available 2>/dev/null | grep -m1 "debian-12.*amd64" | awk '{print $2}')
-if [ -z "$TEMPLATE" ]; then
-    # Try to find any available Debian template
-    TEMPLATE=$(pveam available 2>/dev/null | grep -m1 "debian.*amd64" | awk '{print $2}')
-fi
-if [ -z "$TEMPLATE" ]; then
-    echo "  ${YELLOW}!${NC} No Debian template found. Downloading debian-12-standard..."
-    pveam update 2>/dev/null || true
-    pveam download local debian-12-standard_12.7-1_amd64.tar.zst 2>/dev/null || true
-    TEMPLATE="local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst"
-else
-    # Check if template is downloaded locally
-    TEMPLATE_NAME=$(basename "$TEMPLATE")
-    if [ ! -f "/var/lib/vz/template/cache/${TEMPLATE_NAME}" ]; then
-        echo "  Downloading template: ${TEMPLATE_NAME}..."
-        pveam download local "$TEMPLATE_NAME" 2>/dev/null || {
-            echo "  ${YELLOW}!${NC} Auto-download failed. Trying to continue anyway..."
-        }
-    fi
-    TEMPLATE="local:vztmpl/${TEMPLATE_NAME}"
-fi
-
-echo "  Using template: ${TEMPLATE}"
-
-# ── Create container ──────────────────────────────────────────────
-echo ""
-echo "  Creating container ${CTID}..."
-
-pct create "$CTID" "$TEMPLATE" \
-    --arch amd64 \
-    --cores 3 \
-    --hostname vm-bench \
-    --memory 6144 \
-    --net0 "$NET_CFG" \
-    --ostype debian \
-    --rootfs "$ROOTFS" \
-    --swap 2048 \
-    --unprivileged 0 \
-    --features nesting=1 \
-    --mp0 ${STORAGE_PATH},mp=/mnt/converter \
-    --onboot 1 \
-    --start 0
-
-echo "  ${GREEN}✓${NC} Container created."
-
-# ── Write LXC config overrides ────────────────────────────────────
-CONF_FILE="/etc/pve/lxc/${CTID}.conf"
-echo ""
-echo "  Writing LXC config overrides..."
-
-# Add lines only if not already present
-add_conf_line() {
-    local key="$1"
-    local val="$2"
-    if ! grep -q "^${key}:" "$CONF_FILE" 2>/dev/null; then
-        echo "${key}: ${val}" >> "$CONF_FILE"
-        echo "    + ${key}: ${val}"
-    fi
-}
-
-add_conf_line "lxc.cgroup2.devices.allow" "a"
-add_conf_line "lxc.mount.auto" "proc:rw sys:rw cgroup:rw"
-add_conf_line "lxc.apparmor.profile" "unconfined"
-
-echo "  ${GREEN}✓${NC} LXC config written."
-
-# ── Start the container ───────────────────────────────────────────
-echo ""
-read -p "  Start container now? [Y/n]: " START
-if [ "$START" != "n" ] && [ "$START" != "N" ]; then
-    pct start "$CTID"
-    sleep 3
-    if pct status "$CTID" | grep -q running; then
-        echo "  ${GREEN}✓${NC} Container ${CTID} is running."
-        echo ""
-        echo "  ${CYAN}Next steps:${NC}"
-        echo "    1. Enter the container:  pct enter ${CTID}"
-        echo "    2. Run the frontend installer:"
-        echo "       bash /mnt/converter/frontend/install.sh"
-        echo ""
-        echo "    3. Open the web UI:  http://${CT_IP%/*}:5000"
-        echo ""
-    else
-        echo "  ${YELLOW}!${NC} Container may not have started. Check: pct status ${CTID}"
-    fi
-fi
-
-# ───────────────────────────────────────────────────────────────────
-# Done
-# ───────────────────────────────────────────────────────────────────
-echo ""
-echo "╔═════════════════════════════════════════════════════════╗"
-echo "║  ${GREEN}Deployment complete.${NC}                                   ║"
-echo "║                                                         ║"
-echo "║  Backend:  http://$(hostname -I | awk '{print $1}'):9000       ║"
-echo "║  Frontend: http://${CT_IP%/*}:5000               ║"
-echo "║                                                         ║"
-echo "║  Backend logs:  journalctl -u ${SERVICE_NAME} -f        ║"
-echo "║  Container:     pct enter ${CTID}                       ║"
-echo "╚═════════════════════════════════════════════════════════╝"
-echo ""

backend/vm-bench-backend.service

@@ -1,15 +1,16 @@
 [Unit]
-Description=VM Bench Backend — Proxmox Image Conversion Engine
+Description=VM Bench Converter Backend Service
 After=network.target
 
 [Service]
 Type=simple
 User=root
-WorkingDirectory=/mnt/converter/backend
+WorkingDirectory=/mnt/pve/host-shared/converter/backend
-Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ExecStart=/mnt/pve/host-shared/converter/venv/bin/python /mnt/pve/host-shared/converter/backend/app.py
-ExecStart=/usr/bin/python3 -m uvicorn app:app --host 0.0.0.0 --port 9000 --timeout-keep-alive 300
 Restart=always
-RestartSec=3
+RestartSec=5
+StandardOutput=append:/mnt/pve/host-shared/converter/logs/vm-bench-backend.log
+StandardError=append:/mnt/pve/host-shared/converter/logs/vm-bench-backend.log
 
 [Install]
 WantedBy=multi-user.target

install.sh

@@ -0,0 +1,402 @@
+#!/usr/bin/env bash
+# ==============================================================================
+# VM Bench Stack Provisioning & Lifecycle Script
+# Target: Proxmox VE node (any version)
+# Usage:
+#   ./install.sh --deploy   [Interactive Setup & Deployment]
+#   ./install.sh --update   [Pull Updates & Restart Services]
+#   ./install.sh --remove   [Teardown Everything]
+# ==============================================================================
+
+set -euo pipefail
+# Uncomment next line for debugging:
+# set -x
+
+# ------------------------------------------------------------------------------
+# Formatting & Constants
+# ------------------------------------------------------------------------------
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[0;33m'
+CYAN='\033[0;36m'
+NC='\033[0m'
+
+REPO_URL="https://git.lohmar.co.uk/cclohmar/vm-bench.git"
+SYSTEMD_DIR="/etc/systemd/system"
+SERVICE_NAME="vm-bench-backend"
+
+CT_CORES=1
+CT_MEMORY=512
+CT_SWAP=512
+CT_DISK_SIZE=8
+
+# ------------------------------------------------------------------------------
+# Helpers
+# ------------------------------------------------------------------------------
+has_cmd() {
+    command -v "$1" >/dev/null 2>&1
+}
+
+get_default_bridge() {
+    ip -o link show type bridge | awk -F': ' '{print $2; exit}' 2>/dev/null || echo "vmbr0"
+}
+
+guess_gateway_from_bridge() {
+    local bridge="$1"
+    local ip_cidr
+    ip_cidr=$(ip -o -4 addr show dev "$bridge" 2>/dev/null | awk '{print $4; exit}')
+    if [[ -n "$ip_cidr" ]]; then
+        local ip="${ip_cidr%/*}"
+        local prefix="${ip%.*}"
+        echo "${prefix}.1"
+    else
+        echo ""
+    fi
+}
+
+# ------------------------------------------------------------------------------
+# Environment Discovery & User Parameter Gathering
+# ------------------------------------------------------------------------------
+gather_environment_info() {
+    echo -e "${CYAN}==> [1/3] Environment Inspection & Parameter Collection${NC}"
+
+    # ---- Storage Selection ----
+    echo ""
+    echo "Inspecting available storage pools from /etc/pve/storage.cfg..."
+    declare -a storage_ids=()
+    declare -a storage_contents=()
+    declare -a storage_has_rootdir=()
+
+    # Parse storage.cfg for container-capable storage types
+    if [ -f /etc/pve/storage.cfg ]; then
+        # Extract storage IDs for types: lvmthin, zfspool, rbd, dir
+        while IFS= read -r id; do
+            storage_ids+=("$id")
+            # Get content for this pool (ignore errors)
+            local content
+            content=$(pvesm config "$id" 2>/dev/null | awk '/^content/ {$1=""; print $0}' | xargs | tr ' ' ',' || true)
+            storage_contents+=("${content:-none}")
+            if [[ "$content" == *"rootdir"* ]]; then
+                storage_has_rootdir+=("yes")
+            else
+                storage_has_rootdir+=("no")
+            fi
+        done < <(grep -E "^(lvmthin|zfspool|rbd|dir):" /etc/pve/storage.cfg | awk '{print $2}')
+    fi
+
+    # Fallback: if no storage found in config, use active pools from pvesm
+    if [ ${#storage_ids[@]} -eq 0 ]; then
+        echo -e "${YELLOW}No storage definitions found in /etc/pve/storage.cfg. Falling back to active pools from 'pvesm status'.${NC}"
+        while IFS= read -r pool; do
+            storage_ids+=("$pool")
+            local content
+            content=$(pvesm config "$pool" 2>/dev/null | awk '/^content/ {$1=""; print $0}' | xargs | tr ' ' ',' || true)
+            storage_contents+=("${content:-none}")
+            if [[ "$content" == *"rootdir"* ]]; then
+                storage_has_rootdir+=("yes")
+            else
+                storage_has_rootdir+=("no")
+            fi
+        done < <(pvesm status 2>/dev/null | awk 'NR>1 && $3=="active" {print $1}' || true)
+    fi
+
+    # If still empty, abort
+    if [ ${#storage_ids[@]} -eq 0 ]; then
+        echo -e "${RED}No usable storage pools found. Please create a storage pool (e.g., 'local' directory) and try again.${NC}"
+        exit 1
+    fi
+
+    # Show menu
+    echo "Available Storage Options Detected:"
+    for i in "${!storage_ids[@]}"; do
+        id="${storage_ids[$i]}"
+        content="${storage_contents[$i]}"
+        marker=""
+        [ "${storage_has_rootdir[$i]}" = "yes" ] && marker=" ✓ (has rootdir)"
+        printf "  [%d] %-18s (content: %s)%s\n" "$((i+1))" "$id" "$content" "$marker"
+    done
+
+    # Pick default: first with rootdir, else first
+    default_idx=0
+    for i in "${!storage_ids[@]}"; do
+        if [ "${storage_has_rootdir[$i]}" = "yes" ]; then
+            default_idx=$i
+            break
+        fi
+    done
+    default_num=$((default_idx+1))
+
+    read -p "Select storage pool [1-${#storage_ids[@]}] (default: ${default_num}): " POOL_CHOICE
+    POOL_CHOICE="${POOL_CHOICE:-$default_num}"
+    if [[ "$POOL_CHOICE" =~ ^[0-9]+$ ]] && [ "$POOL_CHOICE" -ge 1 ] && [ "$POOL_CHOICE" -le "${#storage_ids[@]}" ]; then
+        TARGET_STORAGE="${storage_ids[$((POOL_CHOICE-1))]}"
+        TARGET_HAS_ROOTDIR="${storage_has_rootdir[$((POOL_CHOICE-1))]}"
+    else
+        echo -e "${YELLOW}Invalid choice. Defaulting to '${storage_ids[$default_idx]}'.${NC}"
+        TARGET_STORAGE="${storage_ids[$default_idx]}"
+        TARGET_HAS_ROOTDIR="${storage_has_rootdir[$default_idx]}"
+    fi
+
+    # Ensure rootdir capability
+    if [ "$TARGET_HAS_ROOTDIR" != "yes" ]; then
+        echo "Enabling 'rootdir' capability on storage pool '${TARGET_STORAGE}'..."
+        current_content="${storage_contents[$((POOL_CHOICE-1))]:-}"
+        if [ -n "$current_content" ]; then
+            new_content="${current_content},rootdir"
+        else
+            new_content="rootdir"
+        fi
+        pvesm set "$TARGET_STORAGE" --content "$new_content" || {
+            echo -e "${RED}Failed to add rootdir. Please enable it manually.${NC}"
+            exit 1
+        }
+    fi
+
+    # ---- Path & Container Configuration ----
+    echo ""
+    read -p "Base Storage Path [/mnt/pve/host-shared/converter]: " PATH_INPUT
+    STORAGE_PATH="${PATH_INPUT:-/mnt/pve/host-shared/converter}"
+    STORAGE_PATH="${STORAGE_PATH%/}"
+
+    read -p "Container ID [20020]: " CTID_INPUT
+    CTID="${CTID_INPUT:-20020}"
+
+    DEFAULT_BRIDGE=$(get_default_bridge)
+    read -p "Network Bridge [default: ${DEFAULT_BRIDGE}]: " BRIDGE_INPUT
+    CT_BRIDGE="${BRIDGE_INPUT:-$DEFAULT_BRIDGE}"
+
+    SUGGESTED_GW=$(guess_gateway_from_bridge "$CT_BRIDGE")
+    [ -z "$SUGGESTED_GW" ] && SUGGESTED_GW="10.2.0.1"
+
+    read -p "Container IP/CIDR (e.g., 10.2.0.20/16): " CT_IP
+    while [[ -z "$CT_IP" ]]; do
+        echo -e "${RED}Error: Static IP with CIDR mask is required (e.g., 10.2.0.20/16).${NC}"
+        read -p "Container IP/CIDR: " CT_IP
+    done
+
+    read -p "Gateway IP [${SUGGESTED_GW}]: " GW_INPUT
+    CT_GW="${GW_INPUT:-$SUGGESTED_GW}"
+
+    read -p "MAC Address (Press Enter to auto-generate): " MAC_INPUT
+    CT_MAC="${MAC_INPUT:-}"
+
+    echo ""
+    echo -e "${GREEN}✓ Environment configuration gathered successfully.${NC}"
+}
+
+# ------------------------------------------------------------------------------
+# Action Functions (Deploy, Update, Remove)
+# ------------------------------------------------------------------------------
+do_deploy() {
+    echo -e "${GREEN}==> Beginning Deployment on Host $(hostname)...${NC}"
+
+    if ! command -v pct &>/dev/null; then
+        echo -e "${RED}Error: 'pct' CLI not found. Script must be executed on a Proxmox host.${NC}"
+        exit 1
+    fi
+
+    gather_environment_info
+
+    REAL_PATH=$(readlink -f "$STORAGE_PATH" 2>/dev/null || echo "$STORAGE_PATH")
+
+    # Clean previous state
+    if [ -d "$REAL_PATH" ] || pct status "$CTID" &>/dev/null; then
+        echo -e "${YELLOW}! Cleaning pre-existing installation state at ${REAL_PATH} / CTID ${CTID}...${NC}"
+        pct stop "$CTID" --skiplock 2>/dev/null || true
+        pct destroy "$CTID" --purge 2>/dev/null || true
+        systemctl stop "$SERVICE_NAME" 2>/dev/null || true
+        rm -rf "$REAL_PATH"
+    fi
+
+    # Fix APT sources for Proxmox (no subscription)
+    rm -f /etc/apt/sources.list.d/pve-enterprise.sources \
+          /etc/apt/sources.list.d/ceph.sources \
+          /etc/apt/sources.list.d/pve-enterprise.list \
+          /etc/apt/sources.list.d/pve-no-subscription.list 2>/dev/null || true
+    cat <<'EOF' > /etc/apt/sources.list.d/proxmox-no-sub.sources
+Types: deb
+URIs: http://download.proxmox.com/debian/pve
+Suites: trixie
+Components: pve-no-subscription
+Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg
+EOF
+    apt-get update -qq
+
+    echo -e "${CYAN}==> [2/3] Provisioning Directory Structure & Backend Service${NC}"
+    mkdir -p "$REAL_PATH"
+    echo "Cloning source code from ${REPO_URL}..."
+    git clone "$REPO_URL" "$REAL_PATH"
+
+    CONVERTER_ROOT="$REAL_PATH"
+    SCRIPT_DIR="${CONVERTER_ROOT}/backend"
+    SERVICE_FILE="${SCRIPT_DIR}/${SERVICE_NAME}.service"
+
+    mkdir -p "${CONVERTER_ROOT}/logs" "${CONVERTER_ROOT}/tmp"
+    chmod 777 "${CONVERTER_ROOT}/tmp"
+
+    apt-get install -y -qq --no-install-recommends \
+        python3 python3-pip python3-venv libguestfs-tools \
+        p7zip-full unzip unrar-free tar gzip bzip2 xz-utils git
+
+    VENV_DIR="${CONVERTER_ROOT}/venv"
+    echo "Setting up Python virtual environment..."
+    python3 -m venv "$VENV_DIR"
+    "$VENV_DIR/bin/pip" install --upgrade pip -q
+    if [ -f "${SCRIPT_DIR}/requirements.txt" ]; then
+        "$VENV_DIR/bin/pip" install -r "${SCRIPT_DIR}/requirements.txt" -q
+    fi
+
+    cat <<EOF > "$SERVICE_FILE"
+[Unit]
+Description=VM Bench Converter Backend Service
+After=network.target
+
+[Service]
+Type=simple
+User=root
+WorkingDirectory=${SCRIPT_DIR}
+ExecStart=${VENV_DIR}/bin/python ${SCRIPT_DIR}/app.py
+Restart=always
+RestartSec=5
+StandardOutput=append:${CONVERTER_ROOT}/logs/vm-bench-backend.log
+StandardError=append:${CONVERTER_ROOT}/logs/vm-bench-backend.log
+
+[Install]
+WantedBy=multi-user.target
+EOF
+
+    cp "$SERVICE_FILE" "${SYSTEMD_DIR}/${SERVICE_NAME}.service"
+    systemctl daemon-reload
+    systemctl enable --now "$SERVICE_NAME"
+    echo -e "${GREEN}✓ Host Backend systemd service initialized and running.${NC}"
+
+    echo -e "${CYAN}==> [3/3] Creating and Starting Frontend LXC Container (${CTID})${NC}"
+
+    # Get LXC template
+    TEMPLATE=$(pveam available 2>/dev/null | grep -m1 "debian-12.*amd64" | awk '{print $2}' || true)
+    if [ -z "$TEMPLATE" ]; then
+        pveam update 2>/dev/null || true
+        pveam download local debian-12-standard_12.7-1_amd64.tar.zst 2>/dev/null || true
+        TEMPLATE="local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst"
+    else
+        TEMPLATE_NAME=$(basename "$TEMPLATE")
+        [ ! -f "/var/lib/vz/template/cache/${TEMPLATE_NAME}" ] && pveam download local "$TEMPLATE_NAME" 2>/dev/null || true
+        TEMPLATE="local:vztmpl/${TEMPLATE_NAME}"
+    fi
+
+    NET_CFG="name=eth0,bridge=${CT_BRIDGE},firewall=1,gw=${CT_GW},ip=${CT_IP},type=veth"
+    [ -n "$CT_MAC" ] && NET_CFG="${NET_CFG},hwaddr=${CT_MAC}"
+
+    pct create "$CTID" "$TEMPLATE" \
+        --arch amd64 \
+        --cores "$CT_CORES" \
+        --hostname vm-bench \
+        --memory "$CT_MEMORY" \
+        --net0 "$NET_CFG" \
+        --ostype debian \
+        --storage "$TARGET_STORAGE" \
+        --rootfs "${TARGET_STORAGE}:${CT_DISK_SIZE}" \
+        --swap "$CT_SWAP" \
+        --unprivileged 0 \
+        --features nesting=1 \
+        --mp0 "${REAL_PATH},mp=/mnt/converter" \
+        --onboot 1 \
+        --start 0
+
+    # Extra LXC config
+    CONF_FILE="/etc/pve/lxc/${CTID}.conf"
+    echo "lxc.cgroup2.devices.allow: a" >> "$CONF_FILE"
+    echo "lxc.mount.auto: proc:rw sys:rw cgroup:rw" >> "$CONF_FILE"
+    echo "lxc.apparmor.profile: unconfined" >> "$CONF_FILE"
+
+    pct start "$CTID" 2>/dev/null || true
+
+    echo ""
+    echo -e "${GREEN}======================================================================${NC}"
+    echo -e "${GREEN} Deployment Successful on $(hostname)!${NC}"
+    echo " - Host Backend API    : http://127.0.0.1:9000"
+    echo " - LXC Container IP    : ${CT_IP%/*}"
+    echo " - Container Storage   : ${TARGET_STORAGE}"
+    echo " - Mount Directory     : ${REAL_PATH} -> /mnt/converter"
+    echo -e "${GREEN}======================================================================${NC}"
+}
+
+do_update() {
+    echo -e "${CYAN}==> Updating VM Bench Stack...${NC}"
+    read -p "Base Storage Path [/mnt/pve/host-shared/converter]: " PATH_INPUT
+    STORAGE_PATH="${PATH_INPUT:-/mnt/pve/host-shared/converter}"
+    REAL_PATH=$(readlink -f "$STORAGE_PATH" 2>/dev/null || echo "$STORAGE_PATH")
+
+    read -p "Container ID [20020]: " CTID_INPUT
+    CTID="${CTID_INPUT:-20020}"
+
+    if [ -d "$REAL_PATH/.git" ]; then
+        echo "Pulling latest code from Git..."
+        git -C "$REAL_PATH" pull --ff-only
+    fi
+
+    if [ -d "${REAL_PATH}/venv" ]; then
+        echo "Updating Python environment..."
+        "${REAL_PATH}/venv/bin/pip" install --upgrade pip -q
+        [ -f "${REAL_PATH}/backend/requirements.txt" ] && "${REAL_PATH}/venv/bin/pip" install -r "${REAL_PATH}/backend/requirements.txt" -q
+    fi
+
+    echo "Restarting Services..."
+    systemctl restart "$SERVICE_NAME"
+    pct restart "$CTID" 2>/dev/null || true
+    echo -e "${GREEN}✓ Stack update complete.${NC}"
+}
+
+do_remove() {
+    echo -e "${YELLOW}==> Tearing Down VM Bench Stack...${NC}"
+    read -p "Base Storage Path [/mnt/pve/host-shared/converter]: " PATH_INPUT
+    STORAGE_PATH="${PATH_INPUT:-/mnt/pve/host-shared/converter}"
+    REAL_PATH=$(readlink -f "$STORAGE_PATH" 2>/dev/null || echo "$STORAGE_PATH")
+
+    read -p "Container ID to Destroy [20020]: " CTID_INPUT
+    CTID="${CTID_INPUT:-20020}"
+
+    if pct status "$CTID" &>/dev/null; then
+        echo "Destroying LXC Container ${CTID}..."
+        pct stop "$CTID" --skiplock 2>/dev/null || true
+        pct destroy "$CTID" --purge 2>/dev/null || true
+    fi
+
+    if systemctl is-active --quiet "$SERVICE_NAME" 2>/dev/null || systemctl is-enabled "$SERVICE_NAME" 2>/dev/null; then
+        echo "Removing Host Backend Systemd Service..."
+        systemctl stop "$SERVICE_NAME" 2>/dev/null || true
+        systemctl disable "$SERVICE_NAME" 2>/dev/null || true
+        rm -f "${SYSTEMD_DIR}/${SERVICE_NAME}.service"
+        systemctl daemon-reload
+    fi
+
+    if [ -d "$REAL_PATH" ]; then
+        echo "Purging directory ${REAL_PATH}..."
+        rm -rf "$REAL_PATH"
+    fi
+
+    echo -e "${GREEN}✓ Removal complete.${NC}"
+}
+
+# ------------------------------------------------------------------------------
+# CLI Dispatcher
+# ------------------------------------------------------------------------------
+case "${1:-}" in
+    --deploy)
+        do_deploy
+        ;;
+    --update)
+        do_update
+        ;;
+    --remove)
+        do_remove
+        ;;
+    *)
+        echo -e "${CYAN}VM Bench Stack Lifecycle Manager (${HOSTNAME})${NC}"
+        echo "Usage: $0 {--deploy|--update|--remove}"
+        echo "  --deploy : Interactive installation, storage selection, and deployment"
+        echo "  --update : Pull latest repository changes and refresh stack services"
+        echo "  --remove : Purge LXC container, host systemd backend, and storage directories"
+        exit 1
+        ;;
+esac