chore: add AGENTS.md, move install.sh to root, rename frontend install.sh → setup.sh

This commit is contained in:
Claus Lohmar 2026-07-25 17:40:59 +00:00
parent ef2d74a62b
commit f7c6527d3d
5 changed files with 622 additions and 349 deletions

214
AGENTS.md Normal file
View file

@ -0,0 +1,214 @@
# AGENTS.md — VM Bench (VMware → Proxmox Image Converter)
## Project Overview
VM Bench is a two-tier web application for converting VMware disk images (VMDK, VHD,
etc.) into Proxmox-compatible QCOW2 disks and provisioning VMs with automatic OS and
boot type detection.
- **Repository**: `https://git.lohmar.co.uk/cclohmar/vm-bench.git`
- **Deployment root**: `/mnt/converter/` (bind-mounted between host and LXC)
- **Frontend**: FastAPI + Jinja2 web UI in an LXC container (`vm-bench`) on port 5000
- **Backend**: FastAPI REST API on the Proxmox host (`srv2`) on port 9000
- **API spec**: `open-api.yaml` (single source of truth — derive models from this)
- **Python version**: 3.13 (backend), 3.11 (frontend LXC)
## Architecture
```
Browser vm-bench LXC Proxmox Host (srv2)
(this container)
──→ :5000 ──→ frontend/app.py ──→ :9000 backend/app.py
├─ qemu-img (format, size)
├─ guestfish (OS, EFI detection)
├─ 7z/unzip (archive extraction)
└─ qm create (VM provisioning)
```
- **Frontend** (`vm-bench` LXC) — user-facing web UI, manages sessions, proxies API calls
- **Backend** (Proxmox host) — disk conversion, analysis, VM provisioning
- **Shared storage**`/mnt/converter/tmp/` (staging), `/mnt/converter/logs/` (logs)
## Directory Layout
```
/mnt/converter/
├── install.sh # Main deployment script (host-level)
├── clean.sh # Clean tmp dirs + restart backend
├── open-api.yaml # API specification (single source of truth)
├── README.md
├── AGENTS.md # This file
├── backend/ # Backend API (runs on Proxmox host)
│ ├── app.py # FastAPI app, routes, logging
│ ├── models.py # Pydantic request/response models
│ ├── converter.py # Archive extraction, disk probing, OS/EFI detection
│ ├── provisioner.py # VM provisioning (qm create/importdisk/convert)
│ ├── requirements.txt # fastapi, uvicorn, pydantic
│ └── vm-bench-backend.service # Systemd unit
├── frontend/ # Web UI (runs on vm-bench LXC)
│ ├── app.py # FastAPI app, routes, download/SCP handling
│ ├── api_client.py # Typed REST client → backend
│ ├── setup.sh # Frontend installer (inside LXC)
│ ├── requirements.txt # fastapi, uvicorn, jinja2, requests, python-multipart
│ ├── vm-bench.service # Systemd unit
│ ├── templates/ # Jinja2 HTML templates
│ │ ├── base.html # Base layout with session GUID + header/footer
│ │ ├── index.html # New session form + download + progress
│ │ ├── _analysis.html # Analysis result + confirm form (HTML fragment)
│ │ ├── polling.html # Progress bar + job status + reuse section
│ │ └── scp.html # SCP pull page
│ └── static/
│ └── proxmox.css # Proxmox VE-inspired CSS theme
├── logs/ # Shared log files (gitignored)
├── tmp/ # Staging files (gitignored)
└── venv/ # Backend Python venv (gitignored)
```
## Key Files by Responsibility
### Backend (`backend/`)
| File | Purpose |
|------|---------|
| `backend/app.py:1` | FastAPI app with routes: `/api/v1/health`, `/analyze`, `/jobs`, `/jobs/{id}`, `/jobs/{id}/cleanup`, `/clone` |
| `backend/models.py:1` | Pydantic models: `JobSubmissionRequest`, `JobStatusResponse`, `AnalyzeRequest`, `AnalyzeResponse`, `DiskSpec`, `CleanupRequest`, `CloneRequest` |
| `backend/converter.py:1` | Archive extraction (`extract_if_needed`), disk discovery (`discover_disk`), OS detection (`detect_os`), EFI detection (`detect_efi`) |
| `backend/provisioner.py:1` | Background job processor: `submit_job`, `get_job_status`, `cleanup_staging`, `clone_vm`, `_process_job` (qemu-img convert → virt-resize → qm create → qm importdisk) |
### Frontend (`frontend/`)
| File | Purpose |
|------|---------|
| `frontend/app.py:1` | FastAPI app with routes: `/`, `/session/upload`, `/session/progress/{f}`, `/session/analyze`, `/session/confirm`, `/session/status/{id}`, `/session/cleanup/{id}`, `/session/clone`, `/scp`, `/scp/start`, `/scp/progress/{sid}/{f}` |
| `frontend/api_client.py:1` | `ApiClient` class — typed HTTP client for backend API with `ApiError` exception |
### Deployment (`install.sh`)
| File | Purpose |
|------|---------|
| `install.sh:1` | Master deployment script (Proxmox host). Modes: `--deploy`, `--update`, `--remove`. Creates directory tree, clones repo, installs system packages (qemu-img, guestfish, archive tools), Python venv, backend systemd service, LXC container with bind mount. |
| `frontend/setup.sh:1` | Frontend installer (run inside LXC). Installs system packages (python3, aria2, openssh-client, sshpass), Python deps, systemd service. |
### Systemd Units
| File | Port | Env Vars |
|------|------|----------|
| `backend/vm-bench-backend.service` | 9000 | None |
| `frontend/vm-bench.service` | 5000 | `BACKEND_URL=http://10.2.0.2:9000`, `TMPDIR=/mnt/converter/tmp` |
## Usage Flow
1. User opens web UI → session form
2. Upload or download a source image → lands in `/mnt/converter/tmp/{session_id}/in/`
3. Backend analyses image → shows format, size, OS, EFI status
4. Configure VM settings → name, CPU, RAM, storage, boot type
5. Submit job → backend converts, shrinks (if needed), creates VM
6. Poll progress → real-time status bar updates every 2 seconds via AJAX
7. Reuse or cleanup → create another VM or delete staging files
## Disk Sizing & Auto-Shrink Logic
- If `target_disk_size_gb` is omitted and the source disk is larger than its
auto-shrink target (10% of virtual size, minimum 20 GB), the disk is shrunk
using `virt-resize --shrink --resize-force`.
- If `virt-resize` fails, falls back to `qemu-img resize --shrink`.
- If `target_disk_size_gb` is larger than current, disk is expanded.
## Session Management
- Frontend generates a UUID session ID stored in `localStorage` + cookie (`vm_bench_sid`)
- Files are staged at `/mnt/converter/tmp/{session_id}/in/` and `out/`
- Cleanup can delete the entire session directory or preserve for reuse
## Source Image Ingestion Methods
1. **Download URL** (`index.html`) — uses `aria2c` with 8 connections, kills download if ETA > 1 hour
2. **SCP Pull** (`scp.html`) — uses `sshpass` + `scp` directly from remote servers
3. File upload (legacy, referenced but not primary flow)
## Backend Tools Required (on Proxmox host)
- `qemu-img` — disk format and size detection, format conversion
- `guestfish` / `virt-inspector` — OS type and EFI boot detection
- `virt-resize` — safe disk shrinking
- `7z`, `unzip`, `unrar`, `tar`, `gunzip`, `bunzip2`, `xz` — archive extraction
- `qm` (Proxmox CLI) — VM creation, disk import, configuration
## Frontend Tools Required (in LXC)
- `python3`, `pip3`, `wget`, `curl`, `aria2c`, `openssh-client`, `sshpass`
## Configuration
| Variable | Default | Description |
|----------|---------|-------------|
| `BACKEND_URL` | `http://10.2.0.2:9000` | Backend API base URL (set in systemd unit + env) |
| `VM_ID_MIN` | `21000` | Minimum allowed Proxmox VM ID |
| `VM_ID_MAX` | `21100` | Maximum allowed Proxmox VM ID |
| `DOWNLOAD_TIMEOUT` | `14400` (4h) | Max download time (safety net) |
| `SPEED_CHECK_AFTER` | `30` (seconds) | Wait before judging download speed |
| `MAX_ETA_SECONDS` | `3600` (1h) | Kill download if ETA exceeds this |
## Job States
```
queued → processing_conversion → importing_storage → completed
→ failed
```
## Logging
Both services log to console (systemd journal) and rotating file handler:
- Frontend: `/mnt/converter/logs/vm-bench.log`
- Backend: `/mnt/converter/logs/vm-bench-backend.log`
- Rotation: 10 MB max, 5 backup files
## Development Commands
### Backend (on Proxmox host)
```bash
# Install/update
bash install.sh --deploy
# Restart
systemctl restart vm-bench-backend
# View logs
tail -f /mnt/converter/logs/vm-bench-backend.log
journalctl -u vm-bench-backend -f
# Health check
curl http://127.0.0.1:9000/api/v1/health
```
### Frontend (inside LXC)
```bash
# Install/update
bash /mnt/converter/frontend/setup.sh
# Restart
systemctl restart vm-bench
# View logs
tail -f /mnt/converter/logs/vm-bench.log
journalctl -u vm-bench -f
```
### Cleanup script
```bash
bash /mnt/converter/clean.sh # Clears tmp dirs + restarts backend
```
## Project Conventions
- **No code comments** unless strictly necessary (follow existing patterns)
- Models derived from `open-api.yaml` — update spec first, then code
- `backend/provisioner.py` uses in-memory job store (`_jobs` dict) with `threading.Lock`
- All paths use `pathlib.Path`
- Loggers use module-level naming: `logging.getLogger("backend")`, `logging.getLogger("backend.converter")`, etc.
- Frontend templates use `pve-` CSS prefix mimicking Proxmox VE design system
- HTML fragments (e.g. `_analysis.html`) are injected via `innerHTML` — no `<script>` tags
- Session-based isolation: each browser tab/session gets a UUID for file staging
- VM names are sanitized to DNS-safe characters: `[^a-zA-Z0-9-]` replaced with `-`
- VM ID range is validated client-side (2100021100) and enforced in frontend routes

View file

@ -1,344 +0,0 @@
#!/bin/bash
set -e
# ╔════════════════════════════════════════════════════════════════════╗
# ║ VM Bench — Full Deployment Script (run on Proxmox host srv2) ║
# ║ ║
# ║ Usage: bash /mnt/converter/backend/install.sh ║
# ║ ║
# ║ Does everything: ║
# ║ 1. Creates /mnt/converter/ directory tree ║
# ║ 2. Clones (or pulls) the git repository ║
# ║ 3. Installs system packages + Python deps + systemd service ║
# ║ 4. Prompts for container ID + network config ║
# ║ 5. Creates the vm-bench LXC container with bind mount ║
# ╚════════════════════════════════════════════════════════════════════╝
# ───────────────────────────────────────────────────────────────────
# Colours & paths
# ───────────────────────────────────────────────────────────────────
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[0;33m'; CYAN='\033[0;36m'; NC='\033[0m'
REPO_URL="https://git.lohmar.co.uk/cclohmar/vm-bench.git"
CONVERTER_ROOT="/mnt/converter"
SCRIPT_DIR="${CONVERTER_ROOT}/backend"
SERVICE_NAME="vm-bench-backend"
SERVICE_FILE="${SCRIPT_DIR}/${SERVICE_NAME}.service"
SYSTEMD_DIR="/etc/systemd/system"
# ───────────────────────────────────────────────────────────────────
# Banner
# ───────────────────────────────────────────────────────────────────
echo ""
echo "╔═════════════════════════════════════════════════════════╗"
echo "║ VM Bench — Full Deployment Installer ║"
echo "║ Target: Proxmox host (srv2) ║"
echo "╚═════════════════════════════════════════════════════════╝"
echo ""
# ───────────────────────────────────────────────────────────────────
# Pre-flight: must run on a Proxmox host
# ───────────────────────────────────────────────────────────────────
echo "[preflight] Checking environment..."
if ! command -v pct &>/dev/null; then
echo " ${RED}${NC} 'pct' not found — this script must run on a Proxmox host."
exit 1
fi
if ! command -v git &>/dev/null; then
echo " Installing git..."
apt-get update -qq && apt-get install -y -qq git
fi
if ! command -v qemu-img &>/dev/null; then
echo " ${RED}${NC} 'qemu-img' not found — Proxmox ships this. Do NOT install qemu-utils (conflicts with Proxmox)."
exit 1
fi
echo " ${GREEN}${NC} Running on Proxmox host."
# ───────────────────────────────────────────────────────────────────
# Step 1: Create directory tree
# ───────────────────────────────────────────────────────────────────
echo ""
echo "[1/6] Creating directory structure..."
for d in \
"$CONVERTER_ROOT" \
"$CONVERTER_ROOT/logs" \
"$CONVERTER_ROOT/tmp" \
"$CONVERTER_ROOT/backend" \
"$CONVERTER_ROOT/frontend" \
"$CONVERTER_ROOT/frontend/templates" \
"$CONVERTER_ROOT/frontend/static"; do
if [ ! -d "$d" ]; then
mkdir -p "$d"
echo " Created: $d"
fi
done
echo " ${GREEN}${NC} Directories ready."
# ───────────────────────────────────────────────────────────────────
# Step 2: Clone or update the git repository
# ───────────────────────────────────────────────────────────────────
echo ""
echo "[2/6] Cloning / updating repository..."
if [ -d "$CONVERTER_ROOT/.git" ]; then
echo " Existing repo found, pulling latest..."
git -C "$CONVERTER_ROOT" pull --ff-only
echo " ${GREEN}${NC} Repository updated."
else
# Clone into a temp location, then move if /mnt/converter is not empty
if [ -z "$(ls -A "$CONVERTER_ROOT" 2>/dev/null)" ]; then
git clone "$REPO_URL" "$CONVERTER_ROOT"
else
echo " ${YELLOW}!${NC} ${CONVERTER_ROOT} is not empty — cloning into temp and copying..."
TMP_CLONE=$(mktemp -d)
git clone "$REPO_URL" "$TMP_CLONE"
cp -a "$TMP_CLONE"/. "$CONVERTER_ROOT"/
rm -rf "$TMP_CLONE"
fi
echo " ${GREEN}${NC} Repository cloned."
fi
# ───────────────────────────────────────────────────────────────────
# Step 3: System packages
# ───────────────────────────────────────────────────────────────────
echo ""
echo "[3/6] Checking system packages..."
PACKAGES=()
REQUIRED=(
python3 "python3"
python3-pip "pip3"
libguestfs-tools "guestfish"
p7zip-full "7z"
unzip "unzip"
unrar-free "unrar"
tar "tar"
gzip "gunzip"
bzip2 "bunzip2"
xz-utils "xz"
)
for ((i=0; i<${#REQUIRED[@]}; i+=2)); do
pkg="${REQUIRED[$i]}"
bin="${REQUIRED[$i+1]}"
if command -v "$bin" &>/dev/null; then
echo " ${GREEN}${NC} $bin"
else
echo " ${YELLOW}${NC} $bin (will install ${pkg})"
PACKAGES+=("$pkg")
fi
done
if [ ${#PACKAGES[@]} -gt 0 ]; then
echo ""
echo " Installing: ${PACKAGES[*]}"
apt-get update -qq
apt-get install -y -qq "${PACKAGES[@]}"
echo " ${GREEN}${NC} System packages installed."
else
echo " ${GREEN}${NC} All system packages present."
fi
# ───────────────────────────────────────────────────────────────────
# Step 4: Python dependencies + systemd service
# ───────────────────────────────────────────────────────────────────
echo ""
echo "[4/6] Installing Python dependencies..."
cd "$SCRIPT_DIR"
python3 -m pip install --break-system-packages -r requirements.txt -q
echo " ${GREEN}${NC} Python dependencies installed."
echo ""
echo "[5/6] Installing systemd service..."
if [ ! -f "$SERVICE_FILE" ]; then
echo " ${RED}${NC} Service file not found: $SERVICE_FILE"
echo " Make sure the repository was cloned correctly."
exit 1
fi
cp "$SERVICE_FILE" "$SYSTEMD_DIR/${SERVICE_NAME}.service"
systemctl daemon-reload
systemctl enable "$SERVICE_NAME"
systemctl restart "$SERVICE_NAME"
sleep 2
if systemctl is-active --quiet "$SERVICE_NAME"; then
echo " ${GREEN}${NC} Backend is RUNNING on port 9000"
echo " Verify: curl http://127.0.0.1:9000/api/v1/health"
else
echo " ${RED}${NC} Backend failed to start. Check logs:"
echo " journalctl -u ${SERVICE_NAME} -n 30"
exit 1
fi
# ───────────────────────────────────────────────────────────────────
# Step 6: Create the vm-bench LXC container
# ───────────────────────────────────────────────────────────────────
echo ""
echo "╔═════════════════════════════════════════════════════════╗"
echo "║ LXC Container Setup (vm-bench frontend) ║"
echo "╚═════════════════════════════════════════════════════════╝"
echo ""
# ── Prompt for container ID ──────────────────────────────────────
read -p " Container ID (e.g. 20020): " CTID
if [ -z "$CTID" ]; then
echo " ${RED}${NC} Container ID is required."
exit 1
fi
# Check if ID already exists
if pct status "$CTID" &>/dev/null; then
echo " ${YELLOW}!${NC} Container $CTID already exists."
read -p " Overwrite? [y/N]: " OVERWRITE
if [ "$OVERWRITE" != "y" ] && [ "$OVERWRITE" != "Y" ]; then
echo " Aborted."
exit 0
fi
echo " Destroying existing container $CTID..."
pct stop "$CTID" --skiplock 2>/dev/null || true
pct destroy "$CTID" --purge 2>/dev/null || true
fi
# ── Network settings ─────────────────────────────────────────────
echo ""
echo " ${CYAN}Network configuration:${NC}"
read -p " IP address/CIDR [10.2.0.20/16]: " CT_IP
read -p " Gateway [10.2.0.2]: " CT_GW
read -p " Bridge [vmbr0]: " CT_BRIDGE
read -p " MAC address [BC:24:11:80:5A:B1]: " CT_MAC
CT_IP="${CT_IP:-10.2.0.20/16}"
CT_GW="${CT_GW:-10.2.0.2}"
CT_BRIDGE="${CT_BRIDGE:-vmbr0}"
CT_MAC="${CT_MAC:-BC:24:11:80:5A:B1}"
NET_CFG="name=eth0,bridge=${CT_BRIDGE},firewall=1,gw=${CT_GW},hwaddr=${CT_MAC},ip=${CT_IP},type=veth"
ROOTFS="local-lvm:vm-${CTID}-disk-0,size=24G"
# ── Storage path ──────────────────────────────────────────────────
echo ""
echo " ${CYAN}Storage mount:${NC}"
echo " Path on the Proxmox host where VM Bench stores its data."
echo " The LXC will see this as /mnt/converter internally."
read -p " Host storage path [/mnt/pve/hosted-thin]: " STORAGE_PATH
STORAGE_PATH="${STORAGE_PATH:-/mnt/pve/hosted-thin}"
# Remove trailing slash
STORAGE_PATH="${STORAGE_PATH%/}"
# Create the directory on the host if it doesn't exist
if [ ! -d "$STORAGE_PATH" ]; then
echo " Creating: $STORAGE_PATH"
mkdir -p "$STORAGE_PATH"
fi
# ── Template selection ────────────────────────────────────────────
echo ""
echo " ${CYAN}Available Debian templates:${NC}"
pveam available 2>/dev/null | grep debian-12 || echo " (none found locally, will attempt to download)"
TEMPLATE=$(pveam available 2>/dev/null | grep -m1 "debian-12.*amd64" | awk '{print $2}')
if [ -z "$TEMPLATE" ]; then
# Try to find any available Debian template
TEMPLATE=$(pveam available 2>/dev/null | grep -m1 "debian.*amd64" | awk '{print $2}')
fi
if [ -z "$TEMPLATE" ]; then
echo " ${YELLOW}!${NC} No Debian template found. Downloading debian-12-standard..."
pveam update 2>/dev/null || true
pveam download local debian-12-standard_12.7-1_amd64.tar.zst 2>/dev/null || true
TEMPLATE="local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst"
else
# Check if template is downloaded locally
TEMPLATE_NAME=$(basename "$TEMPLATE")
if [ ! -f "/var/lib/vz/template/cache/${TEMPLATE_NAME}" ]; then
echo " Downloading template: ${TEMPLATE_NAME}..."
pveam download local "$TEMPLATE_NAME" 2>/dev/null || {
echo " ${YELLOW}!${NC} Auto-download failed. Trying to continue anyway..."
}
fi
TEMPLATE="local:vztmpl/${TEMPLATE_NAME}"
fi
echo " Using template: ${TEMPLATE}"
# ── Create container ──────────────────────────────────────────────
echo ""
echo " Creating container ${CTID}..."
pct create "$CTID" "$TEMPLATE" \
--arch amd64 \
--cores 3 \
--hostname vm-bench \
--memory 6144 \
--net0 "$NET_CFG" \
--ostype debian \
--rootfs "$ROOTFS" \
--swap 2048 \
--unprivileged 0 \
--features nesting=1 \
--mp0 ${STORAGE_PATH},mp=/mnt/converter \
--onboot 1 \
--start 0
echo " ${GREEN}${NC} Container created."
# ── Write LXC config overrides ────────────────────────────────────
CONF_FILE="/etc/pve/lxc/${CTID}.conf"
echo ""
echo " Writing LXC config overrides..."
# Add lines only if not already present
add_conf_line() {
local key="$1"
local val="$2"
if ! grep -q "^${key}:" "$CONF_FILE" 2>/dev/null; then
echo "${key}: ${val}" >> "$CONF_FILE"
echo " + ${key}: ${val}"
fi
}
add_conf_line "lxc.cgroup2.devices.allow" "a"
add_conf_line "lxc.mount.auto" "proc:rw sys:rw cgroup:rw"
add_conf_line "lxc.apparmor.profile" "unconfined"
echo " ${GREEN}${NC} LXC config written."
# ── Start the container ───────────────────────────────────────────
echo ""
read -p " Start container now? [Y/n]: " START
if [ "$START" != "n" ] && [ "$START" != "N" ]; then
pct start "$CTID"
sleep 3
if pct status "$CTID" | grep -q running; then
echo " ${GREEN}${NC} Container ${CTID} is running."
echo ""
echo " ${CYAN}Next steps:${NC}"
echo " 1. Enter the container: pct enter ${CTID}"
echo " 2. Run the frontend installer:"
echo " bash /mnt/converter/frontend/install.sh"
echo ""
echo " 3. Open the web UI: http://${CT_IP%/*}:5000"
echo ""
else
echo " ${YELLOW}!${NC} Container may not have started. Check: pct status ${CTID}"
fi
fi
# ───────────────────────────────────────────────────────────────────
# Done
# ───────────────────────────────────────────────────────────────────
echo ""
echo "╔═════════════════════════════════════════════════════════╗"
echo "${GREEN}Deployment complete.${NC}"
echo "║ ║"
echo "║ Backend: http://$(hostname -I | awk '{print $1}'):9000 ║"
echo "║ Frontend: http://${CT_IP%/*}:5000 ║"
echo "║ ║"
echo "║ Backend logs: journalctl -u ${SERVICE_NAME} -f ║"
echo "║ Container: pct enter ${CTID}"
echo "╚═════════════════════════════════════════════════════════╝"
echo ""

View file

@ -1,15 +1,16 @@
[Unit]
Description=VM Bench Backend — Proxmox Image Conversion Engine
Description=VM Bench Converter Backend Service
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/mnt/converter/backend
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ExecStart=/usr/bin/python3 -m uvicorn app:app --host 0.0.0.0 --port 9000 --timeout-keep-alive 300
WorkingDirectory=/mnt/pve/host-shared/converter/backend
ExecStart=/mnt/pve/host-shared/converter/venv/bin/python /mnt/pve/host-shared/converter/backend/app.py
Restart=always
RestartSec=3
RestartSec=5
StandardOutput=append:/mnt/pve/host-shared/converter/logs/vm-bench-backend.log
StandardError=append:/mnt/pve/host-shared/converter/logs/vm-bench-backend.log
[Install]
WantedBy=multi-user.target

402
install.sh Executable file
View file

@ -0,0 +1,402 @@
#!/usr/bin/env bash
# ==============================================================================
# VM Bench Stack Provisioning & Lifecycle Script
# Target: Proxmox VE node (any version)
# Usage:
# ./install.sh --deploy [Interactive Setup & Deployment]
# ./install.sh --update [Pull Updates & Restart Services]
# ./install.sh --remove [Teardown Everything]
# ==============================================================================
set -euo pipefail
# Uncomment next line for debugging:
# set -x
# ------------------------------------------------------------------------------
# Formatting & Constants
# ------------------------------------------------------------------------------
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
CYAN='\033[0;36m'
NC='\033[0m'
REPO_URL="https://git.lohmar.co.uk/cclohmar/vm-bench.git"
SYSTEMD_DIR="/etc/systemd/system"
SERVICE_NAME="vm-bench-backend"
CT_CORES=1
CT_MEMORY=512
CT_SWAP=512
CT_DISK_SIZE=8
# ------------------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------------------
has_cmd() {
command -v "$1" >/dev/null 2>&1
}
get_default_bridge() {
ip -o link show type bridge | awk -F': ' '{print $2; exit}' 2>/dev/null || echo "vmbr0"
}
guess_gateway_from_bridge() {
local bridge="$1"
local ip_cidr
ip_cidr=$(ip -o -4 addr show dev "$bridge" 2>/dev/null | awk '{print $4; exit}')
if [[ -n "$ip_cidr" ]]; then
local ip="${ip_cidr%/*}"
local prefix="${ip%.*}"
echo "${prefix}.1"
else
echo ""
fi
}
# ------------------------------------------------------------------------------
# Environment Discovery & User Parameter Gathering
# ------------------------------------------------------------------------------
gather_environment_info() {
echo -e "${CYAN}==> [1/3] Environment Inspection & Parameter Collection${NC}"
# ---- Storage Selection ----
echo ""
echo "Inspecting available storage pools from /etc/pve/storage.cfg..."
declare -a storage_ids=()
declare -a storage_contents=()
declare -a storage_has_rootdir=()
# Parse storage.cfg for container-capable storage types
if [ -f /etc/pve/storage.cfg ]; then
# Extract storage IDs for types: lvmthin, zfspool, rbd, dir
while IFS= read -r id; do
storage_ids+=("$id")
# Get content for this pool (ignore errors)
local content
content=$(pvesm config "$id" 2>/dev/null | awk '/^content/ {$1=""; print $0}' | xargs | tr ' ' ',' || true)
storage_contents+=("${content:-none}")
if [[ "$content" == *"rootdir"* ]]; then
storage_has_rootdir+=("yes")
else
storage_has_rootdir+=("no")
fi
done < <(grep -E "^(lvmthin|zfspool|rbd|dir):" /etc/pve/storage.cfg | awk '{print $2}')
fi
# Fallback: if no storage found in config, use active pools from pvesm
if [ ${#storage_ids[@]} -eq 0 ]; then
echo -e "${YELLOW}No storage definitions found in /etc/pve/storage.cfg. Falling back to active pools from 'pvesm status'.${NC}"
while IFS= read -r pool; do
storage_ids+=("$pool")
local content
content=$(pvesm config "$pool" 2>/dev/null | awk '/^content/ {$1=""; print $0}' | xargs | tr ' ' ',' || true)
storage_contents+=("${content:-none}")
if [[ "$content" == *"rootdir"* ]]; then
storage_has_rootdir+=("yes")
else
storage_has_rootdir+=("no")
fi
done < <(pvesm status 2>/dev/null | awk 'NR>1 && $3=="active" {print $1}' || true)
fi
# If still empty, abort
if [ ${#storage_ids[@]} -eq 0 ]; then
echo -e "${RED}No usable storage pools found. Please create a storage pool (e.g., 'local' directory) and try again.${NC}"
exit 1
fi
# Show menu
echo "Available Storage Options Detected:"
for i in "${!storage_ids[@]}"; do
id="${storage_ids[$i]}"
content="${storage_contents[$i]}"
marker=""
[ "${storage_has_rootdir[$i]}" = "yes" ] && marker=" ✓ (has rootdir)"
printf " [%d] %-18s (content: %s)%s\n" "$((i+1))" "$id" "$content" "$marker"
done
# Pick default: first with rootdir, else first
default_idx=0
for i in "${!storage_ids[@]}"; do
if [ "${storage_has_rootdir[$i]}" = "yes" ]; then
default_idx=$i
break
fi
done
default_num=$((default_idx+1))
read -p "Select storage pool [1-${#storage_ids[@]}] (default: ${default_num}): " POOL_CHOICE
POOL_CHOICE="${POOL_CHOICE:-$default_num}"
if [[ "$POOL_CHOICE" =~ ^[0-9]+$ ]] && [ "$POOL_CHOICE" -ge 1 ] && [ "$POOL_CHOICE" -le "${#storage_ids[@]}" ]; then
TARGET_STORAGE="${storage_ids[$((POOL_CHOICE-1))]}"
TARGET_HAS_ROOTDIR="${storage_has_rootdir[$((POOL_CHOICE-1))]}"
else
echo -e "${YELLOW}Invalid choice. Defaulting to '${storage_ids[$default_idx]}'.${NC}"
TARGET_STORAGE="${storage_ids[$default_idx]}"
TARGET_HAS_ROOTDIR="${storage_has_rootdir[$default_idx]}"
fi
# Ensure rootdir capability
if [ "$TARGET_HAS_ROOTDIR" != "yes" ]; then
echo "Enabling 'rootdir' capability on storage pool '${TARGET_STORAGE}'..."
current_content="${storage_contents[$((POOL_CHOICE-1))]:-}"
if [ -n "$current_content" ]; then
new_content="${current_content},rootdir"
else
new_content="rootdir"
fi
pvesm set "$TARGET_STORAGE" --content "$new_content" || {
echo -e "${RED}Failed to add rootdir. Please enable it manually.${NC}"
exit 1
}
fi
# ---- Path & Container Configuration ----
echo ""
read -p "Base Storage Path [/mnt/pve/host-shared/converter]: " PATH_INPUT
STORAGE_PATH="${PATH_INPUT:-/mnt/pve/host-shared/converter}"
STORAGE_PATH="${STORAGE_PATH%/}"
read -p "Container ID [20020]: " CTID_INPUT
CTID="${CTID_INPUT:-20020}"
DEFAULT_BRIDGE=$(get_default_bridge)
read -p "Network Bridge [default: ${DEFAULT_BRIDGE}]: " BRIDGE_INPUT
CT_BRIDGE="${BRIDGE_INPUT:-$DEFAULT_BRIDGE}"
SUGGESTED_GW=$(guess_gateway_from_bridge "$CT_BRIDGE")
[ -z "$SUGGESTED_GW" ] && SUGGESTED_GW="10.2.0.1"
read -p "Container IP/CIDR (e.g., 10.2.0.20/16): " CT_IP
while [[ -z "$CT_IP" ]]; do
echo -e "${RED}Error: Static IP with CIDR mask is required (e.g., 10.2.0.20/16).${NC}"
read -p "Container IP/CIDR: " CT_IP
done
read -p "Gateway IP [${SUGGESTED_GW}]: " GW_INPUT
CT_GW="${GW_INPUT:-$SUGGESTED_GW}"
read -p "MAC Address (Press Enter to auto-generate): " MAC_INPUT
CT_MAC="${MAC_INPUT:-}"
echo ""
echo -e "${GREEN}✓ Environment configuration gathered successfully.${NC}"
}
# ------------------------------------------------------------------------------
# Action Functions (Deploy, Update, Remove)
# ------------------------------------------------------------------------------
do_deploy() {
echo -e "${GREEN}==> Beginning Deployment on Host $(hostname)...${NC}"
if ! command -v pct &>/dev/null; then
echo -e "${RED}Error: 'pct' CLI not found. Script must be executed on a Proxmox host.${NC}"
exit 1
fi
gather_environment_info
REAL_PATH=$(readlink -f "$STORAGE_PATH" 2>/dev/null || echo "$STORAGE_PATH")
# Clean previous state
if [ -d "$REAL_PATH" ] || pct status "$CTID" &>/dev/null; then
echo -e "${YELLOW}! Cleaning pre-existing installation state at ${REAL_PATH} / CTID ${CTID}...${NC}"
pct stop "$CTID" --skiplock 2>/dev/null || true
pct destroy "$CTID" --purge 2>/dev/null || true
systemctl stop "$SERVICE_NAME" 2>/dev/null || true
rm -rf "$REAL_PATH"
fi
# Fix APT sources for Proxmox (no subscription)
rm -f /etc/apt/sources.list.d/pve-enterprise.sources \
/etc/apt/sources.list.d/ceph.sources \
/etc/apt/sources.list.d/pve-enterprise.list \
/etc/apt/sources.list.d/pve-no-subscription.list 2>/dev/null || true
cat <<'EOF' > /etc/apt/sources.list.d/proxmox-no-sub.sources
Types: deb
URIs: http://download.proxmox.com/debian/pve
Suites: trixie
Components: pve-no-subscription
Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg
EOF
apt-get update -qq
echo -e "${CYAN}==> [2/3] Provisioning Directory Structure & Backend Service${NC}"
mkdir -p "$REAL_PATH"
echo "Cloning source code from ${REPO_URL}..."
git clone "$REPO_URL" "$REAL_PATH"
CONVERTER_ROOT="$REAL_PATH"
SCRIPT_DIR="${CONVERTER_ROOT}/backend"
SERVICE_FILE="${SCRIPT_DIR}/${SERVICE_NAME}.service"
mkdir -p "${CONVERTER_ROOT}/logs" "${CONVERTER_ROOT}/tmp"
chmod 777 "${CONVERTER_ROOT}/tmp"
apt-get install -y -qq --no-install-recommends \
python3 python3-pip python3-venv libguestfs-tools \
p7zip-full unzip unrar-free tar gzip bzip2 xz-utils git
VENV_DIR="${CONVERTER_ROOT}/venv"
echo "Setting up Python virtual environment..."
python3 -m venv "$VENV_DIR"
"$VENV_DIR/bin/pip" install --upgrade pip -q
if [ -f "${SCRIPT_DIR}/requirements.txt" ]; then
"$VENV_DIR/bin/pip" install -r "${SCRIPT_DIR}/requirements.txt" -q
fi
cat <<EOF > "$SERVICE_FILE"
[Unit]
Description=VM Bench Converter Backend Service
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=${SCRIPT_DIR}
ExecStart=${VENV_DIR}/bin/python ${SCRIPT_DIR}/app.py
Restart=always
RestartSec=5
StandardOutput=append:${CONVERTER_ROOT}/logs/vm-bench-backend.log
StandardError=append:${CONVERTER_ROOT}/logs/vm-bench-backend.log
[Install]
WantedBy=multi-user.target
EOF
cp "$SERVICE_FILE" "${SYSTEMD_DIR}/${SERVICE_NAME}.service"
systemctl daemon-reload
systemctl enable --now "$SERVICE_NAME"
echo -e "${GREEN}✓ Host Backend systemd service initialized and running.${NC}"
echo -e "${CYAN}==> [3/3] Creating and Starting Frontend LXC Container (${CTID})${NC}"
# Get LXC template
TEMPLATE=$(pveam available 2>/dev/null | grep -m1 "debian-12.*amd64" | awk '{print $2}' || true)
if [ -z "$TEMPLATE" ]; then
pveam update 2>/dev/null || true
pveam download local debian-12-standard_12.7-1_amd64.tar.zst 2>/dev/null || true
TEMPLATE="local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst"
else
TEMPLATE_NAME=$(basename "$TEMPLATE")
[ ! -f "/var/lib/vz/template/cache/${TEMPLATE_NAME}" ] && pveam download local "$TEMPLATE_NAME" 2>/dev/null || true
TEMPLATE="local:vztmpl/${TEMPLATE_NAME}"
fi
NET_CFG="name=eth0,bridge=${CT_BRIDGE},firewall=1,gw=${CT_GW},ip=${CT_IP},type=veth"
[ -n "$CT_MAC" ] && NET_CFG="${NET_CFG},hwaddr=${CT_MAC}"
pct create "$CTID" "$TEMPLATE" \
--arch amd64 \
--cores "$CT_CORES" \
--hostname vm-bench \
--memory "$CT_MEMORY" \
--net0 "$NET_CFG" \
--ostype debian \
--storage "$TARGET_STORAGE" \
--rootfs "${TARGET_STORAGE}:${CT_DISK_SIZE}" \
--swap "$CT_SWAP" \
--unprivileged 0 \
--features nesting=1 \
--mp0 "${REAL_PATH},mp=/mnt/converter" \
--onboot 1 \
--start 0
# Extra LXC config
CONF_FILE="/etc/pve/lxc/${CTID}.conf"
echo "lxc.cgroup2.devices.allow: a" >> "$CONF_FILE"
echo "lxc.mount.auto: proc:rw sys:rw cgroup:rw" >> "$CONF_FILE"
echo "lxc.apparmor.profile: unconfined" >> "$CONF_FILE"
pct start "$CTID" 2>/dev/null || true
echo ""
echo -e "${GREEN}======================================================================${NC}"
echo -e "${GREEN} Deployment Successful on $(hostname)!${NC}"
echo " - Host Backend API : http://127.0.0.1:9000"
echo " - LXC Container IP : ${CT_IP%/*}"
echo " - Container Storage : ${TARGET_STORAGE}"
echo " - Mount Directory : ${REAL_PATH} -> /mnt/converter"
echo -e "${GREEN}======================================================================${NC}"
}
do_update() {
echo -e "${CYAN}==> Updating VM Bench Stack...${NC}"
read -p "Base Storage Path [/mnt/pve/host-shared/converter]: " PATH_INPUT
STORAGE_PATH="${PATH_INPUT:-/mnt/pve/host-shared/converter}"
REAL_PATH=$(readlink -f "$STORAGE_PATH" 2>/dev/null || echo "$STORAGE_PATH")
read -p "Container ID [20020]: " CTID_INPUT
CTID="${CTID_INPUT:-20020}"
if [ -d "$REAL_PATH/.git" ]; then
echo "Pulling latest code from Git..."
git -C "$REAL_PATH" pull --ff-only
fi
if [ -d "${REAL_PATH}/venv" ]; then
echo "Updating Python environment..."
"${REAL_PATH}/venv/bin/pip" install --upgrade pip -q
[ -f "${REAL_PATH}/backend/requirements.txt" ] && "${REAL_PATH}/venv/bin/pip" install -r "${REAL_PATH}/backend/requirements.txt" -q
fi
echo "Restarting Services..."
systemctl restart "$SERVICE_NAME"
pct restart "$CTID" 2>/dev/null || true
echo -e "${GREEN}✓ Stack update complete.${NC}"
}
do_remove() {
echo -e "${YELLOW}==> Tearing Down VM Bench Stack...${NC}"
read -p "Base Storage Path [/mnt/pve/host-shared/converter]: " PATH_INPUT
STORAGE_PATH="${PATH_INPUT:-/mnt/pve/host-shared/converter}"
REAL_PATH=$(readlink -f "$STORAGE_PATH" 2>/dev/null || echo "$STORAGE_PATH")
read -p "Container ID to Destroy [20020]: " CTID_INPUT
CTID="${CTID_INPUT:-20020}"
if pct status "$CTID" &>/dev/null; then
echo "Destroying LXC Container ${CTID}..."
pct stop "$CTID" --skiplock 2>/dev/null || true
pct destroy "$CTID" --purge 2>/dev/null || true
fi
if systemctl is-active --quiet "$SERVICE_NAME" 2>/dev/null || systemctl is-enabled "$SERVICE_NAME" 2>/dev/null; then
echo "Removing Host Backend Systemd Service..."
systemctl stop "$SERVICE_NAME" 2>/dev/null || true
systemctl disable "$SERVICE_NAME" 2>/dev/null || true
rm -f "${SYSTEMD_DIR}/${SERVICE_NAME}.service"
systemctl daemon-reload
fi
if [ -d "$REAL_PATH" ]; then
echo "Purging directory ${REAL_PATH}..."
rm -rf "$REAL_PATH"
fi
echo -e "${GREEN}✓ Removal complete.${NC}"
}
# ------------------------------------------------------------------------------
# CLI Dispatcher
# ------------------------------------------------------------------------------
case "${1:-}" in
--deploy)
do_deploy
;;
--update)
do_update
;;
--remove)
do_remove
;;
*)
echo -e "${CYAN}VM Bench Stack Lifecycle Manager (${HOSTNAME})${NC}"
echo "Usage: $0 {--deploy|--update|--remove}"
echo " --deploy : Interactive installation, storage selection, and deployment"
echo " --update : Pull latest repository changes and refresh stack services"
echo " --remove : Purge LXC container, host systemd backend, and storage directories"
exit 1
;;
esac