vm-bench/README.md

VM Bench — Proxmox Image Conversion Frontend

Web GUI and REST API for converting VMware disk images (VMDK, VHD, etc.) into Proxmox-compatible QCOW2 disks and provisioning VMs with automatic OS and boot type detection.

Architecture

Browser            vm-bench LXC             Proxmox Host (srv2)
                  (this container)
──→ :5000 ──→  frontend/app.py  ──→  :9000  backend/app.py
                                      │
                                      ├─ qemu-img   (format, size)
                                      ├─ guestfish  (OS, EFI detection)
                                      ├─ 7z/unzip   (archive extraction)
                                      └─ qm create  (VM provisioning)

• Frontend (vm-bench LXC) — FastAPI + Jinja2 web UI on port 5000
• Backend (srv2 Proxmox host) — FastAPI REST API on port 9000
• Shared storage — /mnt/converter/in (staging) and /mnt/converter/out (output)

Directory Layout

/mnt/converter/
├── frontend/                  # Web UI (runs on vm-bench LXC)
│   ├── app.py                 # FastAPI app, routes, session handling
│   ├── api_client.py          # Typed REST client → backend
│   ├── install.sh             # Frontend installer (system deps + service)
│   ├── vm-bench.service       # Systemd unit for the frontend
│   ├── requirements.txt
│   ├── templates/             # Jinja2 HTML templates
│   │   ├── base.html
│   │   ├── index.html         # New session form
│   │   ├── _analysis.html     # Analysis result + confirm form
│   │   └── polling.html       # Progress bar + job status
│   └── static/
│       └── proxmox.css
├── backend/                   # REST API (deploys to Proxmox host srv2)
│   ├── app.py                 # FastAPI app, routes
│   ├── models.py              # Pydantic request/response models
│   ├── converter.py           # Archive extraction, disk probing, OS/EFI detection
│   ├── provisioner.py         # VM provisioning (qm create/importdisk)
│   ├── install.sh             # Full deployment script (backend + LXC creation)
│   ├── vm-bench-backend.service
│   └── requirements.txt
├── open-api.yaml              # API spec (single source of truth)
├── in/                        # Shared staging directory (gitignored)
├── out/                       # Shared output directory (gitignored)
├── logs/                       # Shared log files (gitignored)

Features

• File upload or URL download — ingest .vmdk, .vhd, .7z, .zip, .tar.gz archives
• Automatic archive extraction — 7z, unzip, tar, gunzip
• Disk analysis — detects format, virtual size, guest OS, EFI bootability
• VM provisioning — creates Proxmox VM with automatic disk import and boot config
• Progress polling — real-time job status with progress bar
• Staging reuse — keep source files to create multiple VMs from one image

Prerequisites

Proxmox host (srv2)

The host must be a Proxmox VE node with:

• pct — LXC container management (built-in)
• git — to clone the repository (install script handles this)
• Internet access to download container templates

LXC container (vm-bench)

Created automatically by the install script. Requires:

• 3 CPU cores, 6 GB RAM, 24 GB disk, 2 GB swap
• Debian 12 template (downloaded by script)
• Network bridge vmbr0

Deployment

All deployment is driven by a single script on the Proxmox host. It creates the directory tree, clones the repo, installs the backend, and provisions the vm-bench LXC container ready for the frontend.

Step 1: Clone & deploy (on srv2)

# Create the shared directory and clone the repo
mkdir -p /mnt/converter
cd /mnt/converter
git clone https://git.lohmar.co.uk/cclohmar/vm-bench.git .

# Run the installer — handles everything interactively
bash backend/install.sh

The script will:

1. Create the /mnt/converter/ directory tree
2. Clone or update the git repository
3. Install system packages (qemu-img, guestfish, archive tools, etc.)
4. Install Python dependencies and start the backend systemd service
5. Prompt for container ID + network settings
6. Create and configure the vm-bench LXC container with:
   • 3 cores, 6 GB RAM, 24 GB rootfs, 2 GB swap
   • Bind mount: /mnt/converter shared with host
   • Unprivileged mode off, nesting enabled
   • AppArmor unconfined, cgroup device access allowed
7. Optionally start the container

Example prompts:

Container ID (e.g. 20020): 20020
IP address/CIDR  [10.2.0.20/16]:
Gateway          [10.2.0.2]:
Bridge           [vmbr0]:
MAC address      [BC:24:11:80:5A:B1]:

Verify the backend is running:

curl http://127.0.0.1:9000/api/v1/health
# → {"status":"ok"}

Step 2: Install frontend (inside the LXC)

After the container is created and started, enter it:

pct enter 20020   # use your container ID

Inside the container, run the frontend installer:

bash /mnt/converter/frontend/install.sh

This installs Python, pip, wget, curl, Python dependencies, and starts the vm-bench systemd service on port 5000.

Verify:

curl http://127.0.0.1:5000
# → HTML response (the web UI)

Open http://<container-ip>:5000 in a browser.

Container LXC Config (created by install script)

Setting	Value
Arch	amd64
Cores	3
Memory	6144 MB
Swap	2048 MB
Rootfs	local-lvm:24G
OS type	debian
Unprivileged	0 (disabled)
Features	nesting=1
Network	vmbr0, firewall enabled, veth
Bind mount	mp0: /mnt/converter,mp=/mnt/converter
AppArmor	unconfined
Cgroup	devices.allow: a, mount.auto: proc:rw sys:rw cgroup:rw

Re-deploying / updating

The install scripts are idempotent — safe to re-run:

# On srv2: update backend
cd /mnt/converter && git pull
bash backend/install.sh    # skips LXC creation if container exists

# Inside LXC: update frontend
cd /mnt/converter && git pull
bash frontend/install.sh

API Reference

All endpoints are under /api/v1/. Full specification in open-api.yaml.

Method	Path	Description
GET	/api/v1/health	Health check
POST	/api/v1/analyze	Probe source image (format, size, OS, EFI)
POST	/api/v1/jobs	Submit conversion + provisioning job
GET	/api/v1/jobs/{id}	Poll job status and progress
POST	/api/v1/jobs/{id}/cleanup	Delete or preserve staging files

Example: Analyze a source image

curl -X POST http://10.2.0.2:9000/api/v1/analyze \
  -H "Content-Type: application/json" \
  -d '{"vmid": 21050, "source_filename": "64bit/Debian 12.11.0 (64bit).vmdk"}'

# Response:
# {
#   "vmid": 21050,
#   "filename": "Debian 12.11.0 (64bit).vmdk",
#   "disk_format": "vmdk",
#   "disk_size_gb": 7.5,
#   "os_type": "Debian",
#   "efi_detectable": true
# }

Example: Submit a conversion job

curl -X POST http://10.2.0.2:9000/api/v1/jobs \
  -H "Content-Type: application/json" \
  -d '{
    "vmid": 21050,
    "vm_name": "debian-test",
    "boot_disk": {
      "disk_type": "image_file",
      "source_filename": "64bit/Debian 12.11.0 (64bit).vmdk",
      "format": "vmdk"
    },
    "cpu_cores": 2,
    "ram_mb": 4096,
    "target_storage": "local-lvm"
  }'

# Response:
# { "job_id": "job_21050_1737480000", "vmid": 21050, "status": "queued", ... }

Usage Flow

1. Open the web UI → new session form
2. Upload or download a source image → file lands in /mnt/converter/in/
3. Backend analyses the image → shows format, size, OS, EFI status
4. Configure VM settings → name, CPU, RAM, storage, boot type
5. Submit job → backend converts, shrinks (if needed), creates VM
6. Poll progress → real-time status bar updates every 2 seconds
7. Reuse or cleanup → create another VM from same source, or delete staging files

Disk sizing and auto-shrink

Virtual disk images (VMDK, QCOW2) have two sizes:

	Meaning	Example
Virtual size	Maximum the disk could grow to	500 GB
Actual size	Real data in the image (sparse file)	~7 GB

During analysis, the UI shows the virtual size (e.g. "500.0 GB detected. Omit to auto-shrink to 30 GB"). This is because the source image was created with a large virtual disk, but only a fraction is used.

Target Disk Size field — what happens:

You enter	Result
Blank (default)	Auto-shrinks to 30 GB if the source is larger. Uses virt-resize --shrink --resize-force to safely reduce the disk container.
A number (e.g. 500)	Uses that exact size. No shrinking.
A number < virtual (e.g. 40)	Shrinks the disk to 40 GB.
A number > virtual (e.g. 600)	Expands the disk. Safe operation.

Tip: For most Linux VMs (Debian, Ubuntu, etc.), 30 GB is plenty. You can always expand the disk later in Proxmox if you need more space. If you plan to store large datasets inside the VM, set a higher value.

Configuration

Variable	Default	Description
BACKEND_URL	http://10.2.0.2:9000	Backend API base URL (set in systemd unit)
VM_ID_MIN	21000	Minimum allowed Proxmox VM ID
VM_ID_MAX	21100	Maximum allowed Proxmox VM ID

Development

Source of truth: open-api.yaml — always derive models from this file.

Backend tool requirements (on srv2):

• qemu-img — disk format and size detection
• guestfish / virt-inspector — OS type and EFI boot detection
• 7z, unzip, tar, gunzip, bunzip2, xz — archive extraction

Provisioner note: backend/provisioner.py is currently a stub with simulated progress. Replace with real qm create, qm importdisk, and qm set commands when deploying to the Proxmox host.

Staging files: /mnt/converter/in/, /mnt/converter/out/, and /mnt/converter/logs/ are gitignored — they contain runtime data, not code.

Logging

Both services write to two locations simultaneously:

Destination	How to access
File (shared)	tail -f /mnt/converter/logs/vm-bench.log (frontend) or vm-bench-backend.log (backend)
Journal (per-service)	journalctl -u vm-bench -f (frontend) or journalctl -u vm-bench-backend -f (backend)

Log files rotate automatically at 10 MB, keeping 5 historical files (e.g. vm-bench.log, vm-bench.log.1, vm-bench.log.2, …).

The file logs are the authoritative source — they're written to the shared /mnt/converter/logs/ directory, accessible from both the Proxmox host and the LXC container.