# VM Bench — Proxmox Image Conversion

Web GUI and REST API for converting VMware disk images (VMDK, VHD, etc.) into
Proxmox-compatible QCOW2 disks and provisioning VMs with automatic OS and boot
type detection.

## Quick Start

Download and run the install script on your Proxmox host:

```bash
wget https://git.lohmar.co.uk/cclohmar/vm-bench/raw/branch/main/install.sh
bash install.sh --deploy
```

The script interactively:
1. Selects a storage pool
2. Installs the backend API (port 9000) on the host
3. Creates an LXC container with bind-mounted `/mnt/converter`
4. Inside the LXC, run `bash /mnt/converter/frontend/setup.sh` to start the web UI

## Architecture

```
Browser            vm-bench LXC             Proxmox Host
                   (this container)
──→ :5000 ──→  frontend/app.py  ──→  :9000  backend/app.py
                                       │
                                       ├─ qemu-img   (format, size)
                                       ├─ guestfish  (OS, EFI detection)
                                       ├─ 7z/unzip   (archive extraction)
                                       └─ qm create  (VM provisioning)
```

- **Frontend** (`vm-bench` LXC) — FastAPI + Jinja2 web UI on port 5000
- **Backend** (Proxmox host) — FastAPI REST API on port 9000
- **Shared storage** — `/mnt/converter/tmp/` (staging, bind-mounted)

## Directory Layout

```
/mnt/converter/
├── install.sh                  # Master deployment script (run on Proxmox host)
├── clean.sh                    # Clean tmp dirs + restart backend
├── open-api.yaml               # API spec (single source of truth)
├── AGENTS.md                   # Developer guide
├── README.md                   # This file
├── backend/                    # REST API (runs on Proxmox host)
│   ├── app.py                  # FastAPI app, routes, logging
│   ├── models.py               # Pydantic request/response models
│   ├── converter.py            # Archive extraction, disk probing, OS/EFI detection
│   ├── provisioner.py          # VM provisioning (qemu-img convert, qm create/importdisk)
│   ├── vm-bench-backend.service
│   └── requirements.txt
├── frontend/                   # Web UI (runs on vm-bench LXC)
│   ├── app.py                  # FastAPI app, routes, download/SCP handling
│   ├── api_client.py           # Typed REST client → backend
│   ├── setup.sh                # Frontend installer (inside LXC)
│   ├── vm-bench.service        # Systemd unit
│   ├── requirements.txt
│   ├── templates/              # Jinja2 HTML templates
│   │   ├── base.html           # Base layout with session GUID
│   │   ├── index.html          # New session form + download + progress
│   │   ├── _analysis.html      # Analysis result + confirm form
│   │   ├── polling.html        # Progress bar + job status + reuse section
│   │   └── scp.html            # SCP pull page
│   └── static/
│       └── proxmox.css         # Proxmox VE-inspired theme
├── logs/                       # Shared log files (gitignored)
└── tmp/                        # Staging files per session (gitignored)
```

## Features

- **URL download** — ingest archives via `aria2c` with 8 connections
- **SCP pull** — transfer files directly from remote servers, no middle-hop
- **Archive extraction** — 7z, zip, rar, tar, gz, bz2, xz
- **Disk analysis** — detects format, virtual size, guest OS, EFI bootability
- **VM provisioning** — qemu-img convert + optional virt-resize shrink + qm create/importdisk
- **VM cloning** — instant full clone of an existing VM
- **Progress polling** — real-time status bar updates every 2 seconds
- **Session isolation** — UUID per browser session for file staging
- **Staging reuse** — keep source files to create multiple VMs from one image

## Deployment

### Step 1: Backend (on Proxmox host)

```bash
wget https://git.lohmar.co.uk/cclohmar/vm-bench/raw/branch/main/install.sh
bash install.sh --deploy
```

The script interactively:
1. Scans available Proxmox storage pools and lets you pick one
2. Prompts for install path, container ID, IP, gateway, bridge, MAC
3. Installs system packages (qemu-img, guestfish, 7z, unzip, etc.)
4. Creates Python venv, installs deps, generates and starts `vm-bench-backend` service
5. Creates the `vm-bench` LXC container with bind mount

Verify:
```bash
curl http://127.0.0.1:9000/api/v1/health
# → {"status":"ok"}
```

### Step 2: Frontend (inside the LXC)

```bash
pct enter 20020   # use your container ID
bash /mnt/converter/frontend/setup.sh
```

Installs system tools (python3, aria2, sshpass, etc.), Python deps, and starts
`vm-bench` on port 5000.

Verify:
```bash
curl http://127.0.0.1:5000
# → HTML response (the web UI)
```

Open `http://<container-ip>:5000` in a browser.

### Re-deploying / updating

```bash
# On Proxmox host
bash install.sh --update

# Inside LXC
bash /mnt/converter/frontend/setup.sh
```

### Uninstall

```bash
bash install.sh --remove
```

## Usage Flow

1. **Open web UI** → new session form (UUID auto-generated)
2. **Download or SCP pull** a source image → `/mnt/converter/tmp/{session_id}/in/`
3. **Analyse** → backend probes format, size, OS, EFI
4. **Configure VM** → name, CPU, RAM, storage, boot type, target disk size
5. **Submit job** → backend converts, shrinks (if needed), creates VM
6. **Poll progress** → real-time updates every 2 seconds
7. **Reuse or cleanup** → clone, re-convert from same source, or delete staging

### Disk sizing and auto-shrink

| You enter | Result |
|-----------|--------|
| **Blank** (default) | Auto-shrinks to 10% of virtual size (min 20 GB) via `virt-resize --shrink --resize-force` |
| **A number** (e.g. `500`) | Uses that exact size. No shrinking. |
| **A number < virtual** (e.g. `40`) | Shrinks the disk to 40 GB. |
| **A number > virtual** (e.g. `600`) | Expands the disk. |

## API Reference

All endpoints under `/api/v1/`. Full spec: [`open-api.yaml`](open-api.yaml).

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/health` | Health check |
| `POST` | `/api/v1/analyze` | Probe source image (format, size, OS, EFI) |
| `POST` | `/api/v1/jobs` | Submit conversion + provisioning job |
| `GET` | `/api/v1/jobs/{id}` | Poll job status and progress |
| `POST` | `/api/v1/jobs/{id}/cleanup` | Delete or preserve staging files |
| `POST` | `/api/v1/clone` | Clone an existing VM |

### Example: Analyze a source image

```bash
curl -X POST http://10.2.0.2:9000/api/v1/analyze \
  -H "Content-Type: application/json" \
  -d '{"vmid": 21050, "source_filename": "session-id/in/image.vmdk"}'

# → {"vmid": 21050, "filename": "image.vmdk", "disk_format": "vmdk",
#    "disk_size_gb": 7.5, "os_type": "Debian", "efi_detectable": true}
```

### Example: Submit a conversion job

```bash
curl -X POST http://10.2.0.2:9000/api/v1/jobs \
  -H "Content-Type: application/json" \
  -d '{
    "vmid": 21050, "vm_name": "debian-test",
    "boot_disk": {"disk_type": "image_file", "source_filename": "session/in/image.vmdk", "format": "vmdk"},
    "cpu_cores": 2, "ram_mb": 4096, "target_storage": "local-lvm"
  }'

# → {"job_id": "job_21050_1737480000", "vmid": 21050, "status": "queued", ...}
```

## Configuration

| Variable | Default | Description |
|----------|---------|-------------|
| `BACKEND_URL` | `http://10.2.0.2:9000` | Backend API base URL |
| `VM_ID_MIN` | `21000` | Minimum allowed Proxmox VM ID |
| `VM_ID_MAX` | `21100` | Maximum allowed Proxmox VM ID |
| `DOWNLOAD_TIMEOUT` | `14400` (4h) | Max download time |
| `MAX_ETA_SECONDS` | `3600` (1h) | Kill download if ETA exceeds this |

## Development

**Source of truth**: `open-api.yaml` — update spec first, then derive models.

### Backend tools required (on Proxmox host)
- `qemu-img` — disk format, size detection, format conversion
- `guestfish` / `virt-inspector` — OS type and EFI boot detection
- `virt-resize` — safe disk shrinking
- `7z`, `unzip`, `unrar`, `tar`, `gunzip`, `bunzip2`, `xz` — archive extraction
- `qm` (Proxmox CLI) — VM creation, disk import, configuration

### Frontend tools required (in LXC)
- `python3`, `pip3`, `wget`, `curl`, `aria2c`, `openssh-client`, `sshpass`

### Job states

```
queued → processing_conversion → importing_storage → completed
                                                  → failed
```

## Logging

Both services log to systemd journal and rotating file:

| Service | File Log | Journal |
|---------|----------|---------|
| Backend | `/mnt/converter/logs/vm-bench-backend.log` | `journalctl -u vm-bench-backend -f` |
| Frontend | `/mnt/converter/logs/vm-bench.log` | `journalctl -u vm-bench -f` |

Rotation: 10 MB max, 5 backup files.

## License

MIT