Runs 'ssh stat -c%s' or 'ls -l' to get total bytes before starting
SCP transfer. Progress bar now shows real percentage and ETA
instead of indeterminate spinner.
Replaces wget with aria2c -x8 -s8 for significantly faster
downloads, especially on high-latency connections. Falls back to
single-connection if the server doesn't support range requests.
- Backend cleanup_staging accepts session_id, deletes whole
STAGING_ROOT/{guid}/ dir when provided
- CleanupRequest model gets session_id field
- Frontend cleanup proxy passes session_id from body
- api_client passes session_id to backend
- reuseImage JS includes session_id in cleanup payload
When files are stored in /mnt/converter/in/{guid}/, the backend
needs the full relative path. The analyze endpoint now constructs
'{guid}/{filename}' before passing to the backend API.
Each browser gets a UUID stored in localStorage, set as a cookie,
and attached to every API call. Files go to session-specific dirs:
/mnt/converter/in/{guid}/ — uploads, downloads, SCP pulls
/mnt/converter/out/{guid}/ — converted QCOW2s
Changes:
- base.html: generates UUID via crypto.randomUUID(), stores in
localStorage + cookie, exposes as window.VM_BENCH_SID
- Frontend: all endpoints accept session_id, _staging() helper
creates session-aware paths on demand
- JavaScript: session_id appended to all FormData, set as
X-Session-ID header on raw uploads
- Backend models: JobSubmissionRequest.session_id field added
- Provisioner: _staging_in/_staging_out helpers, source path
resolution uses session-aware directory
- Converter: extract_if_needed skips re-extraction if dir exists
After conversion completes, the user sees a form with VM Name and ID
prefilled, and three buttons:
- Clone (instant): qm clone --full via backend, takes seconds
- Copy (re-convert): shows CPU/RAM/Disk fields, re-runs full pipeline
- Clean Up & Finish: deletes staging, returns to start page
Backend: new /api/v1/clone endpoint → provisioner.clone_vm()
Frontend: new /session/clone proxy, api_client.clone_vm()
The 10s HEAD request for Content-Length blocked the download endpoint
response. nginx proxy in front (bench.srv2.sechpoint.app:443 → :5000)
timed out waiting, returned empty response → 'data is null' JS error.
Now wget starts immediately, response returns instantly, and HEAD
runs in a daemon thread to populate content_length for ETA later.
Starlette's multipart parser (pure Python) is too slow for >1 GB uploads —
boundary scanning over gigabytes blocks the event loop indefinitely.
New /session/upload-raw endpoint:
- Receives raw binary body via request.stream() — no parsing overhead
- Metadata (filename, vmid, vm_name) passed in HTTP headers
- Async chunked write directly to staging — true zero-copy streaming
- Same progress logging as before
Frontend now sends File object directly via xhr.send(file)
instead of FormData — eliminates multipart encoding on the client too.
New /scp page with form for host, user, password, remote path.
Runs scp via sshpass in background with progress polling.
Files go to /mnt/converter/in/{session_id}/ for multi-user isolation.
- Added sshpass + openssh-client to frontend installer
- scp.html template with full form + progress bar
- POST /scp/start — launches background SCP via sshpass -e
- GET /scp/progress/{sid}/{file} — polls file size + speed
- 'SCP Pull (+10 GB)' button on start page
- HTTP middleware logs every incoming request (method, path, body size)
BEFORE handler runs — catches silent failures at the network/parser layer
- Changed upload handler from async to sync so FastAPI runs it
in a thread pool, preventing event loop blockage during large
file reads
Uploads already logged start, every-1-GiB progress, and completion.
Downloads only logged start — now also log:
- Completion with file size (Download complete: file.7z (1.9 GiB))
- Failure with wget exit code
- Periodic progress every ~1 GiB with speed and ETA
Both frontend and backend now write rotating log files to
/mnt/converter/logs/ (shared between host and LXC):
/mnt/converter/logs/vm-bench.log (frontend)
/mnt/converter/logs/vm-bench-backend.log (backend)
- RotatingFileHandler: 10 MB per file, 5 backups
- Console handler still writes to systemd journal
- Logs/ directory is gitignored and auto-created on startup
- Install script creates logs/ directory
Large-file handling:
- Set TMPDIR=/mnt/converter/in in service to spool uploads to
shared storage instead of 24 GB LXC rootfs (critical for >24GB)
- Chunked upload streaming (8 MiB) with progress logging every 1 GiB
- Pre-flight disk space check via Content-Length header
- Clean up partial files on upload/download failure
- Download timeout extended to 7200s (2 hours) for 88 GB images
- Switched wget from --show-progress to --progress=dot:giga
(compact output, won't fill memory on large transfers)
- uvicorn --timeout-keep-alive 300 on both frontend and backend
VM name:
- Added vm_name field to initial session form (step 1)
- Falls back to auto-generated 'os_type-vmid' if left blank
- Pre-filled & editable in confirm form (step 2)
