The 10s HEAD request for Content-Length blocked the download endpoint
response. nginx proxy in front (bench.srv2.sechpoint.app:443 → :5000)
timed out waiting, returned empty response → 'data is null' JS error.
Now wget starts immediately, response returns instantly, and HEAD
runs in a daemon thread to populate content_length for ETA later.
Starlette's multipart parser (pure Python) is too slow for >1 GB uploads —
boundary scanning over gigabytes blocks the event loop indefinitely.
New /session/upload-raw endpoint:
- Receives raw binary body via request.stream() — no parsing overhead
- Metadata (filename, vmid, vm_name) passed in HTTP headers
- Async chunked write directly to staging — true zero-copy streaming
- Same progress logging as before
Frontend now sends File object directly via xhr.send(file)
instead of FormData — eliminates multipart encoding on the client too.
New /scp page with form for host, user, password, remote path.
Runs scp via sshpass in background with progress polling.
Files go to /mnt/converter/in/{session_id}/ for multi-user isolation.
- Added sshpass + openssh-client to frontend installer
- scp.html template with full form + progress bar
- POST /scp/start — launches background SCP via sshpass -e
- GET /scp/progress/{sid}/{file} — polls file size + speed
- 'SCP Pull (+10 GB)' button on start page
- HTTP middleware logs every incoming request (method, path, body size)
BEFORE handler runs — catches silent failures at the network/parser layer
- Changed upload handler from async to sync so FastAPI runs it
in a thread pool, preventing event loop blockage during large
file reads
Uploads already logged start, every-1-GiB progress, and completion.
Downloads only logged start — now also log:
- Completion with file size (Download complete: file.7z (1.9 GiB))
- Failure with wget exit code
- Periodic progress every ~1 GiB with speed and ETA
Both frontend and backend now write rotating log files to
/mnt/converter/logs/ (shared between host and LXC):
/mnt/converter/logs/vm-bench.log (frontend)
/mnt/converter/logs/vm-bench-backend.log (backend)
- RotatingFileHandler: 10 MB per file, 5 backups
- Console handler still writes to systemd journal
- Logs/ directory is gitignored and auto-created on startup
- Install script creates logs/ directory
Large-file handling:
- Set TMPDIR=/mnt/converter/in in service to spool uploads to
shared storage instead of 24 GB LXC rootfs (critical for >24GB)
- Chunked upload streaming (8 MiB) with progress logging every 1 GiB
- Pre-flight disk space check via Content-Length header
- Clean up partial files on upload/download failure
- Download timeout extended to 7200s (2 hours) for 88 GB images
- Switched wget from --show-progress to --progress=dot:giga
(compact output, won't fill memory on large transfers)
- uvicorn --timeout-keep-alive 300 on both frontend and backend
VM name:
- Added vm_name field to initial session form (step 1)
- Falls back to auto-generated 'os_type-vmid' if left blank
- Pre-filled & editable in confirm form (step 2)
