Model

vLLM vs llama.cpp on a rented RTX 5090

Serve Qwen3.8-27B at 4-bit on vLLM and on llama.cpp on the same rented RTX 5090. Per engine: billed seconds from launch to ready and to first token, stable tokens/s at one request; then raise concurrent requests step by step and record requests/hour, total output tokens/s and median time-to-first-token; mark the load where latency becomes unacceptable (our threshold: median TTFT > 2 s).