Skip to content
Cloud Provider Benchmarks

Benchmarks · GPU & AI

NVIDIA L40S cloud benchmarks: 48 GB Ada GPU for cost-efficient inference

The L40S is an Ada Lovelace PCIe GPU with 48 GB of GDDR6 at 864 GB/s and FP8 Tensor Cores, without NVLink or MIG. Its lower price and power make it attractive for small and mid-size models, embeddings, image generation and mixed graphics and AI workloads.

Updated:

Direct answer

L40S serves 7B–8B models in BF16 and models up to about 32B in FP8 or INT4 on one GPU. Its 864 GB/s bandwidth gives roughly a quarter of the H100 SXM single-stream decode ceiling, so it suits throughput-tolerant or smaller-model workloads. PortvMind Cloud, AWS, DigitalOcean and Vultr list L40S among the providers tracked here.

What is measured

  • Executable model × context × concurrency scenarios on 1, 2, 4 and 8 GPUs
  • Single-stream decode ceiling from memory bandwidth
  • Provider availability and published per-GPU-hour list prices
  • Verified vLLM serving runs (when published)

Overview

The L40S is a PCIe card with 48 GB of GDDR6 (ECC) at 864 GB/s, fourth-generation Tensor Cores with FP8 and third-generation RT cores. Its dense FP8 throughput is roughly 37% of an H100 SXM, while its memory bandwidth is roughly 26%. It has no NVLink and does not support MIG, so multi-GPU inference runs over PCIe. It is well suited to 7B–14B-class models at BF16, quantised 30B-class models, embedding and reranking services, diffusion models and mixed graphics/AI workloads.

Suitable workloads

  • Inference of 7B–14B models at BF16 and quantised ~30B models
  • Embedding, reranking and vision models
  • Image and video generation, rendering and graphics
  • Cost-sensitive fine-tuning of small models (LoRA/QLoRA)

Considerations

  • 864 GB/s bandwidth caps single-stream decode speed well below Hopper GPUs
  • No NVLink; tensor parallelism across cards uses PCIe Gen4
  • No MIG; sharing relies on vGPU or time slicing

Specifications

GPU specifications (NVIDIA datasheets)
GPU Architecture Memory (GB) Memory type Bandwidth (GB/s) FP8 (TFLOPS) BF16 (TFLOPS) FP32 (TFLOPS) Interconnect TDP MIG
L40S PCIe dual-slot Ada Lovelace 48 GDDR6 (ECC) 864 733 362 91.6 PCIe only 350 W Not supported

Tensor values are dense (non-sparse) peaks; NVIDIA publishes sparse figures at twice these values.

Providers offering NVIDIA L40S

GPU availability and published per-GPU-hour list prices by provider
Provider NVIDIA L40S Türkiye infrastructure
PortvMind Cloud Generally available GPU / AI Compute (L40S option) Listed as an available GPU option; configurations and prices are not included in the public calculator table and are provided via the console or sales. Yes
Amazon Web Services Generally available EC2 G6e 1 to 8 L40S GPUs per instance. Yes
DigitalOcean Generally available GPU Droplet — NVIDIA L40S $1.570 / GPU-h 1× L40S 48 GB with 8 vCPU and 64 GiB RAM. No
Vultr Generally available Cloud GPU — NVIDIA L40S $1.671 / GPU-h 1, 2, 4 or 8× L40S 48 GB; 1× plan with 16 vCPU and 180 GB RAM. No

Decode ceiling (theoretical)

Single-stream upper bound derived from memory bandwidth. Not a measurement.

Memory-bandwidth ceiling for single-stream decode (FP8, tok/s) Theoretical upper bound — not a measurement
Model L40S
Llama 3.1 8B Instruct 108
Llama 3.3 70B Instruct 24 2× TP
Qwen2.5 7B Instruct 114
Qwen3 32B 26
Qwen3 30B-A3B 262
Qwen2.5 72B Instruct 24 2× TP
DeepSeek-R1-Distill-Qwen-32B 26
DeepSeek-V3 does not fit
MiniMax-M1-80k does not fit

Ceiling = aggregate memory bandwidth ÷ active weight bytes read per token (batch 1, excluding KV reads and communication, assuming ideal tensor parallelism). Real engines reach a fraction of this bound.

Memory-bandwidth ceiling for single-stream decode (BF16, tok/s) Theoretical upper bound — not a measurement
Model L40S
Llama 3.1 8B Instruct 54
Llama 3.3 70B Instruct 24 4× TP
Qwen2.5 7B Instruct 57
Qwen3 32B 26 2× TP
Qwen3 30B-A3B 262 2× TP
Qwen2.5 72B Instruct 24 4× TP
DeepSeek-R1-Distill-Qwen-32B 26 2× TP
DeepSeek-V3 does not fit
MiniMax-M1-80k does not fit

Ceiling = aggregate memory bandwidth ÷ active weight bytes read per token (batch 1, excluding KV reads and communication, assuming ideal tensor parallelism). Real engines reach a fraction of this bound.

Executable scenarios

Which context and concurrency combinations fit in memory on one GPU.

Llama 3.1 8B Instruct · BF16 · 1× L40S Executable scenario matrix (KV-cache capacity model) · KV budget ≈ 24.7 GB
Context 15102550100 Max conc.
8K ✓ ✓ ✓ — — — 23
16K ✓ ✓ ✓ — — — 11
32K ✓ ✓ — — — — 5
64K ✓ — — — — — 2
128K ✓ — — — — — 1

✓ scenario fits the memory model · — KV cache exceeded (not executed) · ✕ weights do not fit · n/a context above model maximum. Model: 90% memory utilisation, 5% runtime reserve, 16-bit KV cache.

Qwen3 32B · FP8 · 1× L40S Executable scenario matrix (KV-cache capacity model) · KV budget ≈ 8.0 GB
Context 15102550100 Max conc.
8K ✓ — — — — — 3
16K ✓ — — — — — 1
32K — — — — — — 0
64K — — — — — — 0
128K — — — — — — 0

✓ scenario fits the memory model · — KV cache exceeded (not executed) · ✕ weights do not fit · n/a context above model maximum. Model: 90% memory utilisation, 5% runtime reserve, 16-bit KV cache.

Llama 3.3 70B Instruct · FP8 · 1× L40S Executable scenario matrix (KV-cache capacity model) · KV budget ≈ -29.8 GB
Context 15102550100 Max conc.
8K ✕ ✕ ✕ ✕ ✕ ✕ 0
16K ✕ ✕ ✕ ✕ ✕ ✕ 0
32K ✕ ✕ ✕ ✕ ✕ ✕ 0
64K ✕ ✕ ✕ ✕ ✕ ✕ 0
128K ✕ ✕ ✕ ✕ ✕ ✕ 0

✓ scenario fits the memory model · — KV cache exceeded (not executed) · ✕ weights do not fit · n/a context above model maximum. Model: 90% memory utilisation, 5% runtime reserve, 16-bit KV cache.

Verified results