These functions can now operate on EC2 G-series instances, including G4dn, G5, G6, G6f, Gr6, Gr6f, and G6e at initial release. The platform supports up to 48 concurrent invocations sharing a single model instance per GPU.
The implementation utilizes vLLM and SGLang base images, meaning no custom handler code is required. Furthermore, these functions support asynchronous timeouts extending up to 90 minutes.
The cost structure involves the EC2 On-Demand rate plus a 9.75 percent management charge, alongside the standard $0.20 per million requests.
A senior specialist solutions architect at AWS noted the operational simplicity: "You build an OCI image, point the function at it, and let Lambda handle provisioning, scaling, and event integration." This removes complexities like managing GPU drivers, CUDA versions, or device plugins.
The function configuration allows declaration of GPU memory via a new AcceleratorConfig block, alongside standard settings like memory and timeout. Concurrent requests leverage one model copy in GPU memory, with scaling governed by GPU utilization, CPU utilization, and concurrency.
First practical tests showed that one environment successfully served all 48 simultaneous requests, achieving approximately 130 tokens per second for individual calls and 3,400 to 3,600 tokens per second when handling all 48.
The author concluded that for models triggered by AWS services like queues or events, the Lambda GPU function offers a simpler deployment path: "one image, one function, and Lambda keeps the server alive and routes the requests."
This development aligns with the broader trend of abstracting infrastructure management, allowing platform participants to leverage specialized hardware without direct operational overhead.
Why it matters
This integration simplifies deploying GPU-intensive workloads by abstracting away complex infrastructure like drivers and cluster management. For operators, this means a path to serverless ML inference without managing underlying EC2 provisioning or scaling logic.






