Article

Inferact Inc. has launched to commercialize the open-source vLLM project, securing $150 million in seed funding led by Andreessen Horowitz and Lightspeed, alongside Databricks Inc.'s venture capital arm and others, valuing the startup at $800 million. The founding team includes Ion Stoica, a computer science professor and Databricks co-founder, who directed the University of California at Berkeley's Sky Computing Lab that developed vLLM in 2023. vLLM optimizes large language models (LLMs) for faster inference by reducing memory usage through features like PagedAttention, allowing KV cache data to be stored efficiently.

It also enhances inference speed by enabling the generation of multiple tokens simultaneously. Inferact aims to simplify AI deployment and plans to enhance the open-source version of vLLM with new optimizations and support for various data center hardware.