UC San Diego's Hao AI Lab Enhances LLM Research with NVIDIA DGX B200 System

The Hao AI Lab at the University of California San Diego has integrated the NVIDIA DGX B200 system to enhance its large language model (LLM) inference capabilities. This system, noted for its high performance, allows for faster prototyping and experimentation within the School of Computing, Information and Data Sciences and the San Diego Supercomputer Center.

Key projects benefitting from the DGX B200 include FastVideo, which generates video content from text prompts, and the Lmgame benchmark for evaluating LLMs. The lab is also exploring low-latency LLM serving through disaggregated inference, which optimizes system throughput and user latency by separating prefill and decode tasks across different GPUs. This approach, involving the new metric