Research

Energy-Aware Scheduling for Serverless LLM Serving on Shared GPUs

Hewlett Packard Enterprise Labs

Developed an energy-aware LLM serving prototype deployed on HPE GreenLake, integrating workload dispatch and consolidation across shared GPUs. Reduced energy consumption by 56% while maintaining SLO attainment on par with state-of-the-art serving systems.

56% lower energy consumption at comparable SLO attainment

Accelerating Dynamic Sparse Attention for Long-Context Decoding

University of Pittsburgh

Designed a temporal-locality-aware runtime that speculatively computes attention over predicted top-K KV blocks while overlapping execution with KV selection. Combined this scheduling approach with a custom FA3-style kernel, reducing per-token decoding latency by over 40%.

Over 40% lower per-token decoding latency

KV Cache Compression for Long Video Generation

University of Pittsburgh

Developed an online KV-cache compression runtime integrated into SGLang-Diffusion to manage growing KV histories during long-form video generation. Combined on-the-fly compression with prefix-tree-based cache reuse to avoid redundant KV computation, achieving a >20x compression ratio and >5x lower next-chunk latency.

>20x KV compression and >5x lower next-chunk latency

RL Training Acceleration for Multi-Agent Systems

University of Pittsburgh

Implemented a dependency-aware rollout policy on top of Slime and OWL, coordinating multi-agent execution around inter-agent dependencies to avoid blocked rollouts during RL training.

Efficient RAG Serving

University of Pittsburgh

Designed a multi-GPU prefix KV-cache management policy for RAG serving, combining workload redistribution with out-of-order query execution to coordinate cache usage and query scheduling. Reduced time to first token (TTFT) by over 64%.

Over 64% lower time to first token

Multi-Tenant Continual Learning

University of Pittsburgh

Proposed an inference-training co-scheduling framework that uses integer linear programming to dynamically reconfigure GPU MIG slices. Coordinated resource allocation across inference and continual-learning workloads, improving SLO attainment and accuracy by over 1.4x and 1.49x, respectively.

Over 1.4x SLO attainment and 1.49x accuracy

Raptor Framework

Enflame Technology Inc., Shanghai, CN

Developed a Rust-based scheduling framework for distributed DNN training. Formulated parallelism strategy selection as an integer linear program, enabling automatic, optimization-driven planning of parallel execution for distributed training workloads.

MLIR Operator Conversion

Enflame Technology Inc., Shanghai, CN

Implemented conversions between custom MLIR dialects and built-in dialects, bridging custom operator representations with the MLIR compilation infrastructure.