Demo Theater

See how TorchTPU provides a native, high-performance compilation path for PyTorch models on Cloud TPUs. This 10-minute demo highlights how TorchTPU serves as a unified backend for leading open-source inference engines like vLLM and SGLang, enabling deployment of state-of-the-art models with minimal code modifications.

Presenters:

• Rob Mulla — Developer Relations Engineer, Google

View in the Schedule

Fine-tuning LoRA adapters on large Mixture-of-Experts (MoE) models is often bottlenecked not by compute but by HBM capacity: expert weights, long-context activations, and optimizer state can easily exceed what a small cluster provides. In this demo, we give a quick primer on expert parallelism and context parallelism, then show how to combine them in DeepSpeed to fit large MoE fine-tuning jobs onto just a few nodes.

Presenters:

• Suman Debnath — Director, Developer Relations and Product, Crusoe, Crusoe AI
• Jiale Huang — Staff Software Engineer, Crusoe

View in the Schedule

When a Ray training job fails, the cluster is usually already gone, and with it your evidence. We'll show what a job-centric view of distributed AI looks like, and why infrastructure-first observability keeps missing the failure.

Presenters:

• Marie Antons — Enterprise Sales Engineer, Baseten

View in the Schedule

Come see me beat up (or get beaten up by) an LLM in Street Fighter III! Along the way, I'll show how Modal's infrastructure enables low-latency inference and high-throughput RL post-training for elite gameplay. This same infrastructure powers real-time voice AI at Decagon and reinforcement learning at Applied Compute.

Presenters:

• Andrew Hinh — Developer Relations Engineer, Modal

View in the Schedule

AI agents are increasingly expected to move beyond software and interact with the physical world. This live demonstration showcases how the PyTorch ecosystem can be used to build intelligent applications that seamlessly span cloud, edge and embedded devices.

We'll build an end-to-end workflow combining PyTorch, ExecuTorch, vLLM and Arm Device Connect. Starting with a natural language request, a large language model will reason about the task, discover available devices, invoke edge AI models where appropriate, retrieve live sensor data and coordinate real hardware through a unified programming model.

The demo highlights practical techniques for combining cloud-hosted foundation models with efficient edge inference while simplifying integration with heterogeneous hardware. Attendees will see how modern AI applications can orchestrate cameras, sensors and embedded devices without requiring device-specific programming, providing a blueprint for building scalable, hardware-aware AI systems across the entire compute continuum.

Presenters:

• Kavya Sri Chennoju — Staff AI Engineer, Arm

View in the Schedule

Discover how to next-level your PyTorch inference with advanced optimization techniques like quantization and speculative decoding. Learn how tools like LLM Compressor and vLLM make these techniques production-ready. The focus will be on seeing these techniques in action: we will do a live stress test these techniques under real load to understand the performance, cost, and quality trade-offs that matter for your production deployments.

Presenters:

• Markell Rawls — Technical Marketing Engineer, Red Hat

View in the Schedule

PyTorch runs natively on Trainium via TorchNeuron. In this live demo, see how to train on TorchTitan or HuggingFace Transformers v5 with zero code changes, deploy with vLLM-Neuron, profile with Neuron Explorer, and add custom NKI kernels directly into your PyTorch code — including ready-made kernels from the HuggingFace Kernel Hub or NKI Library. We'll also show how Neuron Agentic Development — AI-assisted tooling — accelerates kernel authoring and optimization where engineers already work. Walk away with a clear picture of how standard PyTorch workflows — training, serving, profiling, and kernel development — run end-to-end on Trainium without modification. Come see it live.

Presenters:

• Maen Suleiman — Principal Product Manager, Amazon Web Services

View in the Schedule

Wan 2.2 can now generate a five-second video in less than three seconds, a 53.6x improvement over the baseline implementation. Crossing the real-time threshold turns video generation from a batch workload into an interactive creative loop. The work combines three performance techniques: timestep distillation, custom kernel engineering and four-bit quantization. This talk provides a breakdown of techniques across model, kernel, and runtime that unlock real-time inference for video generation models.

Presenters:

• Ali Taha — Software Engineer, Baseten

View in the Schedule

dmx-compressor is a PyTorch 2.0 based quantization framework that allows ML engineers to obtain numerical reference output to validate models running on ASIC hardware. Developing AI/ML hardware involves low-level kernel programming, a very challenging process.

dmx-compressor facilitates a mapping between its own PyTorch based GPU quantization routines and kernel libraries in the d-Matrix Aviator software stack. Aviator encompasses interfaces that model block-floating point hardware circuits, core MatMul operations, as well as various SIMD kernels (such as add, sub, gelu, etc…).

dmx-compressor's PyTorch 2.0 program capture ingest modern PyTorch models, and then its FXIR interpreters substitute standard torch modules with modules from our custom DmxModules library wrapping the hardware interface. dmx-compressor then allows users to measure the numerical error between its own reference GPU implementations and the accurate model of our hardware.

Our approach finds and fixes bugs earlier in the development process, has increased our organizational velocity, and showcases how the modern PyTorch ecosystem works across all layers of the AI stack, from HuggingFace to silicon.

Presenters:

• Tristan Webb — Principal Engineer, ML Research, d-Matrix

View in the Schedule

What if you didn't have to restart from the last checkpoint, wait for minutes, and roll back steps every time your training hits a failure?

This talk shows it live, twice. Mid-run on a real cluster, we pull a network cable from a switch port, on camera, in the server room. LinkPass steers traffic onto healthy links and the job trains straight through it. Then we pull the power cable on a node running the job. TorchPass migrates the affected rank's state to a healthy node, throughput dips but never flatlines, and the job climbs back to full speed on the same training step. No restart, no lost progress, no wasted GPU hours.

The numbers behind the demo come from the published TorchPass benchmark, run at Llama 4 Scout scale on 64 H200 GPUs and independently validated by SemiAnalysis. You will leave with three checks to run on your own cluster: does a failure resume from the same step or restart from scratch, does throughput ever hit zero, and can the claim be verified the way this one was?

Presenters:

• Lin She — Software Engineer, Clockwork.io

View in the Schedule

Experience AI Playground Home Agent, a new way to access PyTorch-powered AI workflows running on your Intel AI PC from anywhere. Generate character concepts, create 3D meshes, and interact with generative AI applications directly from your phone while computation executes locally on your AI PC.

Presenters:

• Ashok Emani — AI Frameworks Engineer, Intel
• Qiacheng Li — Technical Marketing Engineer, Intel Corporation

View in the Schedule

Building upon machine learning research requires reproducing existing results before modification. This first step is often a tedious process that involves locating the reference repository, configuring the environment, provisioning hardware, and debugging toward a working multi-node run, often across a patchwork of disconnected tools. The promise of agentic coding assistants is full automation of such pipelines, with dry runs and experiment tracking. In this live demo, we present an agent operating entirely within one platform, Lightning AI. We show the agent working in real time, where it succeeds autonomously, where it needs intervention, and what that means for agentic ML engineering today.

Presenters:

• Teja Pulagam — Research Engineer, Lightning AI

View in the Schedule

Somewhere out there, a GPU, an NPU, and a custom accelerator walk into your ML stack — and none of them speak the same language. Every new chip shows up promising better perf-per-watt, and every time, your team ends up rewriting kernels, re-tuning schedules, and re-learning pain from scratch. It's 2026. Why are we still doing this?

Join Lemurian Labs for a talk about the actual reasons heterogeneous compute is such a nightmare — and the (surprisingly good) news that most of that pain isn't fundamental. We'll dig into where today's compiler and runtime abstractions quietly fall apart, what a truly hardware-agnostic stack has to get right, and why "write once, run fast everywhere" isn't a fantasy — it's an engineering problem we're actively solving.

Come for the hot takes on why your accelerator roadmap keeps costing you sleep. Stay for the argument that it doesn't have to.

Presenters:

• Jay Dawani — CEO & Co-Founder, Lemurian Labs

View in the Schedule

Data movement between host and device is often overlooked in accelerator integration, quietly eating into LLM serving performance. In this talk, we share how we extended PyTorch's device abstraction to support device-persistent tensors, invoked simply via Tensor.to('rbln'), to keep tensors resident on-device and minimize costly HtoD/DtoH transfers alongside our torch.compile-based integration for pre-compiled graph execution.

Presenters:

• Minwook Ahn — Head of Framework Software, Rebellions

View in the Schedule

Distributed inference is changing what AI frameworks need from RDMA networking. Mixture-of-Experts models repeatedly dispatch and combine tokens across GPUs, creating dynamic, latency-sensitive all-to-all traffic unlike the large-message xCCL collectives of distributed training. Current solutions lower latency through tightly integrated GPU, communication, and networking stacks built around connection-oriented RDMA transports, trading portability and developer choice for performance. These approaches remain a poor structural match for the many-destination communication patterns of expert-parallel inference, often consuming GPU execution resources for communication that could otherwise be devoted to inference.

A more open and scalable path starts from a different question: what should inference communication look like if it were designed for connectionless RDMA networks from the ground up, rather than adapted from connection-oriented ones? The emerging Ultra Ethernet Transport—modeled on the HPE Slingshot interconnect deployed in many of the world’s most powerful supercomputers—avoids per-peer connections to improve multipathing, loss recovery, and scalability. It provides an open foundation for innovation across accelerators, networks, and software.

This presentation will introduce OpenCLIC, an open inference communication framework for connectionless networks being integrated with vLLM. Expert-parallel dispatch and combine are its first use case, with broader support planned for one-sided operations such as KV-cache transfer, cache reuse and offload, and model-state movement. OpenCLIC separates framework interfaces, GPU kernels, and transport implementations so developers can use a common interface for adding communication-aware inference capabilities without tying them to a single accelerator or network stack.

Its initial HPE Slingshot implementation uses kernel-initiated OpenSHMEM communication to expose network resources directly to GPU code and keep the CPU out of the critical path. Designed to extend across accelerators and fabrics, OpenCLIC demonstrates how portable GPU-initiated communication and scalable connectionless networking can support expert parallelism and the data-movement needs of distributed inference. By separating communication algorithms from transport implementations,OpenCLIC preserves flexibility in hardware choices while creating a path for inference software to exploit more capable communication architectures as they emerge.

Presenters:

• Jesse Treger — Strategist, HPE HPC & AI Business Unit, HPE

View in the Schedule