Demo Theater

See how TorchTPU provides a native, high-performance compilation path for PyTorch models on Cloud TPUs. This 10-minute demo highlights how TorchTPU serves as a unified backend for leading open-source inference engines like vLLM and SGLang, enabling deployment of state-of-the-art models with minimal code modifications.

Presenters:

• Rob Mulla — Developer Relations Engineer, Google

View in the Schedule

Large fused graphs create a fundamental tension for accelerator compilers: wider fusion enables optimization across operator boundaries, but also causes a combinatorial search over execution engine, tiling, computational lowering, and memory scheduling. This talk presents furiosa-torch’s approach to narrowing that search with TCL (Tensor Contraction Language) and VISA (Virtual Instruction Set Architecture), two custom-kernel DSLs whose kernels remain composable with automatically compiled ATen operators and GraphModules rather than becoming opaque boundaries to graph compilation. In our internal development workflow, we begin with automatic compilation, identify expensive or performance-critical regions, use an AI agent to write or improve their TCL implementations, validate them against the original PyTorch code, and compose them back into eager execution and torch.compile. TCL makes engine selection, tiling, and lowering explicit while leaving cross-operator memory scheduling to the compiler; VISA additionally exposes memory placement, with a type system that rejects mappings unsupported by the hardware. Using this workflow on the same fused computation, we examine how the compiler, developer, and agent can divide implementation decisions to reduce compilation time while balancing execution performance and programming effort.

Presenters:

• Sol Kim — Senior Software Engineer, FuriosaAI

View in the Schedule

See how to go from a deployed model to an agent doing real work. In this live demo, we’ll deploy an open-weight model on Nscale Dedicated Inference, connect it to the open-source OpenCode coding agent, and watch it pick up and work through a real issue. It’s a practical demonstration of running agentic coding workflows on dedicated GPU infrastructure using open tooling, and a hands-on companion to our breakout session, Lessons at Scale: Building an Open PyTorch Inference Stack.

Presenters:

• Cam Flanagan — Senior AI Product Engineer, NScale

View in the Schedule

Fine-tuning LoRA adapters on large Mixture-of-Experts (MoE) models is often bottlenecked not by compute but by HBM capacity: expert weights, long-context activations, and optimizer state can easily exceed what a small cluster provides. In this demo, we give a quick primer on expert parallelism and context parallelism, then show how to combine them in DeepSpeed to fit large MoE fine-tuning jobs onto just a few nodes.

Presenters:

• Suman Debnath — Director, Developer Relations and Product, Crusoe AI
• Jiale Huang — Staff Software Engineer, Crusoe

View in the Schedule

This session showcases how Qualcomm enables developers to build AI-accelerated applications with ExecuTorch once and seamlessly deploy them across a broad range of Qualcomm-powered devices, including compute platforms, smartphones, XR devices, and IoT form factors. By leveraging a unified PyTorch-native deployment workflow, developers can preserve portability while scaling performance and power efficiency across heterogeneous compute resources, including CPU, GPU, NPU, and Low Power AI (LPAI) accelerators.

The live demonstration features a fully on-device, agentic robotics application powered by Gemma 4, running on both a laptop and a smartphone from a common ExecuTorch deployment architecture. Using the Model Context Protocol (MCP), either device can interact with and control a robot by combining multimodal AI capabilities, including real-time perception, gesture recognition, language understanding as well as low-latency control. The robot can switch between behaviors such as Training, Follow, and Sentry modes, illustrating how a single AI application can scale across devices while taking advantage of the optimal hardware acceleration available on each platform.

Attendees will see how ExecuTorch, combined with Qualcomm AI acceleration, enables a “”build once, deploy everywhere”” approach for next-generation edge AI applications, reducing development complexity while delivering scalable performance, power efficiency, and portability across diverse device ecosystems.

Presenters:

• Derrick Johnson — Staff Engineer and Developer Advocate, Qualcomm
• Felix Baum — Senior Director, Product Management, Qualcomm

View in the Schedule

What if your AI agent knew how to get the most out of AMD hardware? Could it help you deploy a model on an AMD Instinct GPU? Find bottlenecks and optimize GPU kernels? Diagnose and fix stubborn issues on your local device? See AMD Skills in action and turn your coding assistant into an AMD expert.

Presenters:

• Daniel Holanda — Principal AI Engineer, AMD

View in the Schedule

Frontier model pre-training has converged on MoE, and communication increasingly determines how efficiently models scale across thousands of GPUs. In this demo, we walk through how NVIDIA’s contributions to TorchTitan unlock world-record MoE pre-training performance on NVIDIA GPUs covering the key optimizations that compound into real throughput gains you can measure and reproduce yourself.

Presenters:

• Elfie Guo US — Senior Software Engineer, Nvidia

View in the Schedule

This talk presents how IBM Spyre streamlines its AI acceleration stack by leveraging PyTorch’s open-source ecosystem. We showcase how PyTorch’s Inductor compiler, combined with open-source extensions, provides a unified compilation framework that enables Spyre to be registered as an out-of-tree accelerator fully aligned with PyTorch’s standardized interfaces. By integrating seamlessly with vLLM and adopting broader open-source model optimization techniques, we demonstrate a cohesive opensource approach that maximizes performance across diverse AI workloads on x86, IBM Z and Power platforms. In this expo theater presentation, we’ll demonstrate the new stack through two compelling use cases. First, we’ll showcase an Insurance Claim Processing system leveraging Granite and Gemma models on IBM Z with Telum II processor, performing real-time multi-model AI inference for claim analysis and damage assessment including visual damage detection. Second, we’ll present Document Intelligence with AI Optimizer, featuring multi-language support through IBM Granite Embedding Models, RAG-based document processing, and intelligent summarization capabilities across enterprise documents. Both demos highlight the practical impact of seamless PyTorch ecosystem integration for accelerating production AI applications.

Presenters:

• Mudhakar Srivatsa — Distinguished Engineer, IBM

View in the Schedule

When a Ray training job fails, the cluster is usually already gone, and with it your evidence. We’ll show what a job-centric view of distributed AI looks like, and why infrastructure-first observability keeps missing the failure.

Presenters:

• Marie Antons — Enterprise Sales Engineer, Datadog

View in the Schedule

Come see me beat up (or get beaten up by) an LLM in Street Fighter III! Along the way, I’ll show how Modal’s infrastructure enables low-latency inference and high-throughput RL post-training for elite gameplay. This same infrastructure powers real-time voice AI at Decagon and reinforcement learning at Applied Compute.

Presenters:

• Andrew Hinh — Developer Relations Engineer, Modal

View in the Schedule

AI agents are increasingly expected to move beyond software and interact with the physical world. This live demonstration showcases how the PyTorch ecosystem can be used to build intelligent applications that seamlessly span cloud, edge and embedded devices.

We’ll build an end-to-end workflow combining PyTorch, ExecuTorch, vLLM and Arm Device Connect. Starting with a natural language request, a large language model will reason about the task, discover available devices, invoke edge AI models where appropriate, retrieve live sensor data and coordinate real hardware through a unified programming model.

The demo highlights practical techniques for combining cloud-hosted foundation models with efficient edge inference while simplifying integration with heterogeneous hardware. Attendees will see how modern AI applications can orchestrate cameras, sensors and embedded devices without requiring device-specific programming, providing a blueprint for building scalable, hardware-aware AI systems across the entire compute continuum.

Presenters:

• Kavya Sri Chennoju — Staff AI Engineer, Arm

View in the Schedule

Discover how to next-level your PyTorch inference with advanced optimization techniques like quantization and speculative decoding. Learn how tools like LLM Compressor and vLLM make these techniques production-ready. The focus will be on seeing these techniques in action: we will do a live stress test these techniques under real load to understand the performance, cost, and quality trade-offs that matter for your production deployments.

Presenters:

• Markell Rawls — Technical Marketing Engineer, Red Hat

View in the Schedule

PyTorch runs natively on Trainium via TorchNeuron. In this live demo, see how to train on TorchTitan or HuggingFace Transformers v5 with zero code changes, deploy with vLLM-Neuron, profile with Neuron Explorer, and add custom NKI kernels directly into your PyTorch code — including ready-made kernels from the HuggingFace Kernel Hub or NKI Library. We’ll also show how Neuron Agentic Development — AI-assisted tooling — accelerates kernel authoring and optimization where engineers already work. Walk away with a clear picture of how standard PyTorch workflows — training, serving, profiling, and kernel development — run end-to-end on Trainium without modification. Come see it live.

Presenters:

• Maen Suleiman — Principal Product Manager, Amazon Web Services

View in the Schedule

Wan 2.2 can now generate a five-second video in less than three seconds, a 53.6x improvement over the baseline implementation. Crossing the real-time threshold turns video generation from a batch workload into an interactive creative loop. The work combines three performance techniques: timestep distillation, custom kernel engineering and four-bit quantization. This talk provides a breakdown of techniques across model, kernel, and runtime that unlock real-time inference for video generation models.

Presenters:

• Ali Taha — Software Engineer, Baseten

View in the Schedule

dmx-compressor is a PyTorch 2.0 based quantization framework that allows ML engineers to obtain numerical reference output to validate models running on ASIC hardware. Developing AI/ML hardware involves low-level kernel programming, a very challenging process.

dmx-compressor facilitates a mapping between its own PyTorch based GPU quantization routines and kernel libraries in the d-Matrix Aviator software stack. Aviator encompasses interfaces that model block-floating point hardware circuits, core MatMul operations, as well as various SIMD kernels (such as add, sub, gelu, etc…).

dmx-compressor’s PyTorch 2.0 program capture ingest modern PyTorch models, and then its FXIR interpreters substitute standard torch modules with modules from our custom DmxModules library wrapping the hardware interface. dmx-compressor then allows users to measure the numerical error between its own reference GPU implementations and the accurate model of our hardware.

Our approach finds and fixes bugs earlier in the development process, has increased our organizational velocity, and showcases how the modern PyTorch ecosystem works across all layers of the AI stack, from HuggingFace to silicon.

Presenters:

• Tristan Webb — Principal Engineer, ML Research, d-Matrix

View in the Schedule

What if you didn’t have to restart from the last checkpoint, wait for minutes, and roll back steps every time your training hits a failure?

This talk shows it live, twice. Mid-run on a real cluster, we pull a network cable from a switch port, on camera, in the server room. LinkPass steers traffic onto healthy links and the job trains straight through it. Then we pull the power cable on a node running the job. TorchPass migrates the affected rank’s state to a healthy node, throughput dips but never flatlines, and the job climbs back to full speed on the same training step. No restart, no lost progress, no wasted GPU hours.

The numbers behind the demo come from the published TorchPass benchmark, run at Llama 4 Scout scale on 64 H200 GPUs and independently validated by SemiAnalysis. You will leave with three checks to run on your own cluster: does a failure resume from the same step or restart from scratch, does throughput ever hit zero, and can the claim be verified the way this one was?

Presenters:

• Lin She — Software Engineer, Clockwork.io

View in the Schedule

Experience AI Playground Home Agent, a new way to access PyTorch-powered AI workflows running on your Intel AI PC from anywhere. Generate character concepts, create 3D meshes, and interact with generative AI applications directly from your phone while computation executes locally on your AI PC.

Presenters:

• Ashok Emani — AI Frameworks Engineer, Intel
• Qiacheng Li — Technical Marketing Engineer, Intel

View in the Schedule

Building upon machine learning research requires reproducing existing results before modification. This first step is often a tedious process that involves locating the reference repository, configuring the environment, provisioning hardware, and debugging toward a working multi-node run, often across a patchwork of disconnected tools. The promise of agentic coding assistants is full automation of such pipelines, with dry runs and experiment tracking. In this live demo, we present an agent operating entirely within one platform, Lightning AI. We show the agent working in real time, where it succeeds autonomously, where it needs intervention, and what that means for agentic ML engineering today.

Presenters:

• Teja Pulagam — Research Engineer, Lightning AI

View in the Schedule

Somewhere out there, a GPU, an NPU, and a custom accelerator walk into your ML stack — and none of them speak the same language. Every new chip shows up promising better perf-per-watt, and every time, your team ends up rewriting kernels, re-tuning schedules, and re-learning pain from scratch. It’s 2026. Why are we still doing this?

Join Lemurian Labs for a talk about the actual reasons heterogeneous compute is such a nightmare — and the (surprisingly good) news that most of that pain isn’t fundamental. We’ll dig into where today’s compiler and runtime abstractions quietly fall apart, what a truly hardware-agnostic stack has to get right, and why “write once, run fast everywhere” isn’t a fantasy — it’s an engineering problem we’re actively solving.

Come for the hot takes on why your accelerator roadmap keeps costing you sleep. Stay for the argument that it doesn’t have to.

Presenters:

• Jay Dawani — CEO & Co-Founder, Lemurian Labs

View in the Schedule

Data movement between host and device is often overlooked in accelerator integration, quietly eating into LLM serving performance. In this talk, we share how we extended PyTorch’s device abstraction to support device-persistent tensors, invoked simply via Tensor.to(‘rbln’), to keep tensors resident on-device and minimize costly HtoD/DtoH transfers alongside our torch.compile-based integration for pre-compiled graph execution.

Presenters:

• Minwook Ahn — Head of Framework Software, Rebellions

View in the Schedule

Distributed inference is changing what AI frameworks need from RDMA networking. Mixture-of-Experts models repeatedly dispatch and combine tokens across GPUs, creating dynamic, latency-sensitive all-to-all traffic unlike the large-message xCCL collectives of distributed training. Current solutions lower latency through tightly integrated GPU, communication, and networking stacks built around connection-oriented RDMA transports, trading portability and developer choice for performance. These approaches remain a poor structural match for the many-destination communication patterns of expert-parallel inference, often consuming GPU execution resources for communication that could otherwise be devoted to inference.

A more open and scalable path starts from a different question: what should inference communication look like if it were designed for connectionless RDMA networks from the ground up, rather than adapted from connection-oriented ones? The emerging Ultra Ethernet Transport—modeled on the HPE Slingshot interconnect deployed in many of the world’s most powerful supercomputers—avoids per-peer connections to improve multipathing, loss recovery, and scalability. It provides an open foundation for innovation across accelerators, networks, and software.

This presentation will introduce OpenCLIC, an open inference communication framework for connectionless networks being integrated with vLLM. Expert-parallel dispatch and combine are its first use case, with broader support planned for one-sided operations such as KV-cache transfer, cache reuse and offload, and model-state movement. OpenCLIC separates framework interfaces, GPU kernels, and transport implementations so developers can use a common interface for adding communication-aware inference capabilities without tying them to a single accelerator or network stack.

Its initial HPE Slingshot implementation uses kernel-initiated OpenSHMEM communication to expose network resources directly to GPU code and keep the CPU out of the critical path. Designed to extend across accelerators and fabrics, OpenCLIC demonstrates how portable GPU-initiated communication and scalable connectionless networking can support expert parallelism and the data-movement needs of distributed inference. By separating communication algorithms from transport implementations,OpenCLIC preserves flexibility in hardware choices while creating a path for inference software to exploit more capable communication architectures as they emerge.

Presenters:

• Jesse Treger — Strategist, HPE HPC & AI Business Unit, HPE

View in the Schedule

Before a post-training job trains a single step, its trainers and inference engines must be scheduled onto the cluster, and every step after that, fresh weights must move between them. We evolve Ray Core by starting from the workloads it runs, and for post-training, the trends point to two needs: better scalability and better performance. We have been improving Ray Core on both fronts. This talk starts with a quick introduction to Ray Core, the open source runtime underneath post-training frameworks like SkyRL, Miles, and NeMo RL, and how its building blocks map onto a post-training loop. We then cover scalability, including scaling Ray’s scheduling to 10,000-node clusters, and performance, including moving PyTorch tensors directly between GPUs with Ray Direct Transport (RDT). We close with where Ray Core is heading next and how to get involved.

Presenters:

• Mengjin Yan — Engineering Lead, Ray Core, Anyscale
• Josh Lee — Software Engineer, Anyscale

View in the Schedule