September 22 – 24 | San Jose, California
Learn how to use the event app to plan your personal schedule.
Join us for a Meet the Ambassador's Breakfast on Day 1 of OpenSearchCon North America.
Learn about our current Ambassadors: https://opensearch.org/ambassadors/
Light breakfast will be provided.
Five years ago, OpenSearch launched with a bold vision. Today, the community's momentum is undeniable. Tia White, General Manager for OpenSearch at AWS, opens with a high-energy look at the evolution of OpenSearch — the contributions, the community milestones, and the innovations driving the project forward. Then comes the challenge: what will it take to become the open-source project engineers love to contribute to? Drawing on her experience leading teams building on open source projects at scale, Tia explores the trifecta that separates good communities from great ones — fast time-to-value, genuine collaboration, and radical transparency — and lays out a vision for the future of the OpenSearch Project. This is a keynote for anyone who's ready to shape what comes next.
Vector search is the fastest-growing workload in OpenSearch — and many clusters are leaving hardware performance untapped. For 2+ years, Intel and the OpenSearch community have been working together through a hardware/software co-design effort spanning compression, SIMD acceleration, and graph-based index methods.
Intel's contributions span the full stack: QAT-Java 2.5.0 integration with the Custom-Codecs plugin delivers 1.35× search throughput gains; AVX-512 SIMD kernels contributed to FAISS and the k-NN plugin make vector search up to 30% faster on standard Intel Xeon hardware; and Intel Scalable Vector Search (SVS) is landing in the k-NN plugin as a new engine — delivering 2.57× the throughput of HNSW FP16, 36% less graph memory, and 44% lower P99 latency at matched recall.
The roadmap includes SVS on-disk and filtered search extensions, automatic LeanVec encoder training, and optimizations for next-generation Intel architectures.
Attendees will learn what this collaboration has produced, what is coming, and how to get the most out of OpenSearch on Intel hardware today.
CERN operates the world's largest particle accelerator and a computing infrastructure spread across 160 sites in 40 countries. Alongside the physics data itself, that infrastructure produces a second stream: the logs of the systems that capture and distribute the physics output, the monitoring of the grid that processes it, and the search engines that make the resulting papers and records findable.
In 2016, CERN consolidated that work into a single centralised search service. This keynote traces its evolution — from scattered Elasticsearch clusters, through the 2021 move to OpenDistro and then OpenSearch, to today's 140 clusters holding 1.4 PB. We'll look at what OpenSearch powers at CERN: log analytics and observability, the search engines behind high-energy physics portals, and more experimental ground in security analytics, semantic search, and AI applications.
The OpenSearch community has been open, fast-moving, and easy to contribute to — which is why CERN joined the OpenSearch Software Foundation as an associate member earlier this year. We'll close on what we hope to bring to it: our experience in observability and Kubernetes-based deployments in a global research environment.
The line between search engine and AI platform is disappearing. OpenSearch is no longer just where you query data — it's where AI applications think, retrieve, observe, and protect.
This customer panel brings together practitioners pushing OpenSearch into the next era across search, observability, and security. Panelists will share how they've moved from lexical search to semantic retrieval with vector embeddings and hybrid search, architected RAG pipelines with OpenSearch as the knowledge store, and built agentic AI workflows where agents query, reason over, and act on data in real time.
The AI story doesn't stop at search. We'll explore how teams apply generative AI to operational challenges — using OpenSearch for intelligent log analysis where AI surfaces anomalies that pattern-matching misses — and for security analytics, where threat detection moves from static rules to AI-powered correlation across logs, traces, and security events.
Expect an honest conversation — what worked, what broke, and what they'd do differently. Audience Q&A encouraged.
Autonomous agents are no longer chat demos. With OpenClaw and Hermes-agent, we can run real workloads: an agent gets a task, holds memory, calls tools, fetches context, and decides what to do next. When something goes wrong, the chat output tells you almost nothing. We had an agent “complete” a dozen tasks in a row without doing any of them (emitting work under the wrong message role, with no error, no useful log, just silent drift).
This talk shows how we instrument that world with OpenSearch. We model agent activity as OpenTelemetry traces (every tool call, model invocation, and context retrieval is a span) and ship logs and metrics into the same cluster. Traces are the spine: the honest record of an agent’s execution path.
We walk through dashboards for tool usage, model latency, retry storms, cost-per-task, and “golden paths”: reference runs of a task done right, which newer executions are scored against to flag drift automatically.
Then the part we like most: agents observing agents. Via the OpenSearch MCP server, an agent queries its own telemetry, asks why a run cost 4x the median, and proposes a fix.
Agentic Search in OpenSearch represents a major leap toward adaptive, tool‑aware search flows. This session takes a deep dive into the internal design and implementation of agentic search types—conversational, plan‑execute‑reflect, and flow agents—showing how they interact seamlessly with MCP tools for secure discovery and protocol interoperability. We’ll explore architectural diagrams, highlight critical code snippets from ML Commons agentic search and MCP, and unpack design decisions that drive extensibility and performance. By blending high‑level design with practical implementation details, the talk aims to make the “black box” transparent. Attendees will walk away with insights into how agentic search works under the hood, how to extend or debug it, and how to leverage it in real‑world scenarios. A live demo will showcase one or more agentic search types in action, bridging theory with practice and empowering developers to innovate on top of OpenSearch’s agentic ecosystem.
—
Key Takeaways
• Internal design patterns of agentic search
• How MCP tools integrate with agents
• Code snippets from ML Commons agentic search and MCP
• Diagrams and demos to bridge theory with practice
Agentic search lets you ask in Natural Language and have OpenSearch plan and execute retrieval. The Query Planner re-writes Natural Language to OpenSearch DSL using SOTA LLMs. This works amazingly well but not without important limitations:
1/ The biggest factor is added latency for remote inference. eCommerce Search demands sub-100ms response times. Even the best results when slow lead to search abandonment.
2/ We'll demo overcoming latency by hosting SLMs locally within OpenSearch nodes using vLLM/Ollama. This is faster & cheaper, but SLMs suffer quality decline in query re-writes. We'll explore fine-tuning with domain-specific data—proving fine-tuned SLMs beat SOTA LLMs.
3/ Relevance remains generic even with Agentic Search. We'll show how to improve it using user and business contexts with hybrid search and reranking.
This hands-on talk covers:
1/ Search Latency with Remote Inference
2/ Locally hosting SLMs using vLLM/Ollama
3/ Agentic search with SLM Local Inference
4/ Improving relevance with user and business contexts
5/ Fine-tuning SLMs for domain-specific query re-writing via Axolotl/LLaMA-Factory
Anomaly detection is the easy part. An alert fires at 2AM — now what?
The real cost isn't detection lag, it's investigation lag: the 20 minutes an on-call SRE spends correlating logs, cross-referencing metrics, and forming hypotheses before knowing where to look. OpenSearch's anomaly detection plugin surfaces the signal well. It has no opinion on what you do next.
This talk explores how agent skills in OpenSearch's ML framework can collapse that investigation window. Instead of a human triaging an alert, an agent picks up where the detector left off: querying correlated log indices, identifying co-occurring error patterns, cross-referencing recent deploys, and surfacing ranked probable causes — autonomously, before the SRE opens a dashboard.
We'll walk through the architecture of a plan-execute-reflect agent wired to an anomaly detection workflow, the tool design decisions that make agents useful vs. noisy in on-call contexts, and the failure modes that will humble you in production.
You'll leave with a clear pattern for connecting OpenSearch's anomaly detection and agent skills layers, and an honest assessment of where human judgment still wins.
Phishing attacks have evolved far beyond traditional email lures. Modern techniques such as Adversary-in-the-Middle (AITM) and ClickFix operate directly in the browser, allowing attackers to steal credentials and session tokens while bypassing traditional defenses like email security gateways and secure web gateways (SWG). At the same time, attackers are increasingly using AI to craft more convincing phishing campaigns, making these attacks more scalable and effective.
This talk demonstrates how these advanced phishing techniques work through a live walkthrough of an AITM/ClickFix attack, showing why conventional security controls often fail to detect them. The session will then focus on the browser as a critical security control point, outlining practical browser hardening rules and protections that organizations can implement to reduce risk.
Finally, the talk will explore how browser telemetry and security signals can be analyzed with OpenSearch to detect suspicious behavior, hunt for indicators, and improve visibility into modern phishing attacks. Attendees will leave with practical strategies to better protect users against the rapidly evolving phishing landscape.
Agentic coding is transforming software development, but how do you validate AI agents before production? Unlike traditional software with clear pass/fail tests, agents operate probabilistically across reasoning, tool selection, and multi-step decisions. Manual testing doesn't scale—it slows iterations and makes debugging nearly impossible.
OpenSearch's Agentic AI Evaluation Platform brings systematic Automatic Relevance Tuning (ART) to agent development. Define test suites, run benchmarks, and measure performance across reasoning quality, tool accuracy, and task completion.
We'll share a compelling case study: one agent built with observability shipped in 6 weeks; another retrofitted later took 12 months. See how systematic evaluation transforms development cycles.
High-cardinality aggregations in OpenSearch face a performance-memory trade-off at the collector level: OrdinalsCollector leverages Lucene's SortedSetDocValues for fast ordinal-based collection but is memory-intensive, while DirectCollector uses direct hash computation and is memory-efficient but slower.
Current logic conservatively selects DirectCollector when relative memory overhead is high, sacrificing performance even when sufficient memory is available—resulting in up to 5x slower queries.
This session introduces the Hybrid Cardinality Collector that intelligently starts with Lucene's ordinal-based collection and dynamically switches to direct hashing only when memory consumption exceeds a configurable threshold. Using exception-based control flow for zero overhead during normal operation, the hybrid collector preserves already-computed aggregation state during the switch, eliminating the need to rebuild results.
Learn how this adaptive collector strategy delivers up to 5x faster query performance while maintaining memory safety, and discover the configurable settings to optimize for your workloads.
As the OpenSearch engine evolves, the platform surrounding the search service where most integrations take place is the first line of defense where some of the hardest problems show up: operator watch scope, namespaced RBAC, ingress/auth handoffs, packaging rules, and release dependencies on other components.
In this session, we'll walk the audience through several challenges faced while integrating OpenSearch in a constrained Kubernetes platform at IBM, and the lessons learnt from them. We’ll cover what broke in multi-namespace deployments, why RBAC models that work with cluster-admin access become fragile on shared platforms, and how to systematically trace failures across ingress, auth, and backend services.
We’ll also share the changes that made the system more operable and secure: safer chart assumptions, cleaner deployment boundaries, better handoffs, and tighter scope control.
Attendees will leave with concrete debugging patterns and deployment tactics that can be applied when OpenSearch has to operate in their own environments, such as on-prem deployments.
*About the Better Together Lunch Group*
The Better Together Lunch Group offers an opportunity for all event participants from underrepresented or marginalized communities (including race, gender, sexual orientation, and disability), along with their allies, to come together and build meaningful connections that extend beyond the event. We hope that this gathering will help foster greater representation and inclusion both at the event and in the open source community over time.
*Who Can Attend?*
Any event participant from an underrepresented or marginalized community (including race, gender, sexual orientation, and disability) and their ally guests.
Grab your lunch and meet us at the reserved tables!
High availability in distributed systems is easy to claim but difficult to achieve under real-world failures. Outages, network partitions, and slow failure detection can silently degrade both search latency and indexing throughput.
This session presents a deep operational look at running OpenSearch with a zone-aware active–standby architecture that delivers predictable performance during failures. We explore how separating traffic routing from replication enables fast recovery without emergency shard movement or reactive capacity changes.
The talk covers how weighted shard copy selection keeps standby zones hot but idle during steady state, while safely failing open during read impairments. We then dive into write-path failures, where synchronous replication can amplify outages, and show how controlled I/O fencing isolate faulty nodes to restore write availability.
We will walk through coordinator behaviour, leader election during failovers, graceful isolation, and recovery back to steady state. Attendees will leave with practical techniques for minimising blast radius, reducing recovery time, and operating OpenSearch clusters that remain available when infrastructure fails.
Every Splunk-to-OpenSearch migration hits the same wall: thousands of SPL queries rewritten one by one, and every new migration starting from scratch. The real problem isn't conversion — it's that migration knowledge never compounds. What if each conversion taught the system something it could reuse on the next?
We built AI-powered migration skills — modular, reusable units of knowledge that encode how specific SPL patterns map to PPL on OpenSearch 3.4+. Skills are authored once and invoked automatically across migrations. Given a Splunk dashboard XML, the system selects the right skills to extract, convert, generate test data, and validate queries against a live cluster. We tested against a production SOC environment — 71 detection rules for threat hunting, incident triage, and alerting — achieving 86% full parity with zero manual rewriting, entirely skill-driven.
Attendees will learn how to author, test, and share migration skills, how skills compound across engagements making each migration faster than the last, and how to identify the 14% where skills flag architectural decisions rather than syntax fixes. Stop rewriting queries and start using skills.
As vector search workloads scale into hundreds of millions of high-dimensional embeddings, memory becomes the binding constraint. Even with compression techniques like SQ FP16 or PQ, there's room for techniques that compress further without sacrificing recall.
This talk covers the integration of Intel's Scalable Vector Search (SVS) into the OpenSearch k-NN plugin as a new method under the existing Faiss engine. SVS combines the Vamana graph algorithm with two Intel-developed compression techniques: LVQ (Locally-adaptive Vector Quantization) and LeanVec (learned dimensionality reduction plus LVQ). Both run entirely on compressed vectors at query time, meaning no full-precision data is touched during search.
On a 1M-vector, 768-dimensional benchmark (cohere-1M), SVS LeanVec delivered 2.57x the QPS of HNSW FP16 at comparable recall (~95.5%), with 36% less graph memory and 44% lower P99 latency.
Join this session to walk away with a clear understanding of when to choose SVS over HNSW, how to configure it in OpenSearch, what trade-offs to consider, and what performance gains to expect for semantic search and RAG workloads at scale.
Modern search has outgrown the false choice between exact words and intent. Technical terms often lack synonyms; vector search captures intent beyond keywords.
Hybrid search acknowledges language is both precise and ambiguous. The question is not whether to go hybrid, but how to do it economically when indexes no longer fit in memory.
In OpenSearch, hybrid search is orchestrated via the neural-search plugin, currently tightly coupled to the k-NN plugin as its vector backend, limiting newer engines like opensearch-jvector from using hybrid capabilities.
This talk shows how we enabled neural-search to work with opensearch-jvector, allowing hybrid search with an alternative vector engine.
We cover key challenges: feature parity gaps with k-NN, plugin dependency conflicts, and maintaining compatibility with existing neural-search workflows.
Attendees will get a behind-the-scenes view of integrating a new vector engine into OpenSearch hybrid search, tradeoffs between short- and long-term architecture, and lessons from building parallel JVector-enabled releases.
Audience: OpenSearch contributors and search engineers focused on hybrid search, vector engines, and plugin design.
How do you migrate a mission-critical autocomplete engine serving millions of users without breaking relevance, latency, or user trust? In this session, engineers from Booking.com share how they migrated their decade-old Lucene-based autocomplete platform to managed Amazon OpenSearch Service. The legacy system relied on tightly coupled custom ranking logic embedded in application servers, making relevance tuning and scalability increasingly difficult. This talk walks through the architecture of the new OpenSearch-based solution, including multilingual autocomplete modeling, custom analyzers, ranking replication using query DSL and function scoring, and the operational challenges of achieving ranking parity at scale. The session also covers how experimentation frameworks and phased rollouts were used to safely migrate traffic with zero user-visible regressions. Finally, the speakers will discuss the new capabilities unlocked after migration, including semantic search, AI-powered query completion, and cross-vertical search experiences. Attendees will leave with practical migration patterns for modernizing large-scale, latency-sensitive search systems using OpenSearch.
Security isn't just about firewalls and alerts, it’s about curiosity, context, and catching the weird stuff before it gets weird. But let’s be real: most teams don’t have the budget (or patience) for a million-dollar SIEM.
In this talk, I’ll walk you through how we turned OpenSearch into a scrappy, surprisingly powerful threat-hunting platform. Using native tools, open-source plugins, and a healthy dose of creativity, we built real-time alerting and investigation flows without blowing up costs. We'll cover:
• Designing log schemas that highlight anomalies.
• Building threat-detection pipelines using ingest processors and OpenSearch Dashboards.
• Real-life incident where OpenSearch helped us catch something our cloud provider missed.
If you’ve ever felt like security tools are either overkill or underwhelming, this session is for you. You’ll walk away with practical patterns and open-source recipes for turning OpenSearch into your security command center — no license key required.
Monitoring clusters is a challenging task, and having spent a decade operating hundreds of them at CERN, we know it. Fortunately, the open source ecosystem offers a wealth of tools that expose all necessary logs & metrics, allowing teams to reliably detect issues before they become critical.
In this talk, we’ll present our battle-tested, fully open source approach to monitoring large-scale OpenSearch deployments. We’ll cover:
- The architecture of our monitoring stack and how its components integrate end-to-end
- How we use GitOps to ensure every change is reviewable, traceable, and safely deployed
- The key metrics and signals we rely on to identify issues before they impact users
- Real-world alerting strategies that minimize false positives while catching genuine issues
- Visualization techniques and alerts documentation that accelerate root-cause analysis in complex environments
- Potential future enhancements, benefiting from OpenSearch roadmap
Whether you're building a monitoring stack from scratch or scaling an existing one, this session will provide actionable insights and practical examples you can implement immediately.
This is not just about single organisation but for every organisation as E-commerce platforms with catalogues in the hundreds of millions have tried pure vector search and found it beats BM25 on semantic recall but loses on exact-match queries, SKUs, brand names, model numbers. Pure BM25 wins on exact match but misses intent. Neither alone is good enough at sub-100ms latency and published tuning guidance for hybrid scoring in OpenSearch at this scale is essentially non-existent. This session presents a production hybrid scoring architecture: HNSW for semantic recall using OpenSearch's k-NN engine with a Faiss HNSW index, BM25 for exact-match precision, and a learned rank fusion layer that combines both scores using a lightweight cross-encoder re-ranker deployed via ML Commons. Benchmarks run at 500 million documents on a 6-node Kubernetes-deployed cluster show p99 latency below 80ms with measurably higher NDCG@10 than either approach alone.
In January we launched the Search Technical Advisory Group , the third TAG in the OpenSearch Project. This group, like other TAGS, is to provide strategic direction and technical guidance across an area. In this talk I'll share more about what TAG's provide, I'll give some specific results of the work of the Search TAG in 2026, and finally some lessons learned on how we got a diverse group of people all to think about the future of OpenSearch.
I'll also lay out how you can get involved in one of the four different TAGs in the OpenSearch Project.
At Transunion, we build solutions that generate actionable insights and provide “Information for good” for customers .
As Organizations adopt and build out Platforms and Apps that leverage AI Platforms, It becomes a key imperative to orchestrate across provisioning , realtime inference & feature tuning based on Operational and Business data stores.
Through this session we introduce Nextgen Orchestration with Opensearch Agent server in an AI native Enterprise service mesh comparing with OpenClaw and NemoClaw. The OpenSearch Agent Server is a multi-agent orchestration platform that enables you to build specialized AI agents ( like relevance agent ) that work together within OpenSearch.
We look at Leveraging OpenSearch and customizations that enable Agentic PoDs and AgenticOps – with necessary support for GenAI Commons driven Feature stores (Feast) , Model registries and Execution environments .Challenges & Lesson learnt in evolving App , Model / Inference Monitoring will help the Enterprise GenAI arch community.
Co-Author: Ramesh Kumar Manickam, TransUnion
This session presents a production framework for transforming retail purchase patterns into population-level health risk intelligence using OpenSearch — drawing on published research in JMIR AI on LLM-powered health signal detection in underserved communities.
Attendees will explore:
Indexing retail behavioral data in OpenSearch for health signal retrieval
Vector similarity search for detecting chronic disease risk patterns — CKD, cardiovascular disease, and mental health crisis indicators
Privacy-preserving query design on sensitive behavioral data
LLM-powered pipelines layered on OpenSearch for signal enrichment and health inference
This is not theoretical. These pipelines operate against production-scale retail data where retrieval precision directly affects whether an at-risk patient receives an intervention.
Search infrastructure has always connected people to information. This session shows how OpenSearch can connect people to care.
Most hybrid search implementations combine BM25 scoring on keyword matches with semantic search on dense vector embeddings and then fuse the results into a combined results list. Most agentic search approaches, likewise, treat lexical/BM25 and semantic/embedding search as independent tools that return separate sets of search results.
Wormhole vectors, in contrast, enable utilizing the underlying documents in your search engine to find shared meaning(s) and query intention across these different vector spaces (sparse lexical space vs. dense semantic space).
In this talk, we’ll show how wormhole vectors work and how to use them in OpenSearch to traverse between disparate vector spaces for better hybrid search. You’ll see how to jump back and forth between multiple dense and sparse vector spaces in the same query, and we’ll show benchmarks and examples of how wormhole vectors compare to other leading query models and approaches like SPLADE and hybrid fusion algorithms. We’ll show with open source code how to implement these wormhole vectors in OpenSearch, demonstrating their impact on search quality for hybrid and agentic search.
By invitation only.
The Partner Roundtable is a great way to interact with OpenSearch leadership and have an open dialogue about the project. We will discuss strategy, challenges and opportunities, community and collaboration, future trends, training and enablement, and project growth. And of course, you are encouraged to openly exchange ideas and experiences, ask questions, share your thoughts, and engage in meaningful discussion.
Running an observability and logging stack on kubernetes sounds straightforward until retention, indexing pressure, disk usage, cluster state, and performance all collide. This talk dives into the real world experience of deploying, operating, and scaling OpenSearch on kubernetes as the primary logging and search platform across multiple clusters and cloud providers.
We begin by exploring the challenges that arise when ingesting logs from diverse workloads: high volume control plane logs, application logs with unpredictable spikes, and multi-cluster ingestion. We’ll walk through how the OpenSearch cluster was designed, tuned, and hardened to handle the load; covering everything from sharding strategies, ISM policies to storage choices.
We also cover the ecosystem around OpenSearch: FluentBit/FluentD ingestion, index templates, dashboards, alerting, and the operational guardrails that keep the system reliable. Expect practical insights into failure modes, disk pressure, cluster red/yellow recovery, snapshot/restore strategies, and zero-downtime operations on Kubernetes.
OpenSearch Dashboards (OSD) ships as the analytics UI for an OpenSearch cluster, and most teams stop there. This talk covers two OSD extension patterns from a compliance platform built on OSD.
Part one: a registry-driven visualization library. Custom React and D3 visuals register through OSD's vis builder, with declared dependencies driving the editor's UI. Decoupled fetching lets a single dashboard mix native panels with custom ones reading from external data sources, with animations, loading states, and tooltips.
Part two: getting those visuals in front of your users. End users author dashboards in OSD's tooling, leveraging aspects from part one. Your plugin embeds them directly into plugin pages, surrounded by custom UI.
Live code walkthroughs, architectural gotchas, and what’s next.
Key Takeaways
1. The Registry Pattern: Auto-derive configuration UIs from declared dependencies, eliminating per-visual boilerplate.
2. Multi-Source Rendering: Visualize Postgres or API data while utilizing OSD's drag-and-drop layout, resizable panels, filter bar, and time picker.
3. Seamless Embedding: Embed a dashboard inside a plugin page from its ID.
Something uncomfortable is happening in our industry. Search practitioners — people who have spent years mastering relevance, ranking, and retrieval — are finding themselves excluded from conversations about RAG pipelines, agent architectures, and AI knowledge layers. The assumption, often unspoken, is that retrieval is an AI problem now — a solved one, at that.
This keynote argues the opposite.
The "R" in RAG is not a footnote — it is the hardest part. Fetching a document is easy. Orchestrating the right context across structured and unstructured data, permissions, freshness, and intent — at scale, in real time, for both humans and AI agents — is a search problem. It always was. The expertise that this community has built over years is not becoming obsolete. It is becoming essential.
But there is a catch. AI practitioners are not going to become search experts. They want good retrieval as a service — fast, accurate, and invisible. If OpenSearch is to own that layer, we need to think differently about how we position what we do: not as a search engine that also supports vectors, but as the system that makes AI systems trustworthy.
This talk introduces a simple frame: little-r retrieval (fetching documents) versus big-R Retrieval (the context infrastructure that AI depends on). It explores what it means for OpenSearch — and for this community — to deliberately claim the big-R role. And it makes the case that the rise of AI is not the moment we get left behind. It is the moment we have been building towards.
At Aiven, we've built our OpenSearch offering to meet developers and enterprises wherever they run, on whichever cloud they choose, and at whatever stage they're at. We’re committed to making OpenSearch an option for everyone, whether they are migrating from another platform, starting from scratch or adding OpenSearch to existing infrastructure.
This talk shows how we do it: from a newly merged Terraform provider update that brings GCP into our Elasticsearch-to-OpenSearch migration path, to full multi-cloud coverage across nine providers including (AWS, GCP, Azure, and others). We'll share how our startup program fuels the OpenSearch community, spotlighting Klipy, which runs on Aiven for OpenSearch to power search across Discord, Bluesky, and BeReal. We'll cover what's new under the hood — expanded node roles, tiered storage, and our path toward ML node support with CPU and GPU inference — and how we serve everyone from large enterprise customers to startups to solo builders on our free and developer tiers. Along the way, we'll showcase our commitment to open source, including our recent work on the OpenSearch Migration Assistant. Finally, we'll explain why we commit to LTS versions like 2.19 and 3.6, rolling every minor and patch release, so teams can upgrade with confidence.
Our job at Aiven is to make OpenSearch accessible, dependable, portable and capable – for anyone building on it.
OpenSearch has quietly become critical AI infrastructure, powering RAG, semantic search, and agent memory across production systems. As it shifts from indexing web pages to storing vector embeddings of proprietary data, the stakes change: a compromised vector index isn't a search problem, it's a window into everything your AI systems know.
This talk examines why data protection deserves fresh scrutiny in the AI era: what makes vector embeddings a sensitive attack surface, how encryption at rest and in transit intersects with retrieval performance, and what governance teams should require before connecting OpenSearch to production AI pipelines. We'll use the open source Security plugin's encryption features as one example of where the ecosystem is headed, plus a framework for assessing whether an AI deployment is truly protected.
Engineers, architects, and governance leaders will leave with concrete questions for their own security posture, before an incident forces the conversation.
Organizations increasingly use Apache Iceberg as the source of truth in their data platforms, with curated and transformed data managed in lakehouse tables. Making this data searchable in OpenSearch, or writing pipeline data into Iceberg for long-term analytics, typically requires custom jobs and separate tooling.
This session introduces two new Data Prepper plugins that address both directions natively.
The Iceberg CDC Source captures row-level changes from Iceberg tables and delivers them to OpenSearch, keeping it synchronized with the lakehouse without dual-writing.
The Iceberg Sink writes Data Prepper events into Iceberg tables, supporting row-level operations. We will cover the coordinated commit architecture for multi-node writes and equality delete handling.
We will walk through how to use these plugins and dive into the internals, including how they efficiently detect and process changes at scale through distributed coordination and fault tolerance.
Operating massive streaming clusters shouldn't require an army of middleware. Today’s Pull-Based Ingestion often hits two walls: rigid 1:1 shard-to-partition mapping and a lack of native data transformation. This forces teams to manage costly, complex external layers like Flink or Kafka Streams just to prep data for the cluster.
We’re tearing down these walls. This session introduces a production-ready evolution of PBI that decouples your search cluster from your streaming infrastructure. By wiring Ingest Pipelines directly into the pull path and enabling flexible multi-partition mapping, we eliminate the "middleware tax" and operational friction.
We’ll dive into:
🔹 Flexible Sharding: Decoupling OpenSearch shards from Kafka partitions for truly independent scaling.
🔹 Native Transformation: Replacing external processors with built-in Ingest Pipelines to simplify architecture.
🔹 Operational Resilience: Advanced offset management and partition-aware tracking for real-world debugging.
Learn how to shrink your infrastructure footprint, slash TCO, and build a self-scaling ingestion engine that handles enterprise-scale spikes without the architectural bloat.
What if your search index could improve itself? Your OpenSearch documents are the best training data for your own search model — but turning that insight into a production system requires solving hard engineering challenges.
Earlier this year at OpenSearchCon China, community members introduced the vision of using index data for model fine-tuning. In this talk, we take it from concept to reality: we present the production pipeline we built and are shipping as part of Amazon OpenSearch Service Serverless, and show you how to apply the same pattern on any OpenSearch cluster.
Through a live demo, we'll show the full closed loop in action:
• Extracting documents and generating synthetic training queries via LLMs
• Fine-tuning an embedding model on your domain-specific data
• Deploying new embeddings back to a live index with zero downtime
• Guaranteeing 100% consistency with automatic retry and orchestration
Whether you run OpenSearch Serverless or a self-hosted cluster, you'll leave with a proven blueprint for continuously improving search relevance — using the data you already have.
Late interaction models such as ColBERT have pushed semantic retrieval quality forward by using multiple embeddings per document instead of a single dense vector. However, production deployments face significant challenges: large token-level indexes, expensive MaxSim, high memory consumption, and increased query latency.
This session explores how MUVERA (Multi-Vector Retrieval via Fixed Dimensional Encodings) can make multi-vector retrieval substantially more efficient in OpenSearch. Rather than searching directly over token embeddings, MUVERA transforms multi-vector representations into compact fixed-dimensional encodings that preserve late-interaction semantics while enabling retrieval through traditional ANN pipelines such as HNSW.
Covers the architectural implications of this approach, including candidate generation, reranking stages, memory and latency tradeoffs, and how MUVERA changes the scalability profile of late interaction retrieval systems. The session focuses on practical system design considerations for building large-scale semantic search and RAG applications in OpenSearch without the infrastructure overhead traditionally associated with multi-vector retrieval.
Most engineers building AI agents hit the same wall. The agent gives a wrong answer, or a slow one, or behaves differently on 15% of queries. You look at the logs and see: success. No exception. No timeout. Nothing to go on.
This talk is about applying what the distributed systems community already knows — instrument it, trace it, query it — to a class of failures that logs were never designed to catch.
We walk through a Python incident-investigation agent along with OpenTelemetry spans flowing into OpenSearch 3.6. OpenSearch 3.6 ships Agent Traces, which gives you a DAG view and token usage for a single execution. On stage, we show what comes next: PPL queries across 500 runs that surface a silent retry storm, a latency boundary hidden inside a healthy-looking average, and a context overflow that caused the agent to ignore everything it retrieved.
All three are patterns any team running agents in production will encounter. None of them show up in application logs.
The goal is that anyone in the room can apply this to their own agents the following week.
Repo – github.com/eilhamv/cfp-opensearch
Running OpenSearch in multiple geographical regions is important for high availability and for localized queries. There are a few approaches to cross-region replication in OpenSearch. Cross-cluster replication provides one solution, and we'll look at where it fits best. Another approach that works for many use cases is to use Data Prepper along with Kafka as the cross-region replication mechanism. We will explore how to use Data Prepper with OpenSearch's pull-based ingestion as one approach, and we will also see how you can craft another solution with forward_to pipelines. We will discuss the advantages of each approach and help you decide which is right for your workload.
Most developers treat OpenSearch Dashboards (OSD) strictly as a visualization layer—a place for charts and graphs. They are missing the bigger picture. OSD is a powerful, untapped environment for full-stack application development. In this session, we peel back the architecture of an enterprise-grade risk, compliance, and security platform built entirely on top of OSD. We moved beyond simple plugins and pushed OSD to its limits to create a seamless, interactive application experience.
Join us for a deep dive into the engineering reality of the OpenSearch stack, including:
The Analysis: Why the OpenSearch stack beat out the competition.
The Build, Beyond Visualization: How to implement custom React components to build complex workflows and interactive UIs within OSD.
The Integrations: Integrating Wazuh for agent management and ingesting data at scale.
The “Gotchas”: Honest, real-world lessons on data handling, state management, and what to look out for.
If you are ready to push your plugins from “reporting tools” to “robust software,” do not miss this session!
As privacy regulations reduce access to traditional tracking data, organizations need new ways to generate marketing insights while protecting user information. This presentation explores how OpenSearch supports AI-driven advertising analytics through scalable indexing, real-time querying, and intelligent observability pipelines.
The session highlights modeled attribution, vector search, anomaly detection, and predictive analytics built on OpenSearch infrastructure. Attendees will learn how AI techniques such as semantic search and neural inference improve campaign analysis despite data signal loss. The presentation also covers operational strategies for deploying OpenSearch clusters for privacy-aware analytics and high-performance monitoring. By combining search technology with AI, organizations can build secure and efficient measurement systems for modern digital advertising.
Migrating a production search cluster isn't just a technical "copy-paste" of data; it’s a high-stakes business transition. If a migration is technically successful but search relevance drops the project is a failure because of lower conversion rates or frustrated users.
In this session, we introduce the OpenSearch Migration Companion, a set of tools designed to ensure that migrations are guided by business requirements and search quality parity. We move beyond simple data transfer to a migration experience that is fully autonomous. We will demonstrate how to move from legacy search technologies to OpenSearch using a framework that prioritizes:
- Functional Parity: Not just "Does the query run?" but "Do the results match?"
- Search Quality Guardrails: Using automated traffic replay and comparison to detect relevance regressions before you cut over.
- Operational Confidence: Managing business risk through gradual rollovers and automated rollback triggers.
Whether you are migrating from Solr or older versions of Elasticsearch, this talk provides the blueprint for a "no-surprises" migration that keeps your business goals and your search quality intact.
Combining keyword and semantic search brings clear relevance gains, but choosing how to combine them is where teams get stuck. This session compares three pipeline architectures on OpenSearch: classic keyword + vector, sparse + dense neural retrieval, and multi-stage pipelines with reranking. We run each architecture against the same datasets and measure relevance, latency, and infrastructure cost. We walk through how OpenSearch's search pipeline processors normalize and merge scores from different retrieval methods, and share practical guidance on which architecture fits which workload. Attendees leave with a practical framework for evaluating hybrid search tradeoffs and choosing the right architecture for their workload.
Most RAG systems have a retrieval problem. The language model gets the blame, but the real culprit is upstream in the search layer deciding what context the model ever sees in the first place. Fix the retrieval, and the answers fix themselves.
This talk is for developers and ML engineers who have shipped a RAG system with OpenSearch and are wondering why the answers aren't as good as the demos suggested. No retrieval evaluation experience needed.
Through live demos on a real OpenSearch index, the session will show you how to see exactly what your retrieval layer is returning, identify the failure modes that silently kill answer quality, and apply targeted fixes without rearchitecting your stack. You’ll leave knowing where to look, what to measure, and what to do about it.
Teams running Prometheus for metrics and OpenSearch for logs and traces operate two separate monitoring stacks. Metrics in one silo, everything else in another. This session closes that gap.
We will see how to build a Data Prepper pipeline that ingests Prometheus metrics into OpenSearch's TSDB engine. In testing with over 30 million real samples, the TSDB index used 22x less storage than a standard index for the same data. The per metric aggregation query that timed out after 30 seconds on the standard index completed on TSDB in under a second. All four query paths work: PromQL, M3QL, timeseriesunfold, and DSL.
I will walk through the ingestion architecture, the two pass algorithm that reconstructs histograms and summaries from Remote Write's flat format, and the TSDB document conversion. I will run a live demo with PromQL queries against continuously ingested data and show the storage and query comparison side by side.
Key Takeaways
– A working Prometheus to OpenSearch TSDB pipeline with 22x storage reduction and PromQL support
– The two pass grouping algo that solves Remote Write's missing type metadata problem
– How TSDB changes the cost and query model for metrics at scale
Modern AI systems can appear confident while being surprisingly fragile. Change the order of the same evidence, rearrange the prompt, or present supporting facts in a slightly different sequence, and the model may produce a different answer. That is not always a retrieval problem. It is often a context presentation problem. This session explores why the layout of information matters, why repeated exposure to the same input can improve results, and what that tells us about how models process evidence before generating a response.
Using OpenSearch as the foundation, this session examines how RAG agent responses change when the same knowledge is retrieved, ranked, and presented in different ways. We will explore practical patterns for structuring search results, document chunks, tool outputs, and evidence chains to make models less sensitive to arbitrary ordering. Attendees will see how changes in context presentation can alter an agent's answer, then learn techniques for testing input-order sensitivity, reducing permutation-induced hallucinations, and building production AI systems that reason from stable evidence instead of whatever sequence the pipeline happened to cough up.
It’s safe to say Unconference is now an OpenSearchCon tradition. Each time the community gets together for this event, we learn from one another in an engaging environment with interesting talks and open discussions.
Make plans to join us again and come ready to pitch your favorite speaking topic! This is an opportunity for the community to come together at OpenSearchCon North America for an action-packed afternoon of sharing and discovery. With no pre-planned talks, what you want to hear about will be determined by you and your fellow conference-goers.
Attendees/Speakers
This is a first-come, first-served event, and anyone registered for OpenSearchCon is welcome to attend until the room is full. We advise you to show up a little early to make sure you get your spot.
Each speaker has 15 minutes to do with as they see fit! Want to talk for 10 minutes and have 5 minutes of questions? Great! Have a lot to say and want to talk for the whole 15? That works too! Are you a maintainer and want to hold a lightning round of questions with the audience? Fantastic! You get the picture. Just be mindful of your 15 minutes!
Voting
– At 1:30 PM, each attendee will receive a card and three gold-star stickers. Those who would like to give a talk will write their title and brief description on the card, put their name on the back, and post them to the board for voting.
– At 1:45 PM, you’ll have 15 minutes to walk the board and place a gold star on one of the talks you would like to hear. One rule: please do not vote for yourself.
– At 2:00 PM, our host, Kris Freedain, and a lucky volunteer will collect and sort the cards, then select the day’s talks from the cards receiving the most votes while ensuring the widest arrangement of topics are covered.
Presentations
At 2:15PM, we will return the cards to the wall in the order each talk will be given. Be ready to talk and participate!
Questions?
We look forward to seeing you there! Please reach out to [email protected] with any questions.
OpenSearch ingestion pipelines have traditionally been built around document-oriented JSON processing, optimized for indexing and search workloads. However, modern data systems are increasingly adopting columnar and vectorized architectures to improve throughput, interoperability, and analytical efficiency at scale.
In this talk, we explore how Apache Arrow can enable a new ingestion model for OpenSearch based on columnar RecordBatches instead of individual documents. We will discuss the architectural motivations behind moving toward columnar ingestion, the performance and memory-efficiency benefits of vectorized processing, and how Arrow integrates naturally with modern ecosystems such as Parquet, DataFusion, and lakehouse-style analytics platforms.
The session will also cover practical considerations including schema evolution, dynamic mappings, batching strategies, and balancing search-oriented and analytics-oriented workloads. Attendees will gain insight into how columnar data pipelines could reshape the future architecture of OpenSearch ingestion systems.
Agentic search is an exciting new frontier, but most solutions focus only on what happens after the query arrives: agentic orchestration of retrievers, rewriting queries, reranking results. We'll go further: the same agent reasoning that adapts queries at runtime can also reshape what gets indexed and how. Using Lucille, an open source ETL framework, as the ingestion backbone, we'll show how agents can augment documents, analyze content for retrieval-relevant signals, and align the OpenSearch index with the agentic tooling that will query it. On the query side, we'll cover query understanding, adaptive hybrid search, query rewriting, and reranking driven by live diagnostics: confidence gaps, score variance, agent reasoning. And we'll close the loop with LLM-as-judge signals that flow back into ingestion decisions. Demos will draw from real use cases where retrieval problems unsolvable through query tuning alone became tractable by re-modeling the data with agentic help.
Attendees will gain a blueprint for building search systems that think end-to-end, from smarter indexing to self-improving retrieval and relevance, and the open source tools to start building one today.
The OpenSearch Technical Steering Committee (TSC) serves as the governing body responsible for the technical direction and community stewardship of the OpenSearch Project. This presentation will provide an overview of the TSC's structure, responsibilities, and decision-making processes that guide one of the most dynamic open-source search and analytics projects.
As TSC chair, I'll share insights into our work throughout 2025 and our vision for the future. We'll discuss what went well and how we plan to keep improving to ensure OpenSearch remains a truly community-driven project.
Attendees will gain practical knowledge about the project's governance model and leave with a clear understanding of how they can engage with and contribute to the OpenSearch Project's future.
AI inference failures are rarely obvious. A slow response might be a saturated GPU, a bloated prompt, a cold cache, or a backed-up model queue. Without unified visibility, every investigation starts from scratch. This talk shows how to instrument an AI inference stack using OpenTelemetry and OpenSearch so that model serving, context assembly, and cost signals are observable together in one place.
We will trace a single user request from input to generated response, capturing model latency, queue time, token counts, GPU and CPU utilization, cache hit rates, and p95 latency, and show how to correlate these signals into dashboards that make inference behavior explainable rather than assumed. Where retrieval is involved, it appears as one span in the trace, not the centerpiece: the focus stays on what happens inside the inference path.
The talk addresses practical operating questions teams actually face: which spans deserve attention, which attributes are worth indexing, how to keep telemetry cardinality manageable at scale, and what views help separate a prompt-size problem from a capacity problem.
Coding agents perform well in verifiable loops—compilers, tests, and static analysis—but often struggle in large codebases where retrieval becomes the bottleneck. This session explores how improving search, rather than scaling models, can significantly boost agent reliability.
We examine three complementary approaches: lexical (keyword-based), structural (code-aware), and semantic (embedding-driven) search, along with hybrid pipelines that combine their strengths. Attendees will learn where each method excels, where it fails, and how to tune relevance for real-world repositories.
The talk also covers indexing strategies, evaluation techniques, and practical ways to reduce token waste while improving answer quality. A live demo showcases how OpenSearch-powered retrieval enhances coding agents, providing cleaner context and more stable outcomes.
Rather than replacing AI, better search makes it work.
Prometheus and Grafana are excellent, until they are not. Across fleets of hundreds of clusters running thousands of applications, the stack runs into real limits. Cross-cluster aggregations explode cardinality. Log and metric correlation requires two separate query paths. Cost attribution across many tenants becomes something Prometheus was never designed to answer. These are not edge cases. They are the predictable physics of running this stack at large multi-cluster scale, and they show up the same way regardless of which company is hitting them.
This session is a practitioner's honest look at where the Prometheus and Grafana model hits its ceiling at fleet scale, and a concrete look at where alternatives like OpenSearch fit into the tradeoff space. As a project reviewer for kube-state-metrics, the CNCF tool that is the source of truth for Kubernetes object state across every major cloud provider, I will walk through how Kubernetes state data actually flows and why certain query patterns fall apart as cluster counts grow.
You will leave with the query patterns where Prometheus falls over at scale and why, how teams approach routing Kubernetes state data into log and search backends, the schema and retention strategies that hold up across large cluster counts, and a clear eyed take on what Prometheus is still genuinely better at. Real failure modes and real tradeoffs.
At a certain scale, your "standard" indexing strategy stops being an asset and becomes a liability. When you cross the threshold of 10 billion documents, mistakes made in shard sizing, rollover policies, and tiering strategies don't just cause latency—they cause cluster-wide outages. Managing this volume requires moving beyond basic automation to a precision-engineered Index Lifecycle Management (ILM) strategy.
This session dives into the technical "horror stories" of scaling OpenSearch clusters to the multi-billion document range. We will explore the architectural trade-offs between storage density and search performance, the nuances of hot-warm-cold-frozen tiering, and the critical role of Data Prepper and Kafka in buffering ingestion spikes. Attendees will learn how to design a resilient ILM policy that balances cost, performance, and operational sanity.
Most RAG implementations suffer from stale vector stores — knowledge bases that update daily or weekly while AI agents need current context to make good decisions. For use cases like customer experience optimization or live content recommendation, that lag makes agents unreliable.
This session tackles the stale data problem head-on. We'll walk through a real-time ingestion architecture that continuously streams data into OpenSearch's vector database using Data Prepper, keeping embeddings fresh for agentic AI workflows. We'll cover real-time vector indexing with the k-NN plugin, neural search pipelines serving fresh results, and practical patterns for balancing ingestion throughput with search latency.
Attendees will leave with a reusable architecture for building RAG systems that never go stale.
This talk will explore how Opensearch within IBM is addressing the need for search in the AI era for its customers. We will touch upon some key technical innovations involved to bring this to reality.
The opensearch-jvector plugin, which is at the crux of our architecture is not a stranger to OpenSearch anymore. We will go in-depth about our plugin readiness journey to make it enterprise production grade, its building blocks such as the Jvector library, how we depend on core OpenSearch libraries such as Apache Lucene and what sets it apart. Furthermore, this session will go deep into some of the recent innovations and core concepts involved, followed by some quantitative benchmarks to show their effectiveness.
Through this talk, attendees will learn about the broader hybrid search problem space, the concepts and approaches used to build an actual service using OpenSearch, and our learnings through this journey to serve real world workloads.
AI agents are increasingly connecting to OpenSearch clusters via MCP to search, analyze, and manage data. But across the MCP ecosystem, most deployments still rely on static credentials or no authentication at all, giving agents broad access with no per-user identity, limited audit trails, and no defense against prompt injection steering agents into unsafe operations.
This talk examines the current security posture of AI agent access to OpenSearch via MCP and presents a layered approach to hardening it. We will cover OAuth/OIDC integration for per-user token forwarding, role-based tool authorization to enforce least privilege, audit logging that distinguishes AI-initiated from human-initiated actions, and mitigations for prompt injection attacks that could lead to data exfiltration or unintended mutations.
Attendees will walk away with a practical security checklist for deploying AI agents against OpenSearch clusters, an understanding of the threat model unique to agentic access patterns, and concrete architectural patterns for authentication, authorization, and observability in AI-to-OpenSearch workflows.
SAP runs one of the world's largest OpenSearch deployments, with over 18,000 instances powering solutions like SAP Cloud Logging across its Business Technology Platform. In this keynote, Wolfgang Theilmann shares how OpenSearch became the observability foundation for thousands of SAP cloud workloads, some of the enterprise-grade capabilities SAP has contributed back to the project, and how SAP is exploring the next evolution of observability.
Millions of users search in languages that mainstream platforms barely recognise. Low-resource languages, from Twi to Tigrinya, remain second-class citizens in search relevance and NLP support, ignored not out of malice but out of prioritisation. The OpenSearch community can change that.
This session explores how to build OpenSearch plugins that extend search relevance and NLP capabilities for underserved languages, using an Agile delivery mindset and quality-first engineering practices. Drawing from a background in QA and Agile delivery management, this talk walks through how contributors can apply iterative development, acceptance criteria, and structured testing frameworks to ship plugins that are reliable, inclusive, and production-ready.
I am not a linguist or a core search engineer. I am someone who ships quality software iteratively. And that perspective turns out to be exactly what plugin development for underserved languages needs.
Building for underserved languages is not a niche problem. It is one of the most meaningful things this community can do.
Join us for an interactive "Ask the Experts" Maintainer Panel and take advantage of a unique opportunity to gain insights from core maintainers from OpenSearch and across the open source ecosystem.
Moderated by Senior Community Manager Kris Freedain, this session brings maintainers together to explore the work of building an enterprise-grade platform in the open. From navigating complex roadmap trade-offs and scaling search performance to managing contributor dynamics and AI-fueled contributions, panelists will share their firsthand perspectives on the triumphs and realities of open-source software maintenance.
Organizations store critical knowledge across dozens of systems — object stores, ticketing platform, messaging platforms, RDMS and many other sources. Users expect a single search experience that finds the right information regardless of where it lives, respects access permissions instantly, and increasingly, answers questions rather than returning document lists. This session presents a five-component architecture for federated enterprise search: per-source indexing for relevance optimization, built-in access control for sub-second security enforcement, two-phase search for accurate facet counts, Reciprocal Rank Fusion for fair cross-source ranking, and an Agentic AI layer that enables conversational, multi-step queries like "Find all open tickets related to the API migration, cross-reference with design docs, and summarize the blockers." Built on Amazon OpenSearch , this architecture is production-ready today. Attendees will leave with a clear blueprint for transforming their enterprise search from keyword-based document retrieval into an intelligent, conversational knowledge discovery platform.
Most teams use OpenSearch as a sophisticated log dump — data goes in, dashboards show counts, nothing hunts proactively. Yet OpenSearch ships with anomaly detection, log pattern analysis, and ML-powered alerting out of the box, all sitting idle behind a few unchecked configuration switches.
In exactly 15 minutes this session will show — live — how to transform a vanilla log ingestion setup into an active threat intelligence engine. We activate OpenSearch's anomaly detection on authentication logs to catch credential-stuffing attacks invisible to static threshold alerts, use the log pattern analyzer to cluster novel error signatures that precede system compromise, and wire everything into a real-time notification channel that fires before a human would notice. No external SIEM. No added infrastructure. No budget increase. Just OpenSearch features your cluster already has, finally doing work.
Bring your agents and services and learn to trace, evaluate, and stop them from silently going off the rails — with OpenTelemetry, Data Prepper, and OpenSearch.
Your agents and the services they depend on are running in production — but are they actually doing what you think they're doing? In this hands-on workshop, you'll build an end-to-end observability pipeline for AI agents and services, from first span to production monitoring, on a fully OpenTelemetry-native foundation. Guided step-by-step, you'll instrument an agent and its services with OpenTelemetry, route traces through Data Prepper into OpenSearch, and collect OTel-native metrics via Prometheus — no proprietary agents or vendor lock-in — then generate service maps that reveal how requests actually flow across agents and services. From there you'll go beyond dashboards: reconstruct an agent's reasoning path with PPL, correlate it against service RED metrics, score runs against golden paths to catch regressions in correctness, tool selection, cost, and latency, and feed production failures back into your eval set to close the loop. Whether you're replacing fragmented monitoring or bringing agentic workloads under control, you'll leave with a fully functional, OpenTelemetry-native OTel-to-OpenSearch stack — and the know-how to deploy and scale it in your own environment.
What you'll bring: A laptop with docker installed with minimum 8GB Ram for docker and a desire to build.
What you'll leave with: A working OTel-to-OpenSearch pipeline for agents and services, a regression-ready eval suite, and the confidence to scale it.
All lab assets — the multi-framework sample agent (LangGraph, LangChain, LlamaIndex, Bedrock, Strands, and more), observability wiring, dataset, and evaluation criteria — are provided from the workshop repo.
Enterprise RAG systems operate under constraints very different from open-domain AI: documentation changes continuously, versioning impacts answer correctness, and retrieval failures lead directly to customer support escalations.
This talk presents a production RAG architecture deployed for Adobe’s AI Assistant, built on a hybrid sparse and dense retrieval stack designed to maintain quality under continuous corpus change. The system manages 121,000+ enterprise documents with version-aware retrieval to ensure both temporal and semantic correctness as content evolves.
Training data is scaled using synthetic generation and user interaction mining, producing 700,000+ closed-domain QA pairs. The retrieval layer achieves 72.8% nDCG@4 while maintaining sub-200 ms P95 latency to meet production SLAs. Embedding selection balances retrieval quality with inference constraints.
The system integrates automated offline evaluation and structured human review to prevent regressions. Attendees will gain practical design patterns for hybrid retrieval, version-aware indexing, and evaluation strategies for reliable enterprise-scale RAG.
Since OpenSearchCon US 2025, OpenSearch has transformed from a traditional retrieval system into agent-native infrastructure for the generative AI era. Tthis FAST PACED talk walks through what's new across versions 3.3–3.8 and what it means for builders LIKE YOU.
We'll explore how Agentic Search replaces complex query DSL with natural language, how Agentic Memory enables coherent multi-turn conversations, and how protocols like AG-UI and OpenSearch Agent Server and Agentskills simplify agent integration. On performance, we'll cover the gains to search quality and search performance.
We'll also tour the features democratizing search: OpenSearch Launchpad's AI-guided app scaffolding, the Relevance Workbench's "LLM as judge," the newly launched Relevance Agent for automated tuning, and Agent Health's OTel-native trace observability for debugging AI reasoning chains.
Plus we’ll talk about how Migration Companion drastically simplifies the process of moving search workloads from other search engines to OpenSearch with native tooling.
We will briefly touch on what 2027 will hold.
You will leave with a clear map of what's production-ready, what's experimental, and where to start.
OpenSearch provides great tools for scaling immutable data like logs and analytics with components like index lifecycle management. But how can massive amounts of mutable data be scaled most efficiently?
In this session, we will explore dynamic scaling strategies within OpenSearch, evaluating the architectural trade-offs for managing massive volumes of mutable text content.
We will discuss:
1. Automated Shard Splitting: How we leveraged the OpenSearch split-index API to scale indices as they grow.
2. Dynamic Shard Advising: How we built a shard advisor that monitors real-time primary shard metrics and uses data density to trigger scaling actions.
3. State-Machine Orchestration: We explain how to safely transition multi-index ingest jobs, handle updates in a distributed environment, and ensure zero data loss during maintenance.
4. Handling Ingest Backpressure: Strategies for managing backpressure in text-heavy pipelines.
5. Kubernetes-Native Scaling: Leveraging the OpenSearch operator for seamless horizontal expansion.
Ingesting large datasets into OpenSearch can be slow if you do not have the right client architecture. Many developers start with simple scripts in Python or Java but eventually hit performance walls when the data volume grows.
In this talk, I will show how to solve this using Go. The language is naturally good at concurrency and is perfect for building high-performance data pipelines.
I will walk through how to write an ingestion tool from scratch that uses the opensearch-go client effectively. We will look at using the worker pool pattern to send data in parallel, how to handle backpressure so we do not overload the cluster, and how to manage retries without losing data.
This will be a practical session with real code examples. By the end, you will know how to structure a Go application that can index data much faster than standard bulk API scripts.
Engineering teams generate large amounts of operational data through tools such as Jira, including issue histories, workflow transitions, and sprint activities. While this data is useful for tracking work, it is rarely indexed or analyzed at scale for deeper insights.
This session explores how teams can transform Jira data into searchable analytics datasets using modern data pipelines and OpenSearch. Attendees will learn how to extract structured data using JQL queries and the Jira REST API, transform JSON datasets, and index them into OpenSearch for fast querying and visualization.
The presentation will also discuss how automation workflows can continuously push operational project data into OpenSearch, enabling teams to monitor delivery trends, analyze engineering performance, and detect workflow patterns in real time. Practical examples will demonstrate how search based analytics can improve reporting, observability, and operational visibility across engineering organizations.
Participants will leave with practical strategies for building scalable data pipelines that convert operational project data into searchable insights using OpenSearch.
Agentic AI systems are moving from experiments to production—but most teams lack visibility into how agents
think, coordinate, and evolve. This session explores using OpenSearch as the cognitive backbone for modern agent
frameworks like Strands and LangGraph, positioning it as: a long-term memory store, reasoning trace index, multi-
agent coordination layer, and observability platform.
Through a live architecture walkthrough, we'll build a multi-agent system where every planning step, tool
invocation, state transition, and outcome is indexed in OpenSearch. We'll demonstrate how to debug hallucinations
using reasoning traces, replay agent decisions across sessions, analyze performance with hybrid search, detect
behavioral drift, and coordinate multiple agents through search-backed state.
Attendees will gain practical architectural patterns for running stateful, observable, production-grade AI agents
using OpenSearch as core infrastructure—beyond basic retrieval.
Key Takeaways:
• Design stateful agent memory architectures
• Implement agent observability pipelines
• Use OpenSearch for reasoning trace analysis
• Coordinate multi-agent systems via indexed state
Stop settling for basic keyword matches and start building search systems that actually reason. In this hands-on workshop, you’ll roll up your sleeves and construct an autonomous AI search application from scratch—moving beyond traditional retrieval to dynamic, multi-step agent workflows. Guided step-by-step, you’ll learn how to combine vector search, reasoning loops, and tool use to turn raw queries into intelligent, actionable answers. Whether you’re looking to upgrade a basic RAG setup or master the mechanics of agentic data retrieval, you’ll leave this session with a fully functional application—and the practical know-how to deploy it in your own environment.
What you’ll bring: A laptop with Workshop Studio and Jupyter notebooks installed, and a desire to build.
What you’ll leave with: A working agentic AI search app and the confidence to scale it.
To build a good search experience, you need to know exactly what your users are doing. The OpenSearch User Behavior Insights (UBI) framework exists for this – but what happens when the legal department says you can’t store the queries? In high-stakes industries like healthcare and finance, zero-trust is no longer just a buzzword because now it is a blocker to relevance tuning.
This session demonstrates using Local Differential Privacy (LDP) to privatize user search queries and interaction logs directly on the client’s device before they reach an OpenSearch cluster. By injecting noise at the source, we can capture the behavioral trends needed for learning-to-rank (LTR) and query-intent analysis without ever seeing the raw PII. Instead of just hoping the data is actually private, we demonstrate how to verify the injected noise.
Attendees will learn how to balance privacy and search utility and move away from reactive data scrubbing using OpenSearch and UBI.
Vector search has quickly become a foundational capability for modern AI applications, powering semantic search, recommendation engines, RAG pipelines, and AI agents. However, moving vector workloads from prototype to production introduces significant operational and architectural challenges around scale, latency, memory consumption, indexing throughput, and infrastructure cost.
This session explores real-world architecture patterns for running large-scale vector search workloads on OpenSearch efficiently and reliably. Attendees will learn how to design production-ready vector search platforms that support high ingestion rates, low-latency retrieval, and growing embedding volumes while maintaining operational stability.
Topics covered include:
designing scalable vector indexing architectures
shard sizing and distribution strategies for vector-heavy clusters
tuning ANN algorithms for latency and recall
balancing vector search with traditional lexical search
hybrid retrieval architectures for enterprise AI applications
Search has changed. Users expect natural language, semantic answers, and sub-second latency — without a PhD in information retrieval. OpenSearch has quietly shipped a stack that makes this achievable for any team.
A practitioner's tour. No search fundamentals retread — just what each feature does, when to reach for it, and how the pieces connect.
We'll walk through:
Launchpad — AI-driven scaffolding from a sample doc to a running app
Agentic search toolkit — conversational, flow, and plan-execute-reflect agents; QueryPlanningTool and friends; agentic memory; MCP; unified registration
Search Relevance Workbench — query sets, judgments, experiments, new metrics (Recall@K, MRR, DCG@K), Relevance Agent
Query rewriting and Search Pipelines — the composable layer between app and index
AI Search Flows — visual builder for semantic, hybrid, RAG, and multimodal pipelines
User Behavior Insights — the click/event schema that powers relevance measurement
OpenSearch Assistant and Workflow Automation — templates that stand up AI configs in one call
You'll leave with a decision tree: where to start, and which feature solves your current pain.
