GIDS news media, articles, insights and virtual events educate and illuminate its audiences so they can be fully prepared to deal with the new realities at work and in their professions.
Get the latest event updates, and insights from today's leading voices.
< session />
Thu, December 10Production AI SystemsTrust, Security & Governance
Once agentic systems reach production, the model quickly stops being the hardest engineering problem. The greater challenge becomes understanding why the system made particular execution decisions, observing how its behaviour changes over time, and establishing the operational controls needed to keep adaptive agent systems reliable, inspectable, and safe.
This session explores what it takes to operate agentic software in production. Rather than asking only whether the final answer was correct, we will examine the runtime decisions that produced it. Why was one tool selected instead of another? Why did the system choose a particular plan, memory, model, handoff, or recovery strategy? How can engineering teams determine whether those runtime decisions were appropriate, and how can they detect when behaviour begins to drift?
The discussion focuses on the operational layer surrounding agentic systems, including execution traces, runtime decisions, evaluation signals, policy evolution, safety boundaries, observability, and debugging practices for software whose behaviour changes as it encounters new tasks, contexts, tools, users, and information.
The emphasis is on production operations rather than model capability. The model is one component within a larger agent harness that provides context, coordinates tools and skills, applies operational constraints, manages execution boundaries, and supports ongoing improvement. Attendees will leave with a practical framework for operating agentic systems: what to observe, what to evaluate, where runtime controls belong, and how to keep adaptive behaviour inspectable, governable, and understandable as systems evolve.
What You Will Learn:
Who Should Attend:
< speaker_info />
Nicole Koenigstein is an AI Researcher and Practitioner in Agentic Systems, working across research, consulting, teaching, and direct system implementation to build reliable, production-ready AI systems. Her work focuses on multi-agent architectures, evaluation, safety, and long-term system behavior.
She served as an external evaluator for a European Commission AI Grand Challenge and has advised IOSCO on generative AI in regulated environments. She also serves on advisory boards for leading AI and quantitative finance conferences. Nicole regularly delivers invited talks and technical workshops across academia, industry, and international events. She is the author of Math for Machine Learning and Transformers in Action with Manning Publications. Her books Transformers: The Definitive Guide: Applications Beyond NLP and AI Agents: The Definitive Guide have been published by O’Reilly Media, and her forthcoming book Harness Engineering for AI Agents will also be published by O’Reilly Media.
