Conventional wisdom suggests that adding more models should improve prediction quality. Our experience suggested otherwise.

While building a high-fidelity prediction system that combined classical machine learning models and large language models, we encountered a series of unexpected outcomes. In one experiment, a three-model system consistently outperformed every individual model we tested. Adding a fourth model significantly reduced accuracy. Architectures that produced strong results in one prediction domain failed when applied to another. In some cases, additional intelligence introduced disagreement, instability, and reduced explainability rather than better outcomes.

This session examines the multi-model approaches we built, tested, and ultimately abandoned, including ensembles, swarms, generator-critic architectures, and LLM-orchestrated machine learning systems. Using real prediction traces and production results, we will explore where these approaches succeeded, where they failed, and what those failures revealed about model composition, convergence, and decision-making.

Attendees will see examples of contradictory model interactions, performance degradation caused by increasing complexity, and situations where no single component could adequately explain the final outcome. The talk focuses on assumptions that proved incorrect, architectures that behaved differently than expected, and the lessons learned while building systems in which multiple forms of intelligence collaborate to produce a result.

What You Will Learn:

• How different multi-model approaches behaved in practice, including ensembles, swarms, generator-critic architectures, and LLM-orchestrated machine learning systems
• Examples of contradictory model interactions and performance degradation caused by increased complexity
• Lessons learned about model composition, convergence, and decision-making from real prediction systems

Who Should Attend:

• AI/ML engineers
• Applied machine learning practitioners
• Engineers working with LLM-based systems
• Data scientists building prediction systems
• Technical leads evaluating multi-model approaches
• Architects and researchers interested in model composition