State of Open Con 2025

Many orgs are evaluating running open source LLMs on their own infrastructure, and Kubernetes is a natural platform choice. However, running open source LLMs in production on Kubernetes is, honestly, a bit of an undocumented mess.

This technical presentation shares the experience of both speakers in deploying production-grade LLM infrastructure on Kubernetes. Through practical demonstrations, we'll explore the complete deployment lifecycle, from GPU setup to optimization techniques like Flash Attention, quantization tradeoffs and GPU sharing.

You'll learn:

* Architectural patterns for efficient LLM deployment using Ollama and vLLM
* Solutions for model weight management and context length optimization
* Techniques for GPU sharing and improving resource utilization
* Production approaches to fine-tuning with Axolotl and serving multiple models with LoRAX

You'll leave with a complete blueprint for building reliable, scalable LLM infrastructure on Kubernetes.

This session explores the importance of building inclusive AI tools tailored for underserved groups, particularly in the Global South. Drawing from JournoTECH's work, we’ll showcase practical steps for bridging the gap despite limited resources and discuss how inclusive design can empower marginalized communities while driving equitable access to technology.

Software is eating the world, and AI is eating software, transforming the way we build and use technology. Open Source has been at the heart of this transformation, ensuring openness and enabling innovation for the past 25 years. However, modern systems increasingly depend on non-code components such as datasets, models, and APIs—creating challenges to completeness that the Open Source must now address.

This session will outline Open Source 2.0: a vision for evolving to global governance and refining—not redefining—the Open Source Definition to ensure that modern systems remain open. We'll discuss how achieving clear community consensus, similar to a constitutional amendment, could ensure that any updates to the definition don't change its meaning for existing systems and avoid the fragmentation of new and conflicting definitions.

By addressing these challenges, Open Source can fully preserve the four essential freedoms of Free Software and thrive in a world shaped by services (cloud) and data (AI).