My Notes from Attending KubeCon EU 2024

Over the past week, I was in Paris 🥐🥖 for KubeCon + CloudNativeCon. This was my first time attending, but I tagged along with two folks from our platform engineering team—Francisco and Miguel, who are a bit more experienced here.

I previously wrote about my preparation, which was mainly focused on which talks and sub-events I wanted attend. But I didn’t expect the activity and conversations around the vendor booths to be so interesting as well. Finally, even though I had visited before, Paris was a surprise — I had a really good time outside KubeCon in the city (even if extremely short).

On the first day of the conference, I mainly attended various talks from a co-located event called “Data on Kubernetes Day”. A few years ago, it was a bit unclear whether databases should even be run in Kubernetes, but now everyone is doing it. This happened thanks to improvements to the Operator pattern and the StatefulSet API, but also PVs and PVCs have matured to a point that allows databases to be set up really nicely in Kube. Nowadays, the conversation is around running entire data platforms in Kubernetes (which means bringing Spark, batch jobs and other workloads to Kubernetes).

Finally, database operators are still evolving. There was a great talk about the implementation of MongoDB’s operator, as well as one about the CloudNativePG operator for Postgres. Different operators can have quite different approaches to things like replication and high availability, but also the number of custom resources they expose and how they’re laid out. Obviously, there isn’t a “one size fits all” solution, and that’s why there’s quite a few different database operators even for the same underlying database system (more on this later).

At the end of the day, we tried to attend a conference party called “House of Kube” but unfortunately there were more attendees than expected and we weren’t allowed in. We still managed to get a very decent “steak frites” though 😋.

The second day was the first “proper” conference day with vendor booths opening after the morning keynotes. We watched a talk about Vitess as well as a talk about bin-packing pods for a distributed database (ClickHouse Cloud). However, the main focus of this day for us was meeting people at the booths. I distinctly remember great conversations with people from companies like Coder, PlanetScale (thanks for the hat!), TestKube, Splunk, DataDog, KubeDB, Kloudfuse, Grafana Labs, Honeycomb, Hetzner, ClickHouse, Incident.io, Cosmonic and so many more. The energy was buzzing; I learned a lot about the state of the industry, particularly around Kubernetes, databases, and other infrastructure software.

One highlight for me was the amount of vendors that are offering hybrid deployment solutions with a “BYOC” approach. In this model, customers keep their data in their own cloud provider accounts, but there is a control plane running somewhere centrally. It will be interesting to see how this trend develops alongside fully managed services and self-managed software. Of course, this was KubeCon after all, so there is a huge presence of self-managed software too.

Another highlight was the sheer amount of vendors selling internal developer portals and platforms. Whereas in the past, Backstage received all the attention, now there’s a dozen or so of these tools. One personal favorite has been Coder.com which really delivers on making it easy for developers to launch on-demand environments from certain predefined workflows.

At the end of the day, a huge happy hour event broke out in the middle of the booths floor and we walked around and met some people from various walks of life (including some Portuguese VCs). Then, we tried to swing by the official conference after-party, but latecomers like us had to be turned away. We still ended up at a Karaoke event organized by Honeycomb and some other sponsors.

For the third day, we started our day with a talk called “We Tested and Compared 6 Database Operators”. The speakers talked about different MySQL and PostgreSQL Kubernetes operators. I wish the talk had been much more in depth, but it was still a good overview and definitely helpful.

I then attended a couple of different talks about the present and future of Kubernetes APIs like network policies and its various storage primitives. These were given by Kubernetes contributors, and it was also interesting to see how people from different companies work together on these things. One of my favorite talks from this day was called “Zonal Outage Operational Stories”. This talk was about how the engineering team from AWS has worked towards providing high availability of the Kubernetes control plane in EKS. If you like working on distributed systems and thinking about all the different ways they can fail, you’ll surely enjoy this one.

The last talk of my day was “Fantastic Ordinals and How to Avoid Them: Auto-Scaling Challenges in a Cloud Database”, also from the folks at ClickHouse Cloud. It was interesting to see how they’ve faced some very similar challenges to us at SingleStore, but of course there are some important differences between our systems (ClickHouse is pure OLAP and SingleStore is HTAP, so the architecture needs to be quite different).

To end the day, we attended a meetup called “Development & Data Productivity in the Age of AI” (have I mentioned that AI was everywhere at KubeCon?). There was really great food, drinks and networking, but I also thoroughly enjoyed the panel on how AI is impacting software development teams.

As the last day came around, the activity at the booths winded down and so we focused on attending some final talks. We went to the following:

• "How to Stabilize a GenAI-First, Modern Data LakeHouse: Provision 20,000 Ephemeral Data Lakes/Year”
  • This talk was simply fantastic. Shirley Yang, from LinkedIn, did a tremendous job at conveying the size of complexity of their internal data infrastructure, and how they service the needs of hundreds of internal customers (data scientists, data engineers, etc.).
• “High Performance Multi Regions Messaging with Nats”
  • I didn’t know about NATS until now, so it was very important for me to learn about it.
• “Kubernetes as a Data Platform”
• “Deploying with Confidence: Lessons Learned Navigating Deployments of a 100-Strong Development Team”
  • The team at Grafana has been adjusting to cloud-first software engineering methodologies and setting up deployment rings and feature flags. We’ve gone through an extremely similar arc at SingleStore, so it was very helpful to compare their approach to ours.

And that was it! I’ll mainly remember the many great people we’ve met, and how well the conference was organized. I learned a lot about the state of the industry and I highly recommend companies in the space attend (booths seem to start at roughly ~30K which is very good value for money IMO). Finally, all the talks are available on Youtube so it's totally possible to catch up on most of what I talked about.

Feel free to reach out on Twitter!