As you may know, there was a KubeCon Europe recently in London.

I didn’t go there, but here a member of our community shares his thoughts about the conference on LinkedIn.

Also, if you understand Ukrainian and prefer the video format, you can also check out the video from Den Vasyliev as well.

#event #kubernetes

You may have noticed a train of released AI stuff here and there. In recent times, everyone started rushing towards AI agents to vendor-lock you to their solutions even more.

And do you know why they are able to do it so relatively easily?

https://ampcode.com/how-to-build-an-agent

P.S. 400 lines of Go code. If you remove all if err != nil by using something like Python, it will be half or even fewer lines. Yep

#ai

​​For today’s Donations Monday, I’d like to share with y’all a new fundraiser from Dzyga’s Paw - Save the Crabs 🦀

They are raising $110,000 to protect 15 Self-propelled artillery systems Krab with 15 Anti-Drone Systems.

#donations #Ukraine

Long post due issue of the CatOps digest is here!

https://newsletter.catops.dev/p/catops-digest-2025-04-27

#digest #newsletter

​Let's help Hospitallers rebuild their base to continue the work of one of the most outstanding volunteer medical battalions

https://send.monobank.ua/jar/2QrD4xoAsb

Other ways to support Hospitallers:
- https://www.hospitallers.life/needs-hospitallers
- https://www.hospitallers.org.uk/ways-to-help

#donations #Ukraine

A new books bundle on Humble Bundle:

https://www.humblebundle.com/books/devops-2025-oreilly-books

These are O'Reilly books, which are usually good quality. A couple of the books from this bundle were quite popular when they were initially released.

#books #bundle

GitHub has its own container registry for quite some time. Also, starting from the Helm version 3.8, it's possible to use any OCI registry to store your Helm charts. So, why not to store them on GitHub?

Here's an article that describes how to do that.

As a bonus, here's another article that provides an example of a GitHub Action workflow that allows you to publish your multi-architecture images to GHCR using GoReleaser. Actions' versions are outdated in that article, but you can simply set the new ones.

#github #helm #ghcr

Today, I'd like to share with you a thing I've worked on for quite some time. Well, the majority of the time I was procrastinating it, but still.

This small project is called Cost Exporter. It's designed to fetch metrics from AWS Cost Explorer API and present them as Prometheus metrics on an HTTP endpoint.

In theory, it's extensible. So, other cloud providers and formats could be added.

It may have some rough edges, since I haven't tested it in real production, only in my test account. Also, this is a classical pet-project: its main goal is to refresh some knowledge and play with some new technologies - not necessarily provide a production-grade solution. Still, I think it may be a useful tool. At least, the idea itself is definitely useful.

Cost Exporter is written in Go, has a Helm chart, everything is stored in GHCR, and automated using GitHub Actions. I have some other ideas on how to improve it, but I cannot guarantee that I do that.

Hope, you'll find this project interesting!

#go #programming #aws #kubernetes

It’s frequently been said, that when one goes up the career ladder, communication skills aka soft skills become as important, if not more important, as technical skills.

Here’s an article by A Life Engineered with some tips on how to improve your communication skills. Just like any other skills, those can be improved with enough dedication.

#culture

​​Let's close this fundraiser today - there's not much left to reach its goal and buy a re-transmitter for drones.

https://send.monobank.ua/jar/21w3A2UaUH

This fundraiser was shared with me with a long-time member of our community and my colleague from the very first paid work of mine. I'm pretty sure, we can close it today!

#donations #Ukraine

One of my favorite comics books has the phrase: "Who watches the Watchmen?".

Re-phrasing that: how is the oncall schedule organized in a company that provides tools for oncall and observability?

In this article you could take a glimpse on how oncall is organized at Datadog.

#oncall #sre

One thing I really like about Helm is that you can "tune" the level of complexity of your codebase. Or one could say, tune the "DRY-ness" of
your codebase.

Thus, you can have simple templates which are mostly plain YAML with only a few values that are set by a user. Or you can have complex configurations with logic, extensions, libraries, tests, and so on. Go templates are ass, but you can totally do that.

One of the common ways of making Helm codebase DRY-er is to move common specs into separate templates. Thus, in some charts you can see files like _pod-spec.tpl, _job-spec.tpl, and so on. Later on, you can include those templates into higher level objects (this is basically how library charts work).

But what if you want to pass an additional variable, not from
the values file, but from a high-level template itself? Think of a private
variable that controls if some parts are included in the manifests,
depending on from where they were called? Say, you want to enable profiling on a subset of pods, so you create two deployments: with
profiling off and on. This is the same app, so both deployments could share the same spec. You need to tell Helm somehow, that one of the deployments should have additional config to enable profiling.

You can actually do that! Helm template function accepts a single
argument that can be a dictionary of parameters, your usual {{ template "foo" . }}, where dot represents all the values in the current scope, which you could later access as {{ .Value.foo }} in your template. The scope here is a dictionary, so you can extend it with any private
variables you like.

For example:

 include "foo" (merge (dict "myVar" "bar") .) }}

Then, you'll be able to access myVar within the included template.

See:
- One
- Two.

#helm #kubernetes

​​A friend of my close friends is raising funds for a vehicle for the 50th Separate Storm Brigade.

https://send.monobank.ua/jar/3CYuCnWww7

Let’s help him to make that happen!

#donations #Ukraine

Spotify has released a postmortem for their outage that happened on 16th of April, and was almost global.

In nutshell, it was a combination of a bug, and a cascading issue caused by user retries. Here's an interesting bit:

> This change was deemed low risk and as such we applied it to all regions at the same time.

This is something what burned a lot of engineers. So, the take-away is probably never consider any change low-risk, especially if you already have the architecture for gradual rollouts. However, it's much easier to be said than done.

#postmortem #sre

Kubernetes v1.33 Fixes a 10-Year-Old Image Pull Loophole.

While technically a loophole, I wouldn't say that its impact was too high. It would be concerning only if you'd run multi-tenant clusters, where customers' pods run on the shared nodes. And even then, it could have been mitigated with pullPolicy: Always. While I never encountered this, I could imagine such setup in some PaaS company.

The gist is that previously (or still, depends on your K8s version), kubelet doesn't check the correct permissions to use a container image if this image is already present on a node.

#kubernetes #security

If you have some time today and you feel like watching some videos, here is a playlist from KubeCon Europe 2025 (the one that was in London).

https://www.youtube.com/playlist?list=PLj6h78yzYM2MP0QhYFK8HOb8UqgbIkLMc

#slides #event

A new issue of the CatOps Digest is here!

https://newsletter.catops.dev/p/catops-digest-2025-05-18

#digest #newsletter

​​Let's help to close a fundraiser from a member of our community.
This one is from a colleague of mine from my very first paid job. His wife is raising funds for a vehicle.

Here's a link to the Monobank jar:

https://send.monobank.ua/jar/5axqiosSrT

More information is in this Instagram post

#donations #Ukraine

A super-short article about Rate Limiting.

Also, it comes from yet another Substack blog about system design, if you're into such things.

This article doesn't show all the details, but it lists some most common algorithms, so you can continue your journey from there.

#systems #networking

- Again?
- Again!

This time with eBPF, though.

Researchers Expose New Intel CPU Flaws Enabling Memory Leaks and Spectre v2 Attacks.

>> "Attackers can speculatively hijack control flow within the same domain (e.g., kernel) and leak secrets across privilege boundaries, re-enabling
classic Spectre v2 scenarios without relying on powerful sandboxed environments like eBPF," VUSec said.

#security

​​🎉 On this day, 8 years ago, this channel was created 🎉

I find it to be a big accomplishment: being able to take care of it for so long and also keep somewhat consistent posts schedule! In these 8 years, CatOps grew to more than 5k subscribers, we had our voice chats (although irregular), and a newsletter.

I've led CatOps longer than I stayed at any job. Heck! In these years, I've changed jobs 3 times and moved countries. Yet, this channel is still here. This is cool, but also a bit weird at the same time.

It all is possible because of you! Thank you for keep reading CatOps, reacting to the posts, and sharing them. For real, I have an idea of abandoning it for good many times, Each time though, I thought: well, but at least someone finds it interesting.

If you enjoy CatOps, and you want to make us a small present, you can do it by donating to Hospitallers using this Monobank Jar:

https://send.monobank.ua/jar/9aHg73XmQm

#catops #birthday