LLMs ('AI') are coming for our jobs whether or not they work - Chris's Wiki

From here:

> In most non-tech organizations, both internal development and system administration is something similar to janitorial services; you have to have it because otherwise your organization falls over, but you don't like it and you're happy to spend as little on it as possible.

https://redd.it/1kf2kzc
@r_devops

Ibm Event notification question

Hello everyone,

I am having difficulties to configure my alerts with different templates.
Maybe can someone help me?

In Event-notifications i have created a Source.
In this sources i have 2 Topics.
I have 2 subscriptions and 2 templates.

But only one of the template is used to send the alerts to slack.

How can i change that?

Ideally would be to write the Template query to call the alert description on slack.
Is this possible?

https://redd.it/1kf8w1x
@r_devops

Passive FTP into Kubernetes ? Sounds cursed. Works great.

“talk about forcing some ancient tech into some very new tech wow... surely there's a better way” said a VMware admin watching my counter FTP strategy😅

Challenge accepted

I recently needed to run a passive-mode FTP server inside a Kubernetes cluster and quickly hit all the usual problems : random ports, sticky control sessions, health checks failing for no reason… you know the drill.

So i built a Helm chart that deploys vsftpd, exposes everything via stable NodePorts, and even generates a full haproxy.cfg based on your cluster’s node IPs, following the official HAProxy best practices for passive FTP.
You drop that file on your HAProxy box, restart the service, and FTP/FTPS just work.

https://github.com/adrghph/kubeftp-proxy-helm

Originally, this came out of a painful Tanzu/TKG setup (where the built-in HAProxy is locked down), but the chart is generic enough to be used in any Kubernetes cluster with a HAProxy VM in front.

Let me know if anyone else is fighting with FTP in modern infra. bye!

https://redd.it/1kfa7mz
@r_devops

Restart Operator: Schedule K8s Workload Restarts

github: [https://github.com/archsyscall/restart-operator](https://github.com/archsyscall/restart-operator)

Built a simple K8s operator that lets you schedule periodic restarts of Deployments, StatefulSets, and DaemonSets using cron expressions.

apiVersion: restart-operator.k8s/v1alpha1
kind: RestartSchedule
metadata:
name: nightly-restart
spec:
schedule: "0 3 * * *" # 3am daily
targetRef:
kind: Deployment
name: my-application

It works by adding an annotation to the pod template spec, triggering Kubernetes to perform a rolling restart. Useful for apps that need periodic restarts to clear memory, refresh connections, or apply config changes.

helm repo add archsyscall https://archsyscall.github.io/restart-operator
helm repo update
helm install restart-operator archsyscall/restart-operator

Look, we all know restarts aren't always the most elegant solution, but they're surprisingly effective at solving tricky problems in a pinch.

Thank you!

https://redd.it/1kfbkfl
@r_devops

Does anyone here use Humanitec? Feedback wanted!

I’ve been looking into Humanitec and I’m curious to hear from people who are actually using it.

What use case(s) you’re solving with it?
How it's integrated into your workflows?
Any wins or challenges you've encountered?
Would you recommend it to others building platform tooling?

I’m especially interested in any honest pros and cons.
Appreciate any insight you can share!

https://redd.it/1kfcpze
@r_devops

I got my first devops position

I'm really happy about this but I don't have a lot of experience. I'm Actually straight out of college. I studied what kubernetes and docker was and even went to linenode to create a kubernetes cluster to get some experience. After messing around a bit I realized I have no idea what to do with this stuff.

I start working a few weeks and I'm a little worried I'm going to go in just not knowing enough, which they probably know. I was wondering if anyone here had any advice on what I could maybe do in the meantime to get prepared. My current goal right now is to just get better with bash scripting because it seems like that's really important.

Thanks in advance!

https://redd.it/1kfce49
@r_devops

Any experience monitoring Redshift

Does anyone have experience monitoring Redshift? We've been having a series of data incidents and we're lacking visibility for what's happening with various jobs. The team usually resorts to tracking various sys_xxx tables to investigate failures. We're also using dbt, which writes some state to tables in Redshift as well. We're using Datadog and pulling in metrics for both Glue and Redshift, but none of those seem to be particularly helpful. I'm looking for any tips anyone has.

https://redd.it/1kfcfcy
@r_devops

Got a 3hr interview coming up. Tips/advice appreciated.

I got through the recruiter screening, a meeting with their main DevOps guy and CTO. I got notified that I'll be moving forward to the next round which is a 3 hour interview with other members of the team. I doubt it's going to be 3 straight hours and it'll probably be more like 3 1 hour blocks.

Anyways,
Any tips, advice, or suggestions? The interviews I already did were pretty chill and I think this might be the last round. The company is pretty cool and in a space where I have some expertise which I think gave me a leg up, I really want the job so help me get through the final push. A little background, I got about 10 years of full stack engineering experience and about the last 5ish years I've been exclusively doing DevOps

Oh edit to add: this is all completely remote

https://redd.it/1kffm0m
@r_devops

Best CI/CD tool

I love TeamCity, it looks great, it's easy to setup and it's easy to work with. The issue at hand tho, it is written in Java and requires over of 4GB free RAM which is just insane.

Is there a product that is as easy to deploy via Docker Compose, is as quality of a product and is more optimized?

https://redd.it/1kfi1d7
@r_devops

How do you inspect what actually changed in container images? (My Git-based approach)

Hey everyone,

When working with CI images or debugging build issues, I often need to understand *exactly* what changed in a container layer - not just which files were added or removed, but what was inside them.

[**Dive**](https://github.com/wagoodman/dive) is a great tool for exploring layers, but it mainly shows file names and status changes - not full file diffs. I wanted something more powerful and familiar.

So I built [**oci2git**](https://github.com/Virviil/oci2git), a tool that converts any OCI-compatible container image into a Git repo. Each image layer becomes a commit.

With it, you can:

* Run `git diff` between layers and see actual content changes, even better - use VSCode for ex, or [**lazygit**](https://github.com/jesseduffield/lazygit)
* Use `git blame` to find which layer added or modified a file
* Explore the entire filesystem history with regular Git commands

It’s been helpful for auditing, debugging, and understanding image composition more deeply. Would love feedback, and I’m curious how others inspect images: Dive? manual tarballing? something else?

https://redd.it/1kfkb6c
@r_devops

Personal Blog and Portfolio: Feedback?!

I have posted many blog articles on GitHub and other sites before and decided I want to have a personal homepage where they are all to find. I want to use this website as my portfolio as well.

It's fully open source if anyone is interested:

Repo: https://github.com/LukasNiessen/personal-website

Website: https://lukasniessen.com

Any feedback or thoughts are highly welcome :-)

https://redd.it/1kfmav7
@r_devops

Grafana Dashboard + Metrics For MCP Servers

I put together a Grafana Dashboard and metrics implementation for MCP servers. I thought some of you, might find it helpful. full post and code source here

https://redd.it/1kfnh0t
@r_devops

Any advice for fake it till you make it with AWS specifically?

Need some input on how to appear to know what I'm doing with AWS lol

https://redd.it/1kfrc3t
@r_devops

Snyk/Bitbucket?

Anyone here have practical experience using the Snyk integration on Bitbucket? We're pursuing SOC 2 compliance and one of the checks requires CVE scanning of code during CI/CD.

Other major CI/CD platforms offer free scanning like Dependabot, but sadly, we are on Bitbucket (constant irritation/constant disappointment), so we're looking at our options. They offer a Snyk integration, which (at our scale) will require a non-free Snyk plan.

Anyone gone through this? Happy to entertain alternatives, but we are likely to stay on BB because our company is all-in on Atlassian.

https://redd.it/1kftph6
@r_devops

Got ghosted after 3rd round

Hey everyone,

Just wanted to share my recent experience and see if others are going through the same thing.

I’ve been applying for DevOps roles for the past few months, and finally landed an interview. It started with a quick HR screen, followed by a technical round, which went well and I was immediately moved to the next stage.

The third round was a DevOps challenge, which I completed over my weekend. I presented it, answered all their technical questions, and felt the interview went smoothly.

I followed up with HR the next day — no response. I waited a week and followed up again — still nothing. Then I sent a message on LinkedIn just in case, and even followed up with the second HR contact mentioned in the original email — still complete silence.

At this point, I’m feeling pretty frustrated. It’s disappointing to invest so much time and effort, only to be met with no closure. Is this kind of ghosting becoming normal now?

Would appreciate hearing if others have gone through something similar, or any advice on how to deal with it.

https://redd.it/1kfuk0z
@r_devops

Anyone facing issue with Cloudflare recently of suddenly not honoring "Access-Control-Allow-Headers" set by origin?

Is anyone facing this recent issue lately where all the sudden, you're getting thrown Access-Control-Allow-Headers error across all proxied domains. Cloudflare proxy, out-of-the-blue, decided not to honor the Access-Control-Allow-Headers set by origin, and decided to block most headers, including "Authorization". This caused temporary downtime across all our services, totally unacceptable.

We had to remove proxy across multiple of our domains temporary and we can't find any changelogs, issues, etc. regarding any changes or reported issues to Cloudflare proxy anywhere (which is strange).

https://redd.it/1kfue56
@r_devops

Stategies for scaling out MySQL/MariaDB when database gets too large for a single host?

What are your preferred strategies when a MySQL/MariaDB database server grows to have too much traffic for a single host to handle, i.e. scaling CPU/RAM or using regular replication is not an option anymore? Do you deploy ProxySQL to start splitting the traffic according to some rule to two different hosts?

Has anyone migrated to TiDB? In that case, what was the strategy to detect if the SQL your app uses is fully compatible with TiDB?

https://redd.it/1kfwpne
@r_devops

LogWhisperer – AI-powered log summarizer that runs locally (no OpenAI keys, no cloud)

I built an open-source CLI tool called LogWhisperer that uses a local LLM to summarize Linux system logs into human-readable summaries. It’s useful for triaging noisy logs, quick postmortems, or just getting a sense of what the hell happened without manually parsing journalctl.

Key features:

Uses a local model (via [Ollama](https://ollama.com)) — supports `mistral`, `phi`, etc.
Parses logs from journalctl or file paths (e.g. /var/log/syslog)
CLI-friendly with flags for source, priority, model, entries
Outputs markdown reports for easy archiving
Includes a spinner so it doesn't feel frozen when summarizing large logs
100% offline (after install) — no OpenAI keys or cloud dependencies

Use case: you're SSH'd into a flaky VM, and you just want a summary of the last 500 err\-level logs without sifting through pages of noise.

Install it with a one-liner shell script — it sets up the Python env, installs Ollama, and pulls the model.

GitHub: https://github.com/binary-knight/logwhisperer

Would love feedback from fellow infra folks. I'm also thinking of extending this into scheduled cron-based summaries, Slack alerts, and anomaly tagging if anyone’s interested in contributing or ideas.

https://redd.it/1kfyv61
@r_devops

What really makes an Internal Developer Platform succeed?

Hey, I work at Pulumi as a community engineer and as we are doubling down on IDP features I’ve been looking around at various other platform tools and it's hard for me to tell which features are great for demos and which are really the important pieces of an ongoing platform effort.

so, in your experience what features are essential for a real world internal developer platform? and how are you handling infrastructure lifecycle management or how would you like to be handling it? I’m more interested in the day-2-and-beyond messy bits of a platform approach but if you are successfully using a 1-click to provision portals I'd love to hear about that as well.

https://redd.it/1kg3gj4
@r_devops

Was pushed into a Devops role. Never got the chance to learn properly

I was pushed into a devops role. And since then there was always a deadline on head and was never able to learn things properly. I am still good at my job and can do what is required but somewhere feel like I don't know stuff in depth. Or some not trivial things like Istio or monitoring tools or something else.

Want to change that. But because devops is so fast, don't have the slightest clue where to begin or how to start. Should I follow some roadmaps? Or implement things? If yes what?



https://redd.it/1kg53p9
@r_devops

Junior sysadmin looking for project ideas to modernize a simple infra

Junior sysadmin looking for project ideas to modernize a simple on-prem infra

Hey everyone,

I’m a junior sysadmin working with a fairly basic on-prem infrastructure with about 45 users, and I’m looking for ideas to improve, automate, and modernize it, ideally to make it more secure, more efficient, and a bit more DevOps-friendly.
The current setup is kind of “freestyle”: backups aren’t really solid yet, and a lot of things could be more structured

Here’s the current setup:
• 5 Ubuntu servers on-prem, used by data scientists to run AI/GPU workloads and experiments.
• Users currently have sudo access, which isn’t very secure - I’m looking for ways to improve that.
• 1 Proxmox server, where I run personal/admin VMs for Docker apps (Grafana, Prometheus, etc.).
• I occasionally spin up temporary VMs for test environments (no GPU) and give users access.
• Using Snipe-IT for asset management and Intune for endpoints.

Some project ideas I’m considering:
• Securing user access more effectively (e.g. removing full sudo, implementing access control or centralized auth).
• Setting up a Proxmox cluster for better flexibility and redundancy — not sure how well that works with GPU passthrough yet.
• Building a web portal where users can request or deploy their own VMs (via Proxmox API) and get direct access (ansible+terraform?).
• Improving asset and VM lifecycle management, to track what’s running, who owns it, and clean up unused resources automatically.

If you’ve done similar projects or have any ideas especially around automation, user access control, or Proxmox + GPU setups, I’d love to hear your thoughts!

https://redd.it/1kg8puy
@r_devops