Reddit DevOps
278 subscribers
69 photos
32.2K links
Reddit DevOps. #devops
Thanks @reddit2telegram and @r_channels
Download Telegram
What makes test automation CI results trustworthy for devs

Setting up test automation in CI is easy, making devs look at and trust the results is the hard part
Coverage numbers that look great on paper but the team treats the test step like a formality because false positives trained them to ignore failures, its pavlovian at this point, test fails and everyone just hits retry without checking

https://redd.it/1v90nxe
@r_devops
Automated remediation through runbooks in prod does it actually work?

Curious how many of you have actually deployed automated runbooks for incident remediation — not just detection and alerting, but the full loop through to fix and close.

I've been building something in this space and want to understand how well it actually works for teams in practice. Do automated runbooks hold up in production, or do they break down on anything non-trivial?

What's your experience?

https://redd.it/1v9i52j
@r_devops
Should an AI security scanner be allowed to block CI, or only write a report?

I’m trying to draw a line around automated security scans in CI.

A read-only scan feels fine. If it finds a suspicious dependency, a broad permission, or a weird auth path, I want the report. I might even want it to draft a small PR.

The part that makes me nervous is when the same worker can change CI rules, mark a PR as failing, or "fix" security-sensitive code without a separate approval step. In one small project, I asked an agent to scan a dependency and config change. The report was useful, but I still did not want it editing the CI rule or auth code without me reading the diff first.

Maybe this is just normal branch protection with a new actor, but I’m curious how people handle it.

For security automation, do you separate "scan/report" from "block merge/change config"? And if you do, what evidence has to be attached to the approval?

https://redd.it/1v9hryk
@r_devops
Changing one thing about AI

I am "required" to use AI for coding, testing, debugging, terraform....and all the whole nine yards. Not too bad but too good either. Management is shoveling AI down our throats, so we have to use AI. If you can change one thing about AI to make your job easier, what would that be?

https://redd.it/1v9hpn3
@r_devops
is any one else bored of the various cloud providers. old man yells at clouds.

Am I just an old man yelling at clouds, or has DevOps become incredibly boring?

I know building technical solutions is not supposed to exist purely for my entertainment, but I do actually want to enjoy my job.

I am not currently working as a DevOps engineer, and one of the reasons I moved toward development was that I became completely bored with modern cloud infrastructure.

When I started, we still racked physical servers and automated the entire bare-metal imaging process. The underlying operating system mattered. We built systems by combining open-source software, custom code, and whatever hardware we could get our hands on. There was a real sense that we owned the systems we operated and understood how the pieces fit together.

Today, DevOps or platform engineering often, though certainly not always, seems to collapse into being a configuration monkey.

Yes, I still got to write plenty of tools and automation, but eventually I became tired of every architectural discussion turning into: “Should we use this AWS service or that AWS service?”

I became even more frustrated with managers and coworkers who did not really want to own or understand a service. They just wanted to hand as much of it as possible to the cloud provider and call the problem solved.

Do people genuinely enjoy the current state of what we used to call systems administration or systems engineering? Or am I just an old man yelling at clouds?

For what it is worth, I do find some salvation in Kubernetes. Once EKS or GKE is running, there is a huge amount you can build without thinking too deeply about the underlying cloud provider. That separation is reassuring, even if Kubernetes has plenty of complexity of its own.

These days, I mostly choose to work in areas where the managed cloud options are still inadequate, such as real-time media, telephony, high-speed stream processing, and similar systems. At least in those spaces, it still feels like I am engineering something rather than selecting products from a cloud-provider catalog.

I am curious whether other longtime infrastructure engineers feel the same way, or whether people genuinely prefer the managed-service-heavy version of the job.


Edit: I also really miss configuring switches and routers the cloud has made networking far more boring than it used to be though GKE has some interesting SDN features. I probably should have tried harder to get a job at one of the big hyperscalers but again on ownership I never wanted hundreds of co-workers.

https://redd.it/1v9llum
@r_devops
Passed CKA with 88%, sharing my prep notes and exam experience
https://redd.it/1v9n87e
@r_devops
Cloud Native Heidelberg Meetup: Kubernetes Platforms, CNCF Technologies and Kubernetes on Android

Sharing this for anyone interested in platform engineering, Kubernetes and cloud-native technologies.

Cloud Native Heidelberg is hosting a hybrid meetup covering:

Lessons from five years of building and rebuilding Kubernetes platforms
Building production-grade platforms using CNCF technologies
Creating vendor-neutral platforms without hyperscaler lock-in
Running Kubernetes directly on an Android phone

Date: July 30, 2026
Time: 6:30 PM CEST

Event details and registration:
https://ocgroups.dev/cncf/group/cncf-heidelberg/event/9euvqye

https://redd.it/1v9nlev
@r_devops
Thinking of building a tool that fixes vulnerable dependencies for you - want some opinions before I start

I'm a 3rd year CS student and I want to build something real over the next couple of months to learn RAG and agents properly. I've been thinking about one idea for a while and wanted to get opinions before I commit to it.

The problem I noticed is when you install on e package, you end up pulling in a bunch of other dependencies. Then a CVE shows up in one of those, and you're exposed through code. I came across Log4Shell and the Equifax breach and then I got this idea.

Fixing it is the annoying part. You can't just bump the broken package, because versions depend on each other, one thing needs a version below 2.0, another needs above 2.0, so you change one and others start conflicting. Now you have to fix all the issues, run tests, and hope everything works.

So what I want to build is a tool that does that for you and checks the fix actually works. The rough idea is you point it at your project, it reads your dependency tree (there's a free Google API called deps.dev that gives you the whole thing), finds why each vulnerable package is there, and works out the smallest set of version changes that clears all the CVEs without causing conflicts.

Then for the "will this upgrade break my code" question, I want to pull the changelog and release notes for each version jump and check them against the functions my code actually uses, so it can tell you whether the change touches anything you rely on. And the last part, it applies the fix in a Docker container, runs the tests, and only then tells you if it is safe.

The way I'm thinking of stucturing it is most of it is a fixed pipeline (read the tree, solve for the versions, run the tests), but the one part I'd actually let an LLM drive is the retry loop, when the tests fail, it looks at what broke, decides whether to drop that version and re-solve or try something else, and loops until it either works or gives up. That's the part where letting the model make the decision seems worth it. The rest doesn't really need it.

I'm thinking of using Postgres with pgvector, deps.dev and OSV for the data, a SAT solver for the version-conflict part, embeddings plus a reranker for the changelog retrieval, and Docker for running the tests. Later on, the nicest version would be exposing it as an MCP server so something like Claude Code could call it as a tool.

At the end it gives a report, a single Markdown file summarizing every vulnerability it found, why each package was present, the minimal fix it computed, the changelog-based breaking-change assessment, and the test results verifying the fix. Something you can hand to your team or keep as a record.

I know OSV-Scanner already does remediation, but only for npm/Maven, with no changelog check and no test verification, so the gap I'm aiming at is the full combination.

A few questions:

\- Would you use it? Would you actually run this on your own projects? If not, what's missing or in the way?

\- Is it useful? Is this a real problem worth solving, or is it already handled well enough that it doesn't need to exist?

\- Does it already exist? Is there something that already does all of this (minimal fix + breaking-change check + test verification) that I haven't found?

\- Right direction? Is this worth building, or would my time be better spent on a different problem in this space?

\- Do's and don'ts?

TL;DR: An agentic RAG tool that finds vulnerable dependencies, computes the smallest conflict-free fix, uses changelog retrieval to predict whether the upgrade breaks your code, and verifies the fix by running your tests. A verified patch.

https://redd.it/1v9q0oy
@r_devops
SecretZero — Git-native secrets-as-code for bootstrapping and managing project secrets

I've been working on **SecretZero**, an open-source tool to solve a problem most DevOps teams eventually run into:

>How do you bootstrap, document, rotate, and reproduce all the secrets required to run an application without relying on tribal knowledge and manual steps?

SecretZero uses declarative manifests to define a project's secret requirements and helps automate:

* Initial secret generation and seeding
* Secret synchronization across providers
* Rotation tracking and lifecycle management
* Git-friendly auditing and documentation of secret state
* Multi-environment workflows
* Contextless agent secret handling

The goal is to make secrets a first-class part of your infrastructure lifecycle instead of a collection of undocumented values hidden across vaults, CI systems, and environment files without additional infrastructure.

It also includes integrations for modern workflows, including MCP support for tooling that needs controlled access to secret metadata, multiple environments, and an optional on-demand, one-time use web UI.

I'd love feedback from the DevOps/SRE community:

* How are you handling "secret zero" today?
* Do you have a repeatable process for bootstrapping new environments?
* Are your secrets documented, rotated, and auditable?

Repo: [https://github.com/zloeber/secretzero](https://github.com/zloeber/secretzero)

This is an early project that I wrote for my own needs and use daily — feedback, criticism, and ideas welcome!

https://redd.it/1v9w4o6
@r_devops
Devops project as a fresher?

What is the best devops project you made as a fresher from college..

Mine was end to end devops project of a Ecom site --

Infra with terraform, CICD with DevSecOps pipeline in GitHub action, GKE for kubernetes, Gitops with ArgoCD (also this one got me hired back then)

What is your best project would love to hear from fresher perspective..

https://redd.it/1v9z0v9
@r_devops
agentic incident response actually saved me at 11pm last week

6 person team, i carry the pager most weeks cuz nobody else wants it. last tuesday 11pm i get paged, payment latency climbing, checkouts timing out. normally this is me sitting up in bed squinting at grafana half asleep for 15 min trying to find what changed



this time i had an agent wired into our runbook stuff and just let it run. it pulled recent deploys, correlated the error logs against the deploy timeline, and pointed at a connection pool config that got shrunk in a PR that afternoon. like 90 seconds, and it was right



what got me is that same PR had already been flagged that day by coderabbit / bugbot for the exact config value, and we waved it through cuz it was a rushed merge before someone's flight. the warning was sitting right there, we ignored it, and the agent rediscovered it later on. i rolled back from my phone and went back to sleep



i dont fully trust it solo though. its confidently pointed at the wrong deploy before when two things shipped close together, so i read it then verify myself. fast first pass, not a decision maker



anyone letting an agent do remediation on its own yet or you all keeping a human in the loop like me

https://redd.it/1va0x5k
@r_devops
Looking for guidance on choosing programming language

Hello everyone I was working as Devops engineer , Currently I was targetting FAANG companies moving forward, I choose python as my primary scripting language I won't say I do well in python just know the basics and AI to generate it my logic and I validate it never coded anything like leetcide problems

Since I was targetting FAANG I was looking to learn DSA in either ways I need to start learning programing language I was confused whether to pic python or JAVA can anyone have any suggestions it will be useful for me to plan my path

https://redd.it/1v9ykus
@r_devops
I just gave an company's OA and just got rigged, help me with these questions below. Are they really for a fresher in devops?

Application version manager

ABC is on a sprint spree and their development team is continuously rolling out new features and fixing bugs. The DevOps team now needs to ensure that the app updates are installed correctly for every user.

Write a bash script that takes in the user’s current version preferably as a parameter, fetches all the versions which have database updates serially from a separate text file, and lists out the versions which need to be updated.

Version format: major.minor.patch (eg- 2.5.23)

Part 1

Write a bash script where the current version and the latest version do not have a major update. Only minor versions or patch may vary.

Eq- current version is 2.5.23, and latest version is 2.6.0

Part 2

In the same bash script file as for part 1 of the question, include changes so that it identifies changes in major version if any.

Eq- current version is 2.5.23, and latest version is 3.1.2

Input

versions.txt (see filetree)

Output

If current version is 3.1.1

3.1.6

3.1.13

If current version is 3.1.2

3.1.6

3.1.13

If current version is 2.1.9

2.1.10

2.1.11

2.1.18

2.2.0

2.2.10

2.2.20

2.4.5

3.1.0

3.1.1

3.1.6

3.1.13

https://redd.it/1va32l4
@r_devops
From Oil field to DevOps is it possible?

Background: I'm an engineer in Algeria. My degree is in electrical/automation and I currently work in oil & gas as a data acquisition engineer (well testing, data integrity, reporting, that kind of thing). About a year ago, I decided to pivot into a remote role because I've been facing a health condition where on-site/field and mostly any physically demanding work is no longer an option for me. Cloud/DevOps is the direction I chose because I like the work and it's the most realistic path I found to stable remote work that pays in USD or EUR

What I've done so far:

Finished my AWS Cloud Practitioner prep, and I'm about to sit the exam ( scored 85-90% on most mocks, been stuck on a dumb payment/booking issue, but it's coming)
Linux fundamentals, wrote a few bash scripts and put them on GitHub
Hands-on AWS labs in my own account: built VPCs with public/private subnets, security groups vs NACLs, IAM, a two-tier setup with a private RDS MySQL behind an ALB, S3 gateway endpoints, etc. Broke things on purpose to learn how they fail
Currently grinding through Terraform. Kubernetes and more Python are next

Here's where the doubt kicks in. When I actually read job postings, two things hit me:

1. Almost every "junior" role still wants 1 to 3 years of real experience
2. Basically every remote role is geo-locked. "Remote (US only)," "must be authorized to work in X," no sponsorship, etc. The genuinely hire-from-anywhere jobs feel rare

So my question to people actually working in this field: am I chasing something that barely exists for someone in my situation? Or is the hire-from-anywhere market real if you're good enough and know where to look? And if you've done this from outside the US/EU, how did you actually break in?

I like the work, that's not the issue. I just don't want to spend another 6 months on a door that won't open.

TLDR: Non-US engineer pivoting into cloud. Made decent progress but every remote role wants experience + local presence. Is this realistic or am I fooling myself?

https://redd.it/1va7smw
@r_devops
How does your team handle documentation

I'm doing research on how documentation is done in different DevOps teams. What processes exist? How is it valued within your team? Has AI made that better or worse? Are there tools that you use to make it easier (not talking about Confluence here)? If you hqve worked in a professional context then I'd love to hear your input.

Considering the huge amount of shill posts I want to disclose that I'm (considering) making a tool that helps with documentation but I'm still in the process of understanding this problem thoroughly, which is why I'm askibg for your input :)

https://redd.it/1va6r9f
@r_devops
Is anyone burnt out?

Maybe it’s because I haven’t taken a proper vacation in lord know long.

I don’t handle k8s, but I work with a containerized environment and pipelines etc. But with the speed everything seems to be going, and the constant push, I feel overwhelmed and just want nothing to do with technology…maybe I just need to step away for a bit…how are you guys dealing with this. Talking to other devops folks, looks like they too are drinking from the fire hydrant

https://redd.it/1vabhl6
@r_devops
Would you still do DevOps?

Not going to lie. It's been a bad day. I almost tell them "take this job and shove it' after reviewing all these crappy AI codes. Vibe coding is not only a joke. It's a monster that mocks our hard-earned experiences!

I am thinking of leaving the industry! If you could go back ten years from now and know what you're knowing right now, would you still do DevOps?



https://redd.it/1val3ha
@r_devops
Can I transition from a BPO job to DevOps?

Hi everyone,

I'm a 2025 Computer Science graduate from India. I'm struggling to get an IT support or infrastructure support job mainly because my spoken English isn't very strong, even though I can troubleshoot issues practically.

I'm considering joining a BPO (technical support/chat support) for financial stability while continuing to learn Linux, Git, Docker, AWS, Kubernetes, and DevOps after work.

Has anyone here transitioned from BPO or technical support into DevOps or cloud roles? How difficult was it, and what roadmap or advice would you recommend?

I'd really appreciate hearing your experiences.

https://redd.it/1vani6a
@r_devops
How you guys are avoiding tying to any specific GPU vendor for AI workloads?

With more and more teams using LLMs and inference services wondering how everyone is making decisions about their infrastructure like are you choosing one major cloud provider, using Kubernetes across multiple providers or trying platforms like RunPod, CoreWeave, Yotta Labs, Lambda to keep things flexible?

more interested in whether anyone has found a real way to move workloads between different GPU providers without having to maintain completely separate deployment processes.

https://redd.it/1vawhik
@r_devops
How do experienced teams answer "What changed?" during a production incident?

I've been curious how different teams approach this in practice.

Imagine you're on call and something in production starts behaving differently. It's not immediately obvious whether it's a deployment, infrastructure change, configuration change, scaling event, cloud service issue, or something else.

Once the alert fires, how do you actually answer questions like:

What changed?

When did it change?

Was the change intentional?

What's the fastest way to confirm the root cause?

I'm less interested in the tools themselves (Terraform, Kubernetes, CloudTrail, Grafana, etc.) and more interested in the actual investigation workflow that experienced engineers follow.

For example:

Where do you usually look first?

Which sources of truth do you trust the most?

What part of the investigation tends to consume the most time?

Are there still manual steps that you wish were easier?

I'm trying to understand how this is handled in real production environments, especially at companies running on AWS/Kubernetes.

Thanks in advance—I’m hoping to learn from people who've been through enough incidents to know what actually works.

https://redd.it/1vatqr6
@r_devops
How many of you went multi-cloud, or switched cloud providers entirely, for a new job?

I have spent most of my career in AWS, but more and more, I am seeing companies asking for skill sets with multiple cloud providers. I feel like the skill set is transferable; if you know how to spin up EKS then spinning up a GKE cluster shouldn't be that much different. It's still Kubernetes under the hood, still installing helm charts, still using cluster autoscaler to provision new nodes, still installing Argo CD via Helm along with Prometheus, Grafana, Loki, etc. All that changes are the resources used and modules on the Terraform/cloud provider level. Am I correct in thinking this or am I way off base here?

https://redd.it/1vazyx8
@r_devops