Reddit DevOps
273 subscribers
64 photos
32K links
Reddit DevOps. #devops
Thanks @reddit2telegram and @r_channels
Download Telegram
Which sites do you actually find DevOps/SRE jobs nowadays?

I've been job hunting for DevOps/SRE/Platform Engineer roles and I'm curious where people are finding the best opportunities today. Besides LinkedIn, what websites, job boards, recruiters, communities, or other sources have worked well for you? I'm interested in answers from anywhere in the world to compare different markets, but if you're from Chile or Latin America, I'd especially appreciate your recommendations since that's where I'm currently looking. Thanks!

https://redd.it/1v04nvj
@r_devops
How much DevOps implementation is expected from a Freshie to know?

Starting to step in this field because development is seen as some random AI stuff now. I think I am too late. I just know abt the tools their names and work, not implemented much more than simple CI/CD pipelines on Github Actions, Docker. Implemented k8s, prometheus, grafana once, not confident.
What should be the bare minimum implementation and land a package. Or a good tutorial/project suggestion would be very helpful.

https://redd.it/1v060es
@r_devops
CVE-2026-63087: Grafana Oncall is EOL

Heads up if you're running Grafana OnCall's plugin-install flow, CVE-2026-63087 lets anyone reconfigure it with zero auth. Repo's archived, so there's no patch coming from upstream.

If you're still migrating off it and need some time, there's a small fix you can apply yourself and build from source in the meantime. Wrote up the actual vulnerable code + a verified patch: emphere.com/blog/cve-2026-63087-grafana-oncall-install-bypass

The install endpoint has zero auth on it - no token, no check, just the two public default IDs (stack_id 5, org_id 100) mint you a fresh one. The fix proposed adds an install secret only the operator holds, plus stops trusting a client-supplied admin role during the token bootstrap.

One caveat if you apply it: the secret is a contract change, so your own install clients (Helm, provisioning) have to start sending it too, or they'll lock themselves out. That's in the writeup too.

https://redd.it/1v07ef4
@r_devops
How should developer automation handle commands that never exit?

I am running into a boring automation question that matters more now that coding agents run local commands.

When a tool says "the command is still running", that is not enough information. Is the parent process alive? Did a child process inherit stdout? Was there any output in the last five minutes? Is this a watcher that should never exit, or a test that is actually stuck?

I have had an agent start a dev server during a small web task, then sit on "running" long enough that I had to check the port and process list myself.

For normal scripts this is annoying. For an agent, it is worse because the whole review gets blocked behind a vague status line.

What would you want in the run log before trusting it: process tree, last output timestamp, timeout policy, and cleanup result? Or is that too heavy for local dev automation?

https://redd.it/1v0fjbn
@r_devops
Terraform visualizer project

Hi everyone

I'm looking for an idea for my semester project, and I thought about building a Terraform Visualizer.

The idea is simple: it takes a Terraform state file, visualizes the infrastructure, and lets you save it as an image.

My question is if I build this, would you find it useful?

I'm asking because I would like to be able to justify the project's usefulness when presenting it to my teacher.

Also, I plan to keep it open source, so anyone can use it.

https://redd.it/1v0luip
@r_devops
Does anyone actually test their database restores on a schedule?

Every backup thread ends with "an untested backup is not a backup," everyone upvotes, and then (in my experience) nobody has an automated restore test anywhere.



Genuinely curious about the state of practice:

1. Do you restore-test on a schedule, or only when something breaks / an audit demands it?
2. If you automated it, what did you build? (ephemeral instance? scripts like pgbackrest_auto? CI job?)
3. If you didn't, what stopped you? (time? nowhere safe to restore to? nobody asked?)
4. For those with SOC 2/ISO: did the audit change anything, or did you just produce a doc that says you test quarterly?



Trying to understand if this gap is real or if I just keep landing in teams that are bad at it.

https://redd.it/1v0m5nh
@r_devops
Development Environment

Hi everyone,
I have a question about development environments and would love to hear how others handle this.

Our current stack is:
\- Ruby on Rails (authentication)
\- React (frontend)
\- Flask/Python (API)

Right now, every developer has to run all three services locally. I’m considering moving the Rails authentication service to a shared Linux development server and having developers run only the React dev server and Flask API locally. The goal is to reduce the number of services each developer needs to keep running.

For those of you using a similar architecture:
Is this a reasonable approach?

How would you set it up for multiple developers?

Would each developer have their own Rails instance and environment variables, or would you share a single authentication service?

Are there any pitfalls (performance, debugging, authentication issues, etc.) that I should be aware of before going down this path?

I’d appreciate any advice or examples of how your team handles a setup like this.

Thanks in advance!

https://redd.it/1v0swf4
@r_devops
Free Kubernetes beginner course. Learn K8s in an interactive web based terminal with lessons, labs, quizzes, and flash cards. [Free / No email required / 100% mobile friendly]
https://redd.it/1v0us6z
@r_devops
How do your teams manage planning?

For those on platform or DevOps teams, what does your planning process look like?

We have no problem writing docs and roadmaps for product features, but it doesn’t always feel like it aligns with the rest of the business.

Do you have Product Design Docs? Technical design docs? Quarterly roadmaps? Or is it mostly just tickets? Curious how other teams handle it.

I feel like every time I try to do structured planning it just gets blown up in numerous ways.

https://redd.it/1v0vscq
@r_devops
NAT Gateway is somehow 80% of our AWS bill, how are you all keeping this in check?

We’re on AWS Activate credits and just realized NAT Gateway (hourly plus data processing charges) is eating up about 80% of our total AWS spend. Didn’t expect it to be anywhere near that high until we actually looked at Cost Explorer.

We know some of the usual fixes exist, like VPC endpoints for S3/DynamoDB, consolidating NAT Gateways in non-HA environments, maybe switching to NAT instances for low-traffic dev setups, but haven’t touched any of it yet and want to understand what actually works before we start ripping things apart.

For anyone who’s dealt with this:

\- Did VPC endpoints actually move the needle for you, or was it a small dent?

\- Has anyone swapped NAT Gateway for NAT instances and found it worth the extra ops work?

\- Is 80% just absurdly high, or is this more common than it feels right now?

Trying to figure out if we’re missing something obvious in our architecture or if this is just what NAT Gateway costs look like at our traffic level. Any pointers appreciated.

https://redd.it/1v0xgjh
@r_devops
How is CI/CD managed in enterprise environments?

Hi everyone,

I'm learning CI/CD and would love to understand how it's managed in enterprise environments.

A few questions:

Which CI/CD tool(s) do you use (GitHub Actions, Jenkins, GitLab CI, Azure DevOps, CircleCI, etc.), and why did your team choose them?
How do you structure your CI/CD pipelines across dev, QA, stage, and production environments?
Do you use separate CI and CD pipelines, or a single end-to-end pipeline? Why?
How do you promote artifacts between environments (rebuild vs. promote the same artifact/image)?
How do multiple engineers collaborate on pipeline changes (branching strategy, PRs, code reviews, approvals)?
What are the typical stages in your CI and CD pipelines (build, test, security scans, deployment, approvals, etc.)?
How do you manage environment-specific configuration and secrets?
What are the biggest lessons or best practices you've learned from running CI/CD in production?

I'm especially interested in real-world workflows using GitHub Actions, Jenkins, GitLab CI, Azure DevOps, or similar tools.

Thanks!

https://redd.it/1v0v1a4
@r_devops
The cost of completing a task dropped by roughly 10x after switching to an AI pipeline. Here's how I calculated it

I see a lot of posts along the lines of AI saved us a ton of money, but they almost never include actual numbers or explain how those numbers were derived. So I decided to break down my approach and be honest about where it gets shaky, because a 10x improvement means nothing without a baseline.



A quick disclaimer: this isn’t audited accounting. It’s a rough unit-economics estimate for a single task. The numbers are rounded, the logic is what matters.



For task cost, I used an average feature from our backlog: human time × hourly rate + AI expenses.



Before AI: 



planning \~1.5 hours, 



coding \~4 hours, 



review and fixes \~2 hours, 



plus the invisible tax of dragging context between Jira, Slack, and docs \~1 hour. 



Total: \~8.5 hours. At €60/hour, that comes to about €510.



With the AI pipeline: the human shifts from being the executor to being the controller. Agents handle planning → execution → review, and I only step in at the plan-review and code-review checkpoints. My actual time drops to \~1-1.5 hours. 



Total: \~€70. It’s not hard to calculate how much the cost decreased, right?



By the way, when I built this pipeline with roles and checkpoints in BridgeApp, the biggest savings came not from code generation itself, but from eliminating the manual work of moving context around.



A question for those already working this way: what happens when an agent makes a mistake? Does it get caught at the next checkpoint, or does it surface in production three weeks later?



For me, the entire 10x savings depends on errors being caught between stages rather than at the very end, but I’m not sure real life is ever that clean.

https://redd.it/1v0wdtm
@r_devops
What's your actual workflow when a test fails in CI but passes locally

Just curious, standard procedure, you have something working locally, you push your code and your CI/CD pipeline fails. Locally everything seems OK and you fail to reproduce the error.

What do you actually do at that point? Genuinely curious what people's real routine looks like here — not the textbook answer, the actual "it's 5pm and CI fails" routine 😅

Also wondering: is this even still a big thing in 2026 now that lockfiles and containers have gotten so good, or is it basically a solved problem for most of you?

https://redd.it/1v119en
@r_devops
How are you actually tracking token spend across your LLM API calls?

We've got OpenAI and Anthropic calls scattered across a handful of services now, and nobody can tell me month to month what's driving the bill until the invoice shows up. No real attribution back to which service or team is burning the tokens. Started looking at logging and a proxy setup since the vendor dashboards alone aren't cutting it.

https://redd.it/1v1by4l
@r_devops
Weekly Self Promotion Thread

Hey r/devops, welcome to our weekly self-promotion thread!

Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!

https://redd.it/1v1dk0c
@r_devops
I don't trust AGENTS.md as a secret-redaction boundary

I am getting less comfortable with using repo instruction files as a safety boundary.

They are useful for preferences: commands to run, folders to avoid, how to format the final answer. But if an agent can read config files or logs, I do not think "please redact secrets" in a markdown file is enough.

In one small debug task, I asked the agent to inspect config safely, and I still felt nervous when it started summarizing values instead of just saying which setting was wrong. Nothing bad happened, but it made the boundary feel too soft.

A rule file can remind the agent to be careful, but it cannot be the thing that catches the secret after the agent already saw it.

For people running agents near CI, logs, or config: do you handle redaction at the tool/runtime layer, or do you rely on instructions and review?

https://redd.it/1v1fihb
@r_devops
How do you save your secrets?

I'm curious to know how different developers save their API keys, tokens and anything that is used in their web apps and projects, do you store them in notepad like me or have a more elegant way to do things? :)

https://redd.it/1v1mqwe
@r_devops
Your CI fails but passes locally, would debugging/shelling inside the CI environment actually help?

I’m exploring a devtool idea around this and I’m looking for brutally honest feedback, not encouragement.


Say a test/build fails in CI, but passes locally. You can’t reproduce it immediately, probably, so you end up digging through logs, rerunning, adding debug output, tweaking workflow files, trying act, SSH/self-hosted runners, etc.

I’m trying to understand whether direct debugging inside the actual CI runner/container would genuinely help, or whether this is mostly solved well enough with logs, artifacts, containers, and better workflow design.

If you’ve had this happen recently:

1. What exactly failed?
2. How long did it take to debug? How much time did you lose?
3. What did you actually try?
4. Would being able to attach a debugger to the actual CI environment have helped in that specific case?
5. Would you try a rough prototype next time this happens, or is this not painful enough?

Please be blunt. “This is annoying but I wouldn’t use it” is a useful answer.



https://redd.it/1v1tl5z
@r_devops
[meme] IaC and docs matter folks!
https://redd.it/1v1w770
@r_devops