Reddit DevOps
273 subscribers
64 photos
32K links
Reddit DevOps. #devops
Thanks @reddit2telegram and @r_channels
Download Telegram
DevOps professionals in Australia - how much should I ask?

Been working as a DevOps / Cloud engineer for just over 6 years. Mostly in AWS shops. Know a bit of everything but quite good with linux, AWS, containers and orchestration, GitHub actions, Python / Node / PHP. Hold following certifications. AWS Solutions Architect Associate, DevOps Engineer Professional, LF Certified Kubernetes Administrator, Hashicorp Certified Terraform Associate.

Based in Melbourne, VIC, and currently paid AUD 100,000 + super. Wanting to see if I could demand more.

TA

https://redd.it/1uz25xp
@r_devops
How do small teams manage shared AI context without losing their minds?

Hey, we're building a project with 3 people using Cursor and Claude Code and keep tripping over the same thing.

One dev changes a port or finds a bug in a library and patches it with a workaround, the other has no idea and burns three hours figuring it out. The AI in the IDE has zero clue about team context and keeps suggesting stuff we already moved past. How's everyone dealing with this? Running some shared MCP server, a team version of something like claude-mem, or just dumping everything in Slack Telegram Discord manually?Curious especially from tech leads and small teams how do you not lose your mind to context chaos.

https://redd.it/1uz6jw9
@r_devops
How do teams using multiple git provide (github, gitlab, bitbucket) manage their daily workflow?

I'm curious how teams working across multiple Git providers manage their day-to-day work.

If your organization uses GitHub + Bitbucket or GitHub + GitLab (or more), how do you:

Track pull requests?

Find stale branches?

Monitor releases?

Keep track of work across providers?

Is it a pain point, or is it basically a non-issue?

https://redd.it/1uz8gk8
@r_devops
Mobile to backend end-to-end tracing: worth it? How do you keep the public ingest from being a DDoS target?

We're adding tracing to our mobile app and want the spans to connect to our backend traces (same trace\_id, so a user action in the app links to the API calls it triggers).

Backend goes OTLP -> Alloy -> Tempo already.

However, the mobile ingest endpoint has to be public, so it's an open, unauthenticated write endpoint which is an obvious DDoS/abuse target.

For now it's a simple one shared Tempo, and push all the protection to the edge instead: Cloudflare rate limiting, per-IP limits at the proxy, plus a sacrificial Alloy gateway that samples, memory-limits, and strips high-cardinality/PII attributes before anything reaches Tempo.

Questions for anyone who's done this:

* Do you actually ingest **mobile app traces end-to-end into your backend tracing**, or keep them separate (RUM-style) and just link by ID?
* Is chasing the unified mobile to backend waterfall worth it, or should we just drop it?
* How do you protect a public OTLP endpoint in practice?

Genuinely curious what's held up for people in production. Thanks.

https://redd.it/1uza6ko
@r_devops
standing api keys vs scoped tokens for ai agents in prod — what are you actually doing?

disclosure up front: i build an ai agent product, so i live this from the vendor side of security reviews.


been through a few enterprise reviews this year and the credential question is the one that keeps stalling. the reviewer asks what the agent authenticates with, and the honest answer for most teams is the same standing api keys a human would get, just never rotated and never scoped. one compromised prompt and the agent can do anything those keys can.


what actually moved reviews for us was short-lived scoped tokens the agent never sees in the clear, plus a per-action log the reviewer could replay. suddenly the question had a real answer instead of a promise.


curious what this sub is doing. still standing keys, scoped tokens per run, or something in between that actually works?

https://redd.it/1uzc66o
@r_devops
What is the one automation you have added, that you are proud of?

As DevOps, we all work on automating tasks to make our lives easier. What is the most satisfying automation you have ever implemented that truly saved you time?

https://redd.it/1uze2sz
@r_devops
Anyone considered code signing as a problem with respect to the P-Q transition?

Most of the discussions I ve heard regarding postquantum cryptgraphy fcus on TLS or key exchange protocls. I’d argue that code signing will probably be a much bigger problem. There are too many things associated with code signing, such as artifacts, build pipelines, firmware signing, package repositories, EV certificates, HSMs, delegated signing, legacy clients, rollbacks, and long-lived binaries. In fact it is possible that transport encryption will be subject to renegotiation in each session. But the signed artifact can continue to be checked many years later by systems that rely on a specific algorithm or certification chain. As I delved into the topic, I’m still quite hesitant in case I’m missing something fundamental.

https://redd.it/1uzorvm
@r_devops
Which sites do you actually find DevOps/SRE jobs nowadays?

I've been job hunting for DevOps/SRE/Platform Engineer roles and I'm curious where people are finding the best opportunities today. Besides LinkedIn, what websites, job boards, recruiters, communities, or other sources have worked well for you? I'm interested in answers from anywhere in the world to compare different markets, but if you're from Chile or Latin America, I'd especially appreciate your recommendations since that's where I'm currently looking. Thanks!

https://redd.it/1v04nvj
@r_devops
How much DevOps implementation is expected from a Freshie to know?

Starting to step in this field because development is seen as some random AI stuff now. I think I am too late. I just know abt the tools their names and work, not implemented much more than simple CI/CD pipelines on Github Actions, Docker. Implemented k8s, prometheus, grafana once, not confident.
What should be the bare minimum implementation and land a package. Or a good tutorial/project suggestion would be very helpful.

https://redd.it/1v060es
@r_devops
CVE-2026-63087: Grafana Oncall is EOL

Heads up if you're running Grafana OnCall's plugin-install flow, CVE-2026-63087 lets anyone reconfigure it with zero auth. Repo's archived, so there's no patch coming from upstream.

If you're still migrating off it and need some time, there's a small fix you can apply yourself and build from source in the meantime. Wrote up the actual vulnerable code + a verified patch: emphere.com/blog/cve-2026-63087-grafana-oncall-install-bypass

The install endpoint has zero auth on it - no token, no check, just the two public default IDs (stack_id 5, org_id 100) mint you a fresh one. The fix proposed adds an install secret only the operator holds, plus stops trusting a client-supplied admin role during the token bootstrap.

One caveat if you apply it: the secret is a contract change, so your own install clients (Helm, provisioning) have to start sending it too, or they'll lock themselves out. That's in the writeup too.

https://redd.it/1v07ef4
@r_devops
How should developer automation handle commands that never exit?

I am running into a boring automation question that matters more now that coding agents run local commands.

When a tool says "the command is still running", that is not enough information. Is the parent process alive? Did a child process inherit stdout? Was there any output in the last five minutes? Is this a watcher that should never exit, or a test that is actually stuck?

I have had an agent start a dev server during a small web task, then sit on "running" long enough that I had to check the port and process list myself.

For normal scripts this is annoying. For an agent, it is worse because the whole review gets blocked behind a vague status line.

What would you want in the run log before trusting it: process tree, last output timestamp, timeout policy, and cleanup result? Or is that too heavy for local dev automation?

https://redd.it/1v0fjbn
@r_devops
Terraform visualizer project

Hi everyone

I'm looking for an idea for my semester project, and I thought about building a Terraform Visualizer.

The idea is simple: it takes a Terraform state file, visualizes the infrastructure, and lets you save it as an image.

My question is if I build this, would you find it useful?

I'm asking because I would like to be able to justify the project's usefulness when presenting it to my teacher.

Also, I plan to keep it open source, so anyone can use it.

https://redd.it/1v0luip
@r_devops
Does anyone actually test their database restores on a schedule?

Every backup thread ends with "an untested backup is not a backup," everyone upvotes, and then (in my experience) nobody has an automated restore test anywhere.



Genuinely curious about the state of practice:

1. Do you restore-test on a schedule, or only when something breaks / an audit demands it?
2. If you automated it, what did you build? (ephemeral instance? scripts like pgbackrest_auto? CI job?)
3. If you didn't, what stopped you? (time? nowhere safe to restore to? nobody asked?)
4. For those with SOC 2/ISO: did the audit change anything, or did you just produce a doc that says you test quarterly?



Trying to understand if this gap is real or if I just keep landing in teams that are bad at it.

https://redd.it/1v0m5nh
@r_devops
Development Environment

Hi everyone,
I have a question about development environments and would love to hear how others handle this.

Our current stack is:
\- Ruby on Rails (authentication)
\- React (frontend)
\- Flask/Python (API)

Right now, every developer has to run all three services locally. I’m considering moving the Rails authentication service to a shared Linux development server and having developers run only the React dev server and Flask API locally. The goal is to reduce the number of services each developer needs to keep running.

For those of you using a similar architecture:
Is this a reasonable approach?

How would you set it up for multiple developers?

Would each developer have their own Rails instance and environment variables, or would you share a single authentication service?

Are there any pitfalls (performance, debugging, authentication issues, etc.) that I should be aware of before going down this path?

I’d appreciate any advice or examples of how your team handles a setup like this.

Thanks in advance!

https://redd.it/1v0swf4
@r_devops
Free Kubernetes beginner course. Learn K8s in an interactive web based terminal with lessons, labs, quizzes, and flash cards. [Free / No email required / 100% mobile friendly]
https://redd.it/1v0us6z
@r_devops
How do your teams manage planning?

For those on platform or DevOps teams, what does your planning process look like?

We have no problem writing docs and roadmaps for product features, but it doesn’t always feel like it aligns with the rest of the business.

Do you have Product Design Docs? Technical design docs? Quarterly roadmaps? Or is it mostly just tickets? Curious how other teams handle it.

I feel like every time I try to do structured planning it just gets blown up in numerous ways.

https://redd.it/1v0vscq
@r_devops
NAT Gateway is somehow 80% of our AWS bill, how are you all keeping this in check?

We’re on AWS Activate credits and just realized NAT Gateway (hourly plus data processing charges) is eating up about 80% of our total AWS spend. Didn’t expect it to be anywhere near that high until we actually looked at Cost Explorer.

We know some of the usual fixes exist, like VPC endpoints for S3/DynamoDB, consolidating NAT Gateways in non-HA environments, maybe switching to NAT instances for low-traffic dev setups, but haven’t touched any of it yet and want to understand what actually works before we start ripping things apart.

For anyone who’s dealt with this:

\- Did VPC endpoints actually move the needle for you, or was it a small dent?

\- Has anyone swapped NAT Gateway for NAT instances and found it worth the extra ops work?

\- Is 80% just absurdly high, or is this more common than it feels right now?

Trying to figure out if we’re missing something obvious in our architecture or if this is just what NAT Gateway costs look like at our traffic level. Any pointers appreciated.

https://redd.it/1v0xgjh
@r_devops
How is CI/CD managed in enterprise environments?

Hi everyone,

I'm learning CI/CD and would love to understand how it's managed in enterprise environments.

A few questions:

Which CI/CD tool(s) do you use (GitHub Actions, Jenkins, GitLab CI, Azure DevOps, CircleCI, etc.), and why did your team choose them?
How do you structure your CI/CD pipelines across dev, QA, stage, and production environments?
Do you use separate CI and CD pipelines, or a single end-to-end pipeline? Why?
How do you promote artifacts between environments (rebuild vs. promote the same artifact/image)?
How do multiple engineers collaborate on pipeline changes (branching strategy, PRs, code reviews, approvals)?
What are the typical stages in your CI and CD pipelines (build, test, security scans, deployment, approvals, etc.)?
How do you manage environment-specific configuration and secrets?
What are the biggest lessons or best practices you've learned from running CI/CD in production?

I'm especially interested in real-world workflows using GitHub Actions, Jenkins, GitLab CI, Azure DevOps, or similar tools.

Thanks!

https://redd.it/1v0v1a4
@r_devops
The cost of completing a task dropped by roughly 10x after switching to an AI pipeline. Here's how I calculated it

I see a lot of posts along the lines of AI saved us a ton of money, but they almost never include actual numbers or explain how those numbers were derived. So I decided to break down my approach and be honest about where it gets shaky, because a 10x improvement means nothing without a baseline.



A quick disclaimer: this isn’t audited accounting. It’s a rough unit-economics estimate for a single task. The numbers are rounded, the logic is what matters.



For task cost, I used an average feature from our backlog: human time × hourly rate + AI expenses.



Before AI: 



planning \~1.5 hours, 



coding \~4 hours, 



review and fixes \~2 hours, 



plus the invisible tax of dragging context between Jira, Slack, and docs \~1 hour. 



Total: \~8.5 hours. At €60/hour, that comes to about €510.



With the AI pipeline: the human shifts from being the executor to being the controller. Agents handle planning → execution → review, and I only step in at the plan-review and code-review checkpoints. My actual time drops to \~1-1.5 hours. 



Total: \~€70. It’s not hard to calculate how much the cost decreased, right?



By the way, when I built this pipeline with roles and checkpoints in BridgeApp, the biggest savings came not from code generation itself, but from eliminating the manual work of moving context around.



A question for those already working this way: what happens when an agent makes a mistake? Does it get caught at the next checkpoint, or does it surface in production three weeks later?



For me, the entire 10x savings depends on errors being caught between stages rather than at the very end, but I’m not sure real life is ever that clean.

https://redd.it/1v0wdtm
@r_devops
What's your actual workflow when a test fails in CI but passes locally

Just curious, standard procedure, you have something working locally, you push your code and your CI/CD pipeline fails. Locally everything seems OK and you fail to reproduce the error.

What do you actually do at that point? Genuinely curious what people's real routine looks like here — not the textbook answer, the actual "it's 5pm and CI fails" routine 😅

Also wondering: is this even still a big thing in 2026 now that lockfiles and containers have gotten so good, or is it basically a solved problem for most of you?

https://redd.it/1v119en
@r_devops