Weekly Self Promotion Thread
Hey r/devops, welcome to our weekly self-promotion thread!
Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!
https://redd.it/1v7scb2
@r_devops
Hey r/devops, welcome to our weekly self-promotion thread!
Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!
https://redd.it/1v7scb2
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Agentic ai devops after a year
we run a fairly boring b2b product, 20ish services, small platform team. a year ago management got excited about agentic ai devops and told us to "explore aggressively". so we did, and honestly most of it got thrown out
what didnt stay:
\- ai generated terraform. we tried stackgen and messed around with facets too, the natural language to terraform thing is genuinely impressive in a demo. but infra code has to BE right, not look right, and "looks right" is exactly what these are good at. every output still needed a human to read every line so where's the win
\- an ai pipeline optimizer that promised faster CI, mostly just added a service we had to babysit
\- an infra chatbot that answered questions wrong just confidently enough to be dangerous. someone almost ran a suggested command in prod, thats when it got killed
what stayed:
\- ai written deploy changelogs, an agent reads the merged PRs and posts a summary to slack. zero effort, everyone actually reads them
\- coderabbit on every PR. catches missing config changes and deploy risks before merge, this is the one we'd riot over if it got removed. it gets noisy on big PRs tho, we tuned it down
\- alert correlation. instead of 15 alerts when one db degrades we get one summary with a probable cause. right maybe 60% of the time which sounds bad until you remember the old system was right 0% of the time, it just screamed
what we're watching but not touching, auto rollbacks on anomaly detection and agent managed feature flags. maybe next year. the line right now is agents can read and summarize anything but they dont get to touch infra, a human still presses the button
https://redd.it/1v7u28r
@r_devops
we run a fairly boring b2b product, 20ish services, small platform team. a year ago management got excited about agentic ai devops and told us to "explore aggressively". so we did, and honestly most of it got thrown out
what didnt stay:
\- ai generated terraform. we tried stackgen and messed around with facets too, the natural language to terraform thing is genuinely impressive in a demo. but infra code has to BE right, not look right, and "looks right" is exactly what these are good at. every output still needed a human to read every line so where's the win
\- an ai pipeline optimizer that promised faster CI, mostly just added a service we had to babysit
\- an infra chatbot that answered questions wrong just confidently enough to be dangerous. someone almost ran a suggested command in prod, thats when it got killed
what stayed:
\- ai written deploy changelogs, an agent reads the merged PRs and posts a summary to slack. zero effort, everyone actually reads them
\- coderabbit on every PR. catches missing config changes and deploy risks before merge, this is the one we'd riot over if it got removed. it gets noisy on big PRs tho, we tuned it down
\- alert correlation. instead of 15 alerts when one db degrades we get one summary with a probable cause. right maybe 60% of the time which sounds bad until you remember the old system was right 0% of the time, it just screamed
what we're watching but not touching, auto rollbacks on anomaly detection and agent managed feature flags. maybe next year. the line right now is agents can read and summarize anything but they dont get to touch infra, a human still presses the button
https://redd.it/1v7u28r
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Planning an EU launch. Looking for real-world GDPR implementation advice.
I'm currently leading the GDPR implementation for our company, and honestly, the deeper we get into it, the more I realize how much there is to do.
Our platform wasn't originally built with GDPR in mind. Right now our primary database is in Australia, and we serve customers in multiple countries from the same infrastructure. As we're planning an EU launch, we're now figuring out what needs to change to become GDPR compliant.
Initially, I thought this would mostly be about infrastructure and documentation, but it's becoming clear that there are quite a few application-level changes as well. It's made me realize that if there's even a chance your product will serve EU users in the future, it's probably worth considering GDPR from day one rather than trying to retrofit it later.
For those who've actually implemented GDPR, I'd love to hear about your experience.
Some things I'm curious about:
What ended up being the hardest part?
Was it more of a legal/compliance challenge or a technical one?
Did you end up redesigning your architecture?
Did you create a separate AWS account or separate EU infrastructure, or keep everything in the same account?
How did you handle data residency and cross-border data transfers?
If your database was already in another region, did you migrate it or keep it where it was?
How did you handle backups when users requested data deletion?
Were there any code changes that surprised you?
Did any third-party services become a problem?
Did you use tools like CompAI, Vanta, Drata, or Secureframe? Were they worth it?
If you have a single admin portal managing multiple regions (US, AU, CA, EU), how did you handle access to EU customer data? Did you keep one global admin system with role-based access, or did you separate access by region (e.g., EU admins only seeing EU data)? How did you handle this in practice?
Looking back, what do you wish you'd known before starting?
I'm basically looking for the "I wish someone had told me this before we started" kind of advice.
If you've been through a GDPR implementation and wouldn't mind answering a few questions, please leave a comment or send me a DM.
https://redd.it/1v7qey5
@r_devops
I'm currently leading the GDPR implementation for our company, and honestly, the deeper we get into it, the more I realize how much there is to do.
Our platform wasn't originally built with GDPR in mind. Right now our primary database is in Australia, and we serve customers in multiple countries from the same infrastructure. As we're planning an EU launch, we're now figuring out what needs to change to become GDPR compliant.
Initially, I thought this would mostly be about infrastructure and documentation, but it's becoming clear that there are quite a few application-level changes as well. It's made me realize that if there's even a chance your product will serve EU users in the future, it's probably worth considering GDPR from day one rather than trying to retrofit it later.
For those who've actually implemented GDPR, I'd love to hear about your experience.
Some things I'm curious about:
What ended up being the hardest part?
Was it more of a legal/compliance challenge or a technical one?
Did you end up redesigning your architecture?
Did you create a separate AWS account or separate EU infrastructure, or keep everything in the same account?
How did you handle data residency and cross-border data transfers?
If your database was already in another region, did you migrate it or keep it where it was?
How did you handle backups when users requested data deletion?
Were there any code changes that surprised you?
Did any third-party services become a problem?
Did you use tools like CompAI, Vanta, Drata, or Secureframe? Were they worth it?
If you have a single admin portal managing multiple regions (US, AU, CA, EU), how did you handle access to EU customer data? Did you keep one global admin system with role-based access, or did you separate access by region (e.g., EU admins only seeing EU data)? How did you handle this in practice?
Looking back, what do you wish you'd known before starting?
I'm basically looking for the "I wish someone had told me this before we started" kind of advice.
If you've been through a GDPR implementation and wouldn't mind answering a few questions, please leave a comment or send me a DM.
https://redd.it/1v7qey5
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Alternatives to Kubernetes for medium-sized organizations?
Kubernetes is a great platform, but it can be expensive and may be more than what’s needed for many medium-sized organizations.
What are the common alternatives being used in production?
Interested in hearing what solutions have worked well in real-world environments and the reasons behind those choices.
https://redd.it/1v7u665
@r_devops
Kubernetes is a great platform, but it can be expensive and may be more than what’s needed for many medium-sized organizations.
What are the common alternatives being used in production?
Interested in hearing what solutions have worked well in real-world environments and the reasons behind those choices.
https://redd.it/1v7u665
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Automated my tech support / investigation workflow using CLI, MCPs, and custom subagents. Here is the architecture.
**Hey everyone,**
Quick workflow showcase/idea for anyone who gets enterprise access to AI CLIs at work (Gemini CLI, Claude Code, etc.) and still finds themselves manually jumping between tools during investigations or routine support tasks.
I recently set up a workflow that basically turns my CLI into an automated agent that runs investigations for me. If you have enterprise access and tool integrations available, you’re missing out if you aren't doing this.
# The Setup:
1. **CLI + Enterprise Access:** Using the company-provided enterprise CLI directly in the terminal.
2. **MCP (Model Context Protocol) Integration:** Connected the CLI to the actual operational and diagnostic tools used daily (monitoring, databases, logs, ticketing APIs, etc.).
3. **Custom Subagents & Commands:** Built specific subagent prompts and custom CLI commands tailored to my precise, repetitive investigation steps.
# How it works in practice:
Instead of manually opening 4 tabs, running 3 queries, and checking status logs when an issue comes up:
1. I trigger a custom slash command or alias in my CLI.
2. The subagent kicks off, uses the connected MCP tools to pull relevant context, inspect logs, run diagnostic checks, and correlate data.
3. It hands back a summarized breakdown of what happened, root cause hypotheses, and recommended next steps.
# Why do this?
* **Zero Context Switching:** You stay in the terminal.
* **No Manual Data Copy-Pasting:** The MCP server handles fetching real-time data securely.
* **Repeatable Process:** Standardized subagent prompts mean complex triage takes seconds instead of 20 minutes.
If your company already pays for enterprise CLI access and you're only using it to rewrite functions or explain error messages, **hook it up to your tools via MCPs**.
Curious if anyone else is running similar subagent setups in their terminal for operational work? What tools/MCPs are you chaining together?
https://redd.it/1v8bm5l
@r_devops
**Hey everyone,**
Quick workflow showcase/idea for anyone who gets enterprise access to AI CLIs at work (Gemini CLI, Claude Code, etc.) and still finds themselves manually jumping between tools during investigations or routine support tasks.
I recently set up a workflow that basically turns my CLI into an automated agent that runs investigations for me. If you have enterprise access and tool integrations available, you’re missing out if you aren't doing this.
# The Setup:
1. **CLI + Enterprise Access:** Using the company-provided enterprise CLI directly in the terminal.
2. **MCP (Model Context Protocol) Integration:** Connected the CLI to the actual operational and diagnostic tools used daily (monitoring, databases, logs, ticketing APIs, etc.).
3. **Custom Subagents & Commands:** Built specific subagent prompts and custom CLI commands tailored to my precise, repetitive investigation steps.
# How it works in practice:
Instead of manually opening 4 tabs, running 3 queries, and checking status logs when an issue comes up:
1. I trigger a custom slash command or alias in my CLI.
2. The subagent kicks off, uses the connected MCP tools to pull relevant context, inspect logs, run diagnostic checks, and correlate data.
3. It hands back a summarized breakdown of what happened, root cause hypotheses, and recommended next steps.
# Why do this?
* **Zero Context Switching:** You stay in the terminal.
* **No Manual Data Copy-Pasting:** The MCP server handles fetching real-time data securely.
* **Repeatable Process:** Standardized subagent prompts mean complex triage takes seconds instead of 20 minutes.
If your company already pays for enterprise CLI access and you're only using it to rewrite functions or explain error messages, **hook it up to your tools via MCPs**.
Curious if anyone else is running similar subagent setups in their terminal for operational work? What tools/MCPs are you chaining together?
https://redd.it/1v8bm5l
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
What is the change frequency rate for your org? How many deployments do you do on a daily, weekly, or monthly basis?
I have been bit hard in the past when joining an org that moved too slowly for my tastes. Took two weeks to get anything deployed, and if a deployment screwed up it was a rollback and two more weeks of waiting for a change request. I am very fond of this OG article about how quickly Etsy moved back in the day, and likely still to this day though I am sure the technology stack has shifted for them directly. Not sure if they ever did a followup on that post so wondering what change frequency looks for you all with modern tooling like Kubernetes and Docker in place?
https://redd.it/1v8bfpd
@r_devops
I have been bit hard in the past when joining an org that moved too slowly for my tastes. Took two weeks to get anything deployed, and if a deployment screwed up it was a rollback and two more weeks of waiting for a change request. I am very fond of this OG article about how quickly Etsy moved back in the day, and likely still to this day though I am sure the technology stack has shifted for them directly. Not sure if they ever did a followup on that post so wondering what change frequency looks for you all with modern tooling like Kubernetes and Docker in place?
https://redd.it/1v8bfpd
@r_devops
InfoQ
How Etsy Deploys More Than 50 Times a Day
Daniel Schauenberg described at QCon London how Etsy, renowned for its DevOps and Continuous Delivery practices, does 50 deploys/day. A fully automated deployment pipeline, thorough application monitoring and IRC-based collaboration are all important to achieve…
Expectations from devops engineer with 3 YOE in USA
Working as devops engineer from last 3 years in states, mostly work on windows stack, azure devops, yaml pipeline. Recently worked with setting up sonarqube & snyk for all teams & product across firm. Currently working in git migration and github actions. Planning to move back to india, what are the expectations for any senior roles as devops, cloud or platform engineer ? I know linux, kubernets and docker but they are more like tools.
https://redd.it/1v8boe6
@r_devops
Working as devops engineer from last 3 years in states, mostly work on windows stack, azure devops, yaml pipeline. Recently worked with setting up sonarqube & snyk for all teams & product across firm. Currently working in git migration and github actions. Planning to move back to india, what are the expectations for any senior roles as devops, cloud or platform engineer ? I know linux, kubernets and docker but they are more like tools.
https://redd.it/1v8boe6
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Would you push back on this architecture decision?🤔
One of our clients wants to build a commercial web application on Red Hat JBoss and isn’t interested in moving to microservices or Kubernetes. We explained the potential benefits of modern architectures, but based on their previous experience, they still prefer a traditional JBoss-based approach.
The application is expected to handle a large number of users on a daily basis.
For those with experience running JBoss in production, how would you approach this? Is JBoss alone still a solid choice for a high-traffic system, and what challenges should be considered?
https://redd.it/1v8lrdh
@r_devops
One of our clients wants to build a commercial web application on Red Hat JBoss and isn’t interested in moving to microservices or Kubernetes. We explained the potential benefits of modern architectures, but based on their previous experience, they still prefer a traditional JBoss-based approach.
The application is expected to handle a large number of users on a daily basis.
For those with experience running JBoss in production, how would you approach this? Is JBoss alone still a solid choice for a high-traffic system, and what challenges should be considered?
https://redd.it/1v8lrdh
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
What are your best tips for keeping AWS costs under control?
I’ve noticed that many AWS environments become increasingly expensive over time, and I’ve seen this firsthand as well.
One thing that helped was implementing cost allocation tags to better understand where the spending was coming from.
What other practices, tools, or strategies have worked well for keeping AWS costs under control?
https://redd.it/1v8lf4z
@r_devops
I’ve noticed that many AWS environments become increasingly expensive over time, and I’ve seen this firsthand as well.
One thing that helped was implementing cost allocation tags to better understand where the spending was coming from.
What other practices, tools, or strategies have worked well for keeping AWS costs under control?
https://redd.it/1v8lf4z
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
github actions central repo
Hi I am in a new role where they use Github for all their repos. I have been using Gitlab exlusively the past 5 years.
For CI/CD in my previous role we had a central 'pipelines' repo that was reference in the gitlab-ci file of each project to save teams/projects from repeating work.
I want to know if it's possible to do something similar in Github? I'm still learning how it maps to my existing Gitlab knowledge.
https://redd.it/1v8wshu
@r_devops
Hi I am in a new role where they use Github for all their repos. I have been using Gitlab exlusively the past 5 years.
For CI/CD in my previous role we had a central 'pipelines' repo that was reference in the gitlab-ci file of each project to save teams/projects from repeating work.
I want to know if it's possible to do something similar in Github? I'm still learning how it maps to my existing Gitlab knowledge.
https://redd.it/1v8wshu
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Need help regarding nfs-server service
So let me tell the context first, I have an kubernetes environment for production, and for like shared storage solution, I have installed nfs-utils and use the nfs-server service. In my scenario I cannot use cloud as an option. And for long I have been using the nfs-server but I have been facing alot of issues with it.
1. On multiple occasions I found dead or queued nfs processes in worker nodes which made the worker node totally unresponsive sys load was very high but low cpu and ram usage.
2. Something with the nfs4+ client where it uses session. On of the worker node again stopped responding, df -h was not responding rebooted the node still the same issue. And while viewing logs in the nfs node it stated nfsd was blocked and looking it up stated the thread on that worker node was locked due to unknown reason and all the requests from the worker node had just queued up. Restarting the service also hanged and needed to reboot the node itself to resolve the issue.
These are some major events there are also other small issues I have faced. I wanted to understand am I the only one facing these issues or this happens with others too, and what resolutions did you used to fix it? Like any alternative utilities and stuff
https://redd.it/1v8y29y
@r_devops
So let me tell the context first, I have an kubernetes environment for production, and for like shared storage solution, I have installed nfs-utils and use the nfs-server service. In my scenario I cannot use cloud as an option. And for long I have been using the nfs-server but I have been facing alot of issues with it.
1. On multiple occasions I found dead or queued nfs processes in worker nodes which made the worker node totally unresponsive sys load was very high but low cpu and ram usage.
2. Something with the nfs4+ client where it uses session. On of the worker node again stopped responding, df -h was not responding rebooted the node still the same issue. And while viewing logs in the nfs node it stated nfsd was blocked and looking it up stated the thread on that worker node was locked due to unknown reason and all the requests from the worker node had just queued up. Restarting the service also hanged and needed to reboot the node itself to resolve the issue.
These are some major events there are also other small issues I have faced. I wanted to understand am I the only one facing these issues or this happens with others too, and what resolutions did you used to fix it? Like any alternative utilities and stuff
https://redd.it/1v8y29y
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
API returns 504 when I emulate phone in Firefox
I have a frontend/backend working fine when I use any browser.
When I emulate a phone in Chrome, it all works.
But any request to the backend as phone in Firefox returns 504 gateway timeout (instant response). The same request works fine in another tab.
Any idea what could it be?
I use istio for networking (virtual service + gateway). No special config, just routing to the k8s service.
https://redd.it/1v8vdq7
@r_devops
I have a frontend/backend working fine when I use any browser.
When I emulate a phone in Chrome, it all works.
But any request to the backend as phone in Firefox returns 504 gateway timeout (instant response). The same request works fine in another tab.
Any idea what could it be?
I use istio for networking (virtual service + gateway). No special config, just routing to the k8s service.
https://redd.it/1v8vdq7
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Need advice: Did I mess up my architecture? Also, Dokploy vs Coolify vs Komodo?
Hi everyone,
I'm looking for some honest feedback on my current self-hosted architecture. I have a feeling I may have made some poor design decisions, so I'd really appreciate suggestions on how to improve it.
Here's my current setup on AWS:
Frontend EC2: Two React applications (Admin & Tenant) deployed using a single Docker Compose file.
Backend EC2: Two Go API applications (Admin & Tenant) deployed using another Docker Compose file.
Core Services EC2: MySQL, Redis, Vault, Keycloak, and MinIO are all running on the same EC2 using a single Docker Compose file.
Analytics EC2: Apache Superset deployed using Docker Compose.
ETL EC2: Airbyte running via abctl (Kind/Kubernetes).
Everything is currently self-hosted on AWS EC2 instances.
I have a few questions:
Is this architecture reasonable, or did I mess it up?
What would you change or redesign if this were your production environment?
Should I split any of these services onto separate servers or use managed AWS services (such as RDS or ElastiCache)?
Would you recommend staying with Docker Compose, or is it worth moving to Kubernetes at this stage?
For managing deployments across these servers, which would you recommend: Dokploy, Coolify, or Komodo?
What I'm looking for in a deployment platform:
*Multi-server management
*Excellent Docker Compose support
*GitHub integration with automatic deployments
*Rollbacks, logs, and environment variable management
*Lightweight and stable
*Preferably fully open source
*Easy to maintain for a small DevOps team
If you've run a similar setup in production, I'd love to hear what worked well and what you'd do differently.
Thanks!
https://redd.it/1v97rw1
@r_devops
Hi everyone,
I'm looking for some honest feedback on my current self-hosted architecture. I have a feeling I may have made some poor design decisions, so I'd really appreciate suggestions on how to improve it.
Here's my current setup on AWS:
Frontend EC2: Two React applications (Admin & Tenant) deployed using a single Docker Compose file.
Backend EC2: Two Go API applications (Admin & Tenant) deployed using another Docker Compose file.
Core Services EC2: MySQL, Redis, Vault, Keycloak, and MinIO are all running on the same EC2 using a single Docker Compose file.
Analytics EC2: Apache Superset deployed using Docker Compose.
ETL EC2: Airbyte running via abctl (Kind/Kubernetes).
Everything is currently self-hosted on AWS EC2 instances.
I have a few questions:
Is this architecture reasonable, or did I mess it up?
What would you change or redesign if this were your production environment?
Should I split any of these services onto separate servers or use managed AWS services (such as RDS or ElastiCache)?
Would you recommend staying with Docker Compose, or is it worth moving to Kubernetes at this stage?
For managing deployments across these servers, which would you recommend: Dokploy, Coolify, or Komodo?
What I'm looking for in a deployment platform:
*Multi-server management
*Excellent Docker Compose support
*GitHub integration with automatic deployments
*Rollbacks, logs, and environment variable management
*Lightweight and stable
*Preferably fully open source
*Easy to maintain for a small DevOps team
If you've run a similar setup in production, I'd love to hear what worked well and what you'd do differently.
Thanks!
https://redd.it/1v97rw1
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
How your company handle AI data leaks ?
Hi everyone,
I was wondering : In the company where you're currently working, how does the IT guys (or you) manage the potential data leaks between employees and AI online agent like chatGTP, Claude or Gemini I mean, I live in Europe and we recently adopted some very restrictive laws regarding this topic and companies that are using AI,
was wondering how it was working and what are the solutions adopted in other companies ?
https://redd.it/1u91hc9
@r_devops
Hi everyone,
I was wondering : In the company where you're currently working, how does the IT guys (or you) manage the potential data leaks between employees and AI online agent like chatGTP, Claude or Gemini I mean, I live in Europe and we recently adopted some very restrictive laws regarding this topic and companies that are using AI,
was wondering how it was working and what are the solutions adopted in other companies ?
https://redd.it/1u91hc9
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
What do you do with CVEs you can't fix? Auditor wants proof they're 'not exploitable'
One-person security/platform team at a small fintech, going through SOC 2.
Patching what has a fix is fine. The problem is the stuff I can't fix, because there's no patched version, or the fix breaks something. Inspector/Trivy keeps flagging it, Vanta keeps showing it red, and technically every one needs a documented risk acceptance.
Most of these aren't even exploitable in our setup, sometimes it's a vulnerable function never called, transitive dep we don't use, requires network access that doesn't exist. But "trust me it's fine" doesn't fly, and writing a proper exception per CVE takes forever.
So, people who've been through this what do you actually show the auditor for "not exploitable"?
Is there a way to automate this discovery/evidence gathering?
Or does everyone just eat the busywork / quietly ignore them and pray?
https://redd.it/1v996vb
@r_devops
One-person security/platform team at a small fintech, going through SOC 2.
Patching what has a fix is fine. The problem is the stuff I can't fix, because there's no patched version, or the fix breaks something. Inspector/Trivy keeps flagging it, Vanta keeps showing it red, and technically every one needs a documented risk acceptance.
Most of these aren't even exploitable in our setup, sometimes it's a vulnerable function never called, transitive dep we don't use, requires network access that doesn't exist. But "trust me it's fine" doesn't fly, and writing a proper exception per CVE takes forever.
So, people who've been through this what do you actually show the auditor for "not exploitable"?
Is there a way to automate this discovery/evidence gathering?
Or does everyone just eat the busywork / quietly ignore them and pray?
https://redd.it/1v996vb
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
What makes test automation CI results trustworthy for devs
Setting up test automation in CI is easy, making devs look at and trust the results is the hard part
Coverage numbers that look great on paper but the team treats the test step like a formality because false positives trained them to ignore failures, its pavlovian at this point, test fails and everyone just hits retry without checking
https://redd.it/1v90nxe
@r_devops
Setting up test automation in CI is easy, making devs look at and trust the results is the hard part
Coverage numbers that look great on paper but the team treats the test step like a formality because false positives trained them to ignore failures, its pavlovian at this point, test fails and everyone just hits retry without checking
https://redd.it/1v90nxe
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Automated remediation through runbooks in prod does it actually work?
Curious how many of you have actually deployed automated runbooks for incident remediation — not just detection and alerting, but the full loop through to fix and close.
I've been building something in this space and want to understand how well it actually works for teams in practice. Do automated runbooks hold up in production, or do they break down on anything non-trivial?
What's your experience?
https://redd.it/1v9i52j
@r_devops
Curious how many of you have actually deployed automated runbooks for incident remediation — not just detection and alerting, but the full loop through to fix and close.
I've been building something in this space and want to understand how well it actually works for teams in practice. Do automated runbooks hold up in production, or do they break down on anything non-trivial?
What's your experience?
https://redd.it/1v9i52j
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Should an AI security scanner be allowed to block CI, or only write a report?
I’m trying to draw a line around automated security scans in CI.
A read-only scan feels fine. If it finds a suspicious dependency, a broad permission, or a weird auth path, I want the report. I might even want it to draft a small PR.
The part that makes me nervous is when the same worker can change CI rules, mark a PR as failing, or "fix" security-sensitive code without a separate approval step. In one small project, I asked an agent to scan a dependency and config change. The report was useful, but I still did not want it editing the CI rule or auth code without me reading the diff first.
Maybe this is just normal branch protection with a new actor, but I’m curious how people handle it.
For security automation, do you separate "scan/report" from "block merge/change config"? And if you do, what evidence has to be attached to the approval?
https://redd.it/1v9hryk
@r_devops
I’m trying to draw a line around automated security scans in CI.
A read-only scan feels fine. If it finds a suspicious dependency, a broad permission, or a weird auth path, I want the report. I might even want it to draft a small PR.
The part that makes me nervous is when the same worker can change CI rules, mark a PR as failing, or "fix" security-sensitive code without a separate approval step. In one small project, I asked an agent to scan a dependency and config change. The report was useful, but I still did not want it editing the CI rule or auth code without me reading the diff first.
Maybe this is just normal branch protection with a new actor, but I’m curious how people handle it.
For security automation, do you separate "scan/report" from "block merge/change config"? And if you do, what evidence has to be attached to the approval?
https://redd.it/1v9hryk
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Changing one thing about AI
I am "required" to use AI for coding, testing, debugging, terraform....and all the whole nine yards. Not too bad but too good either. Management is shoveling AI down our throats, so we have to use AI. If you can change one thing about AI to make your job easier, what would that be?
https://redd.it/1v9hpn3
@r_devops
I am "required" to use AI for coding, testing, debugging, terraform....and all the whole nine yards. Not too bad but too good either. Management is shoveling AI down our throats, so we have to use AI. If you can change one thing about AI to make your job easier, what would that be?
https://redd.it/1v9hpn3
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
is any one else bored of the various cloud providers. old man yells at clouds.
Am I just an old man yelling at clouds, or has DevOps become incredibly boring?
I know building technical solutions is not supposed to exist purely for my entertainment, but I do actually want to enjoy my job.
I am not currently working as a DevOps engineer, and one of the reasons I moved toward development was that I became completely bored with modern cloud infrastructure.
When I started, we still racked physical servers and automated the entire bare-metal imaging process. The underlying operating system mattered. We built systems by combining open-source software, custom code, and whatever hardware we could get our hands on. There was a real sense that we owned the systems we operated and understood how the pieces fit together.
Today, DevOps or platform engineering often, though certainly not always, seems to collapse into being a configuration monkey.
Yes, I still got to write plenty of tools and automation, but eventually I became tired of every architectural discussion turning into: “Should we use this AWS service or that AWS service?”
I became even more frustrated with managers and coworkers who did not really want to own or understand a service. They just wanted to hand as much of it as possible to the cloud provider and call the problem solved.
Do people genuinely enjoy the current state of what we used to call systems administration or systems engineering? Or am I just an old man yelling at clouds?
For what it is worth, I do find some salvation in Kubernetes. Once EKS or GKE is running, there is a huge amount you can build without thinking too deeply about the underlying cloud provider. That separation is reassuring, even if Kubernetes has plenty of complexity of its own.
These days, I mostly choose to work in areas where the managed cloud options are still inadequate, such as real-time media, telephony, high-speed stream processing, and similar systems. At least in those spaces, it still feels like I am engineering something rather than selecting products from a cloud-provider catalog.
I am curious whether other longtime infrastructure engineers feel the same way, or whether people genuinely prefer the managed-service-heavy version of the job.
Edit: I also really miss configuring switches and routers the cloud has made networking far more boring than it used to be though GKE has some interesting SDN features. I probably should have tried harder to get a job at one of the big hyperscalers but again on ownership I never wanted hundreds of co-workers.
https://redd.it/1v9llum
@r_devops
Am I just an old man yelling at clouds, or has DevOps become incredibly boring?
I know building technical solutions is not supposed to exist purely for my entertainment, but I do actually want to enjoy my job.
I am not currently working as a DevOps engineer, and one of the reasons I moved toward development was that I became completely bored with modern cloud infrastructure.
When I started, we still racked physical servers and automated the entire bare-metal imaging process. The underlying operating system mattered. We built systems by combining open-source software, custom code, and whatever hardware we could get our hands on. There was a real sense that we owned the systems we operated and understood how the pieces fit together.
Today, DevOps or platform engineering often, though certainly not always, seems to collapse into being a configuration monkey.
Yes, I still got to write plenty of tools and automation, but eventually I became tired of every architectural discussion turning into: “Should we use this AWS service or that AWS service?”
I became even more frustrated with managers and coworkers who did not really want to own or understand a service. They just wanted to hand as much of it as possible to the cloud provider and call the problem solved.
Do people genuinely enjoy the current state of what we used to call systems administration or systems engineering? Or am I just an old man yelling at clouds?
For what it is worth, I do find some salvation in Kubernetes. Once EKS or GKE is running, there is a huge amount you can build without thinking too deeply about the underlying cloud provider. That separation is reassuring, even if Kubernetes has plenty of complexity of its own.
These days, I mostly choose to work in areas where the managed cloud options are still inadequate, such as real-time media, telephony, high-speed stream processing, and similar systems. At least in those spaces, it still feels like I am engineering something rather than selecting products from a cloud-provider catalog.
I am curious whether other longtime infrastructure engineers feel the same way, or whether people genuinely prefer the managed-service-heavy version of the job.
Edit: I also really miss configuring switches and routers the cloud has made networking far more boring than it used to be though GKE has some interesting SDN features. I probably should have tried harder to get a job at one of the big hyperscalers but again on ownership I never wanted hundreds of co-workers.
https://redd.it/1v9llum
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community