Cloudflare Proxy + DO droplet
Hello,
I am pretty new in the devop world and I would like some help from those who are experienced 😛.
I am noticing in my Nginx error log a considerable number of requests made using the server IP instead of the hostname. I always used Cloudflare as proxy for this specific server.
I suspect this is maybe because DO droplet IP are public and attackers just scan for http/https ports on the various IP ranges?
I would like to whitelist all the public cloudflare IP in my nginx configuration and update them regularly (via a cron).
Is this something common? Do you have any recommendations?
My only concern is if Claudflare adds a new IP range in between my whitelist automatic update and nginx ends up refusing all cloudflare requests from the new IPs.
Thanks!
https://redd.it/1j5ww2h
@r_devops
Hello,
I am pretty new in the devop world and I would like some help from those who are experienced 😛.
I am noticing in my Nginx error log a considerable number of requests made using the server IP instead of the hostname. I always used Cloudflare as proxy for this specific server.
I suspect this is maybe because DO droplet IP are public and attackers just scan for http/https ports on the various IP ranges?
I would like to whitelist all the public cloudflare IP in my nginx configuration and update them regularly (via a cron).
Is this something common? Do you have any recommendations?
My only concern is if Claudflare adds a new IP range in between my whitelist automatic update and nginx ends up refusing all cloudflare requests from the new IPs.
Thanks!
https://redd.it/1j5ww2h
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Employers too hyper focused on specific tool(s) experience above all else when hiring?
So I've been out of a job since October and basically looking for any combination of Automation Engineer, DevOps Engineer, SRE, or Platform Engineer since there can be a lot of overlap. Without deep diving into my resume I have a lot of strong experience with Infrastructure-as-Code, Configuration-as-Code, programming, scripting, troubleshooting, research & development, and well rounded with a lot of previous ops experience too. Now just due to luck of the draw most of this wasn't with Terraform and Ansible. I've done some projects with these, like them, want to use them more, etc. They're far preferred over something like Azure ARM templates, Azure DSC (Desired State Configuration), or scripting from scratch to do deployments and configuration. In my opinion Terraform and Ansible are far easier too.
Now to the point of the title, it seems like I've lost out on multiple opportunities because I can't speak to extensive project experience with Terraform and/or Ansible. One recent one particularly irked me because I thought the interview went well, everyone was friendly, work culture seemed nice, good pay, etc. It was a local position (I've been working remote for years), and it was only me and one other candidate being interviewed. Ironically during the interview I thought maybe I was a little overqualified because the job sounded like mostly deploying and updating deployed (moslty) local infrastructure via Terraform. It didn't sound like there was any advanced configuration, pipeline creation (on that team), or much that was really going to push my limits. But hey, I need a paycheck, everything else sounded nice, and I could get more hands on experience with Terraform. I was very optimistic with the only real worry being if the other candidate happened to be stronger than me or not. When the external recruiter got back to me he told me the employer wasn't going with me or the other candidate because they didn't think either of us had the skill set they were looking for. The recruiter said at that point he told them their only option was probably going to be to look for someone not local. I was pretty dumbfounded.
I've also had similar experiences (that didn't make it as far) where they're just hyper focused on someone with extensive Terraform and/or Ansible experience with seemingly little regard to broader DevOps experience, even when I try to talk through some very impressive DevOps projects I've done. I'm beginning to wonder if most places are just terrible at hiring, I'm terrible at selling myself. or a combination of both.
https://redd.it/1j61mh7
@r_devops
So I've been out of a job since October and basically looking for any combination of Automation Engineer, DevOps Engineer, SRE, or Platform Engineer since there can be a lot of overlap. Without deep diving into my resume I have a lot of strong experience with Infrastructure-as-Code, Configuration-as-Code, programming, scripting, troubleshooting, research & development, and well rounded with a lot of previous ops experience too. Now just due to luck of the draw most of this wasn't with Terraform and Ansible. I've done some projects with these, like them, want to use them more, etc. They're far preferred over something like Azure ARM templates, Azure DSC (Desired State Configuration), or scripting from scratch to do deployments and configuration. In my opinion Terraform and Ansible are far easier too.
Now to the point of the title, it seems like I've lost out on multiple opportunities because I can't speak to extensive project experience with Terraform and/or Ansible. One recent one particularly irked me because I thought the interview went well, everyone was friendly, work culture seemed nice, good pay, etc. It was a local position (I've been working remote for years), and it was only me and one other candidate being interviewed. Ironically during the interview I thought maybe I was a little overqualified because the job sounded like mostly deploying and updating deployed (moslty) local infrastructure via Terraform. It didn't sound like there was any advanced configuration, pipeline creation (on that team), or much that was really going to push my limits. But hey, I need a paycheck, everything else sounded nice, and I could get more hands on experience with Terraform. I was very optimistic with the only real worry being if the other candidate happened to be stronger than me or not. When the external recruiter got back to me he told me the employer wasn't going with me or the other candidate because they didn't think either of us had the skill set they were looking for. The recruiter said at that point he told them their only option was probably going to be to look for someone not local. I was pretty dumbfounded.
I've also had similar experiences (that didn't make it as far) where they're just hyper focused on someone with extensive Terraform and/or Ansible experience with seemingly little regard to broader DevOps experience, even when I try to talk through some very impressive DevOps projects I've done. I'm beginning to wonder if most places are just terrible at hiring, I'm terrible at selling myself. or a combination of both.
https://redd.it/1j61mh7
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Devops Days
Has anyone attended DevOps Days? Looking to go to the Chicago one.
Love to hear your thoughts / experience?
https://redd.it/1j62uwo
@r_devops
Has anyone attended DevOps Days? Looking to go to the Chicago one.
Love to hear your thoughts / experience?
https://redd.it/1j62uwo
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
on prem containers?
I'm looking to hear from people who are running containers on prem? what is your setup?
https://redd.it/1j64mct
@r_devops
I'm looking to hear from people who are running containers on prem? what is your setup?
https://redd.it/1j64mct
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
How creative can devops work get?
Unemployed right now, but at work I'm usually just on the "dev" side of things unless I have to push my code Github staging or FTP-ing some client's website to their web host. Yeah, I'm doing things old school. Generally I don't see the deployment and automation process to be "creative" stuff unlike application development where I get to figure out engineering problems that keep my mind stimulated.
I build standalone websites /binaries instead of putting them in containers (although I've played around with Docker a bit). Even so, this came at a great cost of job opportunities, where I might apply for a back end role but couldn't satisfactorily explain experience with certain DevOps tools.
Maybe it's more of a thing that solves organizational problems and not technical problems, which can explain a lot about my lack of exposure to DevOps. My dev experience is 95% contract jobs with small teams, for minor staff augmentation work.
I'm not looking for a dedicated role, but some ability of able to apply DevOps to personal work for skill building reasons would be nice. Something that is engaging enough to keep my attention in solo learning purposes.
https://redd.it/1j67eoi
@r_devops
Unemployed right now, but at work I'm usually just on the "dev" side of things unless I have to push my code Github staging or FTP-ing some client's website to their web host. Yeah, I'm doing things old school. Generally I don't see the deployment and automation process to be "creative" stuff unlike application development where I get to figure out engineering problems that keep my mind stimulated.
I build standalone websites /binaries instead of putting them in containers (although I've played around with Docker a bit). Even so, this came at a great cost of job opportunities, where I might apply for a back end role but couldn't satisfactorily explain experience with certain DevOps tools.
Maybe it's more of a thing that solves organizational problems and not technical problems, which can explain a lot about my lack of exposure to DevOps. My dev experience is 95% contract jobs with small teams, for minor staff augmentation work.
I'm not looking for a dedicated role, but some ability of able to apply DevOps to personal work for skill building reasons would be nice. Something that is engaging enough to keep my attention in solo learning purposes.
https://redd.it/1j67eoi
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Next Feature in My Opensource Debugging Tool? Would love feedback!
Hi r/devops,
I'm working on an opensource tool that leverages retrieval augmented generation (RAG) to help diagnose production issues faster (i'm a data scientist by trade so this is my bread and butter).
The tool currently stores Loki and Kubernetes data to a vector db which an LLM then processes to identify bugs and it's root cause - *cutting down debugging time significantly*.
I've found the tool super useful for my use case and I'm now at a stage where **I need input on what to build next** so it can benefit others too.
Here are a few ideas I'm considering:
* **Alerting**: Notify the user via email/slack a bug has appeared.
* **Workflows:** Automate common steps to debugging i.e. get pod health -> get pod logs -> get Loki logs...
* **More Integrations:** Prometheus, Dashboards, GitHub repos...
Which of these features/actions/tools do you already have in your workflow? Or is there something else that you feel would make debugging smoother?
I'd love to hear your thoughts! I'm super keen to take this tool to the next level, so happy to have a chat/demo if anyone’s interested in getting hands on.
Thanks in advance !
the tool: [https://github.com/dingus-technology/CHAT-WITH-LOGS](https://github.com/dingus-technology/CHAT-WITH-LOGS)
https://redd.it/1j6fw3n
@r_devops
Hi r/devops,
I'm working on an opensource tool that leverages retrieval augmented generation (RAG) to help diagnose production issues faster (i'm a data scientist by trade so this is my bread and butter).
The tool currently stores Loki and Kubernetes data to a vector db which an LLM then processes to identify bugs and it's root cause - *cutting down debugging time significantly*.
I've found the tool super useful for my use case and I'm now at a stage where **I need input on what to build next** so it can benefit others too.
Here are a few ideas I'm considering:
* **Alerting**: Notify the user via email/slack a bug has appeared.
* **Workflows:** Automate common steps to debugging i.e. get pod health -> get pod logs -> get Loki logs...
* **More Integrations:** Prometheus, Dashboards, GitHub repos...
Which of these features/actions/tools do you already have in your workflow? Or is there something else that you feel would make debugging smoother?
I'd love to hear your thoughts! I'm super keen to take this tool to the next level, so happy to have a chat/demo if anyone’s interested in getting hands on.
Thanks in advance !
the tool: [https://github.com/dingus-technology/CHAT-WITH-LOGS](https://github.com/dingus-technology/CHAT-WITH-LOGS)
https://redd.it/1j6fw3n
@r_devops
GitHub
GitHub - dingus-technology/CHAT-WITH-LOGS: Identify and solve bugs in your code by talking to your logs!
Identify and solve bugs in your code by talking to your logs! - dingus-technology/CHAT-WITH-LOGS
How do you manage incidents beyond alerting?
At my startup, we've been using PagerDuty to get alerts for high-priority issues, but so far it's mostly just for notifying us. As we're growing, we're thinking of setting up a more structured way to track incidents and make it part of our workflow.
If you've used PagerDuty or any other tool for incident management, how do you approach it? Do you have any recommendations on managing incidents better? What would you say are the most important things to focus on as a company starts scaling?
https://redd.it/1j6gp7g
@r_devops
At my startup, we've been using PagerDuty to get alerts for high-priority issues, but so far it's mostly just for notifying us. As we're growing, we're thinking of setting up a more structured way to track incidents and make it part of our workflow.
If you've used PagerDuty or any other tool for incident management, how do you approach it? Do you have any recommendations on managing incidents better? What would you say are the most important things to focus on as a company starts scaling?
https://redd.it/1j6gp7g
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Learning sysadmin tools feels meaningless
I've had to deploy a MELT solution for a client so I was dealing with networking and devops for a few months. Had to learn a TON to get it to work. Networking, linux, TTYs, computing history etc.
By the end of that period I bought a NUC, and deployed using docker compose an entire stack using plex, radarr, sonarr and other things on it, and made it availalbe via a host domain via /etc/hosts. I was proud of myself. Felt like a sigma engineer.
It hasn't been less than three months ago (work has transitioned into building a fullstack webapp) and my plex server is unreachable. As i'm trying to get it working I figure I forgot like 90% of it all.
Do I use nmap or ip addr to find my NUCs IP? How do I make it have a static IP to add it to /etc/hosts? How again does the docker internal networking differ from localhost?
It all now feels meaningless as any attempts i'm going to make at re-learning how to do those things are going to evaporate whenever my work focus changes. Is this just a part of the work? Am I doing things wrong? WIll it get better with experience in the industry?
https://redd.it/1j6hy7r
@r_devops
I've had to deploy a MELT solution for a client so I was dealing with networking and devops for a few months. Had to learn a TON to get it to work. Networking, linux, TTYs, computing history etc.
By the end of that period I bought a NUC, and deployed using docker compose an entire stack using plex, radarr, sonarr and other things on it, and made it availalbe via a host domain via /etc/hosts. I was proud of myself. Felt like a sigma engineer.
It hasn't been less than three months ago (work has transitioned into building a fullstack webapp) and my plex server is unreachable. As i'm trying to get it working I figure I forgot like 90% of it all.
Do I use nmap or ip addr to find my NUCs IP? How do I make it have a static IP to add it to /etc/hosts? How again does the docker internal networking differ from localhost?
It all now feels meaningless as any attempts i'm going to make at re-learning how to do those things are going to evaporate whenever my work focus changes. Is this just a part of the work? Am I doing things wrong? WIll it get better with experience in the industry?
https://redd.it/1j6hy7r
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Building AI agent for DevOps
I'm building an AI DevOps agent at LocalOps. Curious - what areas/workflows do you think I should automate out of the day to day toil a SRE has to go through otherwise. And why? Here to learn from your personal experiences.
I'm thinking about
\- IaC code gen and self-serve provisioning
\- Incident first response
\- Security scanning and patching
Please share your thoughts.
https://redd.it/1j6ii3g
@r_devops
I'm building an AI DevOps agent at LocalOps. Curious - what areas/workflows do you think I should automate out of the day to day toil a SRE has to go through otherwise. And why? Here to learn from your personal experiences.
I'm thinking about
\- IaC code gen and self-serve provisioning
\- Incident first response
\- Security scanning and patching
Please share your thoughts.
https://redd.it/1j6ii3g
@r_devops
localops.co
LocalOps - App platform
Focus on code. Ship ridiculously fast on your cloud.
How much of a programming are you expected to do as a SRE/Devops?
I checked couple of messages on this reddit - and it looks like there are companies that have a DevOps people that only write pipelines.
It is quite a surprise for me, in my experience it is always that you are expected to be a FULL-full stack engineer. Yes, I started as a software engineer and moved into DevOps because that was a pain point for that team. But even after I worked in small (4 people) and big (4000 people) companies - all the time it was NOT only DevOps, I had to work on back ends, frontends and infra code as well.
Am I really "unlucky" (and I put it in quotes because I still enjoyed all of them!) with my jobs or the opposite is actually quite rare?
https://redd.it/1j6kogf
@r_devops
I checked couple of messages on this reddit - and it looks like there are companies that have a DevOps people that only write pipelines.
It is quite a surprise for me, in my experience it is always that you are expected to be a FULL-full stack engineer. Yes, I started as a software engineer and moved into DevOps because that was a pain point for that team. But even after I worked in small (4 people) and big (4000 people) companies - all the time it was NOT only DevOps, I had to work on back ends, frontends and infra code as well.
Am I really "unlucky" (and I put it in quotes because I still enjoyed all of them!) with my jobs or the opposite is actually quite rare?
https://redd.it/1j6kogf
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Advice before purchasing a networking course
I'm thinking of purchasing this networking course https://classes.pracnet.net/courses/networking/buy. He also runs a youtube channel called practical networking and tbh, it's possibly the best networking videos I've come across. The issue is it's incomplete, hence the lower price. Only 2 of 8 modules are available. I emailed him asking him now if he had timelines on when it'd be complete and said he didn't.
It's only $50 and seems it'll be very comprehensive when finished. Would you purchase an incomplete course?
https://redd.it/1j6og1z
@r_devops
I'm thinking of purchasing this networking course https://classes.pracnet.net/courses/networking/buy. He also runs a youtube channel called practical networking and tbh, it's possibly the best networking videos I've come across. The issue is it's incomplete, hence the lower price. Only 2 of 8 modules are available. I emailed him asking him now if he had timelines on when it'd be complete and said he didn't.
It's only $50 and seems it'll be very comprehensive when finished. Would you purchase an incomplete course?
https://redd.it/1j6og1z
@r_devops
classes.pracnet.net
Networking
The Internet, the Cloud, and Everything in between
Vibe Deployment: Your AI DevOps Guy Has Arrived! 🚀
For all those vibe coders, we are bringing vibe deployment :)
Introducing your AI devops guy so that this doesn't happen
https://res.cloudinary.com/dhpuwy5nd/video/upload/v1741466985/Gass\_mqxgfs.mp4
https://redd.it/1j6qpaz
@r_devops
For all those vibe coders, we are bringing vibe deployment :)
Introducing your AI devops guy so that this doesn't happen
https://res.cloudinary.com/dhpuwy5nd/video/upload/v1741466985/Gass\_mqxgfs.mp4
https://redd.it/1j6qpaz
@r_devops
Reddit
From the cursor community on Reddit
Explore this post and more from the cursor community
Need help on devsecops pipeline and branching strategy
I'm starting my devsecops internship and I was told by our IT architect that we will have 3 environments: development environment, staging environment and production environment. I'm having difficulties trying to understand when will the pipeline trigger and will the deployment to dev env or stage env or prod env be made and what tests of my pipeline will on it.
The deployment will be made on kubernetes clusters on vms on on-premises vmware esxi hosts
this screenshot of branching strategy provided by a devops engineer may be helpful
branching
https://redd.it/1j6sqd7
@r_devops
I'm starting my devsecops internship and I was told by our IT architect that we will have 3 environments: development environment, staging environment and production environment. I'm having difficulties trying to understand when will the pipeline trigger and will the deployment to dev env or stage env or prod env be made and what tests of my pipeline will on it.
The deployment will be made on kubernetes clusters on vms on on-premises vmware esxi hosts
this screenshot of branching strategy provided by a devops engineer may be helpful
branching
https://redd.it/1j6sqd7
@r_devops
ImgBB
s hosted at ImgBB
Image s hosted in ImgBB
When working on migration projects, I encountered an unexpected issue related to the GKE (Google Kubernetes Engine) Ingress controller.
When working on migration projects, I encountered an unexpected issue related to the GKE (Google Kubernetes Engine) Ingress controller. Specifically, I found that the GKE Ingress controller doesn’t support URL path overwriting. Let me explain the issue with an example and walk you through the challenges it caused during my debugging process.
I wrote an article about it, hope this will be helpful for the community
https://medium.com/@rasvihostings/challenges-with-url-path-forwarding-in-gke-ingress-controller-c175057a76d6
https://redd.it/1j6rl5u
@r_devops
When working on migration projects, I encountered an unexpected issue related to the GKE (Google Kubernetes Engine) Ingress controller. Specifically, I found that the GKE Ingress controller doesn’t support URL path overwriting. Let me explain the issue with an example and walk you through the challenges it caused during my debugging process.
I wrote an article about it, hope this will be helpful for the community
https://medium.com/@rasvihostings/challenges-with-url-path-forwarding-in-gke-ingress-controller-c175057a76d6
https://redd.it/1j6rl5u
@r_devops
Medium
Challenges with URL Path Forwarding in GKE Ingress Controller
When working on migration projects, I encountered an unexpected issue related to the GKE (Google Kubernetes Engine) Ingress controller…
As a technical resource how do you deal with sales staff?
The setup here is that I manage a team of support engineers, and a lot of times we're asked to support customer "events" where there is elevated traffic. This is a lot we can do mid-event to mitigate problems and even prevent them, and just a lot more that's well outside our control.
I keep running into situations where something will happen during an event (sudden router failure somewhere on the network, misconfiguration leaves a component vulnerable to a traffic spike, etc), a short lived spike or two in errors results from it, the customer calmly asks for an RFO and the next week of my life is spent dealing with an escalating chain of internal account execs and non-technical customer relations people with escalating temperatures who are all demanding a technical explanation of what happened, but don't like the answer they get.
"I can't spin this" is the phrase that I keep hearing when I explain how the thing broke, why it was impossible for a tier 1 support engineer to predict/prevent, and a step by step of configuration changes that can be made to prevent this from happening in the future. Like, what else did you want if the literal correct technical answer isn't good enough? More often than not we'll triage with an engineering team who is already familiar with the account because 6 months ago they warned the account team about the possibility of exactly what broke and the recommendations were ignored.
Whenever this happens I have a sit down with my own managers and they seem pretty confident that we handled it appropriately. But naturally the sales oriented teams have the ear of upper management and execs, and the story that lives on as canon to both management and the customer is that the support team blew it and didn't flip the switch from "broken" to "fixed" fast enough.
I'll admit there's plenty I don't know about the business end of things, and blaming the first available lowest ranked person you can find will certainly get you off the phone quick enough, but I simply don't see a business upside to painting your support team as incompetent. Is there any approach to navigating this that actually helps or is this just the way it is everywhere?
https://redd.it/1j6u9pq
@r_devops
The setup here is that I manage a team of support engineers, and a lot of times we're asked to support customer "events" where there is elevated traffic. This is a lot we can do mid-event to mitigate problems and even prevent them, and just a lot more that's well outside our control.
I keep running into situations where something will happen during an event (sudden router failure somewhere on the network, misconfiguration leaves a component vulnerable to a traffic spike, etc), a short lived spike or two in errors results from it, the customer calmly asks for an RFO and the next week of my life is spent dealing with an escalating chain of internal account execs and non-technical customer relations people with escalating temperatures who are all demanding a technical explanation of what happened, but don't like the answer they get.
"I can't spin this" is the phrase that I keep hearing when I explain how the thing broke, why it was impossible for a tier 1 support engineer to predict/prevent, and a step by step of configuration changes that can be made to prevent this from happening in the future. Like, what else did you want if the literal correct technical answer isn't good enough? More often than not we'll triage with an engineering team who is already familiar with the account because 6 months ago they warned the account team about the possibility of exactly what broke and the recommendations were ignored.
Whenever this happens I have a sit down with my own managers and they seem pretty confident that we handled it appropriately. But naturally the sales oriented teams have the ear of upper management and execs, and the story that lives on as canon to both management and the customer is that the support team blew it and didn't flip the switch from "broken" to "fixed" fast enough.
I'll admit there's plenty I don't know about the business end of things, and blaming the first available lowest ranked person you can find will certainly get you off the phone quick enough, but I simply don't see a business upside to painting your support team as incompetent. Is there any approach to navigating this that actually helps or is this just the way it is everywhere?
https://redd.it/1j6u9pq
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
PM2 process exits after SSH session ends when deploying via AWS CodeBuild
I’m deploying a Node.js backend using AWS CodeBuild and SSH into an EC2 instance to run the deployment steps. The deployment script successfully:
1. Fetches the latest application code from S3
2. Extracts it to `/home/ubuntu/app`
3. Sets up environment variables from AWS SSM Parameter Store
4. Installs dependencies (`npm ci`)
5. Runs database migrations (`npx prisma generate`)
6. Builds the application (`npm run build`)
7. Starts the application using PM2
**The problem:**
* **When CodeBuild runs the script via SSH, everything executes successfully, and PM2 starts the application.**
* **However, once the SSH session from CodeBuild ends, the process moves to an "errored" state.**
* **Manually running** `pm2 restart backend-app --update-env` **after SSH logs out restores the process to "online."**
# What I’ve tried so far:
✅ Ensured PM2 is running as `ubuntu` user
✅ Used `pm2 save` to persist the process list
✅ Ran `pm2 startup systemd -u ubuntu --hp /home/ubuntu`
✅ Enabled PM2 as a systemd service (`systemctl enable pm2-ubuntu`)
✅ Restarted PM2 service (`systemctl restart pm2-ubuntu`)
✅ Set `export PM2_HOME="/home/ubuntu/.pm2"`
But the issue persists—PM2 starts fine during deployment, yet after CodeBuild finishes, the process moves to "errored." If I log in via SSH as `ubuntu` and manually restart it, **it works perfectly.**
**Why is PM2 treating the process as "errored" when the SSH session ends? How can I ensure it remains running after CodeBuild logs out?**
# Additional Info:
I’ve also tried using **CodeDeploy and SSM** instead of SSH, but both had their own issues:
* **CodeDeploy Agent** doesn't pick up the latest changes properly and causes problems with root user permissions.
* **SSM Run Commands** either behave the same way as SSH (failing after session ends) or stay stuck in an **"in progress"** state indefinitely.
Any insights or suggestions would be greatly appreciated!
PS. I'm limited on using only AWS CI/CD tools.
https://redd.it/1j6z253
@r_devops
I’m deploying a Node.js backend using AWS CodeBuild and SSH into an EC2 instance to run the deployment steps. The deployment script successfully:
1. Fetches the latest application code from S3
2. Extracts it to `/home/ubuntu/app`
3. Sets up environment variables from AWS SSM Parameter Store
4. Installs dependencies (`npm ci`)
5. Runs database migrations (`npx prisma generate`)
6. Builds the application (`npm run build`)
7. Starts the application using PM2
**The problem:**
* **When CodeBuild runs the script via SSH, everything executes successfully, and PM2 starts the application.**
* **However, once the SSH session from CodeBuild ends, the process moves to an "errored" state.**
* **Manually running** `pm2 restart backend-app --update-env` **after SSH logs out restores the process to "online."**
# What I’ve tried so far:
✅ Ensured PM2 is running as `ubuntu` user
✅ Used `pm2 save` to persist the process list
✅ Ran `pm2 startup systemd -u ubuntu --hp /home/ubuntu`
✅ Enabled PM2 as a systemd service (`systemctl enable pm2-ubuntu`)
✅ Restarted PM2 service (`systemctl restart pm2-ubuntu`)
✅ Set `export PM2_HOME="/home/ubuntu/.pm2"`
But the issue persists—PM2 starts fine during deployment, yet after CodeBuild finishes, the process moves to "errored." If I log in via SSH as `ubuntu` and manually restart it, **it works perfectly.**
**Why is PM2 treating the process as "errored" when the SSH session ends? How can I ensure it remains running after CodeBuild logs out?**
# Additional Info:
I’ve also tried using **CodeDeploy and SSM** instead of SSH, but both had their own issues:
* **CodeDeploy Agent** doesn't pick up the latest changes properly and causes problems with root user permissions.
* **SSM Run Commands** either behave the same way as SSH (failing after session ends) or stay stuck in an **"in progress"** state indefinitely.
Any insights or suggestions would be greatly appreciated!
PS. I'm limited on using only AWS CI/CD tools.
https://redd.it/1j6z253
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Creating EC2 security group rules for Pingdom?
I have an EC2 instance hosting a webserver that Pingdom performs uptime tests against.
I need 80/443 open to my web server so Pingdom can hit it, but I don't want the web server to be publicly accessible.
I was thinking of manually adding all of Pingdom's probe IP addresses, but there's a couple hundred.
It seems like people have made projects to get around this issue (see PicnicSupermarket/pingdom-probes-aws-whitelist and andypowe11/AWS-Lambda-Pingdom-SG on GitHub).
However, many of the projects are pretty old. I was curious if someone could suggest a project/method that they know works in 2025. Thanks!
https://redd.it/1j70c2f
@r_devops
I have an EC2 instance hosting a webserver that Pingdom performs uptime tests against.
I need 80/443 open to my web server so Pingdom can hit it, but I don't want the web server to be publicly accessible.
I was thinking of manually adding all of Pingdom's probe IP addresses, but there's a couple hundred.
It seems like people have made projects to get around this issue (see PicnicSupermarket/pingdom-probes-aws-whitelist and andypowe11/AWS-Lambda-Pingdom-SG on GitHub).
However, many of the projects are pretty old. I was curious if someone could suggest a project/method that they know works in 2025. Thanks!
https://redd.it/1j70c2f
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Trying for a DevOps role, offered a software support role, will it help?
It’s a weird situation but I am a fresh grad computer science and really interested in DevOps and have been studying it and bootcamps and what not. Now in my search for jobs, I applied to this role I was told that I am being heavily considered and that I might get accepted soon.
Now the role itself is mostly database support and maybe some customer communications regarding APIs and such (things that can be handled with the company’s documentations as well). Now my issue is while I’d love to finally work and gain experience, I fear that I might have landed a role that may be far from what I want and I fear that later I won’t have the ability and ease of transition to DevOps or similar roles.
I decided to ask the experts here as I personally know no one who even understands what DevOps means or stands for.
Context note: the company isn’t huge, around 500 employees and does have DevOps engineers in it, I’ve seen 4 on LinkedIn so I assumed a team of 4.
https://redd.it/1j738do
@r_devops
It’s a weird situation but I am a fresh grad computer science and really interested in DevOps and have been studying it and bootcamps and what not. Now in my search for jobs, I applied to this role I was told that I am being heavily considered and that I might get accepted soon.
Now the role itself is mostly database support and maybe some customer communications regarding APIs and such (things that can be handled with the company’s documentations as well). Now my issue is while I’d love to finally work and gain experience, I fear that I might have landed a role that may be far from what I want and I fear that later I won’t have the ability and ease of transition to DevOps or similar roles.
I decided to ask the experts here as I personally know no one who even understands what DevOps means or stands for.
Context note: the company isn’t huge, around 500 employees and does have DevOps engineers in it, I’ve seen 4 on LinkedIn so I assumed a team of 4.
https://redd.it/1j738do
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Looking for DevOps Projects Can someone sends a lectures videos link – Recently Completed Basics!
I recently completed the basics of DevOps and have a medium-level understanding of CI/CD, Docker, Terraform, and Kubernetes. Now, I want to work on some real-world projects to solidify my skills. Could u suggest me some videos where i can learn end-to-end make projects add into my portfolio.
https://redd.it/1j75nvk
@r_devops
I recently completed the basics of DevOps and have a medium-level understanding of CI/CD, Docker, Terraform, and Kubernetes. Now, I want to work on some real-world projects to solidify my skills. Could u suggest me some videos where i can learn end-to-end make projects add into my portfolio.
https://redd.it/1j75nvk
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
Considering a Career Shift to IT - DevOps or Other Remote Roles?
Hello friends,
I'm reaching out for some advice and insights. I'm currently in a non-IT role that's at risk of being outsourced, and with a salary of about $120k at 47, I feel the need to diversify my skills to ensure future stability and my mental health.
As I don’t want to loose my job without a plan.
And it would be nice to pursue remote work.
Hypothetically if I don’t loose my job it would nice to have a part time home based side hustle.
In the past, I served as a data systems analyst in the Air Force from 1997 to 2017, so I do have some background in IT although archaic.
To mitigate some of the anxiety about my job security, I've started exploring new skills, particularly in roles that could lead to remote work.
I've been playing around with Perplexity AI asking for suggestions.
And DevOps keeps popping up, but from what I've read, it seems like it's far from entry-level.
There are plenty of "gurus" offering training courses claiming you can be DevOps-ready in just six months, which I'm skeptical.
Currently, I'm taking Python courses and considering moving on to Linux and possibly Kubernetes, though I've heard it takes about a year to master which is a ridiculous time investment.
Perplexity suggested some alternative roles that might be more realistic.
1. Remote Python Developer (Backend/Frontend)
2. AI/ML Engineering Support Roles
3. Cloud Automation Specialist
I'm hoping that by the time I get some skills and certs down the job market will have improved.
Has anyone transitioned into these roles from a non-IT background, especially with a focus on remote work?
Any advice or experiences you can share would be greatly appreciated!
Thanks in advance for your input.
https://redd.it/1j7dcck
@r_devops
Hello friends,
I'm reaching out for some advice and insights. I'm currently in a non-IT role that's at risk of being outsourced, and with a salary of about $120k at 47, I feel the need to diversify my skills to ensure future stability and my mental health.
As I don’t want to loose my job without a plan.
And it would be nice to pursue remote work.
Hypothetically if I don’t loose my job it would nice to have a part time home based side hustle.
In the past, I served as a data systems analyst in the Air Force from 1997 to 2017, so I do have some background in IT although archaic.
To mitigate some of the anxiety about my job security, I've started exploring new skills, particularly in roles that could lead to remote work.
I've been playing around with Perplexity AI asking for suggestions.
And DevOps keeps popping up, but from what I've read, it seems like it's far from entry-level.
There are plenty of "gurus" offering training courses claiming you can be DevOps-ready in just six months, which I'm skeptical.
Currently, I'm taking Python courses and considering moving on to Linux and possibly Kubernetes, though I've heard it takes about a year to master which is a ridiculous time investment.
Perplexity suggested some alternative roles that might be more realistic.
1. Remote Python Developer (Backend/Frontend)
2. AI/ML Engineering Support Roles
3. Cloud Automation Specialist
I'm hoping that by the time I get some skills and certs down the job market will have improved.
Has anyone transitioned into these roles from a non-IT background, especially with a focus on remote work?
Any advice or experiences you can share would be greatly appreciated!
Thanks in advance for your input.
https://redd.it/1j7dcck
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community