Disruption Ahead: AWS Quietly Axing Services, including Cloud9, SimpleDB, CodeCommit and more.

It started a couple of days ago with users reporting services being blocked, or warning banners.

There was no official announcement, but according to Jeff Barr's reply on X (Twitter), he listed S3 Select, CloudSearch, Cloud9, SimpleDB, Forecast, Data Pipeline, and CodeCommit.

Though it may not be the only services.

https://horovits.medium.com/disruption-ahead-aws-quietly-axing-services-033e7518eefb

https://redd.it/1egi010
@r_devops

Moving from FE to DevOps

Has anyone successfully made the switch?

Can you share what you did /studied.

I've got my azure fundamentals 900 booked.

I've worked in Microsoft dynamics 365 and powerapps for 2 years.

Obvious crossover skills from power shell, CMD and git.

How did you find the move?

https://redd.it/1egi8ni
@r_devops

Help me convince my manager to buy me a backend course

Hello everyone,

I am currently a DevOps and Cloud Engineer, primarily working with Terraform, K8s, AWS/Azure. My role also involves coding in Bash and Python to automate tasks, develop new tools and scripts, and interact with cloud providers' SDKs, among other things.

My interest in backend development is growing, although I am not looking to transition into a SWE role at the moment. My goal is to broaden my skill set to become more versatile and excel in my current position. Additionally, I am keen on learning Golang to enhance existing tools or develop new ones.

I am considering this course because it appears comprehensive, and I aim to use this opportunity to strengthen my foundational knowledge in development (DSA, OOP, FP, SQL). What are your thoughts on it? As I mentioned earlier, my primary focus is on K8s, Terraform, and cloud architecture, and while I do engage in coding, it differs from backend development. Could you suggest some arguments that might persuade my manager to sponsor this training for me?

https://redd.it/1egjdz7
@r_devops

Non-ephemeral CI runners

Wondering if anyone else has had experience with the approach of having a CI runner provide an environment that’s not pristine on every run.

I’ve seen this with Jenkins at a previous org and currently with self-host GitHub actions runners.

Usual arguments are things like:
- we run on bare metal for extra performance, spinning up a new VM/container just slows things down
- persistence between jobs is good because caches are hot and shared without needing to upload/download to an object store
- we only run trusted jobs so security issues are not so glaring

Issues frequently encountered:
- orphan processes or containers in the host eventually consuming all memory/disk after days/weeks
- tools which store their config in $HOME get configured by one job and give surprising results in another
- general shared resource collision, e.g. docker container with that name already exists because another job is currently running on the same host

Any other experiences or points you’d add? I’m trying to make a case that the cons outweigh the pros here, but the team running the CI agents don’t seem phased by the repeated incidents/outages/flakiness or have the bandwidth to change things.

https://redd.it/1egimwd
@r_devops

How do I build a useful Grafana dashboard for our devs?

I'm trying to design Grafana dashboards and I want them to be meaningful and useful to any dev who would open them.

Most of our alerts are for unusual error rates and resource consumption so I'll try to focus on those:

- If an app produces unusual amounts of errors over a certain amount of time, the link we add to our Slack messages are usually just logs for that app with a window of the last X minutes.

- For resource consumption, since we're deploying everything in K8s, we just add a link to a generic Grafana dashboard that shows resource usage/limit of said app that alerted.

The links are already filtered by the app/container name, namespace, cluster (stg/cand/prod) and a relevant time window.

At first I thought that would be enough but lately I found out that devs don't even open those links. They might only open the dashboard that shows logs, but I heard some devs get lost in the "wall of text" of those logs.

Are there good examples of what data you usually show in your Grafana dashboards? Any tips are welcome.

https://redd.it/1eglat1
@r_devops

Pipelight - Clean this git-hooks mess!!

You may do sport, eat vegetables, get a good amount of sleep time and even have a colon cleanse once in a while with your extra bucket.
But let me tell you. Hygene is not about beeing clean!
It is about having clean!
Having clean git-hook logic and manageable files!

Fortunately nowadays, you can manage your git-hooks with toml, yaml, hcl and javascript.

Add a config file to your repo.

[[pipelines]]
name = "test"

[[pipelines.steps]]
name = "build"
commands = ["pnpm install", "pnpm build"]

[[pipelines.triggers]]
branches = ["master","dev"]
actions= ["pre-push", "pre-commit"]

Enable git-hooks

pipelight init

Troubleshoot with verbose and colorful logs

pipelight logs -vvvv

Darn this feels great! I like to tinker with bash and python to set up my hooks but sometimes, having such an easy way to just get up and running really makes me... clean!

Written in Rust
13kb binary
GPLv2

https://github.com/pipelight/pipelight

https://redd.it/1eghzhi
@r_devops

AWS WAF for external domain

Has anyone used WAF for an external domain?
My current company hosts the majority of their application in Aptible.

Currently in a planning process to migrate more things to AWS down the road but not soon enough.

Looking to implement WAF/ALB but the domain is external and handled through Aptible. Would it be best to have a lambda as the TargetGroup or is there a better way to handle this?

https://redd.it/1egq0wd
@r_devops

Help with AWS

Hi everyone

I wanted to start learning more about AWS, since the only thing I've been working on has been on Route 53, a lil bit of IAM, EKS and Beanstalk

What would you say it is the most important things on AWS to learn as a DevOps?

Thank you!

https://redd.it/1egprz0
@r_devops

DevOps interviews and coding tasks...

Last time I switched jobs was 3 years ago, and yeah, its job hunting season again. Now, the difference between 3 years ago and now is that no one had leetcode shit on their devops interviews, while I see everyone around me complaining about them now. Really? Apart from having to memorize a billion of network/infra/etc shit, now I have to grind python/bash leetshit in the age of gpt? What's your experience with that, and where would you recommend I start? Is there some collection of really nice devops flashcards I could use, and regarding python/bash, I assume basic level of knowledge is enough?

https://redd.it/1egrogb
@r_devops

What's the best Kubernetes course online?

Hey everyone,

I've come across the course Kubernetes for the Absolute Beginners - Hands-on on Udemy. Has anyone taken this course? Is it good?

Are there any other online courses you would recommend for learning Kubernetes? I'm looking for something that covers both the fundamentals and provides practical, hands-on experience.

Thanks in advance!

https://redd.it/1egt6qc
@r_devops

How often should you job hop?

In a very lucky position, I managed to go from no IT background to a Junior Devops role.
However, I work in a consultancy, and the work is so hit and miss.
I can be on the bench for two months and then shoulder deep in work for two months.
I know the down time is good to get some revision in, however, I really dislike my company.

They have some very dubious work practices and are very nasty to their staff. There is a lot of slagging people off behind there backs. And constant blame culture situations.
I would happily stay doing the work a lot longer but the company is just draining the life out of me.

As a first job in IT, and especially Devops being a more advanced position, I don't want to leave my role until I can put enough time on my CV for future employers to accept me. I have good knowledge, but I think this time exposure is also expected.

I have always been told that 2 years is to aim for. But what do you guys think?

https://redd.it/1egv9yl
@r_devops

Bitbucket Pipelines to remote server via rsync or git / post-recieve

Hi All,

Newbie here who is trying to learn. I've been doing a lot of searching and I feel a bit stuck.

I'm using Bitbucket Pipelines, which is pretty much the same as Github Actions, but at the end of the day, I don't think the runner matters. The DevOps world is new to me and I'm trying to figure out the "best" way to deploy code to a server. To start, if my understanding is correct, both BBP and GHA create a docker environment, lint, do whatever, build your code. I have that working just fine. Now I want to get the compiled code on the remote server.

I first went down the rsync route. I got this working just fine. But I realized, it's syncing all files, every, commit. I understand why, and that's because in each docker instance that's created in the runner, the built files will always be newer than what's on the destination. I've tried a few different flags and sped that up a little bit but still wasn't satisfied. I searched and learned many have encountered this issue. I'm like, oh ok, then I can dynamically create a changed-files.txt with some git magic. Then via rsync, I had `include-from=changed-files.txt`. This worked, each commit, I was able to only sync those few files. But then, tragic struck and I had a build fail due to a linting issue. It happens. I resolved the issue, and then triggered a new deploy, and the changed files were no longer in that new commit. That made sense because the changed-files.txt script was checking THAT commit.... went down the rabbit hole and learned this is an issue people run into. There's no easy way to track the last successful build, create the changed-files.txt, and then deploy. So... I gave up on this route because my brain was hurting.

Onto the old git route, I was thinking I can create a bare repo on the destination server in /repos and create a post-receive hook to deploy the code pushed into /website.com.

I have the YML doing a few things:
- A git repo in the runner's environment will be created.
- A commit containing changed files / build
- It will SSH into the destination,
- create a `/repos` if it doesn't already exist.
- Create a bare repo and then create a post-receive hook.
- add remote remote to runner's newly created repo.
- git push to that repo

I'm not having the best success. Thinking now I might need to clone the repo to the destination server? But with that idea, I think it would clone, every deployment which seems odd to me. The repo in the runner, would be new, every time, so I feel I might be running into a similar issue as rsync?

A last minute thought could be cloning the bitbucket repo to the destination, then, each deployment, compare the two for differences and create a changed-files.txt. But that seems resource intensive too, and overkill.

My head is like scrambled eggs after all the googling. I surprisingly haven't come across any tutorials that are trying to do what I am. I've seen a lot with rsync and git-ftp, but none address the issue of it comparing allll the files, every time.

Any help, guidance, would be appreciated.

https://redd.it/1egv7oe
@r_devops

EC2-based NAT Gateway alternative

I’ve developed a fully functional guide and code base for setting up an EC2 instance as a NAT Gateway, offering a cost-effective alternative to the AWS NAT Gateway. Plus, I’m working on adding a Lambda function to automatically replace instances and ensure seamless routing in case of any issues.

Explore the project on GitHub and let me know what you think! Feedback here and PRs on GitHub welcome.

https://github.com/MikeGarde/aws-ec2-nat-gateway

https://redd.it/1eh0ki4
@r_devops

Where to run unit tests when building inside docker

Trying to determine what would the best approach in the following situation:

Currently I have a CICD pipeline in Jenkins for building a Next.js application. It consists of separate stages like:
1. installing deps
2. running unit tests
3. building the application and finally copying the build output into a docker container and pushing it.

Because steps like building also require a separate base image with node and pnpm installed this adds extra maintanance.

I’m looking into doing the building inside of a multi-stage dockerfile. This seems to work quite well with the exception of the test stage for the unit tests. I’m kind off conflicted if it’s wrong to add an extra stage and run the tests on building of the dockerfile or if it really should be done outside?

Curious to hear how others are approaching this issue.


https://redd.it/1egrmbs
@r_devops

15 Cloud-Native DevOps Tools You Should Know

[https://overcast.blog/15-cloud-native-devops-tools-you-should-know-36129057a15c](https://overcast.blog/15-cloud-native-devops-tools-you-should-know-36129057a15c)


These tools are particularly beneficial in several scenarios. They are essential when managing microservices-based applications, requiring continuous integration and delivery, needing robust monitoring and alerting, seeking to automate infrastructure management, and ensuring security and compliance in complex environments. Utilizing these tools helps maintain seamless operations and quickly adapt to changes.

* **Managing microservices-based applications.**
* **Requiring continuous integration and delivery.**
* **Needing robust monitoring and alerting.**
* **Seeking to automate infrastructure management.**
* **Ensuring security and compliance in complex environments.**

https://redd.it/1ehb0fb
@r_devops

Applying for DevOps positions as a East Indian-American

Disclaimer: this is a rant, but I'd like some advice, please.

Hey there, I know there's similar threads for job application strategies on r/cscareerquestions and r/jobs, but I wanted input specifically from other DevOps/Platform Engineers.

I was born and raised in America, but my ethnicity is Indian- and my name is pretty Indian sounding. I've been hearing SO many reports of (from Reddit) about rampant fraud happening from other Indians or people on a Visa, that I believe my application has a high chance of just being tossed because of my name.

I've never had this problem before, and my non-indian coworkers have been getting blasted with job offers and have a similar background as me. I don't want to blame it on my race, but I'm thinking it's crazy I can't even get an interview! I'd hate to do a bait and switch by changing my resume to an Anglo name just to test out my hypothesis, but I have bills to pay and my current position is destroying my mental/physical health.

Any advice?

My tech stack is .NET, Azure, Bicep (IAC), GitHub Actions, Octopus Deploy, Windows IIS, PowerShell and Bash.

I'm applying to jobs that don't have Kuberketes while I study for the CKA. I was a network engineer for 1 year before I ventured into DevOps 3 years ago. Any advice on how to make my resume stand out? I have a pretty nice looking one I made with LaTeX and has the standard jobs, personal projects, skills, and education.

Also, I'm a woman- Idk if that helps, but wanted to mention since there's literally no other women in my team of 40 engineers, so I'm either doing something right or I'm getting in because of my gender- which UGH.

https://redd.it/1ehax2n
@r_devops

Should I leave or stay?

I am feeling quite burnt out now. I am currently in a DevOps position in a startup and the hours + responsibilites are taking a toll on me. Being expected to debug application issues, manage kubernetes environments, manage cloud infra, CICD across 3 different platforms, supporting various projects as well, maintaining observability by creating reports from SLA metrics and usages. All without getting the support I need.

Its only been 1 year and 3 months. I currently have a 1 year contract offer for a DevSecOps position. Should I just take this role? Or stick around longer and toughen it out and hope it gets calmer?

https://redd.it/1ehcvsx
@r_devops

Transitioning from Jira Admin to Devops

Hello, I wil probably accept an offer as a "System Engineer" for which the main responsibilities are Jira administration and integrations. I currenty work a similar role and before that I have worked as a backend developer (Java/Spring) for 3 years. The company I join is a very big e-marketing one, and the team I would be in is actually DevOps.

The System Engineer job is described like this:

full stack admin of jira+confluence
maintain and optimize tools
automation of processes
develop enhanchements
support requests triaging
They require: jira/confl admin experience, MySQL/Postgres, Linux OS, nice but not necessary: API, Python, Groovy

The role is focused on Jira but I would like to evolve from there as a DevOps engineer. In the team here are people who take care of jira (2 + me if I join) and then 6 people who are more "pure" devops (I assume CI/CD, Infra, integrations, etc.).
One thing that made me successful during the interviews was my engineering background and Java/Groovy knowledge.

Here's my question - how likely do you think it is that I can evolve into a ""pure"" devops position? Don't get me wrong, I know there is no actual "pure" devops as it is a very generic term, but I mean that as taking care of many more technologies and automation rather than only Jira/Confluence. Can Jira get me closer to infra tools and CI/CD?

The company I currently work for is rather small and Jira is not integrated with CI/CD, it is not even used by developers, so it is probably very different in a company like the one I will join.

https://redd.it/1ehdpq2
@r_devops

10 Examples Why cURL is an Awesome CLI Tool

Hi /r/DevOps,

Today I published an article titled "10 Examples Why cURL is an Awesome CLI Tool" - Where I explore some of the lesser known or unexpected features of cURL, such as globbing, parallelization, output formatting, URL parsing and much more.

Here's the link: https://levelup.gitconnected.com/10-examples-why-curl-is-an-awesome-cli-tool-0b23223ec845

Feedback is very much appreciated!

https://redd.it/1ehhihj
@r_devops

AWS code build stuck at npm install and getting timeout with no errors, No error on code build logs as well as cloud watch.

Using CodeBuild On-demand
Running node v16.13.0

The same code works previously now I'm not able to build since last few days.

Since there is no error how do I debug this?

https://redd.it/1ehirh7
@r_devops

How hackerrank works across organizations?

I am back in market looking for open roles and started noticing firms sending out hackerrank tests to gauge candidates.

I am trying to understand how this works?

Basically will it have a standard questions (at different levels) or can each company set their own questions for the roles they publish out in market?

How does it work people of this group? 🙋‍♂️

https://redd.it/1ehih4u
@r_devops