file: pipeline/transformations/abc/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: "${{ env.DOCKER_USERNAME }}/${{ env.DOCKER_REPOSITORY }}:${{ env.IMAGE_TAG }}"
cache-from: type=gha
cache-to: type=gha,mode=max

https://redd.it/1ejb7ed
@r_devops

Acquiring a New AWS Environment: Seeking Insights on Best Practices for Smooth Transition and Integration

Hello AWS community,

Our company is in the process of acquiring another firm, and part of this acquisition involves taking over their AWS environment. The services they use include EKS, RDS, and Elastic Beanstalk, among others. We'll receive a replica of their system on a new AWS account that will be handed over to us.

What do you guys recommend for us to stay on a lookout, anybody experienced with such transitions?

https://redd.it/1ejd2gj
@r_devops

dynamic vs single database approach for beta version

Hi everyone,

We're about to launch a beta version of our product and have implemented a dynamic database approach: creating a separate database for each company using our platform. We chose this method to enhance security and stability, ensuring that if one database goes down, the others remain unaffected.

However, a friend of a friend recently commented that this approach might be overkill and expensive, suggesting that a single database could be more efficient and effective.

I would really like to hear you advice on this matter, for those who have experience with both approaches, which is better? In the long run or at all? I really want to avoid a pitfall with this, please if you have some knowledge to share it will be really appreciated!

https://redd.it/1ejcyin
@r_devops

Do any of you actually genuinely care about your employer’s core business?

Always been a technical oriented person, and find myself sometimes zoning out once co-workers on the non-technical side of the firm start talking about stuff they find interesting.

I might get drawn in if I sense they really know their stuff, but on the day to day unless the core business is related to anything I am familiar with I would be fighting for my life to stay awake in the workshops with the project managers.

Anyone else experience something similar, if so what motivated you to overcome it. E.g might be that you are a backend engineer at AirBnb that really cares about the service industry and the how AirBnb could introduce new features to improve.

In contrast, I have enough on my plate and if the PM thinks that some cool feature should go into a sprint then ok. But I would rather stay up to date on AWS, Docker, K8s than discuss the review system of AirBnb.

Background: been in tech my whole adult life, and got some opportunities to do some customer facing tech related jobs i.e customer support. But once I finished my master’s in Comp Science, the technical part of my brain just kept churning and got thrown into the deep end of a steep learning curve, technically speaking. I

https://redd.it/1eji6l9
@r_devops

Monitoring Recs?

Hey All

Just spun up some infrastructure for a project im working on (everything is currently hosted on aws in case it matters).

I want to wire in some monitoring. Since it is somewhat of a side project at this point, im curious to see if anyone has any recs on monitoring frameworks, or had their eye on a cool monitoring project that they may or may not have had a chance to check out yet.

Typically I either lean Prometheus+Grafana or Icinga2, but figure i might as well spice things up a bit.

Thanks in advance!

https://redd.it/1eji2nn
@r_devops

What does your DevOps look like, and what can I propose to my company to improve our DevOps

For some context, our company is a BPO (Business Process Outsourcing) provider. We write custom software solution to sell to business. Our core business is data scanning and document formatting. Most of our applications are monoliths and some shared services like a GPU farm. (I am also newish of 1.5 year, so I may not know all our stuff). What I guess you could need in a DevOps environment:

A central place for issues, tickets

We have mantis, helpdesk and "Espaceprojet" a tool to manage clients demands. We have been promised Jira, but I don't think it's happening. Nothing is linked with DevOps pipelines...

A central place for code, code reviews, version management

We have self-hosted GitLab, we use a gitflow workflow. We have runners for code linting / testing. Our git projects look like this in general :

- project-core

- project-iac (100% puppet)

- project-terraform

- project-confjobs (jobs are for Rundeck) confs are a bunch of yaml file that will be ingested at provision time



A central place for credentials

Vault (self-hosted) vault work well for stuff like renewing certificates

A central place for configurations

We use git, but we are moving to Vault even for basic stuff. Vault agent can dynamically load configurations, however, many of our application need a restart to apply the modifications. I am not sure vault should be used like that

A central place for pipelines :

We use Jenkins, we build "project-core" and "project-iac" as Linux packages automatically with it. We need to launch "project-terraform" manually to create all the VM that will be necessary for a project (no docker... 🙁)

A central place to store deployment ready images (artifacts)

We use nexus for all our packages and artifacts. We have many, many puppet modules to manage the installation of tools on our VMs.

A virtualization platform or bare-bones metal

We have both, we have two big Nutanix clusters, a lot of VMs on Outscale (https://fr.outscale.com/) some few VMware and HyperV here and there. We also have a developer cloud (OpenStack). We also have a GPU farms which is used for some of our products.

A service discovery service : We don't have that
An orchestrator per project

We use Rundeck with scheduled jobs (Backups, Restores, Reboot, Tenable Scans, Jobs to export logs, many other things...) We also use the Rundeck for provisioning VMs and deploy applications to different environments. Most of this is manual, but it's always just one button to press

A central monitoring platform

We have Centreon and an internal tool that leverage Rundeck jobs to check service health, then create a report and then send it to Nagios, which can send us alerts or even calls.

A place to store procedures, on call procedures, contacts and Architecture documents

on SharePoint and in a wiki

The local developer environment: We have a project skeleton and an openstack cloud for devs to bootstrap a dev environment and do their own test.

Please roast us.

https://redd.it/1ejkdux
@r_devops

How many of you have seen the original Flickr/Velocity-2009 presentation?

https://www.youtube.com/watch?v=LdOe18KhtT4

this is essentially the moment 'devops' was born as a popular buzzword concept. patrick dubois started the eponymous conference a few months later. we all know that the term grew to mean everything and anything and then eventually nothing, but I'm curious what % of devops redditors ever even saw the original meaning/purpose in the first pace?

View Poll

https://redd.it/1ejklvt
@r_devops

DevOps Security Best Practices - Fortifying Your Development Pipeline

A comprehensive guide to implementing DevSecOps practices for enhanced security in your software development lifecycle.

https://redd.it/1ejrzxn
@r_devops

Requesting feedback and advice on my transition plan into DevSecOps

I understand there is no one set path. I would appreciate any comments or real tal on the likelihood of this transition plan:

Background:

- BA in nontechnical field,
- Air Force SIGINT analyst for 20 years
- TS/SCI (Top Secret) with CI polygraph <-- I understand it's irrelevant in the civilian sector, but I am hoping to find a cleared role where I can get my feet wet
- PMP, A+, some Python scripting exp, RHCSA (taking it in Oct)
- IT project management as a gov rep for contractors (netadmin, sysadmin)

In 2025:

- Start DevSecOps internship (SkillBridge) with a DoD contractor with a 3-letter agency (Feb - Aug)
- SSCP (DoDM 8140.03 IAT level II, Security+ equivalent) by Feb 2025
- CISSP by Aug 2025 <-- I understand this falls more on the GRC side but wanted to highlight my security background. Already endorsed by another CISSP professional
- AZ-204, AZ-400 or AWS SAA
- CCSP
- Learn, learn, learn both in and out of work with labs, hands-on experience

I will retire from the Air Force in Aug 2025 with the 6-month internship under my belt. My goal is to aim for sysadmin or netadmin as a starter, and work my way up through cloud engineer or any company that would give me a chance for a DevOps-related position.

Is there anything I can do differently? I am also considering these in 2025:

1. CCNA on behalf of AWS or Azure, to learn the fundamentals of networking and shoot for netadmin, because it will give me solid experience for cloud and all aspects of networking.

2. CDP, because I find it very helpful for a roadmap-style learning experience.

Thank you in advance!

https://redd.it/1ejsub0
@r_devops

Anyone test blacksmith.sh

Hello, we are thinking of testing blacksmith.sh, has anyone used this or another website?

For Github action

https://redd.it/1ejsq2a
@r_devops

How different is DevSecOps from DevOps?

Please elaborate

https://redd.it/1ejxfkj
@r_devops

Masters in Devops

Hi Everyone,
I am curious to know whether Universities have a masters program for Devops. Or is there no such thing called masters in Devops. I am interested in this field and tried to home school myself on basics of Devops from online courses and YT tutorials.
My parents want me to go for higher studies, which even I want to do but I need some advice to know what I should go for and proceed in which direction. I think I'm still a fresher and I currently have just 1yoe in a company where I do V&V for sensors. I don't see too much of a growth in this line and I'm not very interested in this line of work.
Thus, reaching out to the community to get some guidance or information on how to transition myself into Devops since I'd like to do this for a long term growth.
Best regards, Thanks!

https://redd.it/1ejy3ia
@r_devops

Greenfield EKS Setup and App Modernization Suggestions

Greetings DevOps community,

Our team has a rare opportunity to standup the infrastructure for an application modernization initiative from scratch. I wanted to come to the community to solicit suggestions, advice, or general best practices that we should adhere to, to avoid the biggest pains when I comes to managing kubernetes. Please note, my main goals are simplicity as the team needs to ramp up of course.

Some background. We are leveraging GitHub Actions primarily for our CI/CD and another team at the company already uses ArgoCD. So, we figured we’d take advantage and use that as well. The apps are all on older versions of dotnet, so we are working with the development team to get those updated to dotnet 8 and containerize them. We do not plan on breaking down all those apps into microservices, the majority will remain monolithic, as they are small in nature. We just plan to break out the database layer into RDS and for a few of the apps break the front end and backend into separate pods, for example. Nothing too fancy. There are one or two apps which would lend themselves to maybe a citadel model, but this wouldn’t be common.

Anyway, this is what we thought of so far:

- continue to use GitHub actions for CI and perhaps leverage ArgoCD for deployment/GitOps.

- deploy and manage the clusters using terraform.

- leverage ECR for our registry and EKS as our platform.

- utilize wiz for container scanning and runtime security.


What id like advice on is:

- How should we organize this cod?
- Should we utilize helm or kustomize( or both)?
- What would local development look like for our devs? Should we get them minikube?
- Anything else the community could offer advice on.

Approaches I’ve seen are to keep all the kubernetes manifests in one repository, separate apps by folders, and then each app into environment folders using kustomize? Then keep the application code separated into its own repository where the only processes that would occur is Ci, building the dockerfile and deploying the image to EcR? Thus keeping app code and infra code separated?


Thank you!



https://redd.it/1ejwues
@r_devops

Starting a DevOps role here shortly... Advice?

Hey guys,

So I'm starting a new job here this week, and it's for a DevOps role. My background is in Software engineering and programming, mostly with Python, with approx 4 years or so experience.

I went in for the interview and explained that I don't really have any experience in DevOps, and I left the interview thinking there's no way in hell I'm gonna get this job. To my suprise, the company actually made me an offer and I accepted it. They said that in the early stages it's gonna be a lot of learning for me to do, but I'm sat here baffled that they've offered me the role when they could likely find someone who is actually experienced in DevOps instead...

So I'm wondering, does software engineering lend itself to DevOps much? I'm a bit nervous at this point, but at the same time, I didn't lie to them about my capabilities and they know that it's pretty much gonna be a fresh start for me. Any advice would be awesome

https://redd.it/1ek1uzm
@r_devops

OpenTelemetry Tracing on Spring Boot, Java Agent vs. Micrometer Tracing

>My demo of OpenTelemetry Tracing features two Spring Boot components. One uses the Java agent, and I noticed a different behavior when I recently upgraded it from v1.x to v2.x. In the other one, I’m using Micrometer Tracing because I compile to GraalVM native, and it can’t process Java agents.
>
>I want to compare these three different ways in this post: Java agent v1, Java agent v2, and Micrometer Tracing.

https://blog.frankel.ch/opentelemetry-tracing-spring-boot/

https://redd.it/1ejzkcd
@r_devops

Mimir or Thanos at Scale

I'm kicking off an investigation to build a observability system for our data platform. We run self-hosted Trino, Hadoop (YARN and HDFS), and Spark. It feels like taking advantage of Prometheus makes sense to minimize our need to build a totally custom solution.

One of the requirements is persistent storage for the metrics. We need to build aggregations to create business level metrics. We want to allow these metrics to be used for debugging. DataDog is expensive. The system needs to support high throughput and should have the ability to retrieve metrics for up to 90 days.

With the above in mind, I'm looking at Thanos, Mimir, and TimescaleDB. I understand that all three are different. How's the scalability of Thanos or Mimir with S3? Have you experienced any issues at scale? How about Timescale? If I went with Timescale, we could use Kafka Connect for metric ingest. Also, their tiered storage looks attrative. Any thoughts?

Edit: I forgot to add TimescaleDB to the title.

https://redd.it/1ek42xi
@r_devops

Windows for work?

Hi there!

I've been working in the DevOps field for two years. I've been using macOS, and it's been working well for me, and I've enjoyed it. However, I'm about to start a new job.

I've been given a Windows laptop, and I'm aware it can cause issues, like the base64 encoding in Terraform (for instance, if the VM is generated with Linux, it can make you recreate the machine).


What would you recommend I do?

On one hand, I feel it's not right to change the OS on a company-issued laptop. But on the other hand, I'm not sure if Windows will be able to handle the job...

https://redd.it/1ek5cbk
@r_devops

Creating AWS AutoScaling Group with Terraform

Good tutorial for anyone learning AWS and terraform

https://youtu.be/1UtEyacje98?feature=shared

https://redd.it/1ekbbqg
@r_devops

Backend dev here. How many Dockerfiles and docker-compose files should a microservice have?

I am learning about CI/CD and I would like to know if the same Dockerfile and docker-compose file should be shared across testing, staging, production, or if you need multiple files for these?

https://redd.it/1ekfhlh
@r_devops

Is there any open source replacement for the full MuleSoft Studio web-based platform which is free to use?

Hello,

I need to test complex API endpoints and would like to find a way to quickly build such tests. It can be pure HTTP, REST, WebSockets, web services, or plugin-based and extendable. I am aware of some minimalistic and crippled versions of Mule, but I need one with a web-based GUI.

Thanks.

https://redd.it/1ekh534
@r_devops

How do you manage a "large" amount of docker environments and containers?

I did not want this.

We're producing just the software for our customers and deploy it manually or per the tooling of the customers choosing - like their Jenkins - on their servers that they control. That's because access is secured per VPN (and/or the server being 'managed' by another provider), so our Jenkins instance won't have access to the customer's systems for deployment.

Yes, we're using Jenkins. Yes, our customers don't care if their services aren't available for 2 days.

The bar is so brutally low, you won't believe it. Monitoring for PROD? Nonono, only if the customer wants it and pays for it (which, I mean, makes sense).

Now we have over two dozen servers to manage (seven of them are our customer's) and I don't even know how many containers, running on Docker.

Every container gets its own folder for its volumes, the .env file and the docker compose file.

One service per file. On every server.

If we want to deploy a new version (automatically), we use Jenkins to run a script or to directly replace the VERSION variable and then run the compose.

* GitOps? Nah, what if someone changes the config on the server? (wtf) I have to save/backup the configs MANUALLY (really funny if i have to edit 20 f\*\*\*\*\* compose files).
* Secrets? PLAINTEXT.
* Docker Swarm (for the secrets)? Isn't compatible with Spring - Tomcat hates the swarm host naming convention.
* When we decide that we have to do xyz another way I have to connect to every goddamn system that exists and DO THE CHANGES MANUALLY.

Whyyyyyyy.

So, now, let's ̵t̸r̷y̴ ̶t̸o̶ ̶s̵m̵i̸l̴e̷ again.

Ok. How do you guys manage - let's say - between 50 and 100 containers (just the beginning) that don't have to scale and are hosted on many different systems?



https://redd.it/1eki570
@r_devops