Is it possible to deploy Argo workflows a job scheduler to AWS and have the control plane in GCP?

Been reading around this all day and I have been getting opinions that it’s not possible and yet it is. I am not sure currently what to believe. I was wondering if anyone has done this before because there doesn’t seem to be much out there around this topic.


https://redd.it/1cn9oxx
@r_devops

PC setup questions

Hi All!

Some background:
I'm a newbie with respect to devops but am a .net software engineer with lots of coding experience - so apologies if I get terms wrong or am even asking in the wrong place!

I've recently been given a task to create a "setup" for an entire new PC that controls a scientific instrument. The Windows 11 PC setup will require installing everything from several 3rd Party MSIs, Nvidia CUDA, numerous pre-compiled python applications, dozens of configuration files, creating a user, and setting permissions and firewall rules. The current culture at the company I was asked to help does everything command line and would like to use mostly scripting and a package manager. They have a JFrog artifactory that they will (probably - not sure yet)post their production packages to in some form. I'm a tiny bit familiar with winget, but it looks like JFrog doesn't support winget. So I'm kind of at a loss on how to proceed with JFrog and a package manager. The final solution will need to support new installs and upgrades of any given item in the artifactory.

So here are my (naive) questions to try to get me pointed in the right direction:

1) First, is this the right place to ask these questions?

2) Would using a generic JFrog repository for everything be best?

3) Are artifact versions controlled by how you define the folder/directory structure and their names?

4) Since JFrog doesn't support winget, what package manager can do the downloads and installs (on Windows)?

I've watched a bunch of JFrog videos, but they don't seem to have any content for this scenario (that I could find). Am I even in the ballpark?

Thanks for any advice! 🙂

https://redd.it/1cnajyr
@r_devops

How to now get swallowed up by the job

Anyone have any recommendations on how to make DevOps/development work less consuming? I currently work as a staff level engineer and do some dev work for some interesting OSS projects on my spare time.

Recently after taking a 2w vacation without much work going on (bless Asia’s cable cuts for creating a 300ms+ latency back to Europe) I realize how much of my life has really been consumed by just dev work instead of doing other things that I want like hiking etc. Sometimes it’s just a bug or something I’m too focused on and end up spending a lot of time on, or some new feature that I’m building…

Anyone else have this issue or have any advice about it?


https://redd.it/1cncx7v
@r_devops

Junior SRE/Devops losing his mind over database replication.

Hello, I'm a junior devops from Argentina. I've been working as a SRE/Devops for like a year as my first IT job, which has been a challenge. I work for a state company, so it's a shitshow as you can imagine. I have to create a database replication using docker and MySQL. The idea is having two DB, each running in differents servers, for load balancing a wordpress page. The master as a write/read and the slave as a read only. But for the love of god, I can't do it. The containers dont communicate with each other, the master works fine, but the slave is useless. Any ideas of what can I do? Thanks in advance and sorry for bad english, is not my first language.

https://redd.it/1cne215
@r_devops

2 projects/products from a single source repo, thoughts!

So I am running into a particular scenario where I need some suggestions on how to put a strategy on managing multiple repositories. Imagine a SaaS product (Alpha) developed and managed by a vendor X, which is purchased by a client Y. Y now needs to have a different version of product Alpha with their customizations and product roadmap, naming SaaS as Gamma. Y has received the source code from X and hosted it in their git infra, managed by Y's internal IT.

Now comes the tricky part, at least for me.

Y is in a contract with X for getting support on bug fixes on existing features and also inheriting the product features which X develops for 6 months. Meanwhile, Y has their developers onboarded and is developing other features on Y's roadmap, which are quite different from X's, but Y needs to receive the periodic updates from X as well. The developers working on this project are different

The question is, what is the recommended way to structure these repositories to avoid conflicts (I agree there would be code overlap which needs to be considered case-by-case) and have it in a neat way?



https://redd.it/1cnd80f
@r_devops

Resources other than Oracle for hosting self hosted runners

What resources would you suggest for hosting self hosted runners for CI/CD workflows other than the free instance of Oracle.

I do want to use an instance but I don't have the budget to spring up lots of VMs (I definitely will in the future as I scale up). So in the meantime, if you have any resources in mind, do let me know!

The best I found was from Hetzner.

https://redd.it/1cnh0rz
@r_devops

How to improve communication skills?

Quick background: I have 15+ of experience but most of those years have been working on my own. I didn't have to answer to anyone, I just did things I felt make sense. So even if I have a lot experience technically I don't have much experience with dealing with people.


For the last 3 years I have been in a huge company as a consultant and have been fairly successful in a platform team. Recently I got asked a question from a dev-team if we had the capability to do "x" in our solution. And I answered no we don't, and that would be too much work to do right now. I honestly expected a follow up question from them but they just answered "Ok, I can understand that". A few weeks later it has come back to our team via back channels (various managers) and they have taken that reply out of context to make me look really bad. What I really meant is that we can't prioritize that since you can do this with y and z instead and having to implement that is not something we want to prioritize since their pattern is questionable at best.


Is anyone of you aware of any literature on how to deal with this in a proactive manner? I'm learning as I go to be more political but I wish upskill without having to learn the hard way. I'm getting really stressed out by this political stuff. I've noticed other respected people in the organization are experts at putting up smoke-screens without ever having to actually deliver much. It's so far away from what I'm used to.

https://redd.it/1cnggy8
@r_devops

How should I split infrastructure in state files?

We are planning an automation to deploy and configure a product through IaC and some scripting. I'm thinking how to break up Infra into different state files, simplest option being Infrastructure and Kubernetes cluster level declaration, and more complex being Base Infra, Networking, App Infra, Database Infra and app configuration. I don't want to overengineer this thing, but I also want to decouple it's components to avoid a gigantic state file working as a bottleneck and a single point of failure.

Are there any patterns for this? How do I know when enough separation of concerns is enough? If I break it up, should I connect everything in a single pipeline with different stages? Or different pipelines triggered independently?

https://redd.it/1cniy84
@r_devops

CI/CD Pipelines vs Deplyoment Pipelines

I watched a couple of introductory videos to CI/CD pipeline design from "Devops Journey". He described the process of creating a CI/CD pipeline first, and then moved on to a "Deployment Pipeline (+GitOps)". The CI/CD pipeline appeared to produce some artifact; so is this not just a CI pipeline and the Deployment Pipeline is the CD, which is triggered on a successful artifact. video: https://www.youtube.com/watch?v=pJ9f7w4AxtU

https://redd.it/1cnl2m3
@r_devops

Why do companies select Azure as their cloud service rather than aws?

As title say, anyone has clues?

https://redd.it/1cnkk96
@r_devops

Understand AWS Public IPv4 Address Usage and Pricing

AWS charges for all public IPv4 addresses, including those associated with running instances and elastic IP addresses. learn more about AWS public IPV4 Address usage and its pricing.

https://vishalvyas.com/understand-aws-public-ipv4-address-usage-and-pricing

https://redd.it/1cnpguc
@r_devops

Ahrefs claims saving 1 billion by not going Cloud

Link: https://tech.ahrefs.com/how-ahrefs-gets-a-billion-dollar-worth-infrastructure-with-a-90-discount-5edd473b2399

HN discussion: https://news.ycombinator.com/item?id=40304549

Excerpt:

> Choosing colocation for the infrastructure was the right decision for Ahrefs. Reflecting on the past six years, the data centers with our own servers and network cost us $122 million, a figure that would have ballooned to an astronomical $1.1 billion if we had opted for the cheapest AWS cloud offering. Our powerful on-prem servers enable our developers to focus on enhancing our products for clients, rather than getting caught in a never-ending cycle of trying to optimize cloud costs with little hope of success.

https://redd.it/1cnqi0x
@r_devops

Beat Practice to in setting up Monitoring ECS tasks

Hello, Junior DevOps here. I would just like to ask what are the best possible solutions for monitoring ECS tasks where it would notify us if the containers reached maximum CPU or memory usage and if there are any failed tasks. I would want it to be cost efficient.

https://redd.it/1cns5z8
@r_devops

Salaries for Southern Europe

Fellow DevOps engineers, what are the (or your) salaries for Junior (if there can be a junior Devops engineer), Mid & Senior.
I know that there are a lot of factors to consider, such as skills, YOE, benefits , working situation (remote, office, hybrid), so let's say average.
Curious since I feel my company offers less the average market, although they claim it's normal for the Country.

https://redd.it/1cntjs1
@r_devops

Automatic Tagging in a multi-repo setup

So in this scenario , you have 2 repos A and B that have a pipeline to create an image that is uploaded and named after the specific commit on its respective main branch.We have a repo C that uses docker compose for example and utilises a repository to access the images of each repo A and B. So my question is what is the best method for updating C so it's aware of any new images from A and B and even better if it's aware of each environment that the image was designated to such as prod / dev.

https://redd.it/1cnuqdp
@r_devops

Best books/resources for understanding true scale and events

Hello I am a mid level engineer with a few years in the DevOps realm. I find myself still struggling to understand the consideration and challenges that goes into designing absolutely massive distributed system.

Any books/resources? In particular, I am interested in learning more about FinTech payment processing systems, and they deal with trillions of transactions per month.

My current understanding of scalability is simply observing and monitoring the CPU and other metrics of a server, make use of autoscaling groups and built-in features of EC2 etc...

I have an AWS solutions architect cert, and I still feel woefully unprepared to even begin to understand the problem at hand. I have been in some medium-sized companies so I have no experience with anything that comes close to the scale that I'm trying to describe here.

Questions:

What are the limitations and constraints that engineers need to know when designing solutions when the scale and events are so big?

Any books that can answer this question more comprehensively?

I know this is a broad question so ANY TIPS would be massively appreciated :)

Thank you all!

PS: Both kubernetes and AWS-native design patterns would be welcome!

https://redd.it/1cnvmu4
@r_devops

CI/CD Pipeline for Node.js Backend on Autoscaling Group

I'm currently in the process of deploying a Node.js backend application and creating a Jenkins CI/CD pipeline to streamline the process. With my existing pipeline, I can deploy to a single instance by SSHing into it via its IP.

Now, my goal is to extend this deployment to the autoscaling group. Since I can't SSH to every single instance that will be created by the autoscaling group when requests increase, how do I achieve this?

Sorry if this is a very basic/dumb question.

PS: If this isn't doable by Jenkins, please let me know if there's another way to accomplish it.

https://redd.it/1cnx5q3
@r_devops

Step Functions vs something else

I follow this guy on YT, and he showed how chosing the wrong step function type (Regular vs Express) sent his cloud way too high... I like to build using Step Functions but I'm now wondering if there's another service that does achieves similar things without the huge bill

https://redd.it/1cnxklv
@r_devops

How do you deal with the cost and maintenance of multiple different staging, dev and other environments?

I am wondering how people here manage the existence of additional environments in addition to production - staging, testing, dev environments.

In my experience these can be heavy compute cost wise and this can also add up maintenance wise. I've in the past merged a few of these environments to reduce maintenance cost.

How do you work around this?

https://redd.it/1co0hti
@r_devops

Need help for first technical test

So I am new in DevOps and this is gonna be my first job, I have technical test in few days and it is written that it will also have some ,,practical tasks with automation checks'' so probably they mean writing automation code, where can I practice it? probably it will be bash maybe python but I don't understand should I just go through some leetcode/codewars tasks or through some special tasks about automation? and if yes then where can I find them?

https://redd.it/1co1hz8
@r_devops

I want to get into DevOps after college. Is my experience enough to progress to a junior sysadmin or cloud engineering role that I can use as a pivot?

https://imgur.com/a/1B1AvLd

Graduating this upcoming Saturday and about to start applying. I've been doing helpdesk/some sysadmin tasks for my college IT department for about two years and will be finishing my last week today. I have an internship as a cloud engineer that gave me some extra hands-on experience as well as personal AWS lab projects involving popular devop tools (Kubernetes, jenkins, docker, etc). Got my SA-A and Network+ last year. Went through the DevOps roadmap and am familiar with majority of the concepts. Will all this be enough to at least be considered for the roles i'm going for? I know that Devops right out of school isn't likely, so I want to try pivoting in a few years.



https://redd.it/1co329x
@r_devops