Folder structure, Deployment best practices with bicep across multiple subscriptions with azure devops. Help

Hi Guys,

I'm kinda new with Bicep and I'm working on the folder structure, for Infra deployment with Azure DevOPS. This is my current folder structure. It seems I'm missing something that I cant point out . I tried googling it but I need some feedback. Anyone with experience? Thank you.

-infra
--->project1
-------main.bicep
-------params.json
--->project2
-------main.bicep
-------params.json
--->project3 (same subscription with project2)
-------main.bicep
-------params.json

-modules
-pipelines
-scripts

https://redd.it/1g5r4a9
@r_devops

Fraud detection data pipeline (ETL) on GCP

# How to create a scalable fraud detection steaming data pipeline

[https://medium.com/@rasvihostings/fraud-detection-data-pipeline-etl-on-gcp-2b15b8f3d65b](https://medium.com/@rasvihostings/fraud-detection-data-pipeline-etl-on-gcp-2b15b8f3d65b)

When we think about large volume streaming data pipeline three things come to our mind

* **Scalability and resilience**
* **Cost**
* **Infrastructure maintenance**

I designed a solution which can scale easily, use much as possible GCP managed services and finally reducing the cloud cost 😉

[read my blog post here](https://medium.com/@rasvihostings/fraud-detection-data-pipeline-etl-on-gcp-2b15b8f3d65b)

let me know what type of articles you guys are more interested on

https://redd.it/1g5t04r
@r_devops

Should I get into devops?

I am a frontend software engineer and I hate my life, I am sick and tired of putting buttons, input boxes, etc. on a web app... And I feel like frontend web development is a dead end job, or at-least it is not suitable for me, because all the frameworks do the same thing... I hate most of my coworkers (including developers, design, product managers, etc.) because they fight over trivial issues like, why use a forEach instead of reduce? Why is this button small, etc.

Should I switch to devops? I feel like there is much more variety of tasks available in this space, but I am afraid of AI, platforms like vercel, that abstracts all the complexity of managing infra, etc. Also currently my salary is quite high, so changing role will be extremely difficult...

Is Devops worth it? Or are all the IT jobs the same?

https://redd.it/1g5uf5x
@r_devops

Manage spot nodepool in GKE

Hi Everyone,

We run about 60-65% of our workloads on spot VMs, but during peak hours we usually hit stock out and new pods are usually in pending state for long hours waiting for a spot VM. So we have implemented 2 ways to improve this state.

1. deploy the same deployment on payg nodepool with a higher hpa threshold, so it scales when spot pods doesnt scale.

2. create a nodepool with different series of machines with same configurations, taints and labels, but at times one nodepool doesnt scale even if it isnt hitting stock out, whereas the other nodepool would have stocked out.

Are there any better ways you guys tackle the stock out situation ? Kindly advice.

Thanks !

https://redd.it/1g5wxze
@r_devops

How do you handle urgent communication when a team leader from another team doesn't respond?

Hey all,

I’m curious about how you handle situations where you need an urgent response from a team leader in another department or team, but they take hours to reply. For example, if you're waiting for critical input and it's been over two hours with no answer—how do you proceed?

Do you escalate the issue, follow up with other team members, or try to find alternative solutions? How do you balance urgency without coming across as too pushy?

https://redd.it/1g5z5y6
@r_devops

Move from Sprints to KanBan?

For a long time I have hated sprints and pretend agile for DevOps in our organisation. We plan, refine, retro, demo - blah blah blah - the whole thing in our org is pointless because we plan/refine but when the sprint starts we abandon the sprint for another team’s work because they didn’t think to involve us early enough or they say no when we asked if they had any requirements, and now we’ve blockers.

Instead, we’re going to use KanBan. Anything in to-do can be picked up by anyone at anytime. Items only move from backlog to to-do after it has been refined, backlog items don’t show on the board, and board is DevOps only work. We no longer include other teams work on our KanBan board.

I’m not saying we won’t do work for other team. Instead we rotate through other teams. Embedding in their process and our tasks in their epics belong on their board because they belong to that team.

My fear is - how will be prioritise our own board if we’re embedding in other teams. What happens to unfinished work after a rotation.

What are peoples thoughts? Is there a better approach?

https://redd.it/1g61gzq
@r_devops

What happens if I have multiple IP addresses in a single weighted routing record in route 53?

Basically the title.

I am in the process of migrating from simple routing to weighted routing and wanted to test using a few servers.

Currently, we have a single A record which is simple routing, it consists of all the server IPs.

I am trying to take out some servers and add some weighted routing entries for the same.

If I have 3 records,
Record A - weighted, 2 IPs, weight 50
Record B - weighted, 1 IP, weight 50

Will each of the IPs in record A get equal traffic, I.e 25%?

I was not able to replicate the above.

Please help.

Thanks in advance.

https://redd.it/1g67daj
@r_devops

First time creating a simple CI/CD pipeline help

Hey guys,

I'm a self learning developer trying to broaden my skills and learn more about cloud services and CI/CD pipelines and all that. To start simple I want to host my personal website on an EC2 instance and set up github actions to containerize and deploy to my instance such that when I push changes to my repository, that all happens automatically. My first basic question is how do you handle private environment variables? Do you use AWS parameter store? I'm using Go and the 'godotenv' package, but if I change my code around to use the AWS Parameter store, how would I test it locally? Since I wouldn't have access to the parameter store, right?

Thanks!

(you'll probably see me a lot around here in the near future 🙌)

https://redd.it/1g6auuk
@r_devops

Interview experience

I recently interviewed in mid level company where they rejected me in second round stating that I do have experience in CI/CD tool like Jenkins , harness but i don't have experience in azure cicd. Why can't they understand it's just a tool, logic and workflow is same everywhere.

https://redd.it/1g6b7yf
@r_devops

How Well Does "all in the same repo" CDK approach Scale?

I asked this in the r/aws (here) but will also post on here.


I am in the process of adopting and learning about CDK for our large-scale microservices architecture. What I want to know is how well does it scale when used in an environment with 100s+ of microservices and pipelines.

Has anyone got any recommendations on best practices in terms of structuring and managing CDK for scale? Does anyone have experience using CDK in environments with 100+ microservices?

I can see that the biggest shift with CDK is essentially coupling the CI/CD config, infra config and application code all in the same repo. How does this approach/recommendation scale?

Let's say I have 100s of microservices and I need to update CI/CD or some infra config across all. Every time you make a change to the pipeline config in the repo, you are potentially "touching" the app and making a release. I can accept the changes to the infra "close" to the app like Lambda config, SQS etc., but I'm not sure CI/CD config is the same.

How do others manage updates to shared infrastructure or CI/CD configurations across multiple services?

Also, regarding self-mutating pipelines: it's something I tried 5 years ago with raw CloudFormation but found that if there was a change to the CodePipeline executing the change to itself, the execution would instantly fail and I would need to rerun it. Has this been fixed?

Lastly, why would a developer want to see the "pipeline update" step execute and do nothing 99% of the time, just wasting time and slowing down the CI/CD cycle?

I'd love to hear about your experiences and best practices for using CDK at scale. Any insights would be greatly appreciated!



https://redd.it/1g6dfye
@r_devops

DevOps/Cloud Engineers: How Do You Manage and Visualize Your Infrastructure?

Hey everyone! 👋

I’m working on a new tool concept to help DevOps teams and cloud engineers better manage their infrastructure. Before diving deeper into development, I’d love to get some feedback from the experts—that’s you! 😊

If you have a few minutes, could you help me by answering some quick questions?

# 1. Visualizing Cloud Infrastructure

How do you currently visualize your infrastructure (e.g., cloud resources, clusters, VMs)?
Do you rely on built-in cloud tools, open-source solutions, or paid platforms?

# 2. Managing Costs and Outdated Components

How do you track cloud costs across different resources?
Do you encounter issues with outdated Terraform modules, providers, or infrastructure drift? If so, how do you handle them?

# 3. What’s Your Biggest Pain Point?

What’s your biggest challenge in managing infrastructure or optimizing cloud costs?
If a tool existed that could simplify visualization and provide actionable cost insights, would that be valuable to you?

Thanks a ton in advance! Your insights will help us make something that really solves problems for the community. 🙌

https://redd.it/1g6dazv
@r_devops

How can a devops engineer develop in machine learning?

Hi everyone, I'm a Devops engineer and I really love my job. I love doing linux, writing scripts, configuring networks and so on. But I also love math and algorithms and data structures. I want to participate in the development of artificial intelligence, apply my math knowledge, but still keep doing devops. Any advice?

https://redd.it/1g6gacx
@r_devops

My employer is offering me a 65% raise and a bonus in the next pay cycle if I rescind my 2 weeks notice.


In the past year working in a start up, I had made a transition working as a senior cloud infrastructure engineer to a junior and now mid level full stack engineer. 2 senior cloud guys and 1 senior full stack engineer decided to leave our company to take roles in FAANGs (who also happen to be our customers for our product) these last few months. Although we re’orgd and some duties got divvied out amongst us. I got bombarded doing my job and taking on cloud duties again. My mental health has been killing me with deadlines, and management asking us to push new releases on a Friday, which takes up some of my weekend. I’m just so done. I been offered employment elsewhere and put my notice in so I can take a month off for vacation and reset. Well I got a call almost instantly from the CTO, Product, and CEO about anything they can do to keep me including offering me a promotion to senior, a huge raise, focus on backend development only, and a $25k retention bonus on the next pay cycle. The raise is about 10% more than the new employee is offering.


They want to give me the weekend to think over it. I’m contemplating on whether I should take the offer or not.

https://redd.it/1g6he1w
@r_devops

Promoted to Manager

I've been in the sysadmin space for 7ish years, plus another 3 as DevOps engineer. The change into DevOps at first was a bit rocky and I think I suffered from imposter syndrome. I was the weakest one on the team (in terms of hard technical background and devops "years"). In recent months I realize I actually did beat out my coworkers at other skills: communication, organization/planning, enabling/empowering other engineers, not letting "great be the enemy of good", making actual progress. Recently we had a bit of a reorg and now I am managing the DevOps team. In a way it makes sense to me, I know the principles and our goals and the bite-sized chunks it takes to get there. I'll never be able to write slick bash one-liners on the fly with 5 ppl in a zoom meeting watching. Sorry for the rant and tamble. Long story short: any tips or suggestions for this transition from engineer to managers? Do you think I have enough background to succeed? Any suggested material or reading? (rn I've been reading Radical Candor)

https://redd.it/1g6hdmb
@r_devops

Books for experienced DevOps?

Hey
I would like to hear recommendations about books that go beyond explaining what is DevOps from zero

I'm thinking about The phoenix project as I heard its great for both beginner and experienced engineers in IT, and also heard about SRE Orielly book
What you guys think?




https://redd.it/1g6kon6
@r_devops

The new release of Dockerfile.app has launched.

Visit https://www.dockerfile.app

Features:
→ Save dockerfiles
→ Browse them
→ Upvote them
→ Search for dockerfiles
→ Create an account

All to create a community-driven location to get top-notch dockerfiles for all languages and frameworks.

Bugs? Let me know.

Feedback is welcome.

https://redd.it/1g6m2ce
@r_devops

Are there any DevOps or Infrastructure-type jobs where you work on boats?

Weird Q. But any need for these roles on oil rigs or cruise ships or anything similar? I had a random thought that it might be a fun alternative / break out of normal society for a while. But no idea if there is any demand for these kind of jobs in those environments. Presumably basic IT support, but not sure what else.

https://redd.it/1g6mlxi
@r_devops

How Meta Uses LLMs to Improve Incident Response

I came across this interesting blog post on HN outlining how Meta uses LLMs to get 42% accuracy in root causing incidents: https://www.tryparity.com/blog/how-meta-uses-llms-to-improve-incident-response

https://redd.it/1g6q0fi
@r_devops

Looking for devops role ; need suggestion

Hi there everyone.

I’m a student currently doing a bachelor in here USA and looking for devops role.

Primarily I did devops for 1 year maintaining full stack apps, automated build using GitHub ci cd and reproducible servers with nixos. I’m proficient with rust, python and basic web development but I’m sure I can learn anything quick if there’s any new tech part to learn.

My current financial situation is going really tough right now. if there’s anyone who’s got job or have any opportunities. Please help me out.


Appreciate it 🙏

https://redd.it/1g6seyv
@r_devops

Management platform

Does any integrated open source software exist to manage Servers, K8s clusters, credentials etc. dor technical devops guys? I can’t find something besides portainer, lens and the usual suspects.

https://redd.it/1g6lh4e
@r_devops

Interview Mess - DevOps

I'm giving interviews these days and I've good understanding of DevOps tools and technologies. But whenever I go into the interviews, interviewers start asking troubleshooting questions and other issue and I've not faced this issue so I'm not sure how to answer these questions. And gets rejected. I know I'm not expert in all these but none of them consider basic understanding of the same.. getting rejected day by day... 😕
What should I do..? There is so much to learn.

https://redd.it/1g6ya2c
@r_devops