How much should a Platform/Cloud/"DevOps" Team be responsible for?

In my organization, it sometimes feels like my team is responsible for too many things for us to be able to do a quality job on anything. In my experience of interviewing around for different positions, it has felt this was a shared problem in most companies, whether the employees working there share my perspective on this or not.

Namely, I'm talking about the Platform, or Cloud, or "DevOps" team, who is responsible for Terraform + Kubernetes stuff. Sometimes this includes CI/CD for the services, sometimes that's a separate team. Often, there is no separate team for DBA's and all DB technologies are thrown into this team. Monitoring/Logging is sometimes partially shared with a small SRE team, sometimes it is thrown into this team.

Basically, wanting to work on the layer below the product services often has put me face to face with the risk of becoming the toss-your-problems-in-the-bucket team. I find managing container orchestration, logging and monitoring, tracing, cloud infra management, databases, message queues, endpoint security (ddos prevention etc), and even more things to be too many to manage in a team of 4-8 people no matter the size of the project unless if you are in the first year of a startup, and unless if the other teams are heavily sharing responsibility rather than 90/10 split at best, if they don't outright ignore it unless you reach out and say their service is nuking the database from orbit.

How do you handle this in your team/project? How have you found ways to work better (or worse) with this? How do you share the responsibilities with the various domain (or otherwise) product teams, specifically when it comes to getting management buy in?

https://redd.it/11pev59
@r_devops

Price / performance sweet spot for Mac Mini iOS CI/CD

Hello.

Should be a quick one. Looking for recommendations on which Mac Mini hits the right sweet spot between price and performance for iOS/Android CI/CD.

I should note that I'm not looking to spend a huge amount as although I'm a software engineer in my day job, this is just for my personal projects so I don't need the fastest build times. Main priorities will be running lint, unit tests and deploying to firebase app distribution so nothing too complex (although I'll likely be using react-native so building iOS & Android in parallel would be nice).

Couple of options around the same price:

1. Second hand M1 with 16GB RAM & 512GB storage.
2. New M2 with 8GB RAM and 256GB storage.

Upgrading the M2 mini's to 16GB RAM or 512GB storage feels like awful value for money, so in reality it becomes: is the M2 with 8GB RAM adequate or better to go back a generation and get more RAM?

M1 feels like the better option out of the two but then again it's possible the M2's might have a couple more years support & OS updates in them.

https://redd.it/11pi1fv
@r_devops

How to run a web application hosted in ec2 server on demand? I want to minimise the application cost. Example turning off automatically the application (dns) when the user is away from his computer and only run when is working. I’m not concerned if ec2 runs 24/7h.

How to run a web application hosted in ec2 server on demand? I want to minimise the
application cost.
Example turning off automatically the application (dns) when the user is away from his computer and only run when is working. I’m not concerned if ec2 runs 24/7h.

https://redd.it/11pb7aj
@r_devops

How to run a website hosted in ec2 server on demand? I want to minimise application cost. I’m not concerned if ec2 runs 24/7h.

How to run a web application hosted in ec2 server on demand? I want to minimise the
application cost.
Example turn off automatically the application (dns) when the user is away from his computer and only run when is working.
I’m not concerned if ec2 runs 24/7h.

https://redd.it/11ovcau
@r_devops

creating a namespace using ansible module

Hi guys,
i'm trying to configure namespcae for nginx from ansible like this:

\- name: create ingress-nginx namespace
kubernetes.core.k8s_info:
api_version: v1
kind: Namespace
name: ingress-nginx
state: present


but receiving the following error:
FAILED! => {"msg": "Could not find imported module support code for ansiblemodule. Looked for either AnsibleTurboModule.py or module.py"}


any idea how to overcome this?

https://redd.it/11pnaad
@r_devops

Deploying app with Podman

Can you guys suggest a good article about deploying react app with Buildah, Podman, and Skopeo

https://redd.it/11pop7e
@r_devops

How to deal with devs pushing bad code

It’s became apparent to me, that code is being pushed straight into production without being load tested.

Take for example this week.

Devs pushed code into their repo. This gets built into an image and deployed, fine. But it gets deployed to production, with no change gate and no testing.

As soon as the client starts using this new feature, all hell breaks loose. The autoscaler was set up to track CPU, this new feature uses a damn tonne of memory, which was not relayed to anyone in ops. The pods get overwhelmed, and barely evade the OOM killer, but it means the site is sluggish and barely chugs along.

Someone, please enlighten me, my management don’t seem to understand that dev and ops need to be working together.

https://redd.it/11ppp66
@r_devops

IaC Best Practices with Bicep/Github Actions?

I'm currently leading a team in a small org who has chosen Bicep as their IaC language and Github Actions for pipelines.


While I'm not a technical resource on the team, I do spend a little effort researching and understanding so I can help guide them to technically sound solutions.


I read up on Bicep best practices, and set up some repos that use a main file to call modules for deploying resources, and then I set up a pipeline to push these files into Azure. All fairly standard stuff, but what was not clear to me is how teams tend to execute their bicep against Azure? The following ideas stuck out to me:

* Executing it directly from Visual Studio, add params in dynamically at runtime via VS UI
* Maintain a master branch with a parameters.json file, changes to the bicep code itself would be merged to master, and if a deployment is needed, a feature branch is created, the parameters file is updated and pushed, then the workflow is triggered against the feature branch. Feature branch is disposed upon completion.
* Set up the workflow to trigger when a change is merged to master (likely the param file).

Before you say "use terraform" - we are not now and will not ever be a multi-cloud shop. It will be Azure only until the end of time, a decision that was not in our hands. Because of this we chose Bicep over Terraform as we felt the barrier to entry was lower, and not having to maintain a state file help keep the process simple.

**For folks using Bicep/Github - how do you deploy your templates?**

https://redd.it/11pqwql
@r_devops

Pulled into a meeting… with the customer

Just want to ask you all how common it is, and how you’d react to being pulled into a meeting, to then have a client grill you about the performance of the service.

This happened a week ago. I was asked by delivery to join a meeting discussing why a customers SaaS app was so slow.

I joined the meeting, and almost straight away I was being asked about issues. OK fair enough, we have problems, let’s fix them. As the meeting went on, I started to question who this person was. I naively thought delivery had put me in front of a colleague that I hadn’t yet met. It was not a colleague.

20 minutes in, after repeatedly stating “we need more info to take a look at this , please send us this data”, I felt myself getting more annoyed. My colleague who was also on the call, didn’t say a word.

I left the call pretty unimpressed and quite frankly pissed off.

Who else has had this happen to them?

https://redd.it/11pph5p
@r_devops

Why is NewRelic so bad?

Really, I hate it. We had datadog before, but we were enforced to use NewRelic. We are currently migrating and I have to say that I really hate it. The docs have so many dead links, it’s a nightmare. Then the NewRelic explorer crashing. A SaaS app crashing in the browser was something I have never seen before. Then this whole pixie thing. I was excited at first, but none of the queries work. Seriously. I hate NewRelic.

https://redd.it/11prydv
@r_devops

Made a CLI tool for Markdown URL format check in Github repos

Hi.

Made a tool (called GMUV) which checks brokenURL's in Markdown format for public Github repos. Output can be stored to file or as a console output. Marketplace action is available. To those who wants to see execution example - click here.

https://redd.it/11ps81u
@r_devops

Git repository tags as a source in terraform module

As a source in terraform module we're using our own github repository with specific tag. That looks something like that:

provider "aws" {
profile = "default"
region = "us-east-1"
}

module "iam" {
source = "[email protected]:path/to-repo.git?ref=0.0.8"
name = "foo"
naming = "bar"
}

I want to make changes in terraform code that is tagged as 0.0.8 so can I push it as 0.0.8 tag or is it not possible and I should bump the tag to 0.0.9 for example?
If the answer is yes (make changes, update the tag to 0.0.9 and use that as a source) what to do with other tags that we're not using yet but will be using them in the near future - update those and bump the tags?
Another question - what is the purpose of master/main branch then?

https://redd.it/11pxixu
@r_devops

Are your product teams able to operate their services effectively without your assistance?

Hello all! 🙂 I'm building a new service for DevOps, SREs, and product teams. The product is designed to help product/dev teams run their services and handle incidents with less SRE/DevOps support.

In my case, it has been difficult to guide teams on how to operate their services due to various reasons, such as a lack of cloud knowledge and permission issues.

I wonder if there are others who also struggle to enable their teams to operate their services with confidence. If you face any problems related to this, would you please share your case?

https://redd.it/11pxxxl
@r_devops

Where are you all?

There is a piece of advice I have seen work over and over again and I am not seeing it repeated here. If you want to do something find a way to be around people already doing that thing.

People talk about tutorials a job advice which is great and we should all obviously be learning from those sources. However, I would love to know where to meet people in this career field to arrange to have quick conversations/coffee chats with to just get the opportunity to talk about the details of their careers and lives. Are there any forums or groups where people like me could find other people locally in the field?

https://redd.it/11pz3br
@r_devops

Tools to help developers do their own monitoring/maintenance?

There are two issues my department’s facing: 1) when something goes down in production, we get a phone call from a client and have to scramble to figure it out. Usually it gets picked up quickly but sometimes it takes hours for the team to dig through stack traces. Are there solutions for this, esp at the network level? 2) We’re encouraging dev teams to be even more responsible for their own code in production, to the point where they should do their own monitoring. Are there services you’ve used that lower the barriers for least common denominator developers (esp front end) to do some of their own ops?

https://redd.it/11pxet4
@r_devops

Working with microservices

My current company does not use microservices (because there’s genuinely no need) but i always been fascinated in working with it. I feel majority of other companies whenever i look at their requirements always have “experience with microservices “ and i feel it may put me at a disadvantage.Is there a certain side project or something that can help me understand it and help me whenever i do want to apply to these companies. I know i can ask Google but i l like talking to fellow devops redditors (feels more of a human interaction lool)

https://redd.it/11ppf3r
@r_devops

Help guys...........

I am new to DevOps. Currently I am working with Azure DevOps. How can I learn CI/CD YAML Scripting for Azure DevOps? I am not getting any material other than the official documentation from Microsoft.
Is there any way to learn it efficiently? Cause I have been reading the documentation for the past 1 week🥴. Got to know many things but I don't how I am going to put that in practical.

https://redd.it/11pxg0q
@r_devops

Best architecture to deploy odoo

i need to deploy instances of Odoo for different clients, what is the best Architect using docker kuberenets on eks using eksctl. what is in my mind is like a main odoo instance that i can duplicate for clients or push the updates to all the clients and a staging odoo that i can push to the main after the update

https://redd.it/11orjov
@r_devops

Monitoring System to Company

Which one is better for monitoring IT system like Servers,application,Network, SAN, NAS ? I would like know community opinion :) Best regards

View Poll

https://redd.it/11ojjyz
@r_devops

IAC templates and artifact repositories

Should your IAC templates (cloudformation/terraform) be pushed to your artifact repository alongside your application code in your CI/CD pipeline?

Or is this not necessary, where only app deployments should be pushed as a deployable package to your artifact repo?

https://redd.it/11nr7yj
@r_devops

Worst devops/dataops platform?

I am looking for the absolute worst platform in terms of integration with tools like terraform or kubernetes, or doing ci/cd, the problem is you only ever find praise when googling. So I would like to ask you guys, what do you think are the absolute worst devops platforms, and why? Feel free to share stories if you like, I would be very interested.

https://redd.it/11q6c95
@r_devops