Work split on modules vertical or horizontal

I worked in cooperations where every developer had one module for himself to work on and if they weren't experts in their layer (e. g. databases) they would ask for others so while every developer was responsible for only his module each module was rather the result of teamwork than of just one individuals' work.

I am now working in a cooperation where a whole team of 5 developers works on one module. The database expert does one layer, the frontend expert does one layer, the deployment expert does one layer, etc. If the deployment expert does not have time you will have to wait a few sprints and if the deployment expert does not like a certain module, he might use his authority as an expert to delay things since other employees are not allowed to do his work since there might be errors. Management is scared of angry customers so they do everything to avoid that.

I know that nothing is black or white but which of the following ways of developing software in a company with SAFe and agile is better if one could even say that. My point of view may be highly subjective that is why I want to know the perspective of someone else.

https://redd.it/10ifcsv
@r_devops

I'm running k8s IPv4/IPv6 dual-stack in production

Curious about people's experience with this

View Poll

https://redd.it/10ih1cg
@r_devops

Anyone use AWS Managed Prometheus Grafana? Confused because it looks like a non-managed prometheus solution?

I started looking at Amazon's solution for prometheus to try to get away from managing my own prometheus database and stack and (like most AWS products) it seems to be designed in a roundabout and confusing way.

Looking at the architecture graph here, it looks like you still need to run your own stack and prometheus database, and you forward your data to a write endpoint. I was expecting to not do that and to simple be able to direct their managed prometheus instance to scrape my exporter endpoints.

Am I missing anything here? Has anyone made this work with their use case?

https://redd.it/10hvvwi
@r_devops

From your experience, what are some use cases that make you use an APM provider like Datadog, NewRelic, blah, blah? Instead of lean open-source solutions (Prometheus, Jaeger, Loki)?

Can you guys give me actual use cases instead of a feature list?

https://redd.it/10k0oc8
@r_devops

Taking over internal tools built by ppl who quit (DevOps/SRE)?

So,

Ppl here say that there is no DevOps without Dev part, so lets assume someone wrote very complex internal tooling that is extensively used by the company.

But the ppl who wrote that quit the company some time ago. The knowledge about how stuff works in details was lost.

You are left with various Java and Go services that are required by the platform to operate but you have no idea how they work.
Most stuff works based on magic injections of extensions.

Whats your methodology to reengineer all knowledge that was lost :D ?

https://redd.it/10k4tgj
@r_devops

Datadog questions

Hi guys, where do you ask your datadog related questions? i see their r/datadog channel is dead and their slack has 0 replies in most of the questions, thanks!

https://redd.it/10k1gnc
@r_devops

Help us gather feedback in the annual community driven AWS survey

This is a yearly survey to collect data on the the state of AWS and related services. The data collected will be released publicly as a resource for all. The more people who submit the more meaningful the data will be, which is why I'm posting it here.

This survey is in not supported, sponsored or endorsed by AWS. It is unofficial and is entirely community made.

After using the State of Javascript website many times, Pete wanted something similar for AWS. After asking around at re:invent 2022, he couldn't find anyone who had put something like this together, so he decided to kick it off.

You can help spread the word and fill it out here! Thanks everyone.

https://answersforaws.com

https://redd.it/10kcj20
@r_devops

Best CI Service for Use with Gitea?

Hey all, I'm looking into deploying my own development environment among other tools that I'm currently setting up on Kubernetes (K3s) and was wondering if there have been any new players in the CI space.

Last I checked, Agola and Drone are the most popular. I tried Woodpecker initially, because it's a fork of Drone and I really want something truly FOSS if possible, but immediately had issues with the log output not being shown in my pipelines. I don't know what it's caused by, but the fact that Woodpecker is still far from a v1 release means it'll have a couple of these issues that I'd like to avoid.

Agola looks great, but the docs are still quite slim and don't mention a setup that doesn't rely on volumes. I want the setup to be distributed and want to be certain I can replace SQLite and whatever else Agola uses with Postgres/S3. Does anyone have experience setting that up?

Drone ends up being the default choice. I set it up, and it immediately worked. Still need to up my Containerd fsnotify limit, but other than that it does the job and has the largest ecosystem among the Gitea-supported CI services.

Between Drone and what I currently was able to find, it seems I'll have to either wait for Gitea to complete their own implementation of CI/CD, or Agola/Woodpecker to improve their documentation so I can solve the problems I've had with them.

https://redd.it/10k61p1
@r_devops

Getting put on pip has made my fears turn into a reality and shaken my confidence.

I've been in the industry for 3.5 years. 1.5 years at my current company (consultancy) where I was about to be put onto pip for under performing. Long story short, I've handed in my resignation and got two offers elsewhere.

My issue is that I had fears of this happening when I left my first company of not knowing enough and that fear has become a reality. The two offers I have I'll be one of two engineers. Which means I relied upon and this pip has really shaken me. Which will mean I'll be heav I feel I need to be in a team of engineers so they can support me and I can learn from them.

They also pay around 10% more, which is also worrying me as higher pay means higher expectations. I couldn't even hack my current job so does asking for more money seem silly?

Can anyone provide me with some advice?

https://redd.it/10ketz7
@r_devops

APM / Observability

Can someone please point me to a reseller or Systems Integrator partner that works with different Observability vendors such as Datadog, Dynatrace, New Relic etc. We would like to speak to someone who would be able to compare the pros and cons of these different vendors, and any would have insight into customer feedback on these products. If you used a reseller/partner when choosing your Observability vendor, please share.

Thx.

https://redd.it/10k8wyc
@r_devops

Best practice for building an internal developer platform

Hi guys,

QQ: We have a green-field project where we are planning to build an internal developer platform with backstage, crossplane and argocd. We don't have much experience as everything is new and we are learning as we go. We have some internal discussions on the team, I think the platform idea changes the development processes, therefore such solution needs evangelisation within the organisation and adoption from the teams, which will require a migration to this new approach.

The other part of the team wants to fit the old processes somehow within the platform and I just don't think how it could work. There's some resistance to changes mostly from dev teams and if we tell them here's a new tool, you need to change how you work and now you own your infra might cause a backslash. Suffice to say It would be messy and very complex to fit our current way of doing things to this new concept.

How should we go about it? We are using Azure, should we even have a different subscription, with different architecture? How can we sell this to management and dev teams?

Any hints would be greatly appreciated. Thanks

https://redd.it/10k20bg
@r_devops

Is 80k/year enough for mid level engineering in major cities in Canada?

I been in DevOps for 3 years now, been in IT for 6 years, I make 120k/year remote.

2 years ago I remember finding a lot of remote jobs for 120k+

The last few months I've been getting a lot of LinkedIn recruiters messaging me offering interviews for non-remote roles in Downtown Toronto for 80k/year.

80k a year after tax in Ontario is about 4,500$/month.

Rent in Downtown Toronto is 3k/month for bare minimum places.

I'm not looking for a new job, I'm just wondering what ya'll think of this?

https://redd.it/10kgunc
@r_devops

I recently extended the GitLab director shadow program criteria to include those between jobs (aka layoffs)

I recently extended the engineering director shadow program criteria to include those between jobs (aka layoffs). The free program is a learning and mentoring opportunity to observe someone living the GitLab values of collaboration, results, efficiency, diversity, inclusion & belonging, iteration, and transparency via asynchronous collaboration and being invited to attend most of my meetings for a week.

You can find out more about the program in the handbook: https://about.gitlab.com/handbook/engineering/development/shadow/director-shadow-program.html

https://redd.it/10kh8jn
@r_devops

POV: offered junior devops position, eventually taking over lead responsibility

As the title states, got an offer on hand for a junior devops position. I’ve worked with CI/CD tools and processes but this is an opportunity to get more hands on.

Shop is AWS/Azure and projects are managed in the cloud.

Goal is to eventually delegate all the responsibilities from the lead/senior devops engineer to myself, and I will be POC going forward but not sure how quickly or how slowly that transition will be. I will be supporting a small team of devs that work on project already in the cloud- most of the infrastructure seems to be set up and looks like I may need to help with any future transitions or changes

Outside of this, I plan on getting familiar with the environment and processes. Also getting a cert aws or azure to make it official.

I think I may take the offer but looking for some advice - what to do to best get myself onboard and up to speed. Seems like a ton of information to take in and don’t want to get in over my head …TIA

https://redd.it/10kgchl
@r_devops

Who is responsible for the Kubernetes "infra-part" in your Organisation?

Hi folks,

we are finally at a point that we can enroll K8s-clusters for dev-projects.
Who is managing the infra part like cert-manager, ingress-controller and monitoring-stack in your organisation?
Is there are central Ops/DevOps/Plattform Team? - If yes, how do you manage a multi-Cluster Environment?
Or is it in the hand of the development team?
Are there channels that everbody can participate on the infra stack?

https://redd.it/10k9fd2
@r_devops

Developer guide for best practices (privacy, GDPR)

I need to write a short, straightforward guide for application devs, but wanted to make sure that one didn’t exist already.

Topics would be targeted at the application layer - eg don’t put this in a log, use cloud events for this, encrypt these fields using this tech, don’t ship logs out of cluster, set up this to cleanse logs on the fly (while you chase dev to make the above changes).

Any favorite resources?

https://redd.it/10k8lbb
@r_devops

Should you build or buy your infra? Checklist for calculating the costs

Last week, I made a post in this sub[1\] & a couple of others requesting feedback on my calculation of the total cost of ownership for Apache Kafka when building/managing it yourself vs. buying it.

Based on the feedback (thanks, by the way!), I've updated my blog post[2\] and included the example calculation I posted on Reddit.

Anyway, here's the "checklist" that guided the calculations made in my previous Reddit post, with links at the bottom:

\-

When calculating the TCO, be sure you calculate the cost for each team involved (e.g., if you have separate infrastructure and development teams, consider the TCO for both independently).

#### Up-front costs

software cost & licensing, if applicable
learning & education
implementation & testing (including data migration costs)
documentation & knowledge sharing
customization

#### Ongoing costs

direct infrastructure costs (e.g., hosting & storage)
backup infrastructure costs (e.g., failover & additional AZs)
supporting infrastructure costs (e.g., monitoring & alerting)
maintenance, patches/upgrades, & support
feature additions

#### Team & opportunity costs

hiring to replace the engineers now working with the new software
time spent on infrastructure that could otherwise be spent on core product

\-

[1\]: https://www.reddit.com/r/devops/comments/10g9bk2/feedback\_request\_tco\_calculation\_for\_apache\_kafka (link to the Reddit post in this sub if you're curious)

[2\]: https://blog.mergent.co/building-vs-buying-software-infrastructure-the-true-total-cost-of-ownership (link to the full blog post)

https://redd.it/10k8egr
@r_devops

Struggling at my job

I managed to secure a job somehow by stretching my experience in my resume however I am struggling badly, I have to design an infrastructure in ECS soon. I don't know where to start learning from. So far everyday is a struggle until a day ends. My learning doesn't get me anywhere so I don't feel motivated to do it either. So far I have:

\- Udemy AWS Cloud practitioner course-Terraform Up and Running book I am reading-Linux and Unix System Admin Handbook-Udemy ECS course I started

I try to do this (one of those) after work but I'm not sure if I get anywhere, I either have to catch up quick or will lose my job. What should my focus be on to get some practical knowledge and what should I focus on ?

https://redd.it/10kpbdn
@r_devops

Starting github action in frontend repo when backend repo changes (w/o backend repo knowing the existence of frontend repo)

I was looking at the repository_dispatch webhook event of github actions but this requires the frontend repo to have repository_dispatch from backend repo and backend repo to explicitly notify the frontend repo with the following api call.

curl \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer <YOUR-TOKEN>"\
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/OWNER/REPO/dispatches \
-d '{"eventtype":"on-demand-test","clientpayload":{"unit":false,"integration":true}}'

source

Is there a way for my frontend repo to get notified by backend repo without the backend repo explicitly notifying my frontend? ( I could achieve this with scheduled workflow where I poll the backend repo every x hours but I feel like there is a better way on doing this).

https://redd.it/10kmwbk
@r_devops

Trouble Launching Docker on Windows

So I have been trying to get into the DevOps thing over the past two years. I used to have a laptop that ran Docker Desktop perfectly, allowing me to mess with containers, and run Kubernetes using Minikube.

&#x200B;

Now I lost the laptop and bought me a desktop, which has refused to run the docker engine completely. I have tried a number of options, including running clusters with Hyper-V as the driver to no avail. The Desktop runs on legacy BIOS, but I was told this should not be a problem. After a little troubleshooting I realized that Docker Desktop fails to install dockerd.exe on my system, so the Engine cannot start, and neither does the daemon (am I even getting the terms right?), so it looks like I'll have to build from source. I am told though that this is complicated and I may end up with issues even then.

&#x200B;

It has been a seven month journey of troubleshooting with trial and error and I am just about to give up on this. Has any of you ever faced this? Anyone know a workaround.

My computer's specs are:

HP ProDesk 600 G1 SFF (2014)

Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz 3.20 GHz

Installed RAM: 16.0 GB

BIOS mode: Legacy

Virtualization: Enabled

Hyper-V: Running

&#x200B;

I have tried Docker on WSL, KIND, Minikube and a few other steps from the Docker documentation and stack overflow but haven't had any success. When I have to, I typically spin up a cloud instance, which would be expensive for everyday practice. BTW I'd love to sign up for the CKA and CKAD exam later this year, if anyone is wondering.

https://redd.it/10kr6ll
@r_devops

Alternative to Atlassian Jira and Confluence

Dear all,

Can you recommend a viable alternative to Jira and Confluence? Costs are rising everywhere and I was asked to look into cheaper viable alternatives. Any thoughts?

Context:
Engineering org of about 250 people
Current use of Jira is pretty standard, confluence mainly for documentation (private and for emerging concepts which have not made it to the ‘official’ documentation yet) and exchange of information/ thoughts. Users are mainly software architects, enterprise architects, devs, qa, etc.


Thanks

https://redd.it/10ksowi
@r_devops