group merge requests

When working with polyrepo architecture, a single change to the entire product often happens across multiple projects, and that means the teams have to coordinate multiple merge requests, which all run CI/CD processes that theoretically should be a single CI/CD pipeline, and a single merge request. Gitlab even recognizes this problem, as seen here:

[https://gitlab.com/groups/gitlab-org/-/epics/882](https://gitlab.com/groups/gitlab-org/-/epics/882)
https://gitlab.com/gitlab-org/gitlab/-/issues/3427

However, they claim that they found low demand when interacting with customers. My question is- how is that possible? As they said themselves, most people solve this problem by either using a monorepo (which has its own issues), ci patch-scripting (which is limited and requires a lot of work and decisions) or by using submodules (which might fit the bill for libraries, but not things that need to be deployed like microservices)

So I guess my question is, what do you guys do when you want to use a polyrepo architecture for developing microservices, and want to make a single change to your product that spans multiple repositories?

https://redd.it/zni83l
@r_devops

Automate the deployment of resources in AWS

Hello, newbie here, be nice...

Region: Bahrain + UAE

Services used: CodeCommit, CodeBuild, ECS, ECR

I'm using both of the regions, because CodeCommit and CodeBuild are not yet supported in UAE region. ECS and ECR are in UAE region.

How is it currently working: GitLab repos are mirrored to CodeCommit. I'm getting the source code from CodeCommit, use it in CodeBuild to build an image, push in to ECR and then use the image for a task definion (service) on ECS. But this is manual approach and I don't like it.

So, I want to automate the workflow but sadly, there is no CodePipeline available in either of the regions. Should I use Lambda which is listening to a certain event, which triggers the build process (maybe whenever a push commits to the master?) I don't really know what is the recommended approach.

Thank you in advance.

https://redd.it/zneccv
@r_devops

Artifactory as CDN/Package Manager, how to ask for latest version?

Companies are taking local backups of thousands of 3rd party libraries and storing these in different versions.

These dependencies eventually become vulnerable and clean-up becomes necessary, but simply deleting the old libraries will mess things up for a lot of developers.

I am familiar with NPM and package.json, where develops use tilde (\~) and caret(\^) to get minor or point releases of libraries... is the workflow similar with JFrog Artifactory?

How does one allow developers to get an approximate version of a dependency, and not direct-linking to a specific version?

Is this even possible, or should the company implement another solution?

https://redd.it/zndrom
@r_devops

How much would it cost to run this service on AWS?

I am developing a software service that will have:
a) a phone app with ~800 users
b) a web app for desktop with ~50 users

The phone app will mainly be used to:
a) track the location of an employee during their shift by sending latitude & longitude data to the server every ~5mins/ user / day for 8 hrs a day.
b) upload photos by the user to the database (~10 photos / user / day)

The web app will mainly be used to:
a) monitor the data recorded by the users of the phone app.
b) assign an agenda to each employee.

The tech stack will be:
Flutter for mobile & web apps
Node Js to run the server

How much will it cost to run the app per month?
What are the best ways of optimising cost (Storage/Compute/everything) ?

TL;DR:

How much will it cost per month to run a Node JS service and Flutter app on AWS with :
1) 2,304,000 latitude & longitude data uploads and fetches per month.
2) 240,000 photo uploads and fetches per month.
3) + Compute costs
What's the best way to optimize cost?

https://redd.it/zo2sfb
@r_devops

Atlantis users on EKS, how do you manage account credentials?

I am just curious to see how people have their Atlantis configured. I assumed the easiest way was to have the Atlantis pod log into its own role and other accounts allow that role to assume a deployer-type role in their respective account. However, I do not want to hardcode any credentials for Atlantis.

Is there a way to get Atlantis to use its IRSA role for terraform changes? If so, how would I configure this? Or, by default, does any traffic leaving the pod assume the pod's IRSA? And then I could just allow that role to assume other roles across accounts?

https://redd.it/zmzfhp
@r_devops

I think IaC is a lot better than “ClickOps”!

Infrastructure as Code is imho way better than *ClickOps (*where you manage infrastructure through a GUI which is slow and prone to errors that only accumulate as environments gradually diverge).

ClickOps practices typically lack versioning, eliminating any hope of clean audit trails. Since you can't reuse configs, it becomes impossible to roll them out to multiple environments.

One of ClickOp's biggest weaknesses is that it's highly dependent on individuals. If your knowledgeable engineers who were in charge of configs jump ship, your infrastructure will be dead in the water until you can decipher the configurations they left behind.

Do you use ClickOp? If yes why?

https://redd.it/znfctz
@r_devops

Terraform azure VM connect to Github

Hello Everyone,

​

I just started to learn DevOps recently I have an infrastructure background and also do a bit of coding.

I have a flask web app in GitHub and I'd like to copy it on Linux VMs whenever I create the environment. Previously I had the code on my PC only and I copied them with provisional but now as I use Github whenever I push an update to my flask app, I'd like to terraform to destroy my current VM and install it with the new one(run terraform manually from my PC). My problem is that I don't know how to connect my Linux VM to GitHub and get it to download the latest codes.

So I was thinking to copy over my private key from my PC as the public key is added to GitHub but in that case, I have to use provisional which is not recommended also not too sure copy my private key also the best practice after I copy the private key I'd just run a custom data file which clone the repo and run the app. I could hardcode my login details to Github into the custom data file as well but this is also something I'd not like to do especially I'll upload the whole terraform code to github at some point.

​

What would be the best solution if I run the terraform code from my PC and want the new VMs to get the code from github?

​

Later I'm planning to upload my terraform code to GitHub and use GitHub action to deploy the VMs. when there is a new push on the main Branch. In that case, i think I can just use GitHub secrets to login when i run the github action.

Thank you for the advice in advance I know maybe what I try to do is totally dumb and there is a much better solution to deploy code to VM.

https://redd.it/zo9m6y
@r_devops

What is the difference between a Microservice- & a Cloud Native Application

To me, this seems to be almost the same, maybe with a very (almost neglectable) small difference, in the idea of breaking up an application in smaller bits being a bit more important to Microservice-Apps than to Cloud Native Apps. If that even is the case.

https://redd.it/zobb18
@r_devops

How would you show “Wireframes” for a code deployment tool?

My boss is high-level and not technical. When I pull up code to show him the deployment workflow he immediately freaks out and says he doesn’t follow. But he asked for a “Wireframe demo” of our new code workflow tool, GitHub Actions.

I created documentation for the tool, including high-level capabilities, guardrails, etc. But, he seems to want to see how the tool is working without viewing any code. This is frustrating. I guess I could show him the UI-only which is a small part of the picture. But to understand things like actions; then it requires viewing code.

I’m not sure to show him how the tool is working without showing code.

https://redd.it/zodt7j
@r_devops

Beginning my transition from Cyber to DevOps. Any tips?

Saw the roadmap, and I'm re-sharpening my Java from college as a first step along with linux refresh, then switching to learning Docker and Kubernetes and maybe terraform. Decent plan?

Done some very, very minimal automation with AD/PowerShell to incremenet GPO changes incrementally.

Some projects with Python + Twilio notifications using a docker container to send SMS when a VM was infected with malware. Some basic superclass demonstrations with employees from school.

Have not yet begun to grind leetcode. Need better programming skillset, so doing 100 hour Udemy course to refresh, since I've been doing Cybersecurity for the past 2 years.

Suggestions on improving my workflow plan?

https://redd.it/znec0h
@r_devops

Team Leader Tooling

Hello everyone,
I've been a developer for 9 years now and I was just promoted to a Team Leader of a small dev team of 3-6 devs.
I would love to know what applications you guys use to manage small teams, being 1:1s, scrum calls, work and tickets.
Also, if anyone could help me finding out what you guys decided when going scrum vs kanban, I'd love to know as I am very undecided

https://redd.it/zomk5g
@r_devops

Looking for Project - GitLab, Python/Flask, Terraform, Docker, Kubernetes, Ansible, ELK stack, AWS

Can anyone point me to an example or full tutorial (paid or free) on using the following exact tech stack?

GitLab -- I know this pretty well

Python/Flask -- Don't use either in my day to day.

Terraform -- Decent experience

Docker -- Good experience

Kubernetes - Good experience

Ansible -- very little experience. Don't use in day to day.

ELK Stack -- No experience

AWS - Excellent experience

https://redd.it/zofgai
@r_devops

Unable to use cd in Jenkins

I am using an ec2 instance with Ubuntu AMI.
As the title say, whenever I try to cd to a directory I get the following msg:

Running as SYSTEM Building on the built-in node in workspace /var/lib/jenkins/workspace/todo-dev [todo-dev] $ /bin/sh -xe /tmp/jenkins6737039323529850559.sh + cd /home/ubuntu/project /tmp/jenkins6737039323529850559.sh: 2: cd: can’t cd to /home/ubuntu/project/django-todo Build step ‘Execute shell’ marked build as failure Finished: FAILURE

It's a Jenkins freestyle project, running on built-in node, with no options selected except execute shell and the shell script is:

cd ~
cd /home/ubuntu/project
pwd

I have added the below code to sudoers file but still same issue
> jenkins ALL=(ALL) NOPASSWD: ALL

I also tried this, but nothing happened

chmod 755 project (default option)
chmod 777 project (access to all)

I found almost identical problem on stackoverflow but the given solutions don't work for me.

Kindly, help me out.

https://redd.it/zmqrot
@r_devops

How to you practice Linux skills? What Linux skills are even necessary for DevOps?

Hello everyone, I am a backend dev migrating to devops. One of the skills I always see that you have to have is Linux. I've used Linux for several years and I haven't had many opportunities to really have to use it. It's been mostly treating me like a regular OS. I recently started actually deep diving into it and learning how everything works, but it's all theoretical. I would like to use my newfound knowledge in a project based way but I have no idea what even that would be. How can I actually tinker and empirically get a hang of Linux? I have on old laptop with ubuntu on it, or I could just use VMs, but what do I do with them? Make a server out of them, mount stuff? Any fun projects that I could look into, maybe something like homelab?

https://redd.it/zoskvx
@r_devops

Looking for DevOps mentor

Looking for devops mentor who can guide me as i am students and i want to be a devops. I have learnt different tools like AWS for cloud, jenkins for CI, Ansible for provisioning, docker for containarizing services and K8s for manages those containers but still need guidance as i am new to this field.

https://redd.it/zm65e3
@r_devops

Ansible playbook Devops

I am trying to run an ansible playbook with

ansible-playbook -i hosts.yml playbook.yml --ask-vault-pass

I enter the password of vault and get the following error

I use Bastion and pass the ssh args

ssh_args = -F ./config_dev_cluster_server.cfg -o ControlMaster=auto -o ControlPersist=30m


fatal: [all\]: UNREACHABLE! => {

"changed": false,

"msg": "Failed to connect to the host via ssh: kex_exchange_identification: Connection closed by remote host",

"unreachable": true

}

What could be the issue ?

https://redd.it/zlrjdt
@r_devops

Automate Deployments

Quick question what tools are you guys using to automate deployments in Kubernetes? Just curious, I haven't used any new products in a while.

https://redd.it/zlubtb
@r_devops

I have made a free tool to compare cloud compute offers. Feel free to try it :)

The cloud calculator compares more than 40 000 cloud compute offers from AWS, Azure, GCP, Linode, Scaleway, OVH and Alibaba. I am working on extending the calculator to storage and network offers so that it's possible to design a project and get an accurate cost estimation (top right of the page > project calculator) .

Link: https://app.holori.com/compare

Any feedback on bugs and missing features is welcome!

https://redd.it/zlts2l
@r_devops

Prometheus Stack deployment using private image registry

How can i deploy kube-prometheus-stack helm chart to k8s cluster with no internet access but only access to private image registry.

https://redd.it/zozac8
@r_devops

How limited are your DevOps / Cloud Engineering career options, if you refuse to do LeetCode interviews?

Title. Are LeetCode questions prevalent in this area of SWE? I know in some areas, like web development, to get into a top company it's basically a requirement if you want a high paying job. Wondering if it's the same for DevOps / Cloud / Infra

https://redd.it/zoypyf
@r_devops

Why is my release pipeline triggered on a PR to master?

### Description

I use a simple branching strategy, `master`, `feature/*` and `fix/*` branches. I have three YAML pipelines:

* `pr_pipeline` - triggered when doing a PR to `master`.
* `build_pipeline` - triggered on commit to `master`, builds my library.
* `release_pipeline` - once the `build_pipeline` is done, this pipeline deploys.

When trying stuff out I commit directly to `master`, `build_pipeline` runs and once done, `release_pipeline` gets triggered and deploys.

However, whenever I go the proper route and create a new PR (on GitHub), I get two pipelines triggered: `pr_pipeline` and `release_pipeline`, both at the same time. `release_pipeline` shouldn't get triggered, obviously.


### pr_pipeline

pr:
branches:
include:
- master
exclude:
- feature/*
- fix/*

trigger: none

### release_pipeline

trigger: none

resources:
pipelines:
- pipeline: build_pipeline
source: infra_build_pipeline
trigger:
branches:
- master

Thank you.

https://redd.it/zp3y1f
@r_devops