Set of Powershell scripts to Trigger Build and Deploy releases parallely in Azure Devops From command line

Hello,

I wanted to share with you all about a new script I developed to help streamline the deployment process in Azure DevOps. This is my first script that I have developed and shared with others, so I am still learning and trying to improve.

As a developer, I often found myself manually deploying code to different environments manually by Creating release, approving it and then wait for it to complete deployment, which can be time-consuming and error-prone. To address this issue, I created a script using PowerShell that allows me to deploy code in a parallel fashion in azure-devops from the command line. This means that I can deploy multiple releases at once, and can get status of all triggered release in a single screen. rather than having to deploy each one individually.

I hope that others can find value in this script, and I welcome any suggestions or ideas for improvement. As a first-time script developer, I am open to feedback and am grateful for the opportunity to learn and grow.

Thank you for reading, and I hope this script can be helpful to you in your work with Azure DevOps. Let me know if you have any questions or feedback

https://github.com/thangeshbabu/hydra

https://redd.it/zzrehe
@r_devops

Do Devops Need an Internal Developer Portal?

There is a lot of focus on Internal Developer Portal solutions. There is a handful of use cases for the developer's use of a developer portal.

Im the founder of port (operating in these areas), lately, I have had many conversations with DevOps teams across different organizations that mentioned several compelling use cases for the use of DevOps teams of a developer portal.

For example:

Devops need a centralized, single source of truth of the software architecture (microservices, environments, deployments, cloud resources, regions, and more).
Devops need one interface for change management to keep track of changes that took place and see the history of changes across the entire stack, such as deployments, infrastructure modifications, versions, configurations, etc
Devops need visibility for troubleshooting & root cause analysis \- Since all metadata is managed in a single source of truth, performing root cause analysis becomes more accessible.
FinOps & Cost control \- seeing assets managed within the developer portal with the associated owner allows them to see cloud expenses from the organizational structure point of view.

​

I wrote a short piece about it, I would love to hear your point of view on Portals for DevOps, is it the exact solution for developers and DevOps as one with different views? A separate solution?

​

https://www.getport.io/blog/do-devops-need-an-internal-developer-portal

https://redd.it/100hyr3
@r_devops

Where to learn about about k8s and EKS/ECR?

For one my courseworks I had a cloud project and I was interested in it out of all the courses I have taken. But the kubernetes stuff confused me and the coursework I had , which was about deploying a k8s cluster using minikube and doing a ci/cd pipeline on any cloud service. I attempted this project and deployed a small flask app and dockerized it before getting stuck at the ECR/EKS stuff. I googled a ton of tutorials on youtube and on google but it was super confusing. I guess its because im jumping right in without learning the basics of kubernetes.

I want to learn more about cloud stuff and devops. How do I learn these techologies? I've tried youtube but I feel like half of them are outdated or sort of jump right in and leaves me confused. Any good courses from start to end about kubernetes and using aws services like ECR/EKS? I've looked at some on udemy but unsure which ones are good.


TL:DR Any course recommendations starting from scratch all the way to development of EKS/ECR and Kubernetes.

Thanks in advance.

https://redd.it/100hlu0
@r_devops

Monthly 'Shameless Self Promotion' thread - 2023/01

Feel free to post your personal projects here. Just keep it to one project per comment thread.

https://redd.it/100p6ma
@r_devops

SRE: What tool do you use for Incident Response Runbook/Playbook

Is there any SREs/Admins that can share what they use for their Incident Response automation and playbook?

I am familiar with security incident playbook that we have a category of tool called "SOAR" that can do process-flow based (semi)automation and manual activities during a security incident.
But from the SRE side, what tool do you to document Runbook "checklist" or process flow, and how do you automate some of the responses?

https://redd.it/100le28
@r_devops

Assignment from technical interview could have been used.

Allright.

A while back I asked some questions about what is normal for technical interviews. In the meantime I have landed a job, but I'd like to share an experience I had so others might learn from it as well.

One of the applications I made was for a position of a full stack engineer. They gave me a huge take home assignment after the first interview. I got the weekend to work on it. I didn’t want to spend a lot of time on it, but since the first interview went great, I decided to do it anyways.

I am not very strong in frontend and had stated that in the first interview. They were fine with it and said my work and assignment would be geared towards backend. I got 1 backend question, which basically was just a copy pasta from 1337 code. The rest of the assignment was mostly to fix up a bunch of sh*t in React, like poor performing chat, issues with props drilling, misused hooks. But really a LOT of stuff. I'd say a solid 10 to 12 hours of work.

I managed to get it all sorted over the weekend, albeit I was very annoyed, and handed it in. After that total radio silence. After several weeks the recruiter came back and told me without further feedback I was not hired.

Funny how the chat page on their website is now working correctly.

From now on every employer who comes up with assignments that take more then 2 hours of my time can stick it where the sun don't shine. I'm rather unemployed.

https://redd.it/100xhib
@r_devops

Pass values from Terraform to Argocd/Kustomize

My terraform module spit outputs that I would like to inject into my kubernetes yaml files.

For example cert-manager with aws with irls authentication via kubernetes service accounts.

So first I would need to create the iam with the correct permission, then create the namespace for cert-manager, then create the service account and pass the iam value in the kubernetes provider.

Then in argocd repo, I set the helm chart cert-manager values to use the existing serviceAccount created by terraform.

So far so good, im okay with this.
But now I need to pass the arn of the certificate to the AWS cert manager controller issuer, I cannot create the AWS issuer object without first creating the CRDs, sure I can install the crds alone in terraform and pass the arn there

but now i have this vague and messy setup where some stuff is in the terraform code base and the other is in the argocd repo.

Since the only reason for this splitting is the need to take terraform output and pass them to argocd, is there a way to actually do it without having to port everything into the terraform world? Does kustomize have a way to patch things from a configmap?

https://redd.it/100t7i7
@r_devops

How do the odds of interruption on a spot instance scale over time?

AWS, for instance, quotes a "less than five percent chance" of interruption for many of its spot types. But they provide no information about how long they expect a typical instance to run if it isn't preempted, so I'm not sure what to make of it. Specifically, I'm considering using Google spot instances for ci -- it'd be really useful to know the odds of interruption for a five minute job vs a ten minute one vs one that lasts an hour and so on. Can anyone give anecdotal information on this?

https://redd.it/1011523
@r_devops

Understanding kubernetes labels

I came across following deployment configuration in kubernetes docs:

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx # 1
spec:
replicas: 3
selector:
matchLabels:
app: nginx # 2
template:
metadata:
labels:
app: nginx # 3
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80

I don't get what is the purpose of specifying app: nginx three times in above deployment configuration. The same web page says:

> The .spec.selector field defines how the created ReplicaSet finds which Pods to manage. In this case, you select a label that is defined in the Pod template (app: nginx).

I guess, this maps #2 to #3. If I am correct, then what is the purpose of #1? Can someone please explain?

https://redd.it/100kika
@r_devops

Use Docker with full IPv6 support

I‘ve wrote a blog post how to configure Docker so that container will get the correct IPv6 address of the request.
In the default configuration it is just the IPv4 address of the docker interface.

Maybe helpful for one or the other :)

https://www.manuel-bauer.net/blog/docker-with-full-ipv6-support

https://redd.it/100upzi
@r_devops

Anomaly Detection - A human+machine approach

Hey folks!

I am a statistician/data scientist who just started working in the SRE domain. I see a wide prevalence of blind-use (misuse) of ML algorithms. I mean just slapping Prophet or some other forecasting on some metric in Grafana will obviously detect every slight perturbation. NOISE!

How can we inject more meaning and domain knowledge into this while still enjoying the automated ease of algorithms? With the use of simple algorithms that allow users to get alerts they exactly seek from the metric?

Here are only a few algorithm examples

a) Algo that detects high/low value (could account for seasonal patterns too)

b) Algo that detects a change in baseline behaviour

c) Algo that detect unexpected missing data

d) Algo that detects a rogue metric that behaves differently from its peers

e) Algo that detects increasing/decreasing trend

Domain experts can choose the algorithm that best describes "what is broken" for a metric.

Do you think there is a need of such algorithms that allow users to get alerts that they seek?

A) No!

B) Maybe

C) Yes!

https://redd.it/100owkn
@r_devops

JWT Token

Is it possible to obtain a JWT token if I store user credentials in an RDS Aurora database?

https://redd.it/10197li
@r_devops

Looking for guidance (container management)

Hello there!

I'm currently running a bunch of projects on a close-to-regular docker host using docker compose. This includes traefik (as reverse proxy), home assistant, a matrix setup, and some more things. Nothing business critical, everything for my own private fun. That host is not very efficient, it eats too much power for not enough performance. So, I built a new server.

This new server is running windows server (for reasons) with hyper-v. I'm now looking for suitable alternatives for running the mentioned docker containers. This can go anywhere from running a linux vm with docker and compose up to running a single node k8s cluster.

I'm not sure which way to do from here. So, I'm asking you: Tell me your ideas for this. No matter if they are crazy or not, as long as they make sense I'll consider them.

Oh, and happy new year!

https://redd.it/100ow0n
@r_devops

envsubst with template file vs using CD Tools

Hi, at my work we are using Gitlab CI to build our pipelines where we build, test, upload images, etc AND deploy


The deploy job (k8s resources) would use deployments/services/ingress templates like this:

apiVersion: v1
kind: Service
metadata:
name: NAME
spec:
ports:
- protocol: TCP
port: PORT
targetPort: TARGETPORT

that are stored in a templates repository, then we clone the repo in the pipeline and run ENVSUBST to replace the NAME,PORT for the desired values.

Then we apply the manifiest to the cluster (gitlab pipeline)

deploy:
stage: deploy-dev
image: $IMAGE
script:
- git clone template-repo
- cd template-repo
- envsubst '$NAME' < service.yml
- k apply -f service.yml


I'm aware that CD tools like Flux, Argo exist for the 'deploying part' but I can't understand why that is 'better' than having templates and filling the desired values in the pipeline

https://redd.it/100lwck
@r_devops

Securely Ingressing into Bare Metal Kubernetes Clusters with Traefik and Gateway API ☸️🔌🔒

In this article, we have unleashed the power of the new Gateway API to design a solution that enables a better collaboration between the platform and application teams. We abstracted the platform details in Gateway resources so the application teams just need to declare in the HTTPRoutes which connectivity model they need for their application.

To provide connectivity to those Gateways, we managed to address our LoadBalancer IPs from inside our VPN by using some brilliant features of Metallb and Tailscale. This enables encrypted point-to-point connectivity to our workloads running in Kubernetes, resulting in better latencies compared to traditional centralized VPN solutions.

Let me know your thoughts:
https://medium.com/itnext/securely-ingressing-into-bare-metal-kubernetes-clusters-with-gateway-api-and-tailscale-cc68299b646a

https://redd.it/100ptgj
@r_devops

What programming language to learn as Devops while having some experience in programming and scripting?

Hi all,


I am working as a DevOps engineer and I realized that I love coding. I am mainly working with terraform, some ansible and bash scripts on Azure. I am trying to find which programming language would be great for me but I cannot decide. As I am working mostly with .NET developers I have passed course on how to do APIs in .NET. I really loved learning it and now I can easily troubleshoot some things and see what programmers are missing.
Also, I was playing with python on my own and I have written a few scripts and a discord bot. That was also really great but I don't like how Python is structured ( the writing syntax is weird for me, I love ; ).


I read a lot of posts on like what language is best for DevOps, some say JS some Python, Golang, bash, and others... I think we all know the story :).


Why I have written this post is to see what you guys think I should learn next, where to focus and why..


P.S. I was also thinking of switching to programming but as a backend programmer.

https://redd.it/zzxrj5
@r_devops

Anyone recommend anything other than Jenkins for executing install scripts on AWS EC2?

We need a bash script to run on instance start/reboot.


The GitHub Actions yaml for ECS connects to the instance, but doesn't specify running any install commands.


What do?

https://redd.it/101b6sg
@r_devops

tfstate - A wrapper around terraform state

Hey everyone!

I'm focusing on learning Go this year, and for a project I wrote a small wrapper around terraform state called tfstate.

The purpose of tfstate is to provide some additional functionalities around terraform state mv & terraform state rm.

1. Adds a manual confirmation step when running state commands that alters your state file
2. Outputs a list of affected resources if you were to apply your state command
3. Auto backsup your current state file inside .terraform/tfstate

Because the tool auto backsup your state file, this allows you to easily rollback changes. For example:

Lets say you ran the following:

tfstate rm null_resource.main

An output with a list of all affected resources will be displayed, and it will ask you to confirm if you want to go ahead with the changes. You type "yes" and suddenly realize you made a mistake.

No biggie. You can simply run:

tfstate rollback --latest

The command will get your current serial version, and look for the most previous version inside .terraform/tfstate

If no previous version is found, the command will just error out.

So instead of messing with terraform state pull, push, manually bumping the serial, the tool takes care of it for you.

You can also manually backup your state file with:

tfstate backup

And all state file backups are stored inside .terraform/tfsatte/state-{{ serial }}-{{ timstamp }}

Additionally, you can rollback to a specific state file with:

tfstate rollback state-10-123456.json

Hopefully this tool helps someone from messing up their state file, which we all know is not a fun problem to fix. Cheers!

[https://github.com/alex067/tfstate](https://github.com/alex067/tfstate)

https://redd.it/101eoh5
@r_devops

What is the Basic Requirements for DevOps intern position

I know that Devops requirements ain't specific, it depends on company but still it's kinda hurting my mind to ask this question to someone experienced. Because im on my own learning Devops. As my opinion discussion with a experienced person is so important for a beginner, I don't know about others. I alwas do have discussion but only this time I left alone I don't know anyone who is learning or experienced in Devops.

&#x200B;

SO please tell me anything which you think I need to know

So far I have learned

1.Basic command and concept of Linux

2.Basic of Docker

3. Git and Basic of Github

4. Basic concept of TCP/UDP, DNS, PKI, NMAP, CNAME, PORT

5. Can setup Oracle VM

6. Trying to learn Vagrant & Yaml

https://redd.it/101fzy1
@r_devops

Tried copilot to automate code?

Could automations replace devs in the future? Looking for the community’s opinion on whether AI tools like copilot can do the work of devs faster and better. WDYT?

https://redd.it/101h4fq
@r_devops

DevOps Market is Saturated - Jan 2023?

Any difference ease of landing a job since Jan 2022 to Jan 2023 given market saturation or financial crisis?

View Poll

https://redd.it/101fkt0
@r_devops