mineOps Part 5 Released!

Following up from my original post, https://www.reddit.com/r/devops/comments/rvkh6w/a\_new\_blog\_series\_mineops/.

At long last I've finally finished and released part 5 of the mineOps series, Making Containers Highly Available.

https://blog.kywa.io/mineops-part-5/

https://redd.it/t0uz62
@r_devops

Github ssh access to multiple repos

Im trying to add my ssh key to github so i can clone multiple repos in my organization. I was able to add my ssh key but it only lets me clone 1 repo. The remaining 80 repos give an error message that says

ERROR: Repository not found.

fatal: Could not read from remote repository.

​

Please make sure you have the correct access rights

and the repository exists.

Is there a way that i can automatically login and clone the repo daily without being prompted for my credentials? I

https://redd.it/t0mge5
@r_devops

aws lambda invoke

Hi, lets say i make some simple flask web app where user can generate image, is it possible to invoke lambda function when user click on generate image?Mostly i used lambda for s3 put objects so not sure about this.

Thanks

https://redd.it/t13ejp
@r_devops

Docker build in GH Actions. Check if image digest is the same as previous before pushing

Hello, I'm trying to build a CI with GithubActions:

on:
push:
branches:
- cicd

permissions:
id-token: write
contents: read # This is required for actions/checkout@v2

name: Build images to ECR and deploy them to ECS
jobs:
deploy:
name: deploy
runs-on: ubuntu-20.04

steps:
#Increments the version for the image tag
- name: gh auth login
env:
pattoken: ${{ secrets.REPOACCESSTOKEN }}
shell: bash
run: gh auth login --with-token <<< "${{ enb.pattoken }}"

- name: gh secret set env
env:
secretname: 'MINOR'
secretrepo: Nasini-Trading/ArqLogger-Server
shell: bash
run: gh secret set "${{ enb.secretname }}" --body $((${{secrets.MINOR}} +1)) --repo "${{ enb.secretrepo }}"

- name: Checkout
uses: actions/checkout@v2
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: arn:aws:iam::XXXX:role/GithubActionsRole
role-session-name: GithubActionsSession
aws-region: us-east-1

- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1

- name: Build, tag, and push backend image to Amazon ECR
id: build-backend
env:
ECRREGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECRREPOSITORY: arqlogger-server-backend
IMAGETAG: ${{ secrets.MAJOR }}.${{ secrets.MINOR }}
working-directory: ./backend
run: |
docker build -f Dockerfile -t $ECRREGISTRY/$ECRREPOSITORY:$IMAGETAG .
docker push $ECRREGISTRY/$ECRREPOSITORY:$IMAGETAG
echo "::set-output name=image::$ECRREGISTRY/$ECRREPOSITORY:$IMAGETAG"

- name: Build, tag, and push frontend image to Amazon ECR
id: build-frontend
env:
ECRREGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECRREPOSITORY: arqlogger-server-frontend
IMAGETAG: ${{ secrets.MAJOR }}.${{ secrets.MINOR }}
working-directory: ./frontend
run: |
docker build -f Dockerfile -t $ECRREGISTRY/$ECRREPOSITORY:$IMAGETAG .
docker push $ECRREGISTRY/$ECRREPOSITORY:$IMAGETAG
echo "::set-output name=image::$ECRREGISTRY/$ECRREPOSITORY:$IMAGETAG"


where I'm:

logging into ECR
using a GH secret to incremente the TAG (so each new docker image has a 0.1, 0.2, 0.3... version tag)
building the images (one for the Dockerfile in the frontend folder, one for the backend)
tagging with the previous MAJOR MINOR version secret


My problem is that sometimes I don't change anything in the image build but it would nontheless trigger a new version.

I want to use the Image Digest/checksum of the just built image to compare with the Digest of the ECR previous image. If they are the same (no changes in the content of any layer of the image) the push should not be triggered


Any ideas?


EDIT: For some strange reason Reddit doesn't allow me to write ${{enb}} with the "v". It gives error 403. Odd...

https://redd.it/t15mbu
@r_devops

Just a day of "Why am I bothering with this lunacy?"

Rant I guess. Sorry.

I've had this ticket for a week or so, it's really simple - make sure EBS volumes are encrypted.

I am relatively new in a large org, but I figured - that should be pretty easy, I can do that.

Packer, Terraform, AWS... build a new instance make sure the attached volumes are encrypted.

So I made a branch, put in the relevant...

"No no, on this system we don't use THAT gitlab instance we use THIS gitlab instance..."

Okay, I can't get access to that...

"Ah you need an LDAP account to use the VPN it's behind"

3 days later...

"Okay you have that LDAP account... You should able to use that VPN now..."

Yep, that VPN works but I can't see any projects in that Gitlab...

"Oh yeah, so now you need to get your LDAP account added to that Gitlab instance..."

Okay who do i ask about that?

"Don't know, ask in standup"

Okay so now I have my LDAP account in that gitlab instance but I can't push any code because it requires a GPG key for signing... I have a GPG public key but it's not accepting it because it's associated with an identity for the gitlab SAAS instance...

I create a new GPG pair, try and upload it.

"No we don't recognise this because the machine you're using is not associated with the LDAP identity provided..."

FML

Project manager: "Nezbla we are waiting on this Jira ticket about EBS volumes to be closed so we can include it on the next release!! What have you been doing all this time!! Can you work harder please?!?"

It's 3 lines of code in the packer template and a minor Terraform tweak with IAM policies regarding KMS....

I'm not happy it's taken me a week to get this nonsense merged... In fact I'm verging in furious.

But it's Friday after 5pm so I've a cold pint of beer in hand and considering how much better life could be if I'd decided to be a lumberjack...

https://redd.it/t193fk
@r_devops

Russia traffic

Seems like all leaders are busy figuring out what ‘sanctions’ to announce next against Russia. Devops should not miss out, what about configuring traffic drops?

https://redd.it/t1dnql
@r_devops

Any recommendation for some must-know DevOps skills or fundamentals?

What are some of the fundamentals or skills that you guys think an individual must know in devops field

https://redd.it/t1qztw
@r_devops

How to host HTML / JS / CSS?

Working on a small project to teach myself DevOps. I built a simple "notes" web app with a document each for html, js, and css. What is a good web server platform to use for hosting this? I've used Apache a bit in the past, but that's about it. I want to run it on an AWS ec2 and branch out using DevOps tools from there to learn the ropes.

Note: I know there are simple ways to host static website, like in s3 - but I explicitly want to overengineer it a bit so I can work with more DevOps tools.

https://redd.it/t1hzoc
@r_devops

Scale Jenkins Behind Webhook

We have a Jenkins setup on Kubernetes. Agents leverage Kubernetes pods dynamically. But the Master controller is just one and has become bottleneck and SpOF.

We can shard the controller for sure. But wanted to check if we can completely abstract, decouple the controllers from consumers.

We intend to completely hide Jenkins behind a event handler or webhooks service like svix. So we can distribute the jobs to any jenkins controller.

Is this feasible? May be I am missing something obvious.

https://redd.it/t1qxy5
@r_devops

lambda pipeline and buildpec

So I have an application that I want to run in lambda, my pipeline looks like I pick the code from github and then I want to build in codebuild and I want the "jar" file from codebuild to go into an s3 bucket so that I can create a deploy stage for lambda so lambda can pick up the changes from there.

&#x200B;

Does this sound like a good plan? Also. what should my buildspec.yml should look like, basically I want to copy the jar file present in /targets into s3

https://redd.it/t1pu53
@r_devops

Hashicorp Packer - VMware timeout over 1h

My builds using packer are timing out due to taking over 1h (windows updates...) if I disable the Windows Updates, works fine. Any idea how to overcome this issue? I can't seem to find what I'm looking for anywhere...

https://redd.it/t1xvxk
@r_devops

What Are My Options For Running SonarQube In A Pipeline?

I have previously run SonarQube using the gradle plugin and the server running on localhost.

I have also worked at larger companies where they have a dedicated server instance.

I now need to run it as part of a build pipeline and not just locally. However, managing a server, keeping it up to date with patches, leaving it running all the time when I don't need it etc seems like a pain and I'm on a shoestring without anyone to manage it.

There is a plugin that give the results on the pipeline (which I want), however, it geared all around having a standalone server.

What are my options for running SonarQube in an (Azure) pipeline?

NB: I need a guide or link on how to do the steps of the options too.

https://redd.it/t1o4yx
@r_devops

To Docker Swarm or to K8?

About to dive into the docs for one of these technologies. Start with Docker Swarm?

https://redd.it/t204vt
@r_devops

Custom authentication in Swagger

is this possible?

Essentially we have a custom oauth2 authentication and I am wondering if it is possible to modify the swagger/openapi files to reflect this

https://redd.it/t127ql
@r_devops

Any recommendations to learn Azure pipelines?

Hi guys,

I’m a junior devops engineer. I started two months ago and am still learning on the job. One of my next assignments is to create a pipeline in azure devops.

I was wondering if you guys got some recommendations to check or read (YouTube or other sources) for beginners.

I do have a cs background, don’t know if that matters.

Thanks in advance :)

https://redd.it/t24z1o
@r_devops

kubelet unreachable after switching networks

I created a cluster with kubeadm while at my house on my local machine. When I went to my office, the server was unreachable (kubectl get pods resulted in The connection to the server 192.168.1.0:8862 was refused - did you specify the right host or port?).

What is the reasoning behind this? Are there any links that can dumb this down for me? I was under the impression a cluster running locally wouldn't be affected by a change in network connection.

Note: I ran the usual fix sudo -i && swapoff -a && exit && strace -eopenat kubectl version and the kubelet still wasn't found. systemctl restart kubelet also didn't help. I ended up losing the logs since I returned home and kubeadm reset so I know I'm not giving much to work with, but I'm still curious as to why these issues occurred.

https://redd.it/t25y5l
@r_devops

Migra: Diff for PostgreSQL schemas

I thought the r/devops subreddit might be interested in this project I just found!

https://github.com/djrobstep/migra

https://redd.it/t27c02
@r_devops

Pre-requisite to learning Docker and Kubernetes?

I am not formally a software engineer. I'm a statistician, turned data scientist, turned data engineer, turned whatever my startup needs me to be lol.

I've watched a few videos about docker. And at a 30,000ft view understand the concept of K8s orchestrating docker. But I want to learn and understand enough about them to be able to use them effectively with Airflow.

Are there any pre-requisite things I should learn? And are there any resources you guys have that you would recommend for noobs to go from zero to one on these topics?

Edit: Also I'm familiar with Python and bash. I don't know Java and was hoping I could keep it that way. Would prefer depth than breadth in programming languages.

Thanks in advance

https://redd.it/t254v3
@r_devops

How did you get into Devops?

Interested to hear everyone’s path into the field. On purpose or accidentally?

https://redd.it/t2cp4o
@r_devops

Ukrainians are organizing a cyberarmy and call volunteers to fight Kremlin's regime in the cyberspace.

This is a twitter twitter.com/FedorovMykhailo/status/1497642156076511233 post by Mykhailo Fedorov Vice Prime Minister of Ukraine and Minister of Digital Transformation of Ukraine

https://redd.it/t2mk8i
@r_devops

How to Setup a Jenkins to DockerHub Pipeline with Multi-Arch Images

I recently wrote on how to create a GitHub to Jenkins pipeline. I have also talked about building and pushing multi-architecture images to the Docker registry.

Link to tutorial: https://santoshk.dev/posts/2022/how-to-setup-a-jenkins-to-dockerhub-pipeline-with-multi-arch-images/

https://redd.it/t2szd2
@r_devops