Beginner in DevOps - how do I manage microservices (via Docker)?

I'm more of a programmer, but obviously to implement my code, I have to delve into DevOps.

Say I have 2 microservices, auth service & user service. And a MongoDB database in addition. I can put all these into a docker-compose file, which would be the easiest, wouldn't it? But if I want to replace the image with another, newer version, I have to do docker-compose down and everything goes down. It's my naive look on this, but I would like to get some help regarding this, the DevOps side. Thanks! ;-)

https://redd.it/sx3jak
@r_devops

Gitpod automates the provisioning of ready-to-code development environments.

I thought the r/devops subreddit might be interested in this project I just found!

https://github.com/gitpod-io/gitpod

https://redd.it/sv079l
@r_devops

Testing Github workflows

Hi there, uni student here learning/working on devops practices/engineering. I've recently started putting more time into writing github actions/workflows for CI/CD reasons. A bit of an issue I come across is the fact that I'll be doing a million commits to a github repository so I can test the action/workflow, which doesn't feel like a very handy process. Besides the solution of running github actions locally using nekos/act, are there any tips, solutions or advices you have for testing github workflows? Thanks!

https://redd.it/sx52lq
@r_devops

non deterministic docker container availability

Hi all,

I am running an app on a docker container that runs a specific application that i communicate with through port 10000. The image I am running on has been stable and unchanged for over a year, and it runs on multiple vms which are configured in the same way.

However, I get some unpredicted behaviour. Running from jenkins, on the same hosts, with the same configurations and the same image results in the container up and running in 30-50% of the times, with no constant pattern. There are no other applications or containers on the VMS.

Any thoughts on how to approach this ?

This is the run code but I think it's not so important since it doesn't behave the same

​

set -x

sudo systemctl restart docker

sudo docker network prune --force

sudo docker container stop $(sudo docker container ls -aq)

sudo docker container rm $(sudo docker container ls -aq)

​

port=10000

​

sudo docker run --hostname=quickstart.cloudera --name ${dockerName} --privileged=true -t -i -d -p 8888:8888 -p 8050:8050 -p 8051:8051 -p $port:10000 -p 25000:25000 -p 25010:25010 -p 25020:25020 -p 9870:9870 -p 8088:8088 -p 19888:19888 -p 14000:14000 -p 3306:3306 -p 8042:8042 -p 7180:7180 spark24_1 /tmp/cdh_startup.sh

​

sleep 240s

nc -zv $hostName $port -w 5

https://redd.it/sx9ieq
@r_devops

Managing pre-commits

I'm a primarily solo developer who has been working on improving my environment through devops principles. One of the setup tasks that I tire of is configuring pre-commit whenever I create a new repository. Since I hop languages and frameworks, it's not feasible to create a single copy and share it around my repo's.

Any ideas on how to better manage this aspect? I'm tempted to just make my own management CLI but wanted to hear of some other ideas before going down that road.

https://redd.it/sxeg9v
@r_devops

Job Hopping

Hi Guys,

A friend of mine joined a company just 4 months ago as a DevOps Engineer. Now he has got an offer from somewhere else for a 150% increase in salary. He is pretty confused since this is the second job he is leaving in less then 7 months and he keeps asking me for advice (the company where he is working now also offered him a 100% raise from his previous salary). I told him to stay where he is since he runs the risk of being labelled as a job hopper by potential employees but he is double minded since he thinks might not receive this much raise again as a new entry in the field. What would you guys recommend ?

P.S He is also worried about the company culture since where he is currently working right now is possibly the best culture according to him (not learning wise but the managers are very lenient and supportive and the company has very good reviews on glassdoor) and from where he has got the offer has average reviews on glassdoor (some good some bad).

https://redd.it/sx4hax
@r_devops

A little tool to help manage Elasticsearch/Opensearch clusters.

This tool helps to export/import data, move indices, perform backups and schedule these tasks. It is a docker image with a browser-based scheduler, code editor, and terminal.

https://github.com/bluxmit/alnoda-workspaces/blob/main/workspaces/elasticsearch-workspace/README.md

https://redd.it/sxovqn
@r_devops

Question on CI/CD with Multiple Environments & Corresponding Branches

The flow I want to follow:

PR from Feature to Dev -> Build & Test -> Merge to Dev -> Docker Build, Tag using GitHub Short SHA, Push Container to Registry, Deploy Container to Dev Env

PR from Dev to Main -> Merge to Main -> Docker Pull & Deploy Container to Prod Env

How do I pull the corresponding Docker Image From The Registry for the code I am currently merging to Main? Do I have to build again on the merge to main everytime?

The GitHub SHA is different in the two workflows because of the merge commit that happens. And cannot always pull the latest I believe because the Dev lifecycle could be faster than Prod.

I'm open to alternative flows that allows keeping code for multiple envs in their respective branches as well.

https://redd.it/sxpkni
@r_devops

DevOps Bulletin Newsletter - Issue 39

Hey folks,

DevOps Bulletin - Digest #39 is out, the following topics are covered:

* 🚀 "A ‘Hello World’ GitOps example" - This post walks through a ‘hello world’ GitOps example using Docker, K8s, GitHub Actions and Terraform.
* 💰 Why sometimes you should press the $100k button: With S3's unlimited object storage you can easily end up with a shitload of data and thousand dollar bill.
* ⭐️ hands-on video tutorial introduces ArgoCD, how it works, why you might need it and how to deploy a hello-world application to Kubernetes with it. All of that in less than 20 minutes
* 🚨 "Startup guide to incident management" - If you’re working at an early stage startup and looking to get some good incident management foundations in place without investing excessive time and effort, this guide is quite literally for you
* ☁️ Introducing a Google Cloud architecture diagramming tool; this interface provides a list of all GCP products and services in one spot. No need to worry about finding the right icon or uploading an image from somewhere else – just focus on building the architecture with the components you need right there in the interface.

Complete issue: [https://www.devopsbulletin.com/issues/s3-storage-bill-of-almost-100k-month](https://www.devopsbulletin.com/issues/s3-storage-bill-of-almost-100k-month)

Feedback is welcome :)

https://redd.it/sxs4b7
@r_devops

Youtube in vid ad skip;

A chrome extension for youtube that tracks ⠀other users skip behaviour and uses that data to skip at video ads such as skillshare, brilliant, world of war, raid shadow legends, ect Is this a good idea?

https://redd.it/sxthfe
@r_devops

Local Development with Kubernetes Service Accounts

I'm a devops engineer and i'm trying to convert an application from using an AWS service account w/ key and secret for authentication, over to using a k8 service account to assume an AWS Role.

The challenge im facing is that the app developers use IntelliJ on their local machines to test their code, which requires hitting AWS resources. This means that they have possession of the key and secret for the service account (in our dev environment at least) on their local machines. If their IAM user is terminated, they still have possession of those keys and security is not ok with that. We don't have a good mechanism for key rotation, nor is there a plan for one.

...Hence using kubernetes service accounts to assume an AWS role and grant the application the AWS permissions that it needs to function. Since we are using EKS and the containers already assume the cluster role by default, this has been super easy to implement. But it totally breaks the ability for devs to run the applications locally in intelliJ.

I'd love to set them up with the ability to run the application locally using docker desktop's kubernetes environment or something like that. But then they have to build the app and then deploy it locally, which is far slower and less streamlined than intelliJ. They are used to being able to run the application without even having to build it.

I'd love any and all suggestions as I am totally out of ideas.

https://redd.it/sxtjv4
@r_devops

Is there an easier way to SSH to ECS containers?

When I want to SSH to ECS I use

aws ecs execute-command --cluster <Cluster> \
--task <taskId> \
--container <ContainerName> \
--interactive \
--command "/bin/sh"

&#x200B;

But the ECS container gets rebuilt on every pull request so I have to keep going to AWS and get the new task Id.

Is there any other more convenient way to SSH?

https://redd.it/sxw5gj
@r_devops

What is Developer Experience? a roundup of links and goodness

https://redmonk.com/jgovernor/2022/02/21/what-is-developer-experience-a-roundup-of-links-and-goodness/

https://redd.it/sxwm41
@r_devops

question about internet speeds...

So, I'm looking at moving to a new house. The place is awesome but it's also pretty rural, and it seems difficult to find internet plans that are more than 50mbps. I'm used to having a much faster connection than that, like at least 300.

I need to know if I can do my job from home before I commit to purchasing a property... y'all think 50mbps would cut it?

https://redd.it/sxzd6h
@r_devops

Does anyone here use oath2 proxy in front of Atlantis?

I'm trying to figure out how to insert https://github.com/bitly/oauth2_proxy in front of https://github.com/runatlantis/atlantis via terraform but have several questions. First, is there any publicly existing terraform repo that can set this up automagically on Kubernetes? My google-fu is coming up short. Second, are there are any automagic scripts that can generate the config values for you via github api? The end goal really is to just figure out how to get this as automated as possible so I can add auth to atlantis. Any additional suggestions appreciated!

https://redd.it/sy0o6c
@r_devops

Telepresence with Consul

I've gotten Telepresence working without Consul (and it's great!), but I'm having some trouble when I add Consul connect's Envoy sidecar. I see the `traffic-agent` sidecar for the service I'm intercepting, but all of the traffic is still flowing thru Envoy.

Have any of y'all gotten Telepresence+Consul working? Thanks!

https://redd.it/sy4raf
@r_devops

Does anyone have experience using KodeKloud as a learning tool? What was your experience?

It is on sale right now. I enjoy the structure for learning and it is on sale right now. Any experience? And alternatives you enjoy?

https://redd.it/sy6e5w
@r_devops

Any folks from the zerossl project lurking these forums? Your user signup page cert is expired.

https://app.zerossl.com/signup

I was looking for an alternative to cert-manager/letsencrypt because of rate limiting pains. This does not look as promising :(

https://redd.it/sy9p9f
@r_devops

EKS ingress

I want to set ALB ingress. For some reason Ingress forwards paths to services.for example. If I access mydomain/ng-test/ I get redirected to nginx-test-service but in pod log I see

\[error\] 35#35: \*16 "/usr/share/nginx/html/**ng-test**/index.html" is not found (2: No such file or directory),

Can I somehow force ingress not to add path?

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: alb-app
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/certificate-arn: ***********
nginx.ingress.kubernetes.io/rewrite-target: / #test ....?
spec:
#ingressClassName: alb
rules:
- http:
paths:
- path: /ng-test/
pathType: Prefix
backend:
service:
name: nginx-test-service
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: xxxx
port:
number: 8989

https://redd.it/sy536o
@r_devops

How to build Lakehouse with Azure Synapse

In today's article, we are going to see what lake house is and how azure synapse can be used to implement lakehouse architecture. https://clouddatascience.medium.com/how-to-build-lakehouse-with-azure-synapse-56ddf831c1b9

https://redd.it/sydl5c
@r_devops

Branching strategy for Infrastructure as a code

Hi folks,

I am very curious about people's way to manage infrastructure in a git and looking for a way to improve mine.
So, we are using terraform and store code separately from source code. The infrastructure code is relatively complex and on general divided by 2 modules(global resources, regional resources) then each environment call modules from own environment folder and supplying variables.
I have aproblem for branching strategy for module (it's a separate repo). The software release happens not often (1 in few month, because before going to prod it is tested on like 4 environments).
So we have a master branch, where the prod deployed from SW released version (say 2.6.30).
Then we have a develop branch with the version that reflect current development (let's say, 3.x)
Then sometimes developers start a complete different version (4.x) while releasing fixes and features for the current master and develop branches.
So,we have a sort of branching hell.
Do you guys have any tips on how to support this scheme?

https://redd.it/syi0ye
@r_devops