How do i setup ci cd pipeline from gitlab to ECS 2 for next js?

Hi I’ve been looking for tutorials on this and I can’t seem to find one that incorporate gitlab even if I find one they are either not working or outdated. Can someone tell me how to set one up for next js ? Or if I can may I message any of you that are capable I mostly work on front end and backend but never on devops or ci cd now I am confused. Thanks

https://redd.it/118ls12
@r_devops

what do you use shell scripts in your day to day work as a devops practitioner?

I've learnt the syntax and understood shell scripting, I'm finding it hard to understand where exactly do one use in their every day practices as a devops engineer.

https://redd.it/118n051
@r_devops

TF401025 'repoName' is not a valid name for a Git repository - solved

I'm just dumping this here, hoping to help someone googling this issue in the future.

When trying to create a repository named 'Server' you get the error TF401025
'repoName' is not a valid name for a Git repository or
'newRepositoryName' is not a valid name for a Git repository

You just can't name a repository 'Server' in DevOps, as explained here

https://redd.it/117zykd
@r_devops

Should we use OpenTelemetry traces for running tests?

Writing tests that check that side effects across the system are actually happening was always difficult (for me at least).

Then came OpenTelemetry, which basically gives you amazing visibility on what's happening across your system. So if you want to check that a user gets an e-mail everytime they register - theoretically you just need to search for the right spans.

We thought it might be an interesting concept to explore, so we started writing an open source testing framework that does exactly that - with Jest. We've actually managed to get into YCombinator with that so we can work on it full time.

Does it make sense? Would you be using something like that?

(Adding here our repo - https://github.com/traceloop/jest-opentelemetry, although this honestly doesn't mean to be a self-promoting post. I'm actually looking for honest feedback).

https://redd.it/118q0xn
@r_devops

Is anyone great at drawing (have a strong artistic side)?

So I think networking/devops is all very interesting, but I have this strong skill in drawing very realistic art. I'm not trying to brag, but I can really draw some great stuff - professional portrait artist level.

Sometimes I feel like this skill is going to waste and people say "oh you should just do frontend/UI developer" type of work.

Are there any devops or network/systems engineers that have a strong drawing/artistic side? Maybe there is some link here in terms of visual talent and working with computer systems?

Btw, this post is not a joke. I seriously am at odds sometimes in terms of choosing/settling on a career path due to having a strong artistic/drawing side. If someone could chime in or offer advice it might really help settle a lot of internal conflict that I have.

https://redd.it/118qixb
@r_devops

A Chainlink alternative? How API's work in Web3

https://www.thestatuscode.co/p/a-chainlink-alternative-how-apis

https://redd.it/117vpw0
@r_devops

What do you do if there is a strange behavior or falling tests in staging env and you have 100 recent commits?

i.e., git bisect?

https://redd.it/1174ycb
@r_devops

Getting "unknown manifest name" with docker registry behind CloudFront distribution

Howdy Devops community,


I am having an issue trying to pull images from my private docker registry running on an EC2 instance through my CloudFront distribution, unable to fetch it via digest and having a fallback on image tag, which will be soon deprecated. Below the logs on the different services processing the request.


For info:

cdn.example.com is my cloudfront domain

awsdocker.example.com is my docker registry address


On the client-side, getting:
```
docker pull cdn.example.com/python:3.10.8-slim-bullseye
WARNING: ⚠️ Failed to pull manifest by the resolved digest. This registry does not
appear to conform to the distribution registry specification; falling back to
pull by tag. This fallback is DEPRECATED, and will be removed in a future
release. Please contact admins of https://cdn.example.com. ⚠️
3.10.8-slim-bullseye: Pulling from python
025c56f98b67: Already exists
778656c04542: Already exists
85485c9f43dd: Already exists
23b3c91f0de2: Already exists
fd19b936aab8: Already exists
Digest: sha256:6a34910f419a1fdaf2b6d886a1b303eaaa093a828528e0f59b95ccc71feddfa4
Status: Image is up to date for cdn.example.com/python:3.10.8-slim-bullseye
cdn.example.com/python:3.10.8-slim-bullseye
```

Fetching the image directly from the registry works as expected. But...


Trying to set up a Docker Registry behind an AWS CloudFront Distribution, I'm getting the following error message on the registry:
```
docker-registry-registry-1 | OBFUSCATED_IP - - [22/Feb/2023:08:07:00 +0000\] "HEAD /v2/python/manifests/3.10.8-slim-bullseye HTTP/1.0" 200 13227 "" "Amazon CloudFront"
docker-registry-registry-1 | time="2023-02-22T08:07:00.575898015Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown manifest name=python revision=sha256:6a34910f419a1fdaf2b6d886a1b303eaaa093a828528e0f59b95ccc71feddfa4" err.message="manifest unknown" go.version=go1.16.15 http.request.host=awsdocker.example.com http.request.id=4fec8c41-9afe-40b7-9af2-6023a9518efb http.request.iss.onethod=GET http.request.remoteaddr=OBFUSCATED_IP http.request.uri="/v2/python/manifests/sha256:6a34910f419a1fdaf2b6d886a1b303eaaa093a828528e0f59b95ccc71feddfa4" http.request.useragent="Amazon CloudFront" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.775224ms http.response.status=404 http.response.written=182 vars.name=python vars.reference="sha256:6a34910f419a1fdaf2b6d886a1b303eaaa093a828528e0f59b95ccc71feddfa4"
docker-registry-registry-1 | OBFUSCATED_IP - - [22/Feb/2023:08:07:00 +0000\] "GET /v2/python/manifests/sha256:6a34910f419a1fdaf2b6d886a1b303eaaa093a828528e0f59b95ccc71feddfa4 HTTP/1.0" 404 182 "" "Amazon CloudFront"
docker-registry-registry-1 | time="2023-02-22T08:07:00.714835837Z" level=info msg="rewriting manifest sha256:49749648f4426b31b20fca55ad854caa55ff59dc604f2f76b57d814e0a47c181 in schema1 format to support old client" go.version=go1.16.15 http.request.host=awsdocker.example.com http.request.id=1e59d24b-24c5-4f1f-a968-e80fd2982f18 http.request.iss.onethod=GET http.request.remoteaddr=OBFUSCATED_IP http.request.uri="/v2/python/manifests/3.10.8-slim-bullseye" http.request.useragent="Amazon CloudFront" vars.name=python vars.reference=3.10.8-slim-bullseye
docker-registry-registry-1 | time="2023-02-22T08:07:00.718986265Z" level=info msg="response completed" go.version=go1.16.15 http.request.host=awsdocker.example.com http.request.id=1e59d24b-24c5-4f1f-a968-e80fd2982f18 http.request.iss.onethod=GET http.request.remoteaddr=OBFUSCATED_IP http.request.uri="/v2/python/manifests/3.10.8-slim-bullseye" http.request.useragent="Amazon CloudFront" http.response.contenttype="application/vnd.docker.distribution.manifest.v1+prettyjws" http.response.duration=6.514434ms http.response.status=200 http.response.written=13227

docker-registry-registry-1 | OBFUSCATED_IP - - [22/Feb/2023:08:07:00 +0000\] "GET /v2/python/manifests/3.10.8-slim-bullseye HTTP/1.0" 200 13227 "" "Amazon CloudFront"
```


On the nginx reverse proxy serving my docker registry on the same EC2 instance:
```
OBFUSCATED_IP - - [22/Feb/2023:08:06:59 +0000\] "GET /v2/ HTTP/1.1" 200 2 "-" "Amazon CloudFront"
OBFUSCATED_IP - - [22/Feb/2023:08:07:00 +0000\] "HEAD /v2/python/manifests/3.10.8-slim-bullseye HTTP/1.1" 200 0 "-" "Amazon CloudFront"
OBFUSCATED_IP - - [22/Feb/2023:08:07:00 +0000\] "GET /v2/python/manifests/sha256:6a34910f419a1fdaf2b6d886a1b303eaaa093a828528e0f59b95ccc71feddfa4 HTTP/1.1" 404 182 "-" "Amazon CloudFront"
OBFUSCATED_IP - - [22/Feb/2023:08:07:00 +0000\] "GET /v2/python/manifests/3.10.8-slim-bullseye HTTP/1.1" 200 13227 "-" "Amazon CloudFront"
```
I wonder why this fallback happens when going through cloudfront.
Guessing some header mix up with CloudFront so that my registry cannot find the request digest?


Big thanks in advance for anyone having a clue!

https://redd.it/118t6aq
@r_devops

DevOps: Learning & Development - looking for feedback

My company (start-up, not too big, not too small with 70 tech employees) provides every employee with $2000 in Learning & Development budget.

I have two years of experience as a DevOps Engineer and aim for a senior role in the mid-term.

What L&D resources would you book or recommend?

Fyi: we can spend the budget on every L&D worldwide (books, conferences, E-Learning, coachings, trainings etc.)

https://redd.it/1173qdj
@r_devops

Best tool to deploy a lot of microservices in kuberentes

Hi,

I have a kuberentes running within the cloud (azure) and local (minikube) and I have at the moment around 10 services which I wrote in C# as docker images in a azure docker container.

At the moment I am deploying my services manually with kubectl (local) or kubernetes-task in Azure Devops (Azure kuberentes). It feels very mehhhh because I know the amount of services is fast growing.

Is there a tool to deploy all of that in a "good manner" like with a nice UI or something half automaticly?

https://redd.it/11764g9
@r_devops

Suggest some Devops project

Hello I’m working as intern and I want some Devops advance project with all tools and learn kubernetes in deep because my trainer does not have to much knowledge on kubernetes and my compony does not have any Devops project yet so
Where can I learn improve my self as Devops
Suggest some
Intership currently 3month

https://redd.it/116ckyy
@r_devops

Best options for SonarQube

hello, colleagues. Tell me how you can optimize sonarqube, which is hosted on azure web app (p1v2). Unfortunately, I can't increase the characteristics of the web app because of the money. But I would like to maximize the performance of sonar.

https://redd.it/11671hx
@r_devops

Being underpaid, what should I do?

Hey everyone,

I recently found out that a coworker who has the same level of experience and job responsibilities as me is getting paid 50% more than me. I was shocked and honestly feel pretty upset about it. Especially since my manager mentions I'm doing extremely well.

I always knew I was being paid a bit less than everyone else, but now I realize that I am extremely underpaid. I don't want to cause any trouble, but I also don't want to continue being undervalued and underpaid for the work that I do.

So, I'm wondering if anyone has been in a similar situation before, and what did you do about it? Should I bring it up with my boss or HR? Or should I just accept it and move on?

Any advice or insights would be greatly appreciated.

TLDR; I found out my coworker gets paid 50% more than me. What should I do?

https://redd.it/114lzpb
@r_devops

IaC git repo advice

I've started to use gitlab CI for internal service provisioning for my company's private cloud. Each service consists of templated configuration that is applied to the infrastructure using a mixture of python scripts, ansible, and terraform to configure the end-to-end network, compute, storage, and virtualisation stacks.

For the first service I'm testing with, there is a single repo with the python, ansible and terraform templates and a variable file that the templates reference. The CI pipeline is kicked off manually to begin service provisioning and configuring the infrastructure. The current workflow is for the engineer to create a new branch, update the variables file, commit, and then to run the CI pipeline from the new branch. After the service is provisioned the branch is left for future reference of how the service was provisioned.

I realise this is not best practice as branches should really be used for testing and developing the actual templates and code, not for service instances. What are some suggestions on how I could improve this git CI workflow?

Sticking with one repo - could you create a variable file for each service instance to avoid the per-instance branches? So the engineer would create a new branch, create the new variable file, raise a PR, and once approved/merged to master the CI pipeline would have a method of running the python scripts, ansible, and terraform templates using only the new variable file (as existing services shouldn't be touched)?

https://redd.it/1191vm5
@r_devops

Is it important to know java/python in Devops Engineer role?

Lot of people suggest or lot of Devops courses have a programming language as a tool in Devops journey. As a real practitioner, do you think it is important, if so where do you use it in your day to day job?

https://redd.it/1191bkw
@r_devops

Kubernetes. Rolling updates but for several resources, not only Pods.

So we know that Kubernetes can provide rolling updates, slowly replacing the older deployed components with new ones through a variety of controllers. Deployments, Statefulsets, Daemonsets and so on.

What if I'd like to do the same for a collection of resources, and not only Pods ?

Small example: A deployment with ConfigMaps and Secrets. I'd like to have a gradual deployment that replaces the old Pods and the old config and the old secrets, this while keeping old pods, old config and old secrets there so that the old pods can still use those until they are replaced.

I could script this no problem, but I don't see (yet) how can I declaratively do this either with kubernetes manifests or Helm. Hence the question.

For me it would be like, say there are 3 pods that use also config and secrets.

* T-X: a way to say "up to the moment labeled T-X"
* T-4: 3 old pods, with old config and old secret
* T-3: Deploy and do a rolling update on the entire package (Pod, Config, Secret).
* T-2: 1 new pod with new config and new secret, 2 old pods with old config and old secret
* T-1: 2 new pods with new config and new secret, 1 old pod with old config and old secret
* T-0: 3 new pods with new config and new secret, 0 old pods with old config and old secret
* T+1: remove old config and old secret.

Of course this is only an example, I could have a package involving also services, PVC, and other resources. Thus a subset of the deployed Kubernetes objects.

What are the Kube resources to achieve that, if any? (again I could script this no problem)

https://redd.it/1193kvs
@r_devops

How do you ensure you've understood the task at hand?

Starting a new job soon and I want to set a good impression from the get go (don't we all). One the things I've noticed when picking up a task is not understanding the task or not having enough info, or both. So my usual recourse is to get on call with someone to get the info I need.

Something I've noticed is I sometimes listen to the person explaining something but don't follow along. Quite often I've come off of calls and still clueless. This sounds silly but I've often thought asking basic or too many questions will make me look like I don't know what I'm doing. When getting on call with someone for help, are there any things you do if you don't understand what the other person is saying/explaining. The worst thing for me is coming a way from a call completely clueless just to save face.

Solutions I can think of is:
- Spend some time researching before a call (this isn't always possible).
- Repeating back to the other person to ensure I've understood it.

Tldr: Basically, I'm afraid of asking dumb questions and sometimes suffer in silence.

https://redd.it/11966b8
@r_devops

Save on AWS by deleting untagged ECR images

Learn how unused/untagged ECR images can rack up your AWS bill unnecessarily and how to get rid of them

https://iasql.com/blog/ecr-save/

https://redd.it/1198ako
@r_devops

Do you have self-managed DBs (on a VM instead of a paid service) and which team does manage them?

Hey there,

in my current company I am part of the team who manages a few huger monolithic databases in addition to a few for microservices.

The bigger ones are around 3 proxies, 1 main, around 8 replicas each.

My question is:

If you do use self-hosted DBs, are you having a team of classical DBAs for this or more a team of infrastructure engineers which should also be able to code and write interfaces for teams to access for example dev DBs?

https://redd.it/119awnt
@r_devops

Noob Directories Question

So we have one resource that is formatted like this,

%w(dir1 dir2 dir3 dir4).each do |directory|
directory "#{node'foo''bar'}/#{directory}" do
owner cookbookowner
group cookbookowner
mode 0777
action :create
end
end


I know this is looping through and creating directories dir1, dir2, dir3, and dir4. However, I noticed that when I inspect these directories in the test instance, some of the directories have files in them? I can't seem to find any resources that creates the aforementioned files in certain directories so how do these files exist? Shouldn't they be all empty directories

https://redd.it/119bl5k
@r_devops