Compilation of programming concepts, architecture, design, and study summaries accumulated over 4 years

https://github.com/muhtasimTanmoy/notebook

https://redd.it/1dw6l7g
@r_devops

Gitlab CI: Run job on declined merge request

I'm trying to figure out if it's possible to run a pipeline in Gitlab when a merge request is closed without being merged (i.e. declined or rejected). It doesn't seem like there are any variables that directly determine this, but I'm wondering if there is any way to combine several to get after what I want. Anybody have any ideas?

https://redd.it/1dw8u98
@r_devops

Will I run into issues with Apple Silicon?

Currently I run Linux, macOS Mojave on Intel, Windows, on different machines. I'm thinking of getting an M2 Pro Mac Mini in the near future, but I wonder if I'd have bad luck using it as a development machine. I do mainly web dev and some C/C++ dev. As someone who has worked on lots of various short-term projects that will be untouched for many years and then suddenly I need to run the code again, I wonder if I should be worried. The vibe I get from many "it's fine, don't worry" comments is that I don't need to stress as long as I'm only working on new code - what's the state of all this in 2024?

https://redd.it/1dw9ne3
@r_devops

Python container can't run bash file because it doesn't find it when I run it on Windows, but not on Linux

Had a simple Dockerfile and a simple docker-compose file, I mounted a drive and mapped it to the root of the directory. Then I put an entrypoint.bash file and copied it to a directory inside the container and tried to run it, but when I am on Windows, it doesn't find the file and on Linux it managed to find the file. Is there a reason why that Python container wouldn't work?

https://redd.it/1dw8iyo
@r_devops

How does ArgoCD deploy helm charts?

Hey all,

I'm just starting to work with helm charts and ArgoCD and I noticed something odd. I know ArgoCD deploys our charts successfully to our clusters, but why can't I list the charts it deployed using "helm list -A"? When I or other users try to manually use helm install, I can "list" these installed charts. I assume that because ArgoCD is working on the same charts, I should be able to list it, but clearly I can't. Can someone explain why is that? I'm still a beginner and really curious to know. Thanks.

https://redd.it/1dwbxce
@r_devops

Lists / calendars of virtual DevOps conferences?

Are there any lists or calendars of virtual DevOps conferences, so I can easily work out what might be available for my next expected break in employment (end of contract)?

When I have a break in work (transitioning between companies, layoffs, between contracts), I like to take some of that time to do things that will push my career forward that are also fun for me. Right now, I'm wanting to get some better perspective on the state of the industry more than I'm wanting to build another skill - my resume is already laden with skills. I think I'll get more value from gathering some high-level insight into how people are innovating, what tools are helping them, and how their companies are benefitting from that innovation. I'm thinking a virtual conference might be a good fit.

(an in-person conference is a no-go, my immune system punishes me for being around crowds in ways that would render the value nonexistent)

https://redd.it/1dwdfxg
@r_devops

Does Google Cloud SQL gives you notifications when there are performance issues like slow queries?

Does Google Cloud SQL gives you notifications when there are performance issues like slow queries? I checked the logs, but they don't seem to indicate there is any slow query. Is there a way to look through the logs and find a slow query.

https://redd.it/1dwcpc1
@r_devops

fluxcd ‘git reset’ support ?

Hello guys,
K8S environment with fluxcd + gitlab CE

after running these command:

git reset --hard commit_sha256
git push origin ocloud --force

gitrepository goes to READY FALSE with these error:

failed to checkout and determine revision: unable to clone 'https://xxx.xxx.xxx/xxx-xxx-xxx.git': object not found

024-07-05T21:07:10.792Z error GitRepository/flux-system.flux-system - failed to checkout and determine revision: unable to clone 'https://xxx.xxx.xxx/xxx-xxx-xxx.git': object not found failed to checkout and determine revision: unable to clone 'https://xxx.xxx.xxx/xxx-xxx-xxx.git': object not found
2024-07-05T21:07:10.809Z error GitRepository/flux-system.flux-system - Reconciler error failed to checkout and determine revision: unable to clone 'https://xxx.xxx.xxx/xxx-xxx-xxx.git': object not found

I tried running flux uninstall and re-running bootstrap but the problem persists. Gitrepository remains in ready False

I solve the problem if I recreate a branch from scratch and report the Kubernetes configurations
However, recreating the branch causes problems for our workflow.
How can I solve this problem on this particular branch that is not working?

Steps to reproduce

git reset --hard commit_sha256
git push origin branch_name --force

OS / Distro - Ubuntu 22.04

Flux version - flux: v2.0.0

Flux check
► checking prerequisites
✗ flux 2.0.0 <2.3.0 (new version is available, please upgrade)
✔ Kubernetes 1.27.10 >=1.24.0-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.35.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.0.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.0.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.0.0
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta2
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1beta2
✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1
✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta2
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed


Thanks in advance to anyone who can help us!

https://redd.it/1dwonk2
@r_devops

How important is cloud service "fit" for startup jobs?

Startups are especially picky on tech stack fit for traditional SWE roles, even moreso in this job market. A Python company will likely choose a great Python developer over a great (other language) developer.

Is the fit requirement (e.g. GCP company = GCP candidate) equally strict for cloud services?

Thank you!

https://redd.it/1dwrqmo
@r_devops

Best way to manage auto backup for self hosted database.

Hi guys, so I am running a mysql db in a container on a vm. I have a db file which I mounted as volume to the db container. I am not really sure of the best practice to manage auto back up for the db.

I have two ideas in mind, feel free to add to them.

1. Have a cron job that auto backup the db after certain period i.e every 12 hours, then another cron job to delete the first 3(the db is kinda large, so it will consume space if I dont auto delete older backups).
2. A cron just that dumps the db to something like s3 buckets.

Not sure which approach is the best and if there are better approach than this.

Would love your input guys.

Ideally would love an approach that wont consume money, but I am also open to paid options.

https://redd.it/1dwrzxy
@r_devops

free status page service

Greetings, I would like to share a few tools we have developed here. One of these is the status page tools that we offer for free. You can review it in detail on the page. It can be an alternative solution especially for web masters who carefully monitor server downtime problems.


https://robotalp.com/status-page/

https://redd.it/1dwu8va
@r_devops

I graduated in cyber security two years ago. i work help desk. tHE SOFTWARE DEV HERE WANTS TO GET ME INTO DEVOPS WITH HIM. can i learn on the job?

I programmed some in school, so I'm not a stranger to it. I would say I took like 3 programming courses and spent some time outside of class on udemy.

Well I've ended up in a help desk role. But it's more like a jr sysadmin role. I touch networking, firewall, servers, everything pretty much. Even a little security stuff.

The software dev knows I want to get out of tier 1 support roles, so he's planning to have a meeting with my boss to bring me onto his "project team".

My boss is a little wishy washy, they have said I can get the AWS certs and be the system engineer here. I'm not sure how viable that is. The security team here also said I could be in their department after trying to feel it out.

So I haven't programmed for awhile. Once the software dev here has a meeting with boss, and if my boss agrees. I would like a plan to start studyign to be ready for this role.

I want to either be the system engineer here, dev ops here, or join the security department.

https://redd.it/1dwxdc1
@r_devops

How to do deployment in ASG ?


Guys how workflow that must I use to make a deployment in ASG to EC2? Please be honest with real case, I think to use codepipeline, codedeploy. My source code was in bitbucket.

https://redd.it/1dx5her
@r_devops

How do infrastructure team work with operations team?

I am new to a company and they have this setup below. I would like to know if any else has a similar setup and sees problems with it.

We have an infrastructure team that has built infra for internal dev teams to be able to test and run their apps on. To the product teams have a dev to prod infra with the production environment being a customer facing one. The apps we sell are hosted on the infra we build, but we don't sell the infra with the product. So they are a separate thing.

Infra team doesn't have CI/CD, it just works on features and tech debt and it's a bit of a mess.

The operations team decided that, for the production environment, we all need maintenance windows for whenever we do upgrades to the infra (because there was downtime in one upgrade we did).

Question: If we have maintenance plans will that slow us down? How will that affect the implementation of a CI/CD? Isn't the whole point of a pipeline to do seamless continuous releases (upgrades,changes) with minimal interruptions, safely? If we have a pipeline won't that mean that we will have to do maintenance windows every time there is a realease?

https://redd.it/1dx8qsq
@r_devops

How to Manage a Queue in the Cloud for Audio Manipulation? (Zero DevOps Experience)

I'm a bit out of my depth here and could really use some guidance. I've been developing with Flutter for about a year now and have a decent grip on Python, but DevOps is a whole new world for me.

Here’s the setup: I’ve got a Flutter app that uses a Firebase Cloud Function to handle some audio manipulation tasks with Python. The cloud function works, but I’ve hit a few snags:

- Concurrency Limits: Firebase Cloud Functions can only handle 100 requests at a time, which isn't going to cut it once I get more users.
- Efficiency Concerns: Spinning up a cloud function for each request seems pretty wasteful and not exactly scalable.

I’m thinking it might be better to have a permanent machine to handle a queue of these tasks. But, like I said, I’m new to this side of things.

So, here are my questions:

1. Queue Management: What’s the best way to manage a queue in the cloud for something like audio processing?
2. Infrastructure Advice: Should I be looking at a dedicated VM, or maybe something like Kubernetes for this?
3. Tools and Services: Any must-use tools or services for handling queues and scaling efficiently?

Extra Info:
- I’ve got zero users right now, but I want to be ready for when that changes.
- I’m comfortable with Python and open to using it for managing this if it helps.

Any advice or pointers would be super appreciated! Thanks!

https://redd.it/1dx9ou2
@r_devops

hub-spoke architecture

I understand that the hub contains (all?) services, which are meant to be shared and accessed by the spoke workloads.

Where would I place central services, such as ElasticSearch clusters or Prometheus backend, in this architecture? Should they be placed in the hub or should I place them in a dedicated spoke (which would be different to all the remaining spokes, since the remaining spokes are exclusively contained of workloads)?

https://redd.it/1dxab50
@r_devops

I have created a progress tracker app for DevOps Engineering. It breaks down bigger concepts into smaller topics so that you can keep a track of your learning.

I have created a progress tracker app that comes preloaded with a roadmap for DevOps Engineer. You just need to import the roadmap and you are good to go.

It follows a simple concept where you break down a task into sub-steps and track your progress on the project.

Along with the multi-level tracker it comes with different features like Pomodoro Timer, Timetable creator, note-taking and more.

PlayStore - https://play.google.com/store/apps/details?id=app.vinztech.trackit

Apple App Store - https://apps.apple.com/app/trackit-pomodoro-tracker/id6504750191

Please try it our give your valuable feedback.

https://redd.it/1dxcjca
@r_devops

Azure Blocking access to public aks ingress based on ip?


Hi,


In my current project we have an aks cluster which has ingresses managed by nginx-ingress controller. We also have a load balancer with public frontend ip. Nginx-ingress and load balancer are connected through annotations, so the forwarding rules are generated automatically.

Now, we'd like to block the access to that ingress except for select few ip addresses. I've read in this article (and many others and answers on SO) https://medium.com/nerd-for-tech/restricting-access-to-services-deployed-on-aks-using-nginx-ingress-conroller-89291df69036 that setting externalTrafficPolicy to "Local" on kubernetes load balancer service will allow me to keep client's ip address, which I can later use in "nginx.ingres.kubernetes.io/whitelist-source-range".

The issue here is that even after setting that policy every request is still seen as coming from 127.0.0.1 and it doesn't want to show me anything else. I really want to get it to work as the alternative is creating app gateway + waf, which requries so much terraform code.

Does anyone have experience with this?

https://redd.it/1dxd9cx
@r_devops

Operator vs an application for visualizing k8s cluster resources

There already exist application like the k8s dashboard, octant and lens that help us get an overview of the resources on a k8s cluster. However, can we build an operator that can run seamlessly within a k8s environment and expose a web interface on a designated endpoint to visualize all resources . Would it be better or worse idea than the applications themselves?

https://redd.it/1dxdtmj
@r_devops

HR interview was cancelled 5 minutes before joining the call. They said I should expect a rescheduled interview. It’s been a week and I haven’t heard back. Should I bother sending an email? I already did two technical interviews with them.

This is a multinational company who I had two technical interviews with first, and then was scheduled to have an HR interview after where I was told I’d be discussing the salary and other HR matters. So the next week I get an email from their HR scheduling a “screening” interview. In my mind I was questioning the “screening” part, because usually the screening is done before the technical interviews with the hiring manager. Yet, I was having it done the other way around. Weird. Anyways, a week later and 5 mins before this HR interview was set to begin, I received an email saying the interview was cancelled due to overlapping meetings and that I should expect a rescheduled call for the interview.

Well, it’s been a week since then and I haven’t received any emails or calls. Does this likely mean I’ve been ghosted and/or rejected? I felt that the technical interviews went well, and the hiring manager even called my personal phone after the interview ended to try and convince me even more to join the company and that I shouldn’t let this chance go to waste, telling me to send him text messages if I have any other questions. But the last technical interview was 2 weeks ago, and the cancelled HR interview was 1 week ago, and I haven’t heard back from them at all. Should I follow up and send an email, or have I been ghosted and rejected? If the former, what should I say to not come across as desperate? Or should I just wait another week?

https://redd.it/1dxgat1
@r_devops

Looking for a junit report (XML) ingestion solution for insights/historyon test result data

Hello,
I'm looking to dig in further to insights on test executions (retaining historical data), metrics, etc. I know there's paid offerings out there.

Are there anytools/solutions for free out there? Bonus points if it can allow me to push XML JUnit report files as part of CI. I have hundreds of test result artifacts that I would love for it to process and ingest.

https://redd.it/1dxhles
@r_devops