How does ArgoCD deploy helm charts?

Hey all,

I'm just starting to work with helm charts and ArgoCD and I noticed something odd. I know ArgoCD deploys our charts successfully to our clusters, but why can't I list the charts it deployed using "helm list -A"? When I or other users try to manually use helm install, I can "list" these installed charts. I assume that because ArgoCD is working on the same charts, I should be able to list it, but clearly I can't. Can someone explain why is that? I'm still a beginner and really curious to know. Thanks.

https://redd.it/1dwbxce
@r_devops

Lists / calendars of virtual DevOps conferences?

Are there any lists or calendars of virtual DevOps conferences, so I can easily work out what might be available for my next expected break in employment (end of contract)?

When I have a break in work (transitioning between companies, layoffs, between contracts), I like to take some of that time to do things that will push my career forward that are also fun for me. Right now, I'm wanting to get some better perspective on the state of the industry more than I'm wanting to build another skill - my resume is already laden with skills. I think I'll get more value from gathering some high-level insight into how people are innovating, what tools are helping them, and how their companies are benefitting from that innovation. I'm thinking a virtual conference might be a good fit.

(an in-person conference is a no-go, my immune system punishes me for being around crowds in ways that would render the value nonexistent)

https://redd.it/1dwdfxg
@r_devops

Does Google Cloud SQL gives you notifications when there are performance issues like slow queries?

Does Google Cloud SQL gives you notifications when there are performance issues like slow queries? I checked the logs, but they don't seem to indicate there is any slow query. Is there a way to look through the logs and find a slow query.

https://redd.it/1dwcpc1
@r_devops

fluxcd ‘git reset’ support ?

Hello guys,
K8S environment with fluxcd + gitlab CE

after running these command:

git reset --hard commit_sha256
git push origin ocloud --force

gitrepository goes to READY FALSE with these error:

failed to checkout and determine revision: unable to clone 'https://xxx.xxx.xxx/xxx-xxx-xxx.git': object not found

024-07-05T21:07:10.792Z error GitRepository/flux-system.flux-system - failed to checkout and determine revision: unable to clone 'https://xxx.xxx.xxx/xxx-xxx-xxx.git': object not found failed to checkout and determine revision: unable to clone 'https://xxx.xxx.xxx/xxx-xxx-xxx.git': object not found
2024-07-05T21:07:10.809Z error GitRepository/flux-system.flux-system - Reconciler error failed to checkout and determine revision: unable to clone 'https://xxx.xxx.xxx/xxx-xxx-xxx.git': object not found

I tried running flux uninstall and re-running bootstrap but the problem persists. Gitrepository remains in ready False

I solve the problem if I recreate a branch from scratch and report the Kubernetes configurations
However, recreating the branch causes problems for our workflow.
How can I solve this problem on this particular branch that is not working?

Steps to reproduce

git reset --hard commit_sha256
git push origin branch_name --force

OS / Distro - Ubuntu 22.04

Flux version - flux: v2.0.0

Flux check
► checking prerequisites
✗ flux 2.0.0 <2.3.0 (new version is available, please upgrade)
✔ Kubernetes 1.27.10 >=1.24.0-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.35.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.0.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.0.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.0.0
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta2
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1beta2
✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1
✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta2
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed


Thanks in advance to anyone who can help us!

https://redd.it/1dwonk2
@r_devops

How important is cloud service "fit" for startup jobs?

Startups are especially picky on tech stack fit for traditional SWE roles, even moreso in this job market. A Python company will likely choose a great Python developer over a great (other language) developer.

Is the fit requirement (e.g. GCP company = GCP candidate) equally strict for cloud services?

Thank you!

https://redd.it/1dwrqmo
@r_devops

Best way to manage auto backup for self hosted database.

Hi guys, so I am running a mysql db in a container on a vm. I have a db file which I mounted as volume to the db container. I am not really sure of the best practice to manage auto back up for the db.

I have two ideas in mind, feel free to add to them.

1. Have a cron job that auto backup the db after certain period i.e every 12 hours, then another cron job to delete the first 3(the db is kinda large, so it will consume space if I dont auto delete older backups).
2. A cron just that dumps the db to something like s3 buckets.

Not sure which approach is the best and if there are better approach than this.

Would love your input guys.

Ideally would love an approach that wont consume money, but I am also open to paid options.

https://redd.it/1dwrzxy
@r_devops

free status page service

Greetings, I would like to share a few tools we have developed here. One of these is the status page tools that we offer for free. You can review it in detail on the page. It can be an alternative solution especially for web masters who carefully monitor server downtime problems.


https://robotalp.com/status-page/

https://redd.it/1dwu8va
@r_devops

I graduated in cyber security two years ago. i work help desk. tHE SOFTWARE DEV HERE WANTS TO GET ME INTO DEVOPS WITH HIM. can i learn on the job?

I programmed some in school, so I'm not a stranger to it. I would say I took like 3 programming courses and spent some time outside of class on udemy.

Well I've ended up in a help desk role. But it's more like a jr sysadmin role. I touch networking, firewall, servers, everything pretty much. Even a little security stuff.

The software dev knows I want to get out of tier 1 support roles, so he's planning to have a meeting with my boss to bring me onto his "project team".

My boss is a little wishy washy, they have said I can get the AWS certs and be the system engineer here. I'm not sure how viable that is. The security team here also said I could be in their department after trying to feel it out.

So I haven't programmed for awhile. Once the software dev here has a meeting with boss, and if my boss agrees. I would like a plan to start studyign to be ready for this role.

I want to either be the system engineer here, dev ops here, or join the security department.

https://redd.it/1dwxdc1
@r_devops

How to do deployment in ASG ?


Guys how workflow that must I use to make a deployment in ASG to EC2? Please be honest with real case, I think to use codepipeline, codedeploy. My source code was in bitbucket.

https://redd.it/1dx5her
@r_devops

How do infrastructure team work with operations team?

I am new to a company and they have this setup below. I would like to know if any else has a similar setup and sees problems with it.

We have an infrastructure team that has built infra for internal dev teams to be able to test and run their apps on. To the product teams have a dev to prod infra with the production environment being a customer facing one. The apps we sell are hosted on the infra we build, but we don't sell the infra with the product. So they are a separate thing.

Infra team doesn't have CI/CD, it just works on features and tech debt and it's a bit of a mess.

The operations team decided that, for the production environment, we all need maintenance windows for whenever we do upgrades to the infra (because there was downtime in one upgrade we did).

Question: If we have maintenance plans will that slow us down? How will that affect the implementation of a CI/CD? Isn't the whole point of a pipeline to do seamless continuous releases (upgrades,changes) with minimal interruptions, safely? If we have a pipeline won't that mean that we will have to do maintenance windows every time there is a realease?

https://redd.it/1dx8qsq
@r_devops

How to Manage a Queue in the Cloud for Audio Manipulation? (Zero DevOps Experience)

I'm a bit out of my depth here and could really use some guidance. I've been developing with Flutter for about a year now and have a decent grip on Python, but DevOps is a whole new world for me.

Here’s the setup: I’ve got a Flutter app that uses a Firebase Cloud Function to handle some audio manipulation tasks with Python. The cloud function works, but I’ve hit a few snags:

- Concurrency Limits: Firebase Cloud Functions can only handle 100 requests at a time, which isn't going to cut it once I get more users.
- Efficiency Concerns: Spinning up a cloud function for each request seems pretty wasteful and not exactly scalable.

I’m thinking it might be better to have a permanent machine to handle a queue of these tasks. But, like I said, I’m new to this side of things.

So, here are my questions:

1. Queue Management: What’s the best way to manage a queue in the cloud for something like audio processing?
2. Infrastructure Advice: Should I be looking at a dedicated VM, or maybe something like Kubernetes for this?
3. Tools and Services: Any must-use tools or services for handling queues and scaling efficiently?

Extra Info:
- I’ve got zero users right now, but I want to be ready for when that changes.
- I’m comfortable with Python and open to using it for managing this if it helps.

Any advice or pointers would be super appreciated! Thanks!

https://redd.it/1dx9ou2
@r_devops

hub-spoke architecture

I understand that the hub contains (all?) services, which are meant to be shared and accessed by the spoke workloads.

Where would I place central services, such as ElasticSearch clusters or Prometheus backend, in this architecture? Should they be placed in the hub or should I place them in a dedicated spoke (which would be different to all the remaining spokes, since the remaining spokes are exclusively contained of workloads)?

https://redd.it/1dxab50
@r_devops

I have created a progress tracker app for DevOps Engineering. It breaks down bigger concepts into smaller topics so that you can keep a track of your learning.

I have created a progress tracker app that comes preloaded with a roadmap for DevOps Engineer. You just need to import the roadmap and you are good to go.

It follows a simple concept where you break down a task into sub-steps and track your progress on the project.

Along with the multi-level tracker it comes with different features like Pomodoro Timer, Timetable creator, note-taking and more.

PlayStore - https://play.google.com/store/apps/details?id=app.vinztech.trackit

Apple App Store - https://apps.apple.com/app/trackit-pomodoro-tracker/id6504750191

Please try it our give your valuable feedback.

https://redd.it/1dxcjca
@r_devops

Azure Blocking access to public aks ingress based on ip?


Hi,


In my current project we have an aks cluster which has ingresses managed by nginx-ingress controller. We also have a load balancer with public frontend ip. Nginx-ingress and load balancer are connected through annotations, so the forwarding rules are generated automatically.

Now, we'd like to block the access to that ingress except for select few ip addresses. I've read in this article (and many others and answers on SO) https://medium.com/nerd-for-tech/restricting-access-to-services-deployed-on-aks-using-nginx-ingress-conroller-89291df69036 that setting externalTrafficPolicy to "Local" on kubernetes load balancer service will allow me to keep client's ip address, which I can later use in "nginx.ingres.kubernetes.io/whitelist-source-range".

The issue here is that even after setting that policy every request is still seen as coming from 127.0.0.1 and it doesn't want to show me anything else. I really want to get it to work as the alternative is creating app gateway + waf, which requries so much terraform code.

Does anyone have experience with this?

https://redd.it/1dxd9cx
@r_devops

Operator vs an application for visualizing k8s cluster resources

There already exist application like the k8s dashboard, octant and lens that help us get an overview of the resources on a k8s cluster. However, can we build an operator that can run seamlessly within a k8s environment and expose a web interface on a designated endpoint to visualize all resources . Would it be better or worse idea than the applications themselves?

https://redd.it/1dxdtmj
@r_devops

HR interview was cancelled 5 minutes before joining the call. They said I should expect a rescheduled interview. It’s been a week and I haven’t heard back. Should I bother sending an email? I already did two technical interviews with them.

This is a multinational company who I had two technical interviews with first, and then was scheduled to have an HR interview after where I was told I’d be discussing the salary and other HR matters. So the next week I get an email from their HR scheduling a “screening” interview. In my mind I was questioning the “screening” part, because usually the screening is done before the technical interviews with the hiring manager. Yet, I was having it done the other way around. Weird. Anyways, a week later and 5 mins before this HR interview was set to begin, I received an email saying the interview was cancelled due to overlapping meetings and that I should expect a rescheduled call for the interview.

Well, it’s been a week since then and I haven’t received any emails or calls. Does this likely mean I’ve been ghosted and/or rejected? I felt that the technical interviews went well, and the hiring manager even called my personal phone after the interview ended to try and convince me even more to join the company and that I shouldn’t let this chance go to waste, telling me to send him text messages if I have any other questions. But the last technical interview was 2 weeks ago, and the cancelled HR interview was 1 week ago, and I haven’t heard back from them at all. Should I follow up and send an email, or have I been ghosted and rejected? If the former, what should I say to not come across as desperate? Or should I just wait another week?

https://redd.it/1dxgat1
@r_devops

Looking for a junit report (XML) ingestion solution for insights/historyon test result data

Hello,
I'm looking to dig in further to insights on test executions (retaining historical data), metrics, etc. I know there's paid offerings out there.

Are there anytools/solutions for free out there? Bonus points if it can allow me to push XML JUnit report files as part of CI. I have hundreds of test result artifacts that I would love for it to process and ingest.

https://redd.it/1dxhles
@r_devops

How do I avoid convoluted Prometheus/Alertmanager rules in my setup?

I have pods that need different thresholds for the alert rules. For example:

- alert: highmemoryusageresourcelimits
expr: >
(
sum by (container, pod, namespace, env) (containermemoryworkingsetbytes{namespace="unbiased"})
)
/
(
sum by (container, pod, namespace, env) (kubepodcontainerresourcelimits{resource="memory"})
) > 0.85
for: 3m
labels:
severity: warning
annotations:
summary: "High memory usage on {{{{ $labels.pod }}}}"

Some of my pods would need a different threshold. Currently what I'm doing:

1. Add not <pod name> to the original rule
2. Clone the rule and set the specific pod name I need to monitor with a different threshold.

From a quick Google search I saw that I can use templating/relabeling. Is there a good strategy that I can learn from to understand how to apply it in my environment?

I'm pretty new to Prometheus in general so any tips on this subject would be appreciated.

https://redd.it/1dxgdis
@r_devops

The OSI Model - When do you guys think about it?

So I've been doing research between the differences of load balancers & ingress controllers.

Feel free to add but:

>They are both ways of managing traffic ( This is the heart of what they do )

>Ingress controllers have more features than basic load balancers, allowing the SSL certificate to float within them, this lets them perform the SSL/TLS termination, important as HTTP means less in the headers therefore faster routing later on ( Does anyone have a number? )

>In technical terms, load balancers operate on Layer 4, and Ingress controllers operate on Layer 7.

This is the first time I've encountered the OSI model in around 6 months! What do you guys use it for, does it help visualise infrastructure!?

Because I've used it so little, knowing that load balancers operate on L4 vs Ingress's L7 does little to help me understand, therefore shooting this into the subreddit!

Thanks

https://redd.it/1dxkyl4
@r_devops

How do you handle database deployments?

Hi! I’m running a relatively small architecture for a Django application. I’m standing up all cloud resources via terraform, including a managed database that my app relies on.

At the moment, one terraform apply creates every resource in the cloud. DNS, Load Balancing, server, database etc…

I’m thinking of removing my database out of the terraform apply and instead manual build that resource via CLI rather than including it in the terraform action. My reasoning here is that I dont want to constantly be deleting a database only for me to upload the same data again. As I’m building my application, I’m constantly terraform destroying resources and building everything up again to catch dependencies.

Curious to how others are deploying their databases.

https://redd.it/1dxqdfp
@r_devops

mTLS using a forward proxy?

I have an application that needs to connect to postgresql, using a binary protocol over TCP.

If the postgresql server requires mTLS from the client, is it possible to use HAProxy as a forward proxy to handle the secure TCP TLS connection to postgresql? Thus freeing the application from the obligation to learn how to use mTLS? Is it possible to use Nginx as a forward proxy? For Nginx I believe the TCP forwarding can be done with the stream {} block, but I haven't seen an example where mTLS is used -- I have done this for HTTP traffic. Similarly for HAProxy, I have seen configurations with TCP forwarding only.

APP --- (TCP plain) ---> HAProxy OR Nginx --- (TCP with mTLS) ---> PostgreSQL

https://redd.it/1dxu42m
@r_devops