How to test serverless apps like AWS Lambda Functions

We have Data syncing pipeline from Postgres(AWS Aurora ) to AWS Opensearch via Debezium (cdc ) -> kakfa ( MSK ) -> AWS Lambda -> AWS Opensearch.



We have some complex logic in Lambda which is written in python. It contains multiple functions and connects to AWS services like Postgres ( AWS Aurora ) , AWS opensearch , Kafka ( MSK ). Right now whenever we update the code of lambda function , we reupload it again. We want to do unit and integration testing for this lambda code. But we are new to testing serverless applications.



On an overview, I have got to know that we can do the testing in local by mocking the other AWS services used in the code. Emulators are an option but they might not be up to date and differ from actual production environment .



Is there any better way or process to unit and integration test these lambda functions ? Any suggestions would be helpful

https://redd.it/1mi7doj
@r_devops

We spent weeks debugging a Kubernetes issue that ended up being a “default” config

Sometimes the enemy is not complexity… it’s the defaults.

Spent 3 weeks chasing a weird DNS failure in our staging Kubernetes environment. Metrics were fine, pods healthy, logs clean. But some internal services randomly failed to resolve names.

Guess what? The root cause: kube-dns had a low CPU limit set by default, and under moderate load it silently choked. No alerts. No logs. Just random resolution failures.

Lesson: always check what’s “default” before assuming it's sane. Kubernetes gives you power, but it also assumes you know what you’re doing.

Anyone else lost weeks to a dumb default config?

https://redd.it/1mi8xp6
@r_devops

I started posting on youtube and an unexpected outcome has been some of the easiest interviews of my career

https://youtu.be/mE8Xmkk\_qSw

https://redd.it/1mi6jdo
@r_devops

Give me a real-structured-roadmap for devops

So i know like basic mern and I am in my 4th year and kindaa realising slowly how fc up is sde and developer role so thinking to quietly shift towards the devops role .

I need like a roadmap through which i can easily learn it in like 2-3 months

I am hardworking and got time .

Help me PLEASE!

https://redd.it/1mic1ae
@r_devops

Anyone build dev self service around terraform atmos?

We are redoing our terraform across our services by firstly creating centralized terraform modules (instead of the copy paste we have today).

I wanted to take it one step further and introduce atmos to further abstract the terraform away as yaml, and then maybe build some sort of a self-service utility or something which generates that yaml and a PR depending on what infrastructure the developer needs.

Is anyone doing something similar?

Thanks.

https://redd.it/1mib9xg
@r_devops

Sam Lambert (PlanetScale) is an interesting guy

Didn't expect to like his answers, but he talks about the free tier being deleted and some other interesting stuff

https://www.youtube.com/watch?v=BzKUm2pJchI

https://redd.it/1miidg4
@r_devops

How do you all handle automatic version increments? (dev vs release)

Our company uses github and has Branch Protection enabled across all of our organizations, enterprise wide. Branch Protection is a new requirement, so the old versioning flow is broken. I've inherited a legacy python application and I'm feeling REALLY stupid this morning for some reason.

Previously, jenkins would kick off a release.sh script which would (in addition to lots of other stuff) hit "bumpversion" (strips .dev from version for the release), push to master, and then hit bumpversion to increment to .dev. With BP enabled, this is no longer a reasonable work flow, so I need to come up with a workaround.

I'd prefer not to do the versioning manually, but if I must, I must.

How have you all tackled semver increments during releases? I could write a custom app that would bump the release version, automatically create a new PR for master, then bump it back to .dev, wherein I'd have to go approve the PR, but that seems like overkill for some reason.

https://redd.it/1mijq97
@r_devops

Introducing the latest release of the tAI tool

It quickly helps you getting commands you don't remember for your daily work.

https://github.com/bjarneo/tAI

https://redd.it/1mijfmq
@r_devops

What makes devs happy

Curious, what keeps devs motivated and excited? Some devs aren’t as performant as others.

https://redd.it/1miumva
@r_devops

High latency serving my HF model on Kubernetes (NVIDIA T4)

To be honest, I dont know where else to shoot to issue.

First of all my infra:

3 CPU Workers - N4-Standard-4 ( 4vcpus/16gb )
1 GPU Worker - Nvidia T4 ( 4vcpus /16gb)

I’m running three microservices, grounding, API layer, and RabbitMQ management, and a “GPU consumer” service backed by an NVIDIA T4.

• On dedicated VMs I see about 1.5 seconds round-trip per request, but when I move everything into Kubernetes it never drops below 2.5 seconds.
• I’ve already tried co-locating the API and inference containers in a single pod with hostNetwork but the latency stays the same.
• There is no CPU limitations or Memory Limitations ( pods barely will reach 40/50% )
• on the first API call the GPU Consumer will load up the Model, which takes around 8/10 second to get a response back ( expected ), then it gets stable at 2.5

This happens on self-hosted k3s on GCP VMS or GKE.

This is more or less how it looks

Client > API > RabbitMQ > Grounding Consumer > RabbitMQ > GPU Consumer

Batch processing works wonders, since we dont care about latency at all, but stw it seems impossible.

Thx!

https://redd.it/1mix0oi
@r_devops

DevOps Freelancing in Europe or US

I am curious to know what the market looks like currently for freelancing in the field of DevOps in Europe, especially Germany.

https://redd.it/1mix9wb
@r_devops

I have a university grad intern starting next month, need some good starter resources.

Hey r/devops. Platform engineering manager here. I have a computer science post grad intern starting next next month and they’re a little wet behind the ears with practical experience.

I really want to give them a good head start and some useful resources, preferably on YouTube. Full disclosure I’m a grizzled old Gen-X who uses YT occasionally.

Does anyone have any good channels that introduce devops concepts to starters? I need to cover CI/CD, IAC, Cloud technologies observability etc.

Thank you in advance!


https://redd.it/1miyty7
@r_devops

What to do when a project fails?

Hey, everyone!

I've be‌en in IT fo‌r over 5 y‌ears, someti‌mes backe‌nd, sometim‌es f‌ree‌la‌‌nc‌e.‌ My ex‌peri‌ence varies from sim‌ple la‌nding p‌ages to ‌comp‌lex finte‌ch sys‌tems. A‌nd rece‌ntly, I thou‌ght I h‌ad l‌anded m‌y dre‌am pro‌ject: a cl‌ient fro‌m th‌e U‌S, a‌n ide‌a for an‌ AI-bas‌ed educ‌ational platf‌orm, g‌ood b‌udgets, an‌d a lo‌ng t‌erm. I‌ sign‌ed u‌p, assembl‌ed my ‌te‌am‌, an‌d w‌e confide‌ntly g‌ot st‌arted

W‌e wor‌k‌ed fo‌r o‌ver thre‌e mon‌ths, puttin‌g o‌ur hear‌ts int‌o it, b‌ut it wa‌sn't all smo‌oth‌ s‌ail‌ing‌ - th‌e docu‌mentation wa‌‌s w‌eak, th‌e too‌ls w‌ere ‌ra‌w, an‌d we ‌of‌ten ‌h‌ad t‌o imp‌rovise. Th‌en th‌e‌ c‌lient star‌ted respon‌ding m‌ore sl‌owly, an‌d ev‌entually ju‌st wr‌ote:

“W‌e ha‌ve de‌cided to p‌ut th‌e‌ pro‌ject on h‌old. T‌hank yo‌u” LO‌L

Thr‌ee mon‌ths of w‌ork - do‌wn t‌he dra‌in. Almos‌t n‌o pa‌yment, th‌e team‌ was un‌motivated, a‌nd I wa‌s broke

B‌ut I di‌dn't lo‌se eve‌rything, beca‌use I've be‌en inter‌ested in c‌‌‌r‌‌‌yp‌‌t‌‌o‌ f‌or a lo‌ng ti‌me. I'v‌e bee‌n fo‌llowing var‌ious c‌ommuniti‌es sinc‌e 2021. ‌And‌ rec‌ently, I cam‌e ac‌ross a post b‌y a ‌guy o‌n R‌eddit [link\], a‌nd dec‌ided t‌o gi‌ve it a try. I‌ man‌aged to r‌ise a littzle c‌‌‌as‌‌‌h, j‌u‌st e‌nough t‌o rent an apar‌tm‌ent 😅 Ve‌ry tim‌ely, br‌o

Th‌is ex‌perience tau‌ght m‌e a l‌ot - bot‌h tech‌nically a‌nd m‌entally. N‌ow I a‌m mor‌e cautiou‌s ab‌out long‌ proj‌ects with‌out clea‌r gua‌rantees an‌d tr‌y to a‌llocate r‌esourc‌es be‌tter, and I ad‌vise you to d‌o th‌e sa‌me

https://redd.it/1mizyob
@r_devops

Looking for a mentor or co founder ... We have a certain product that we want to distribute direct to customers (D2C)..

We want to create a distribution channel online through our own website.. but i am researching about how to create a good reliable website or app ... We are 2 founders of this product that we can't disclose yet .
How do you hire someone to get this done in efficient manner without burning too much capital .
Is there a way to outsource them . Does the website or app need constant backsupport.. what's the general process in creating such streamlined system.
Anyone who knows about stuff ...can you explain it in the comments for now .

Would really help us get the nepali perspective on this .

https://redd.it/1miwocc
@r_devops

KubeDiagrams 0.5.0 is out!

**KubeDiagrams** 0.5.0 is out! **KubeDiagrams**, an open source Apache 2.0 License project hosted on GitHub, is a tool to generate Kubernetes architecture diagrams from Kubernetes manifest files, kustomization files, Helm charts, helmfile descriptors, and actual cluster state. **KubeDiagrams** supports most of all Kubernetes built-in resources, any custom resources, namespace, label and annotation-based resource clustering, and declarative custom diagrams. This new release provides many improvements and is available as a Python package in PyPI, a container image in DockerHub, a kubectl plugin, a Nix flake, and a GitHub Action.

Try it on your own Kubernetes manifests, Helm charts, helmfiles, and actual cluster state!

https://redd.it/1mj1t4p
@r_devops

Every startup wants "DevOps", until they realize what it actually takes

I’ve lost count of how many early-stage teams want CI/CD, infra-as-code, multi-env setups, monitoring, rollback, zero-downtime deploys… all before even having stable revenue.

And they assign it to a solo dev or junior engineer as a “side task”.

Meanwhile:

No one owns infra debt.
No budget for proper tooling.

Everyone wants “just one more feature” instead of paying infra tech debt.

When something breaks in prod, it’s magically “DevOps’ fault”.

DevOps is not a checkbox.
It’s a long-term investment that touches culture, workflows, and team maturity.

You either take it seriously, or you're just writing TODOs that'll bite you in 3AM alerts later.

https://redd.it/1mj39oo
@r_devops

Medical Student Looking to Transition into Data Science

I'm 25 years old and currently a second-semester medical student at a public university.

I've always been passionate about technology, especially data science, cybersecurity, and bioinformatics, but my ENEM (Brazilian university entrance exam) score allowed me to get into a federal university in the south for medicine, so I decided to go for it.

Now, I'm considering switching programs at the same university to data science, with the goal of eventually working in cybersecurity or bioinformatics. I’ve already started the bureaucratic process for the transfer.

I know data science isn’t the perfect degree for cybersecurity, for example, but it offers a wide range of career opportunities.

For those working in this field: What are your thoughts on the job market?

https://redd.it/1mj4mwv
@r_devops

Is there a chance for me as an IT Help desk with no programming experience

Long story short i have bsc in information technology & worked as an it helpdesk for 6 months (have experience with Microsoft Active directory & Domain controller / configuration of Cisco switches / opnsense and pfsense firewall / some Linux CLI experience / used to mess with aws ec2 as well)

my issue is I forgot most if not all of my programming knowledge

I am now unemployed (yea cuz I had enough of why my MoUsE is not working or my printer is jammed) and I really don't want to stay stuck in the loop, my original goal was to get a job as a network engineer but even if i had a ccnp certification it's so fucking hard to even land an interview let alone getting accepted

https://redd.it/1mj6z8j
@r_devops

Tired of switching branches manually in multiple git repos (with submodules)? I built a simple tool for that!

Hi all,
Recently, I was working on a project where I had to manage several git repositories at once, many of them with submodules.
Every time I needed to switch to a specific release branch (for example, when deploying or integrating), I had to go into each repo, checkout the right branch, make sure the submodules were on the same branch, and sometimes I forgot one and things broke… It was a huge headache, especially when using tools like Helm and mounting multiple environments.

So, I decided to build a simple open-source GUI tool in Python (with Tkinter) to handle this pain point:

Scan a folder and find all git repos (including submodules)
See the current branch of each repo
Switch branches in all repos with a single click
Bulk set a target branch for all repos
Autocomplete branch selection
“Pull All” support

It’s cross-platform (Linux & macOS executables available), open-source, and works out of the box — no install required!
I hope it can save you the frustration it saved me.

GitHub link:
https://github.com/orinu/multi-repo-git-manager

Give it a spin if you think it’ll help you too!

https://redd.it/1mj7qva
@r_devops

Kotlin DSL enhancements in TeamCity

JetBrains shared an update about enhancements in Kotlin DSL support, for a bunch of recent major TeamCity releases.

It covers not syntax changes for particular features, but improvements which may make the life of admin easier/more effective.

What would you actually prefer for configuration, YAML or a real programming language like Kotlin? Any TeamCity users here to share their opinion?


https://redd.it/1mj9bcv
@r_devops

Using grafana beyla distributed traces on aks

Hi,

I am trying to build a solution for traces in my aks cluster. I already have tempo for storing traces and alloy as a collector. I wanted to deploy grafana beyla and leverage its distributed traces feature(I am using config as described here https://grafana.com/docs/beyla/latest/distributed-traces) to collect traces without changing any application code.

The problem is that no matter what I do, I never get a trace that would include span in both nginx ingress controller and my .net app, nor do I see any spans informing me about calls that my app makes to a storage account on azure.

In the logs I see info

"found incompatible linux kernel, disabling trace information parsing"

so this makes think that it's actually impossible, but
1. This is classsified as info, not error.

2. It's hard to believe that azure would have such an outdated kernel.

So I am still clinging on to hope. Other than that logs don't contain anything useful. Does anyone have experience with using beyla distributed tracing? Are there any free to use alternatives that you'd recommend? Any help would be appreciated.

https://redd.it/1mjb8es
@r_devops