What's your favorite lightweight monitoring stack?

Prometheus feels a bit heavy for small projects. Any go-to minimal setups you like?

https://redd.it/1ksm95r
@r_devops

What would be a better middleware solution or tool we can use?

We are looking for a middleware solution or tool that connects to a server HTTP/WebSocket hosted in our AWS cloud and continuously streams real-time event/log data.
This middleware is hosted in the client’s cloud, has no public IP, and cannot be accessed outside. But it can access our system as ours is publicly accessible. It pulls data from our network.

So problem is we need a solution/tool that we can use that will ensure all data is being pulled/listened, processed (yes we need to process and post to other endpoint in client network), also we need a monitoring for that to view the data in/posted to that solution for better visibility.

https://redd.it/1ksn1o5
@r_devops

Calling Cloud/Cybersecurity Pros: Help My Thesis on Zero Trust Architectures

Hi everyone,

I'm conducting academic research for my thesis on zero trust architectures in cloud security within large enterprises and I need your help!

If you work in cybersecurity or cloud security at a large enterprise, please consider taking a few minutes to complete my survey. Your insights are incredibly valuable for my data collection and your participation would be greatly appreciated.

https://forms.gle/pftNfoPTTDjrBbZf9

Thank you so much for your time and contribution!

https://redd.it/1ksmsew
@r_devops

Loki giving a "Get - deadline exceeded" error

I have a containerized grafana monitoring stack with Grafana Alloy and Loki working over a tailnet, when I curl to https://mytailnet/loki/ready It works and I get a 200 OK message. However, when I try to use POST to loki, I get a 404 page not found, and the loki docker logs contain "caller=mock.go:150 msg=Get key=collectors/compactor wait_index=779

caller=mock.go:186 msg="Get - deadline exceeded" key=collectors/scheduler

caller=mock.go:150 msg=Get key=collectors/scheduler wait_index=781

caller=mock.go:186 msg="Get - deadline exceeded" key=collectors/ring

caller=mock.go:150 msg=Get key=collectors/ring wait_index=780

caller=mock.go:186 msg="Get - deadline exceeded" key=collectors/distributor" can anybody help?

My loki.yaml is

auth_enabled: false  # Enable in production!



server:

  http_listen_address: 0.0.0.0  # e.g., 100.101.102.103

  http_listen_port: 3100

  grpc_listen_port: 9096

  http_server_idle_timeout: 40m

  http_server_read_timeout: 20m

  http_server_write_timeout: 20m

  log_level: debug

common:

  path_prefix: /loki-data

  storage:

filesystem:

chunks_directory: /loki-data/chunks

rules_directory: /loki-data/rules

  replication_factor: 1

  ring:

instance_addr: 127.0.0.1

kvstore:

store: inmemory

limits_config:

  allow_structured_metadata: false

schema_config:

  configs:

\- from: 2025-05-16

store: tsdb

object_store: filesystem

schema: v13

index:

prefix: index_

period: 24h

\#querier:

\#  engine:

\#    timeout: 15m

\#  max_concurrent: 512

\#  query_timeout: 5m

ingester:

  wal: 

  enabled: true

dir: /loki/wal

storage_config:

  tsdb_shipper:

active_index_directory: /loki-data/tsdb-index

cache_location: /loki-data/tsdb-cache

https://redd.it/1ksoh00
@r_devops

Why doesn't crt.sh show the latest Let's Encrypt cert under the base domain?

I noticed that when I query:
https://crt.sh/?q=DOMAIN.COM&exclude=expired&output=json
…it doesn’t include the latest certificate I just renewed via Let's Encrypt.

However, when I directly query the full subdomain, like:
https://crt.sh/?q=api.test.DOMAIN.COM&output=json
…the new cert (and its corresponding precertificate) appear immediately.

For example, the base domain query returns 4 entries, but the subdomain one returns 6 — the two extra entries are the new precert and the issued cert.

Is there a way to query the base domain and receive all subdomain certs (including the latest) without knowing every subdomain in advance?

https://redd.it/1kssjb1
@r_devops

I'm building an audit-ready logging layer for LLM apps, and I need your help!

**What?**

SDK to wrap your OpenAI/Claude/Grok/etc client; auto-masks PII/ePHI, hashes + chains each prompt/response and writes to an immutable ledger with evidence packs for auditors.

**Why?**

**-** HIPAA §164.312(b) now expects tamper-evident audit logs *and* redaction of PHI before storage.

\- FINRA Notice 24-09 explicitly calls out “immutable AI-generated communications.”

\- EU AI Act – Article 13 forces high-risk systems to provide traceability of every prompt/response pair.

Most LLM stacks were built for velocity, not evidence. If “show me an untampered history of every AI interaction” makes you sweat, you’re in my target user group.

**What I need from you**

Got horror stories about:

* masking latency blowing up your RPS?
* auditors frowning at “we keep logs in Splunk, trust us”?
* juggling WORM buckets, retention rules, or Bitcoin anchor scripts?

**DM me** (or drop a comment) with the mess you’re dealing with. I’m lining up a handful of design-partner shops - no hard sell, just want raw pain points.

https://redd.it/1ksuxb3
@r_devops

Rant - Companies are getting more and more entitled about job interviews

Did a quick recruiter screening Monday and a more technical interview on Tuesday and it went well so for the next "round" they sent me a 70 page document outlying an "assessment" that they want me to do before going further.

Requires me to set up an AWS account and provision a bunch of resources that don't fall under the free tier. Wtf? I asked them if they could just create an account for me to use, or if I can just create a local environment that mimics the AWS stuff as close as possible, they said no because part of the evaluation is how familiar I am with AWS. Like ok I'm familiar but I'm not trying to pay for a job interview.

I read over most of the documentation and the whole thing conservatively would take about 2 days to complete (accounting for you know... my actual life). I could probably do it all in one day if neglected all other responsibilities I have.

They gave me a deadline for Tuesday "to give me some time over the weekend." Whelp, Monday is a bank holiday and my family and I planned a vacation months ago (technically decades ago because we've been doing this same trip every year since I was a baby). We fly out early tomorrow morning and come back Monday night and today is mostly running last minute errands and driving about 3hrs to my cousin's house for the night because they live 20mins from the airport and our flight is at 6am and we're all on the same flight.

I got this assignment today at 10am.

I emailed them and politely explained the situation and that it's not going to work for me. Haven't heard back yet but I'm probably just gonna tell them I'm not interested anymore. This job market is exhausting.

https://redd.it/1kswd6v
@r_devops

When the CI pipeline breaks and the team asks, Did you change anything?

You could’ve sworn you didn’t touch anything. You check the logs. Nope. The error message just says, “undefined something something.” You sit there, staring at the screen like a confused raccoon in headlights. Meanwhile, your coworkers are asking if you broke it. Spoiler: You didn’t, but now it’s your problem. Welcome to DevOps!

https://redd.it/1ksww8t
@r_devops

To Flag or Not to Flag? — Second-guessing the feature-flag hype after a month of vendor deep-dives

Hey r/devops (and any friendly lurkers from r/programming & r/softwarearchitecture),

I just finished a (supposed-to-be) quick spike for my team: evaluate which feature-flag/remote-config platform we should standardise on. I kicked the tyres on:

LaunchDarkly
Unleash (self-hosted)
Flagsmith
ConfigCat
[Split.io](https://Split.io)
Statsig
Firebase Remote Config (for our mobile crew)
AWS AppConfig (because… AWS 🤷‍♂️)

# What I love

Kill-switches instead of 3 a.m. hot-fixes
Gradual rollouts / A–B testing baked in
“Turn it on for the marketing team only” sanity
Potential to separate deploy from release (ship dark code, flip later)

# Where my paranoia kicks in



|Pain point|Why I’m twitchy|
|:-|:-|
|Dashboards ≠ Git|We’re a Git-first shop: every change—infra, app code, even docs—flows through PRs. Our CI/CD pipelines run 24×7 and every merge fires audits, tests, and notifications.   Vendor UIs bypass that flow.  You can flip a flag at 5 p.m. Friday and it never shows up in git log or triggers the pipeline.  Now we have two sources of truth, two audit trails, and zero blame granularity.|
|Environment drift|Staging flags copied to prod flags = two diverging JSONs nobody notices until Friday deploy.|
|UI toggles can create untested combos|QA ran “A on + B off”; PM flips B on in prod → unknown state.|
|Write-scope API tokens in every CI job|A leaked token could flip prod for every customer. (LD & friends recommend SDK_KEY everywhere.)|
|Latency & data residency|Some vendors evaluate in the client library, some round-trip to their edge. EU lawyers glare at US PoPs. (DPO = Data Protection Officer, our internal privacy watchdog.)|
|Stale flag debt|Incumbent tools warn, but cleanup is still manual diff-hunting in code. (Zombie flags, anyone?)|
|Rich config is “JSON strings”|Vendors technically let you return arbitrary JSON blobs, but they store it as a string field in the UI—no schema validation, no type safety, and big blobs bloat mobile bundles. Each dev has to parse & validate by hand.|
|No dynamic code|Need a 10-line rule? Either deploy a separate Cloudflare Worker or bake logic into every SDK.|
|Pricing surprises|“$0.20 per 1 M requests” looks cheap—until 1 M rps on Black Friday. Seat-based plans = licence math hell.|



# Am I over-paranoid?

Are these pain points legit show-stoppers, or just “paper cuts you learn to live with”?
How do you folks handle drift + audit + cleanup in the real world?
Anyone moved from dashboard-centric flags to a Git-ops workflow (e.g., custom tool, OpenFeature, home-grown YAML)?  Regrets?
For the EU crowd—did your DPO actually care where flag evaluation happens?

Would love any war stories or “stop worrying and ship the darn flags” pep talks.



Thanks in advance—my team is waiting on a recommendation and I’m stuck between 🚢 and 🛑.

https://redd.it/1kszbs2
@r_devops

Is DevOps ADHD-Friendly work to do

I am php developer and recently I found out that I do not do well having to answer up for 2-3 teams calls. Also I get stressed and feel interogated upon codereviews. I suspect of ADHD and I am considering a career shift (but not yet fully commited).

In my personal projects I noticed I focus on automation and developing releasing rocedures, compared to the actual implementation od code. Therefore I am looking for a devops but the main problem is the same: I do not go well with communication especially on small teams.

So I wonder is this a setback in DevOps, usually most positions are either Cloud Engineer or SRE or a combination od DevOps and require an on-call rotation schedule. Therefore Idk if would be a better choice for me.

What do you reccomend?

https://redd.it/1kt0n33
@r_devops

Next.js deployment with CDKTF

Hi everyone!
I've decided to make "mega" project starter.
And stuck with deployment configuration.

I'm using terraform cdk to create deployment scripts to AWS, GCP and Azure for next.js static site.

Can somebody give some advice / review, am I doing it right or missing something important?

Currently I'm surprised that gcp requires cdn for routing and it's not possible to generate tfstate based on infra.
I can't understand, how to share tfstate without commit in git, what is non-secure.

Here is my [repo\](https://github.com/DrBoria/md-starter), infrastructure stuff lies [here\](https://github.com/DrBoria/md-starter/tree/master/apps/infrastructure)

It should works if you'll just follow the steps from readme.

Thanks a lot!

https://redd.it/1kt3lg7
@r_devops

Is what I’ve been doing devops?

I have been writing a lot of CDK code and maintaining Cloud Formation templates lately, but my background is as a developer. That said, I don’t know anything about maintaining OLAP or AD, nor could I install a drop or a router, nor can I explain if we should use Apache or Nginx, etc. I can write a simple bash script with a lot of help from Google, but that’s about the extent of my skills. Is this what is meant by devops?

https://redd.it/1kt2raf
@r_devops

Why use Travis CI and Circle CI when there's Github Actions?

Many (or most) projects are hosted on Github repositories today. But I still come across many public projects using third party CI like Circle CI or Travis CI.

May I know why? Is it because they were used before GitHub Actions was available, and projects are just sticking to whatever already works?

When should one use a external CI service provider?

https://redd.it/1kt6f0o
@r_devops

Need Career Advice

Hi,
I just completed my second year of college and I'm looking for some career advice. I’m pursuing a Computer Sci degree with a specialization in Cloud Computing, and I'm curious about what kind of role would be fit for me to prepare for. Since this sub has a lot of experienced professionals, I’d really appreciate any insights or advice.

About me:

I’ve built a couple of decent projects (none cloud-related yet)

Currently interning as an SDET-QA intern at a large and well-known product-based company.(I'll try to get cloud experience if I can).


I hope this post fits the sub, apologies if not. Thanks in advance for your time and help!

https://redd.it/1ktajuu
@r_devops

What’s one cloud concept you still find confusing—no matter how many times you’ve learned it?

for me, it’s networking.
VPCs, subnets, route tables, NACLs… I get it on paper, but then I’ll hit some weird issue.

Every time I think I understand it, some subtle edge case reminds me I don’t.

Curious if anyone else has their own “cloud kryptonite.”
Is it IAM? Billing? Containers?
What’s that one concept you keep circling back to over and over?

https://redd.it/1ktb0on
@r_devops

Roast/Review/Suggest

I need to switch to DevOps roles .
Currently only AWS part is left..plz review and add
https://i.postimg.cc/5tyTt4FZ/IMG-20250523-103221.jpg

https://redd.it/1ktcr6n
@r_devops

I’ve worked only in cloud, now got a job managing on-prem. What should I expect?

I’ve been working 100% in the cloud (mostly GCP, a bit of AWS) doing DevOps — Kubernetes, CI/CD, load balancers, secrets, autoscaling, the usual stuff.

But I’ve never touched on-prem seriously. I’m curious what’s it like doing infra on physical servers?

I want to understand the reality, trade-offs, and what skills I might need to adapt. Appreciate your thoughts. Thanks in advance!

https://redd.it/1ktglvy
@r_devops

Burnout (rant)

I just want to get something off my chest, so feel free to judge me if you want.

I recently had a conversation with my manager about my performance at work. Now I acknowledge that my performance has dipped recently as I am dealing with a toddler and a young baby at home, and my sleep has just been wrecked. I did explain to my manager what is going on and that I am working on fixing the issue, but they want to change my work arrangement to come to the office 5 days a week. I am not sure how that will help if the rest of the team don't go there regularly. I am genuinely considering just quitting. Don't get me wrong, I love my job - I have been doing this for more than 15 years - but my God, some managers really lack empathy.

Maybe I should try freelancing and contract work at least clients don't think they own you. Yeah, the pay may be less and it comes with other annoyances but at least you own your time and keep your sovereignty as a human being not a piece of hardware expected to operate at full capacity at all times

Sorry for the rant, just a burnt out fellow devops dad who needed to get this off his chest.

https://redd.it/1kti4gn
@r_devops

AI-DrivenOps Student Seeking Career Advice: Stick to DevOps or Explore More?

Hello everyone,
I recently enrolled in a Computer Science Engineering program with a specialization in AI-DrivenOps. As someone new to this area, I’m eager to understand if this specialization provides strong opportunities for entry-level jobs after graduation.

I would be grateful for your insights on whether this path is sufficient to build a career in DevOps or if gaining prior experience is typically expected. Additionally, I would appreciate any recommendations on what skills, tools, or technologies I should focus on learning right now to enhance my job prospects. If possible, could you kindly suggest reliable resources or websites for building practical DevOps knowledge?

Also, I wonder if it would be wise to simultaneously explore other fields such as full-stack/web development or data science to ensure better job security and wider career options. I sincerely welcome advice from those currently working in the industry or who have recently entered the field. Thank you very much for your time and guidance

https://redd.it/1ktkj3j
@r_devops

My new job just has me reading documentation and taking certification courses

For context, I'm fresh out of college with a ba in computer science and I got this devops position. My knowledge of Linux, kubernetes, RHEL, and Jenkins is pretty low so my mentor / boss is just telling me to do some self-research. For the past 2 weeks I haven't really done anything besides read documentation and take online self learning courses. I don't have much guidance and I've actually just been doing this on my own as they just told me to learn as much as I can.

There is also a production issue going on that's taking up everyone's time so I know everyone's busy but it's all stuff that's way above my head so they're not even bothering to have me on it.

Is this normal for a junior devops engineer or even just software engineer position?

https://redd.it/1ktml6c
@r_devops

Salary transition from Junior to Mid level

Just looking for a bit of advice to what i should realistically aim for, my current salary is around £35000 and for the value i provide want to get £50K. So my question is, is this an unrealistic expectation? If i went somewhere else i don't think i'd have a problem getting it but id ideally like to stay at my current company.

Let me know your thoughts on if this is an outrageous ask im a bit inexperienced in these sorts of salary negotiations so im not sure what to expect so any insight would be appreciated.

https://redd.it/1ktn5e7
@r_devops