OAuth2 Proxy container triggers auth to all endopints

Hello, I hope there are some oauth2-proxy experts!

I have an issue that when i deploy oauth2-proxy into K8S environement and add Ingress route to prefix it with "/oauth2-proxy" then all my endpoints even "/ping" and "/ready" suddenly start triggereing auth cycle.

Do you have any idea why? Locally in docker i can call ping and ready without being auth, same image same version same settings.

name: oauth2-proxy
image: bitnami/oauth2-proxy:7.7.0
path: /oauth2-proxy
replicas: 1
ports:
- 4180
command: "oauth2-proxy"
args: ["--upstream=https://myhost.com/some-service/", "--http-address=0.0.0.0:4180"]
env:
OAUTH2_PROXY_COOKIE_NAME: "_oauth2_proxy"
OAUTH2_PROXY_CLIENT_ID: "123123"
OAUTH2_PROXY_CLIENT_SECRET: "secret"
OAUTH2_PROXY_COOKIE_SECRET: "secret_cookie"
OAUTH2_PROXY_SESSION_STORE_TYPE: "cookie"
OAUTH2_PROXY_PROVIDER: "keycloak-oidc"
OAUTH2_PROXY_OIDC_ISSUER_URL: "https://login.myhost.com/auth/realms/master"
OAUTH2_PROXY_SCOPE: "openid"
OAUTH2_PROXY_EMAIL_DOMAINS: "*"
OAUTH2_PROXY_CODE_CHALLENGE_METHOD: "S256"
OAUTH2_PROXY_REVERSE_PROXY: "true"
OAUTH2_PROXY_COOKIE_DOMAINS: ".myhost.com"
OAUTH2_PROXY_WHITELIST_DOMAINS: ".myhost.com"
OAUTH2_PROXY_REDIRECT_URL: "https://myhost.com/oauth2-proxy/oauth2/callback"
OAUTH2_PROXY_SKIP_PROVIDER_BUTTON: "true"

Ingress rule

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-oauth2-proxy
namespace: some-namespace
spec:
rules:
- host: myhost.com
http:
paths:
- path: /oauth2-proxy
pathType: Prefix
backend:
service:
name: oauth2-proxy
port:
number: 4180
tls:
- secretName: ssl-certificate

https://redd.it/1gddcu7
@r_devops

Can you help me define if I did an okay job?

Hello, I know this isnt strictly devops, but its something I would like to ask this community. I started a job 2.5 years ago and they wanted me to start a qa automation framework on their apps. One wpf and one web.

All in all, alone, in 2.5 years, I went from nothing to building an automation framework that runs automated tests on a wpf app (with around 85 tests) and a web app (with around 15 tests). I built a jenkins in a docker container from scratch without having done so ever before and built deployment and smoke test pipelines on windows VMs. I really had trouble with a few things, but since nobody had expertise on this, I was alone all the way. I struggled a lot with the way they were compiling their multiple libraries since they hardcoded their library paths in a powershell script and used the same script for both apps. I also struggled with jenkins as the connection to the windows agent wasn't something very intuitive, and the resolution made it so the tests on the web app were failing because it was too small. I had to change the connection method three times.

I wanna know if you think this is a reasonable timeframe or was I way too slow?

https://redd.it/1gdefnh
@r_devops

Finally out: from devops burnout to life in the woods

hey,

After more than a decade in the tech grind, I am finally out. And by "out," i mean no more Kubernetes clusters, CI/CD pipelines, or 2 a.m. incidents. i’ve walked away from devops, and the relief is real.

It all started with a Linux sysadmin role at my local university, juggling cron jobs and small network fixes, earning around 25k€/year. Over time, I climbed the ladder (from small teams to the grand "bigcorp"), where my day-to-day transformed into designing pipelines and managing infra for high-stakes projects. On paper, it was success. I was hitting six figures (not by a great margin, bit still), but in reality, I was on the fast track to burnout.

Despite the paycheck, I was exhausted. everything about the industry felt... relentless. Bosses who didn't get the strain of 24/7 operations, product owners who thought a pipeline redesign was "just a quick tweak," and users who had no idea what really went into keeping things running. Eventually, the grind and demands took their toll.

So, I did something drastic. I started preparing civil service exams for a role as a forest guard. After relentless prep and a ton of doubt, I actually landed a permanent position in my region. Now, I make less than half of what I was in Devops, but I’m out there every day, surrounded by trees instead of dashboards. No customers, no endless Jira tickets, just fresh air and open trails.


I wanted to share this in case any of you are feeling trapped or on the brink of burning out. It is possible to get out and start fresh (even at the age of 35). I can only hope that if you're in a rough spot, you’ll eventually find your own way out.

Cheers!

https://redd.it/1gdf9sr
@r_devops

book or other source to kick off a devops journey

I have multiple teams using a lot of legacy workflows. We need to get the sysadmins and developers on board doing something more modern. Is anyone aware of good sources to start to have people read so we can ease into this and start to get some small wins? deploying anything shouldn't be a series of phone calls between sysadmins and developers doing everything manually

https://redd.it/1gdg45p
@r_devops

Interview for Tech Support Engineer role?

A couple of days ago, a recruiter sent me an invitation for an interview for this role:

AWS Cloud Support Analyst
Assist users with login issues, including password resets and account lockouts.
Verify user credentials and ensure secure access to the system.
Maintain and update login access policies and procedures.
Create and configure new user accounts based on requests.
Ensure proper user roles and permissions are assigned per product management directives and approvals.
Maintain accurate records of account creation and modifications.
Capture detailed information about technical issues reported by users.
Perform basic troubleshooting to identify the nature of the problem.
Document and categorize issues for efficient tracking and resolution.
Forward complex issues to the development team with all relevant details.

I already work with infrastructure operations. I set up monitoring tools likes Grafana dashboards with elasticsearch and prometheus, provide L3 tech support on call to solve production breaking issues and deal with day-to-day technical operations, such as restoring lost passwords to users, helping colleagues with technical issues, making performance presentations for clients, etc.

Reading the job description, this looks more like a L1 or L2 tech support job, which would technically be a downgrade of what I do currently. It's nice that I'll get to work with AWS, which is a highly sought skill in the job market, but going by the descriptions, looks like I'll be limited to only using IAM for user permission management and will not be setting up infrastructure or anything like that.

What you guys think of this role? Will I even learn something that I don't already know?

https://redd.it/1gdi69j
@r_devops

Looking for CTO / co-founder for pre-seed Finance-related AI startup Remote

Hi, I am a non-technical founder (but lack of a CTO is making me more and more technical).

Looking for: a co-founder CTO (or just a CTO) to help design/build/maintain the system, and help develop my startup's demo website.

Startup status: boostrapping a MVP to get first clients and do a SEED round in 1Q25 or 2Q25 (post Revenue).

What we do: AI for stock/investment research (Hedge Funds). My cofounder and I both work in Hedge Funds.

Current team = me + a really non-technical biology PhD (who knows perfectly our "target clients"). My cofounder and I both work in Hedge Funds.

Location: we are both based in the NYC area. Being around here is a + but interactions are expected to be remote (Slack/Discord/Zoom).

More details on tech and on responsibilities: Ideally I would want to keep the backend in Elixir/Phoenix and use LiveView for the website (because that's how it is right now), but this is not a hard request. There's also going to be many other languages/tools involved (rabbitmq queues, databases, python for processing, AI LLMs, etc). In terms of responsibilities, I don't see them differing much. The need is to find someone who can help me with the high-level design of the system (I already have a starting draft that I drew myself )... and with the actual hands-on spinning up of the environment / tech architecture (which in the meanwhile I am doing on my own... I am not the type that will stop/pause or going to be stuck). Ideally, this person should also be responsible for some parts of the system working in a certain way

How I think about the roles: a co-founder CTO should accept a lower (or even zero) pay for a short period of time (until the SEED round is completed) but with a material equity (comparable to other 2 cofounders ) ... while a non-founding CTO would be someone primarily on payroll, with a smaller equity package.

Please DM me with your resume/linkedin if you would like to learn more!

https://redd.it/1gdjj6f
@r_devops

What is your biggest secret trick in devops?

Secret, trick, hack, etc... What knowledge do you keep close to your heart and only share once you know someone for a while.

Obviously looking for some great advice but I am expecting a few "I have automated my whole job and everything I do is scripted".

https://redd.it/1gdlq4f
@r_devops

AWS Account Vending strategies?

Just read Scott Piper’s take on AWS Account Vending vs. Landing Zone strategies...interesting stuff. He argues that account vending is a must as you scale beyond a few dozen accounts. Anyone else agree with this approach? Thoughts on benefits or challenges? How does it fare in practice?

https://redd.it/1gdm3hg
@r_devops

Advice on Integrating DevOps Practices in a Traditional Ops Environment?


Hey all,

I’m working at a large energy utility company. It’s a pretty traditional environment with no real DevOps culture yet, and most workflows are manual and siloed, with few automation practices in place. It’s a challenge, but I’m really interested in finding ways to improve the workflow and bring in some DevOps ideas where I can.

About Me:
I’m in IT operations and mostly focused on monitoring systems—specifically Splunk—and doing a fair bit of scripting in Python and shell to streamline tasks where possible. My day-to-day includes managing monitoring for incidents, pushing updates, and handling deployment and maintenance across a Linux environment. At the moment, a lot of what I do is still manual, like deploying scripts to multiple search heads, and I’ve just started exploring Ansible to speed that up.

Why I’m Here:
Since DevOps isn’t something widely practiced here, I’d love to learn what daily tasks and routines you all typically handle in more DevOps-oriented roles. Any insights on tools or frameworks you’ve found useful for automating repetitive tasks in similar setups would be great. And if you have any advice on introducing DevOps principles gradually into a traditional ops environment, I’m all ears.

Thanks a lot for any tips or experiences you can share—hoping to pick up some ideas to bring back and try out here!

https://redd.it/1gdriey
@r_devops

New to devops, unable to learn anything just because there is so much to learn

Switched to devops a year ago. And now I feel I am kinda stuck in terms of knowledge. There is so much to learn that my daily work requires like GCP, kubernetes, Jenkins, terraform, istio, argo etc

I learned jenkins and all and that was super easy

But I realized that even though 12 months have passed, I am fairly BEGINNER in kubernetes. Unable to increase my technical knowledge, probably a productivity issue not sure. Tried few courses but left all in between.


Any tips for a noobie here?

https://redd.it/1gdruze
@r_devops

Is infra team's whole job just running migrations?

I've so run many migrations in my career. This year I think I'm basically just running migrations.. no feature work at all.

* raw terraform to standardized terraform module to managed platform and migrate back and forth in between these options
* cloud migration: this is probably the only migration in my opinion that's worth the work.
* logging platforms, data warehouses : done so many of these migrations in my career even in startup

I wrote down some thoughts [here](https://jarrid.xyz/articles/2024-10-24-never-ending-migrations-path-to-platform-adoption) that most migrations are probably not worth it. I think there's easier ways to do it but we somehow don't really explore it. Curious about people's experience and thoughts on this. Is organic adoption hard because we we build very bad toolings or it's simply too slow and we just end up doing migration. At the same time, I can't imagine any engineering teams are "excited" by migrations.

https://redd.it/1gdv7u7
@r_devops

What are some of the governance tools you use and are you happy with it?

I am looking for tools specifically in Security, Cost, Performance, Reliability and Operation side.

https://redd.it/1gdvz5i
@r_devops

Is anyone feeding their server or application logs to AI?

Either using some paid service, burning API credits or self hosting an LLM? I'm about to start experimenting.

https://redd.it/1gdxo8y
@r_devops

Serverless vs Serverful

Hi all,

Novice full-stack dev here. I need your opinion regarding the tech stack + deployment of a greenfield, multi-tenant web app for which I have 2 interested customers (payment plan pending) whose pain points are resolved, with hope to have many in the future but not more than 10k users globally.

My initial impulse is to have zero deployment costs, with a dockerized monolith backend (hosted on an always-free Oracle cloud VM), an Angular frontend hosted per Netlight / Cloudflare, and database hosted on Supabase. The reasoning is that “if” I’ll have an increased demand, I’ll simply scale these services vertically, and maybe even go cloud-native in the future.

Competing with this thought are my AWS cloud skills from work, which push me to going completely serverless and using managed services to speed up development and not think about infra scaling and security down the line. However, if I do it right, with API GW, WAF, etc. I’ll incur costs from the get go (even with free tier) without having seen a single payment from the customer(s).

In your experience, which option would you recommend in such scenarios? Would you recommend I disregard the minimal costs from AWS and go cloud-first to prevent future headaches when I’m focusing on delivering features / adapting business logic, or should I experiment with all-free services to wait until I have enough customers that support putting in effort/costs to go cloud-native (given that all code needs to be refactored / changed anyway)?

The application needs a REST API to perform CRUD operations on multiple related tables in a PostgreSQL DB, and start many task queue operations per user.

https://redd.it/1ge11ux
@r_devops

the ui testing debate hasn't gone anywhere

still so hard to decide what the best framework is for ui-testing. here are my learnings.

https://redd.it/1ge30x5
@r_devops

GitOps Setup: Security Concerns with Automated Deployments

**Current Setup**:I have a straightforward but powerful GitOps workflow that consists of these steps:

1. Developer pushes code to source repository (GitLab.com)
2. CI/CD pipeline builds Docker image
3. Automated pipeline updates image tag in application repository
4. ArgoCD detects change and deploys new version



**Problem**: While this setup works well, there are security concerns. External developers with access to one repository can theoretically manipulate image tags of other repositories by triggering the pipeline with different values. There's a lack of granular access control for the deployment process.



**Planned Solution**: I'm planning to develop a open-source service to address this security gap. Here's the current design:

API Endpoint for New Image Tags: Route: /api/v1/new-image-tag Parameters needed:

* image\_tag (e.g., 1234567890)
* branch (e.g., main, develop, or feature/my-cool-feature-1)
* secret\_key (from GitLab CI variables)
* repository (e.g., gitlab.com/my-application/frontend)

The service will:

1. Validate permissions using secret key and repository
2. Determine environment from branch name
3. Create feature-branch specific values files automatically
4. Update image tag in values-$STAGE.yaml

API Endpoint for Closed Merge Requests: Route: /api/v1/closed-mr Parameters needed:

* mr\_id (e.g., 45)
* secret\_key (from GitLab CI variables)
* repository (e.g., gitlab.com/my-application/frontend)

The service will:

1. Get branch name from merge request
2. Check for open merge requests in other repos with same branch name
3. Clean up feature branch configuration if no other MRs exist
4. Allow ArgoCD to remove obsolete deployments



**Questions for the community**:

1. Are there existing tools that solve this security issue?
2. What are the best practices for securely handling image tag updates in a GitOps setup?
3. Would developing a new open source solution for this be valuable?

I'm particularly interested in solutions that maintain the simplicity of GitOps while adding proper security controls. Before starting development on a new tool, I want to make sure I'm not reinventing the wheel. Any insights or suggestions would be greatly appreciated.

https://redd.it/1ge0t5g
@r_devops

Anyone using Harness?

How’s it treating so far?

https://developer.harness.io

I saw this app in a CNCF demo and interested to see what it offers.

So browsing here to get feedback from the community

https://redd.it/1ge7i7x
@r_devops

Should we migrate our IaaC from Terraform to OpenTofu and deployment using Terragrunt with Terramate?

We manage all of infrastructure using Terraform only and because of this we have really big Terraform stacks even tho using modules we end up having 3000 lines in main.tf due to so many services and resources.
1. One issue we faced was, whenever we try to deploy the TF using Mac, we get some drift in the plan, but that is not the case in linux or windows machines, not sure of the file handling is different or some other issue,
2. Second issue we faced was that sometimes when planning we see some drift on DB resources and for production it really scares us like why there is showing changes in DB resources even theo all i did is just changed the values for computer resources,

For first problem we moved to gitops and do all the deployment through aws CodePipeline only, for second issue we decided to use terragrunt since it breaks the stacks and due to the structure, we can use singe repo to store multi region and multi environment deployment with less code and bette file structure, but in terragrunt we don't see change detection, for this we need to ise Terramate and tbh there's very less resources available for the same online, so I'm little worried if should we move out production IaaC to Terragrunt with Terramate and migrate from Terraform to OpenTofu?

If any one of you have done something similar, can you please share your experience considering these are somewhat in early stage, not sure how much these tools has become mature.

Please suggest,
Thanks!

https://redd.it/1ge8mwq
@r_devops

Looking for advice

Looking for a bit of advice


Hi all, I am a junior DevOps engineer I have been a DevOps for 3 years and my skills are mostly comprised of Terraform, AWS and GitLab.


Bit of background: I have a degree in Maths so this is my first experience in IT.
I have learned everything pretty much on the job, and in order to learn I used a variety of resources and some certifications (yes they are not everything but, for me certs are a good way of structural learning ).

Currently I have :
- Terraform Associate
- AWS SA Associate
- AWS DV Associate
- AWS Sysops Associate

We don’t have many cloud requests now at my company and I have been presented an opportunity to join a different project that is focused in Linux automation, Containers and Ansible pretty much to be a Sysadmin.

I am torn because:

- I think I have come along way and my AWS and Terraform knowledge is very good.

- I also feel like I don’t have much Linux knowledge, I can google stuff and find solutions but I always feel like I don’t have enough operational Linux knowledge to be a DevOps altho I have barely used Linux on my work.

- Changing environments is not as easy specially when I have to basically learn everything again.

- My ultimate goal is to be a Cloud architect, and I don’t want to go to far from this path (Although I have been assured that I will still be involved with AWS Projects).

Basically I’m just wondering for Seniors or more experienced DevOps engineer, if you were starting again and you were faced with the same situation how would you go about it?

https://redd.it/1gebsbb
@r_devops

Is devops/IT all doom and gloom?

I've been researching getting into the field of IT/devops and will have the 3 basic CompTIA courses in 3 months time and also starting a homelab with specific devops related projects.

I've read so many comments and posts of the industry going down and no jobs, is this genuinely the case and I'm wasting my time starting all this or is there still a future in the industry of course with the right work and effort? I'm based in the UK

https://redd.it/1ged01o
@r_devops

Live Coding for Interviews in DevOps roles

I've been requested for live coding for a DevOps role. I've been told I can use whatever tools I can for presenting my skills, but I'm confused as to what can I present in 30m/1h of an interview? I'm good with python if they want code but I think that DevOps requires more templates and architecture planning etc... Since the challenge is very open I think it has been hard for me to think of a case where I can show my skills as a DevOps because the daily job envolves a lot of troubleshooting and configuring? Do you guys have any tips for mastering this kind of interview? Thanks.

https://redd.it/1geiron
@r_devops