Staying or Leaving as "person working there the longest"

I am an engineer somewhere between junior and mid-level and have been with my team for roughly three years. Throughout this time, my team has gone through a series of layoffs, organization restructures, and people leaving which significantly cut our tenured team members. I am now the only IC who is currently working on our team's platform and has worked on it for longer than a year.

I am concerned about being "the person who has been there the longest" as it results in me being the main point-of-contact for our platform despite having more-senior team members (but are relatively newer to our team).

One specific concern is the expectation to spend non-trivial amounts of time on mentorship and knowledge transfer rather than solely on business deliverables and my own professional development. I have shared these opinions to my team and attempted to handoff my knowledge as best I can, but I definitely still feel the pressure to be the "subject matter expert" when our team needs one.

On one hand, I am getting experience in mentorship/soft skills and I get to see how our platform evolves in the long-term, which seems useful as I progress to senior. However, I am worried that this is stunting my technical growth as an IC since I dedicate time to upskilling others rather than focusing on my own technical skills.

I would really appreciate some feedback on my views of this situation. I am unsure whether it is best to stay or leave in my role/company for my long-term best interest.

https://redd.it/1hx1cdj
@r_devops

Can we give custom response and status code when lambda is throttling.

I'm calling lambda through API gateway(lambda proxy integration), but want to give some custom response when lambda is throttling.

Any possible ways? Through api gateway or anything?

https://redd.it/1hx2m21
@r_devops

Grafana vs Datadog: Cost Comparisons

Recently published blog post with quality examples on Grafana vs Datadog cost considerations. I know a lot of people here have spoken about the high price of Datadog and I was surprised to see how things tilted in favor of Grafana.

Thoughts?

https://www.vantage.sh/blog/datadog-vs-grafana-cost

https://redd.it/1hx2hzc
@r_devops

Become a Data Engineer in 2025 - Based on 100 Jobs data!

Happy New Year, everyone! Reposting a combination of 2 of my most upvoted posts last year at the start of the year for those looking to set ambitious career goals in 2025 assuming lot of new people are looking for this info now. After all, there’s no better time to plan your next big leap into Data Engineering!

Top skills in demand -

I analyzed 100 data engineering job descriptions from Fortune 500 companies to find the most frequently mentioned skills. Here are the top skills in demand:

|Skill Group|Frequency|Constituents with Frequency|
|:-|:-|:-|
|Programming Languages|196|SQL (85), Python (76), Scala (21), Java (14)|
|ETL and Data Pipeline|136|ETL (65), Pipeline (46), Integration (25)|
|Cloud Platforms|85|AWS (45), Azure (26), GCP (14)|
|Data Modeling and Warehousing|83|Data Modeling (40), Warehousing (22), Architecture (21)|
|Big Data Tools|67|Spark (40), Big Data Tools (19), Hadoop (8)|
|DevOps, Version Control and CI/CD|52|Git (14), CI/CD (13), Jenkins (7), Version Control (7), Terraform (6)|
|Data Quality and Governance|42|Data Quality (20), Data Governance (13), Data Validation (9)|
|Data Visualization|23|Data Visualization (11), Tableau (6), Power BI (6)|
|Collaboration and Communication|18|Communication (10), Collaboration (8)|
|API and Microservices|11|API (8), Microservices (3)|
|Machine Learning|10|Machine Learning (7), MLOps (2), AI/ML Model Development (1)|

➡️ Excel Sheet with data - https://docs.google.com/spreadsheets/d/1zB6wocrgxNgjWwo6Jkezje0SgJ3PXMIoCEyJwdY-nLU/edit?usp=sharing

➡️ Checkout the full video with explanation of tasks - https://youtu.be/XzqYdCov-LA

4 Month Study Plan -

Month 1: Foundations

DBMS & SQL: Basics of database concepts, querying, and design.
Python: Focus on Python essentials, including libraries like Pandas and NumPy.
Linux: Basic commands and navigation.
DSA: Data structures and algorithms, especially for big tech roles.

Month 2: Key Concepts & Tools

Data Concepts: Topics such as Data Lake, Data Mart, Fabric, and Mesh.
Data Governance: Management, security, and ethics in data.
Spark: Introductory concepts with Apache Spark.
Distributed Systems: Overview of Hadoop, Hive, and MPP systems.
Cloud Services: Options such as AWS, GCP, or Azure.

Month 3: Advanced Topics

Orchestration: Basics of workflow orchestration with tools like Apache Airflow.
Compute: Databricks, Snowflake, or equivalents like AWS EMR.
Containers: Introduction to Docker and Kubernetes.
CI/CD: Tools such as Jenkins and SonarQube.
Streaming: Fundamentals of Kafka.
ETL/ELT: Tools like dbt and Talend, along with architecture basics.
Terraform: Code-based infrastructure setup.

Month 4: Projects & Portfolio

Build a project portfolio to showcase skills. Examples include:

Bank Data Warehouse
Fraud Detection ETL
Reddit Review Tracker
Retail Analytics
Trip Data Transformation
YouTube Clone

Certifications

AWS Certifications: Cloud Practitioner, Solutions Architect Associate, Data Engineer Associate
Databricks: Data Engineer Associate
Apache Airflow: Airflow Fundamentals

Showcase Your Work

Document projects on GitHub, post on LinkedIn, and network with target companies.

➡️ Full detailed roadmap - https://youtu.be/5b4CIon\_1pY

➡️ For best certifications checkout: https://youtu.be/cfBoR6VY6Rw

Hope you found this helpful, I will try my best to add value to you this year as well, let's kill it fam! 🤝

https://redd.it/1hx3s1w
@r_devops

Building in public - just added Built-In Analytics to Rollout—For Free!

Hey everyone! I have another update to share!

I am Super excited to announce an exciting feature in Rollout—**Privacy-First Analytics** that’s built directly into the platform

💡 **Why This is a Big Dea**l

* **Out-of-the-Box**: Analytics is included for **free** in Rollout—no setup, no additional cost!
* **Better than Competitors**:
* Netlify charges for analytics (yikes).
* Vercel offers analytics, but with **limited free access**.
* Cloudflare’s analytics can be hard to set up and overly complex.
* **Simplicity**: Whether you’re a freelancer, agency, or developer, you’ll get all the insights you need with just a few clicks.
* **Privacy First**: No trackers, no cookies—your data stays private and secure.

🔍 **What Can You Track**?

* Pageviews
* Sessions
* Bandwidth
* Events, and much more.

Rollout continues to focus on **speed, simplicity, and transparency**, and this feature is just another step in that direction.

Try it out today: [https://app.rollout.sh](https://app.rollout.sh/)

Got thoughts or feedback? We’d love to hear from you!

https://redd.it/1hx4v5i
@r_devops

Configuring logrotation for gunicorn running as a systemd service

Folks, I want to configure log rotation for my django gunicorn log files which is running as a custom systemd service on the server, could not find any appropriate solution yet. Seeking help to guide.

https://redd.it/1hx7i4p
@r_devops

How you handled gitops with kubernetes, i use argocd , but i need a tool (not helm ui) that will help me manage the charts on github (team of 4) so collabaration feature also could help

I

https://redd.it/1hx9lf0
@r_devops

I need help :(

I've been working as a Junior DevOps Engineer for 2 years.
I have some experience with Azure, GitLab, and Azure DevOps, but most of the other tools and technologies I’ve learned on my own. I haven’t had the chance to apply them in real-world projects yet :(.

I want to change jobs because my current salary is low (I’m from Argentina). I’d like to work on personal projects or get hands-on experience to improve my skills and grow professionally.

What would you recommend to win practice? Can I find a new job with my current level of experience?

https://redd.it/1hxdnps
@r_devops

How to go about learning and implementing Devops as a backend developer?

Hi, I am a backend developer(student) and am trying to upskill by learning devops. I have recently got a server that I would like to host and work on.
Coming from backend dev, I have a decent-ish grip on the menial server tasks, I would like to learn more about Devops and so how should I go about learning?
I prefer books, so are there any books that explains Devops theory?
What all technologies should I learn to be able to operate my server and also have professional relevance?



https://redd.it/1hxcvsv
@r_devops

Strategies for Containerized development environments?

What tools or strategies have you found most effective for streamlining containerized development environments? I'm curious how others have tackled the challenges mentioned in this blog about improving dev workflows and reducing build times.

Or on the flip side, if you're a container hater, I'd love to know why.

https://redd.it/1hxhel1
@r_devops

Quickwit Joins Datadog

https://quickwit.io/blog/quickwit-joins-datadog

https://redd.it/1hxjgtg
@r_devops

Does anyone here market infrastructure and cloud templates?

I've been in the DevOps space for many years and have worked with many cloud and "DevOps" MSPs over the years, all the while looking at the application side of the equation and noticing how that has evolved seemingly quite differently. All the cloud MSPs I have worked with were very hesitant to use shared frameworks and develop reusable artifacts between projects because their business model was selling time. I've also seem a lot of SaaS offerings spring up. But when I compare that to the application space I notice a thriving market of templates, themes, plugins, etc... So I was just wondering from other experienced DevOps folks here is this is a thing in any circles, because I would think given we try for hyper automation and infrastructure as code, templates would be a perfect balance between fully custom and uncustomizable SaaS.

https://redd.it/1hxkpm6
@r_devops

Implementing LoadBalancer services on Cluster API KubeVirt clusters using Cloud Provider KubeVirt

Hi everyone!
I wrote an article about configuring Kubernetes LoadBalancer services on Cluster API managed KubeVirt clusters with Cloud Provider KubeVirt.

This is the first article in a series I'm starting about taking Kubernetes clusters from where the Cluster API documentation leaves you to GitOps managed production clusters.
The next article in the series will be about configuring workloads on Cluster API managed clusters with Argo CD.

In my opinion the most interesting part of the article hides in
the linked Helm chart configuring a cluster with centralized telemetry exporter, secret management and more.
I use the chart with an Argo CD ApplicationSet for configuring clusters in GitOps style.

I am very much a beginner in technical writing, and would appreciate any feedback you have.

https://redd.it/1hxkjrh
@r_devops

Ephemeral environment for open merge requests on azure with microservices architecture

Hello Everyone,
I am new to DevOps and I want to create a pipeline on azure, that create a deployment when a merge request is created/updated and destroy it when it's closed.
I'm seeking help with any hint or resources that I can read from, and would also appreciate your opinions if that's doable, knowing that currently frontend and backend are on different git repositories, but I can consider bringing both under one repo.
Thanks in advance

https://redd.it/1hxlojy
@r_devops

Recent Interview Experience

So today I had an interview for an Ops Engr role at a company. Going through the job description I felt the requirements aligned well with my background - the JD mentioned the role of an Ops Engineer as someone who would be installing, updating and configuring products.

I have good knowledge on Infrastructure as Code (IaC) and the infrastructure provisioning tools like Terraform and configuration management tool like Ansible. Apart from that I also have high level knowledge on modern devops tools and platforms like docker for containerization and orchestration tools like Kubernetes.

Today as I said I had my interview. While introducing myself when I pointed out that I know all those stuff I was interrupted by one of the interviewers who went on to inform me that since they deal with legacy systems they are yet to adopt all those devops practices and that they are mostly involved in manual maintenance of applications. So, there is little to no automation being used in the process.


Then they went on to grill me on core linux concepts, some linux commands although I did mention that I was familiar with file system and networking linux commands only. I was asked about different linux distributions, about how to schedule processes using linux. Then some qs related to networking were also asked - the basic ones like OSI model, TCP/IP protocol, DNS. I was asked about Ipv4 and ipv6. Unfortunately, I could not recall the difference between ipv4 and ipv6. Until this moment the interview was going fine - the questions were of quite basic level.


Then one of the interviewers asked me to explain how to respond to an incident of spike in CPU usage. I was able to explain him a few steps but he wasn't quite satisfied with the answer and asked me to explain him the steps in a sequential manner. And then there were a few questions on how to respond to a feedback from end user on production related issue and so on...

Honestly I was a bit disappointed at the end of the interview as I was hoping I would be asked questions on containerization, on cloud platforms and on different tools like Terraform and Ansible.



https://redd.it/1hxo1d6
@r_devops

I'd like to transition my small web app which uses docker-compose to kubernetes. My friend tells me it's a full time job/too much overhead. Thoughts?

My expertise is as a full stack Django/React developer. Through Udemy + testdriven.io courses and some grit, I got my backend running last year on a DigitalOcean droplet and managed Postgres db. It works great and I will likely keep it this way for another year.

I would like to learn kubernetes over the next year and transition my app over for these reasons:

1. Downtime. I haven't had much traffic so its been fine to manually upload new builds to ghcr + deploy it + ssh into my droplet and run the migration but I want to minimize that

2. I just want to understand kubernetes. I will eventually hire someone to do this full time (when my business takes off, I'm an optimist!), but since I'm a bit curious/a control freak, the idea of not knowing how to debug my own web application/core business is scary to me

3. If my servers are getting battered or I want to replicate my app to different regions, I'd like to know how to actually scale the pods

My buddy is a professional DevOps developer and he says it's a bad idea, that I'd likely be spending all my time doing DevOps stuff while I should be working on my core business. He specifically mentions how you constantly have to update to new versions of kubernetes. But I also wonder if his experience is from working at big companies.

When I read the threads here a lot of it is over my head. Helm charts, provisioning, different flavors of k8s, Ansible, I've heard a lot of these terms but it seems like a lot. That said, I know a lot of you work at companies with SLAs that require 99.9+% uptime and do traffic I can't even fathom, so maybe I'm psyching myself out for no reason?

This is getting long, so if kubernetizing my app is a bad idea, could anyone recommend a more intermediary approach?

Thanks in advance!

https://redd.it/1hxpxfq
@r_devops

Resume Review for DevOps/Cloud Engineer Positions (Mid)

Hi everyone,

I’ve been updating my resume to improve my chances of securing a DevOps Engineer or Cloud Engineer role and would really appreciate feedback from others in the field.

Unfortunately, most of my friends find the technical details on my resume a bit hard to understand, so I’m hoping someone with relevant experience could offer some advice.

I have 3 years work experience but I've been getting rejected at screening for roles that it seems like I should quite easily qualify for.

Here are a couple of specific areas where I could use some input:

Am I effectively communicating my skills and previous experience in a way that’s clear and engaging for recruiters or hiring managers?
Does the overall layout and structure work well?

Thanks in advance for your help!


Here's my resume: https://imgur.com/a/QhGA8j8

https://redd.it/1hxgtzt
@r_devops

How to Enable Swap in EKS

Hi all,
I just published a quick guide on enabling swap in EKS. check it out

https://medium.com/@eliran89c/how-to-enable-swap-in-your-eks-cluster-in-under-5-minutes-b87524cc821b


https://redd.it/1hxydvz
@r_devops

How to transfer free app to domain name?

I have an app hosted on free pythonanywhere account. I now I also own a domain name via goDaddy. How to link that domain name to my site?

https://redd.it/1hy02sv
@r_devops

Self-Hosted Drone CI with issues.

Greetings everyone.

I am trying to setup a selfhosted CI/CD setup.

Development server that is running Drone CI in Docker is running on Ubuntu 24.04.1 LTS.

Currently i have a Drone CI in a docker container (both server and runner), then i have a Docker Private Registry on a seperate server.

Once a push is sent to Github, it will activate a webhook which starts the Drone CI to work.

Been tinkering with this a few days now, tried various solutions.

In short, i want to be able to push my code to Github, webhook is called and my local development server with Drone CI is activated, where it pulls the code, caches the dependencies for backend and frontend, runs the unit tests and such, security checks and then pushes the image to private registry which are used to spin up the development site.

Been having issues with caching part where it doesn't actually store it in the cache folder.
Also been having issues with when Drone-Runner trying to push the image to the Private Registry suddenly stalling and retrying over and over but not always.

Here is the .drone.yml :

kind: pipeline
type: docker
name: default

steps:
  # Version 0.1
  # Generate Cache Key
  - name: generate-cache-key
    image: alpine
    commands:
      - echo "Generating Cache Key..."
      - echo -n "$(md5sum package.json | awk '{print $1}')" > .cachekey

  # Debug Cache Key Loation
  - name: debug-cache-key
    image: alpine
    commands:
      - echo "Current Directory:"
      - pwd
      - echo "Listing contents of the Directory:"
      - ls -la
      - echo "Cache Key:"
      - cat .cachekey

  # Restore Cache for Backend Dependicies
  # - name: restore-cache-backend
  #   image: meltwater/drone-cache:latest
  #   pull: if-not-exists
  #   environment:
  #     NUGETPACKAGES: /tmp/cache/.nuget/packages
  #   settings:
  #     backend: "filesystem"
  #     restore: true
  #     cachekey: cache-backend-{{ .Commit.Branch }}
  #     archiveformat: "gzip"
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache

  # Build Backend Image for Development
  # - name: build-backend-dev
  #   image: plugins/docker
  #   when:
  #     branch:
  #       - dev
  #   environment:
  #     NUGETPACKAGES: /tmp/cache/.nuget/packages
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache
  #     - name: dockersock
  #       path: /var/run/docker.sock
  #   settings:
  #     dockerfile: ./backend/Dockerfile.dev
  #     context: ./backend
  #     repo: registry.local/my-backend
  #     tags: ${DRONECOMMITSHA}
  #     purge: false

  # Build Backend Image for Production
  # - name: build-backend-prod
  #   image: plugins/docker
  #   when:
  #     branch:
  #       - main
  #   environment:
  #     NUGETPACKAGES: /tmp/cache/.nuget/packages
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache
  #     - name: dockersock
  #       path: /var/run/docker.sock
  #   settings:
  #     dockerfile: ./backend/Dockerfile.prod
  #     context: ./backend
  #     repo: registry.local/my-backend
  #     tags: ${DRONECOMMITSHA}
  #     purge: false

  # Check Debug Cache before Rebuild
  # - name: debug-cache-before-rebuild
  #   image: alpine
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache
  #   commands:
  #     - echo "Checking cache content before rebuild.."
  #     - ls -la /tmp/cache
  #     - ls -la /tmp/cache/.nuget/packages

  # Rebuild Cache for Backend Dependicies
  # - name: rebuild-cache-backend
  #   image: meltwater/drone-cache:latest
  #   pull: if-not-exists
  #   environment:
  #     NUGETPACKAGES: /tmp/cache/.nuget/packages
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache
 

Self-Hosted Drone CI with issues.

Greetings everyone.

I am trying to setup a selfhosted CI/CD setup.

Development server that is running Drone CI in Docker is running on Ubuntu 24.04.1 LTS.

Currently i have a Drone CI in a docker container (both server and runner), then i have a Docker Private Registry on a seperate server.

Once a push is sent to Github, it will activate a webhook which starts the Drone CI to work.

Been tinkering with this a few days now, tried various solutions.

In short, i want to be able to push my code to Github, webhook is called and my local development server with Drone CI is activated, where it pulls the code, caches the dependencies for backend and frontend, runs the unit tests and such, security checks and then pushes the image to private registry which are used to spin up the development site.

Been having issues with caching part where it doesn't actually store it in the cache folder.
Also been having issues with when Drone-Runner trying to push the image to the Private Registry suddenly stalling and retrying over and over but not always.

Here is the .drone.yml :

kind: pipeline
type: docker
name: default

steps:
  # Version 0.1
  # Generate Cache Key
  - name: generate-cache-key
    image: alpine
    commands:
      - echo "Generating Cache Key..."
      - echo -n "$(md5sum package.json | awk '{print $1}')" > .cache_key

  # Debug Cache Key Loation
  - name: debug-cache-key
    image: alpine
    commands:
      - echo "Current Directory:"
      - pwd
      - echo "Listing contents of the Directory:"
      - ls -la
      - echo "Cache Key:"
      - cat .cache_key

  # Restore Cache for Backend Dependicies
  # - name: restore-cache-backend
  #   image: meltwater/drone-cache:latest
  #   pull: if-not-exists
  #   environment:
  #     NUGET_PACKAGES: /tmp/cache/.nuget/packages
  #   settings:
  #     backend: "filesystem"
  #     restore: true
  #     cache_key: cache-backend-{{ .Commit.Branch }}
  #     archive_format: "gzip"
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache

  # Build Backend Image for Development
  # - name: build-backend-dev
  #   image: plugins/docker
  #   when:
  #     branch:
  #       - dev
  #   environment:
  #     NUGET_PACKAGES: /tmp/cache/.nuget/packages
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache
  #     - name: dockersock
  #       path: /var/run/docker.sock
  #   settings:
  #     dockerfile: ./backend/Dockerfile.dev
  #     context: ./backend
  #     repo: registry.local/my-backend
  #     tags: ${DRONE_COMMIT_SHA}
  #     purge: false

  # Build Backend Image for Production
  # - name: build-backend-prod
  #   image: plugins/docker
  #   when:
  #     branch:
  #       - main
  #   environment:
  #     NUGET_PACKAGES: /tmp/cache/.nuget/packages
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache
  #     - name: dockersock
  #       path: /var/run/docker.sock
  #   settings:
  #     dockerfile: ./backend/Dockerfile.prod
  #     context: ./backend
  #     repo: registry.local/my-backend
  #     tags: ${DRONE_COMMIT_SHA}
  #     purge: false

  # Check Debug Cache before Rebuild
  # - name: debug-cache-before-rebuild
  #   image: alpine
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache
  #   commands:
  #     - echo "Checking cache content before rebuild.."
  #     - ls -la /tmp/cache
  #     - ls -la /tmp/cache/.nuget/packages

  # Rebuild Cache for Backend Dependicies
  # - name: rebuild-cache-backend
  #   image: meltwater/drone-cache:latest
  #   pull: if-not-exists
  #   environment:
  #     NUGET_PACKAGES: /tmp/cache/.nuget/packages
  #   volumes:
  #     - name: cache
  #       path: /tmp/cache