Looking for an Open-Source Logging Solution with S3 + Parquet + Querying Support

Hey everyone!

We're currently using OpenSearch for logging, but we frequently need to access older logs. We're looking for an open-source solution that can store logs in AWS S3 in Parquet format while still allowing us to query them directly from S3.

Additionally, we sometimes need to perform upserts on logs, which is much easier in S3 compared to OpenSearch, where it can take days to process.

Any recommendations?

https://redd.it/1ixg9k7
@r_devops

Ephemeral environment companies that support docker compose and helm?

We have about 20-30 uservices and we use k8s to make our production and single staging environment. It has been an issue for a while now, but it’s getting really bad: contention over the single DB is the biggest issue.

We tried to build our own ephemeral environment or preview apps before but my company didn’t give us enough time on it and did not put enough resource towards it resulting in a half done project; total waste of time and money.

Is anyone using release.com or qovery? We met with both and release seems to support our more complicated environment: docker compose, uservices, helm and they support production or staging database clones with rds and cloud sql. This is seems really cool vs seeds or managing it ourselves.

Anyone have experience with either company? Trying to make a decision soon. Any other companies we should look at?

https://redd.it/1ixgkic
@r_devops

Just Got My GCP Professional Cloud DevOps Engineer Cert! 🎉🎉

Super hyped to share that I’m now a Google Cloud Professional Cloud DevOps Engineer!
It’s been a wild ride learning all things CI/CD, automation, and SRE, but totally worth it.

https://redd.it/1ixiute
@r_devops

Managing TLS cert on k8s with subdomain concern

Hello everyone,

I have a question regarding TLS certificate management in Kubernetes. I understand that Cert-Manager uses the ACME protocol to automate certificate issuance and renewal. However, I currently obtain my certificates manually—generating a private key, creating a CSR, and submitting it to a certificate provider for issuance.

# My Setup:

My team manages the domain [abc.example.com](https://abc.example.com), meaning we control everything under this subdomain.
I obtained a wildcard certificate for *.abc.example.com, which I use for all services and Ingress resources in my development environment.

# My Questions:

1. Can Cert-Manager effectively manage wildcard certificates for third-level subdomains?
Since my company uses Entrust as our certificate provider, I found that they offer an ACME server that supports HTTP-01 and DNS-01 challenges.
Would it make sense to configure a ClusterIssuer in Cert-Manager to handle my wildcard certificate (*.abc.example.com)?
2. How does the ACME challenge work for third-level subdomains?
If I create a ClusterIssuer for [abc.example.com](https://abc.example.com), can I request a certificate for [uptime.abc.example.com](https://uptime.abc.example.com) directly by specifying:yamlCopyEdittls: - hosts: - "uptime.abc.example.com"
Or do I need to start at the root domain (example.com) level and work down from there?

​

  tls:
  - hosts:
    - "uptime.dev-k8s.med.ubc.ca"



# Seeking Recommendations:

For those managing Kubernetes TLS certificates at a similar subdomain level, how do you approach this?

Do you use Cert-Manager to automate third-level subdomain certificates, or do you prefer manual wildcard certs?
What’s your recommended best practice to simplify certificate management for a company-wide setup?

Looking forward to your insights and discussion. Thanks in advance for your responses!

https://redd.it/1ixisb1
@r_devops

Why do you want to become a DevOps engineer?

To all of you out there who would like to become DevOps engineers: why so?

I’m seeing a lot of questions in this subreddit like „how do I switch from X to DevOps” or „how do I learn quickly how to do DevOps”, etc.

Are there any particular aspects you find tempting? Money? Cloud? Automation?

Let’s discuss!

https://redd.it/1ixpft7
@r_devops

Installing ImageMagick with libheif for HEIC Support on Amazon Linux 2 (ARM64)

* Check out my medium article[ here](https://medium.com/@kamransharief7/installing-imagemagick-with-libheif-for-heic-support-on-amazon-linux-2-arm64-24143057602b).

https://redd.it/1ixpsuf
@r_devops

Github Enterprise Service Account Management

Looking for some guidance on what others are doing in enterprise environments for service accounts they use for Github. As a security leader, normally I would want to make service accounts passwordless and use keys I can auto-rotate. But github doesn't seem to allow for this. Additionally, if we enable full on SCIM and team sync with Okta, it seems the options are more limited.

I feel like this particular situation must have been dealt with an infinite amount of times but I cannot find great documentation/guidance anywhere.

Basically this is what I am looking to do:

1) Have automated provisioning with SSO enabled for Github

2) Allow for use cases with terraform scripts that need to leverage github accounts but properly secure these. Preferably with secrets/keys and not passwords but open to any solution that can be autorotated to minimize credential exposure.

Has anyone found a good solution for this yet?

https://redd.it/1ixp6a7
@r_devops

Document pipelines

Hi,

I would like to know if it's possible to document CI/CD pipelines?

Are there any best practises?

How to better represent them using standardized techniques?

I would like to know your views or practises that you have in place.

https://redd.it/1ixrj8t
@r_devops

2,160 DevOps jobs scraped from corporate websites (hiring.cafe)

I got sick and tired of ghost jobs & 3rd party offshore agencies on LinkedIn & Indeed. So I wrote a script that fetches jobs from 30k+ company websites' career pages and uses ChatGPT to extract relevant information (ex salary, remote, etc.) from job descriptions. You can use it here: (HiringCafe). Here is a filter for DevOps jobs (2,160 and counting). I'm also scraping every company page 3x/day, so the results will stay fresh if you check back the next day.

Hope this tool is useful! Please lmk how I can improve it. You can follow my progress on r/hiringcafe

Here is my technique for doing so (for analytics nerds):

(1) Identify list of verified companies: I use Apollo.io to identify companies that could be hiring. I wrote a web crawler that crawls each corporate page, and then used ChatGPT o1-mini to classify (binary classification) each page if it contains a job description or not. If it contains a job description, I add it to a list and proceed to step 2

(2) After playing with ChatGPT's API, I realized that you can effectively dump raw job descriptions (in HTML) and ask it to give you formatted information back in JSON (ex salary, yoe, etc). I used this technique to scrape 1.6 million jobs (with over \~50k remote jobs) and built powerful filters.

(3) Once I had the structured JSON data (containing salary, years of experience, remote status, job title, company name, location, and other relevant fields) from ChatGPT's extraction process, I needed a robust search engine to allow users to query and filter jobs efficiently. I chose Elasticsearch due to its powerful full-text search capabilities, filtering, and aggregation features. I built a simple UI on top of this using React/Tailwind.

https://redd.it/1ixsdz1
@r_devops

What are competitors of chef ?

Wonder who are all competitors of chef (aka progress chef)
Can someone please help here?
If you ever used l/considered chef... Which tools did you switch to?

https://redd.it/1ixrej1
@r_devops

How do you automate versioning of your apps for each staging push?

Not talking about how to scheme the version.

Ours is

major version bumber.MMYY.DD.# push per day

But how would you automate that. Suppose it’s on a json file. Like package.json. And you update it. What would you use? To both update the file, for it to show on display. For example in “about”.

I’m currently using something I quickly did. Which I’m not happy at all about because it involves pulling and pushing the updated file back to the repo. Since if I only update the file at the CI stage. The repo won’t be updated.

And I don’t want to trust my dev. To remember to do it. Manually.

So, wondering if you guys know tools that are considered better practice. That use different mechanism than me.






https://redd.it/1ixukvr
@r_devops

Front end dev looking to get into DevOps

I'm a front-end dev with couple of years of experience using react, next.js, git and everything surrounding that.
I'm also finishing computer science bachelor's, so I have a good base in everything a little bit.
I've worked minimally with node, .NET and spring boot, and I'm familiar with requests, basic bash usage etc.
I'm in a web development company that is on the verge of acquiring some new projects and we will need to be able to use at least docker for obvious reasons in the future, so I will be able to get hands on experience with this.
Before that happens, I want to introduce myself to the field, understand the basics of docker, Kubernetes and AWS and do a private project or two so I can familiarize myself with the technologies.
Do you have any courses/ resources you can recommend?
Any suggestions for a general roadmap, what to focus on and what to skip?
Any info and advice are welcome!

https://redd.it/1ixw68g
@r_devops

Can you help me out with my research on the use of automations in DevOps?

Hi! For my master's thesis I'm looking into how developers view automations (specifically GitHub workflows and Maven plugins). My goal is to apply a maturity model; I want to better understand which automations belong to advanced projects, while some other (perhaps easier?) automations belong to projects with basic maturity. With the results I want to try to recommend certain automations to repositories on GitHub.

I thought this subreddit could be the perfect place if you are willing to help me out! Of course, if you have any questions I'd be happy to answer (: The study is at the TU Delft, the Netherlands, and it usually takes 7 minutes to fill out. At the end you can join for a gift card raffle of your choice if you're interested!

This is the link to the automation survey

Cheers!!

https://redd.it/1ixsnf8
@r_devops

Looking for a DevOps Role in the UK – 5+ Years Experience

Hi everyone,

I’m a DevOps Engineer with 5+ years of experience in cloud infrastructure, automation, and security. I have hands-on expertise in:

✅ Cloud Platforms: Azure, AWS
✅ Infrastructure as Code (IaC): Terraform, Bicep, ARM Templates
✅ CI/CD Pipelines: Azure DevOps, GitHub Actions, Jenkins, FluxCD
✅ Containers & Orchestration: Docker, Kubernetes (AKS, EKS)
✅ Monitoring & Logging: Prometheus, Grafana, ELK Stack, Azure Monitor
✅ Security & Compliance: OWASP ZAP, SonarCloud, SAST, DAST
✅ Cloud Cost Optimization: FinOps
✅ Scripting & Automation: PowerShell, Bash, Python

I’m the founder of Nimbus Compute, where I provide DevOps consulting, mentorship, and training. I break down complex DevOps concepts for beginners, using real-life examples and hands-on projects. I also post educational DevOps content on TikTok, helping newcomers navigate the field.

I am currently seeking a DevOps role in the UK and open to opportunities anywhere in the country. I’m ready to start immediately and eager to contribute my expertise to a great team.

If you know of any opportunities or have any advice, please reach out! I appreciate any leads, referrals, or networking connections.

Thanks in advance!

Location: Sheffield, England
Availability: Immediate
LinkedIn: https://linkedin.com/in/ifebuche-omeke-9181b7325


https://redd.it/1ixrxxs
@r_devops

3YOE SRE Struggling to Get Interviews. Resume and Career Advice Needed!

Hey everyone,


I've been applying for entry-level SRE/SWE roles for almost two months, but I haven't heard back from recruiters. I got my first SRE job right after college and now have 3 years of experience in the field.

I enjoy being an SRE, but I do feel stuck in my current company with no growth(I’m doing 70% ops and 30% coding). I've also realized that this field requires deeper domain knowledge than I expected.


I’ve seen a lot of discussion in this subreddit about how SRE isn’t really an entry-level role, which matches what I’m seeing in job postings—most require at least 3+ years of software industry experience.


I’m now considering switching tracks to SWE, since it seems to have more entry-level opportunities and gain some coding experience.


I have two questions:

1. Is there anything wrong with my SRE resume that might be hurting my chances?


2. If I want to gain developer experience to diversify my background. What should I change in my resume to increase my chance for SWE?


My resume is attached. Would really appreciate any feedback! Thank you! 🙏

https://imgur.com/a/XKP9rFZ


https://redd.it/1iy1wph
@r_devops

My company is hiring if you have high clearance and live in Northern Virginia.

Hi, my company is looking to hire junior and mid level DevOps/Cloud engineer. If you have TS/SCI and live in Northern Virginia please dm me.

https://redd.it/1iy4u4q
@r_devops

Good CI/CD Books

I’ve been looking for a good book on software delivery, and came across this book a few times

https://amzn.eu/d/4DtjOwg

Can anyone recommend this, or any others?

Thanks

https://redd.it/1iy5bk1
@r_devops

Does your on-call engineer for the week focus on resolving bugs and operational issues?

In my previous roles whomever was oncall for the given week would be the dedicated engineer to work through resolving any bugs and keeping an eye on our observability. This really helped dedicate time to those not on call to focus on any stories outside of identified issues.

I brought this up to my new team and was shot down with angst. Seems like a reasonable pattern to follow

https://redd.it/1iy6f6x
@r_devops

How Can I Find a Good DevOps Job in the US or Europe?

Hi everyone,

I’ve been working in DevOps for about 5 years, currently leading a DevOps team at a large Russian mall. I’m AWS-certified, and I’ve been working remotely for a while now.

I want to transition to a DevOps job in the US or Europe, but coming from a third-world country, I’m not sure what the best approach is. I see many remote job postings, but I’ve heard that companies are hesitant to hire internationally unless you already have a visa or work authorization.

For those who have made the jump (or hired international DevOps engineers), what’s the best path? Should I focus on remote-first companies, apply to on-site roles and hope for visa sponsorship, or try something else? Any tips on how to stand out as a candidate?

Would appreciate any advice!

https://redd.it/1iy50vt
@r_devops

ec2 and azure vm logs

Security department is telling me we need to retain all logs for X number of days. What are others doing? Do you just send all your logs to cloudwatch in AWS or Azure log analytics ?

https://redd.it/1iy3zjs
@r_devops

How can i truly grow as a fullstack developer in the AI Era?


I’m a solo full-stack developer at my company, managing infrastructure and development with my team lead. While I can deploy applications using Kubernetes, Docker, and other modern tools, I rely heavily on AI (ChatGPT, DeepSeek) to complete tasks. This has made me efficient, but I lack deep technical understanding and struggle to answer in-depth questions, making interviews challenging.

With AI rapidly evolving, I want to future-proof my career. My main concerns:
1. How can I build a deeper understanding of technologies instead of just relying on AI?
2. What skills should I focus on to stay competitive and confident in interviews?
3. Should I transition towards AI-related development, or strengthen core engineering skills?

Looking for advice from experienced developers—how do I break out of this cycle and grow meaningfully?

https://redd.it/1iyc1kw
@r_devops