Broke the Kubernetes cluster, and asked ChatGPT to write me a poem about it in Shakespear style

"The cluster hath been shattered, the pods destroyed,
But fear not, for I shall repair it with joy.
Anon, 'twill be up and running with might,
And all shall be well in the morning light."

https://redd.it/10sfoie
@r_devops

How do you remodel Jenkins for a modern, container based environment?

Things in progress at our company:

Containerization of our large, mostly Windows based monolithic projects, making them modular and not dependent on Windows anymore
Slowly shifting to K8 for the projects, test-systems and tools that support it

But we are still using a 8 year old version of jenkins.
My task is to build everything anew in a better, modern and up-to-date way.

If you were in my shoes and Jenkins also a non-negotiable must-have for CI, how would you start?
Would you still use Jenkins for Continuous Delivery or something else?
Does Jenkins hinder the whole Kubernetes workflow?
Or is this the completely wrong way to start modernizing the toolchain?

What are your experiences with switching from old CICD & software architecture/design to full automation of modular/containerized applications?

https://redd.it/10sfknd
@r_devops

oAuth/OIDC: id token & access token issue

Following scenario:

I want a user to authenticate through a single page application to my plattform. Therefor I will use oAuth/OIDC. The platform contains several services, so I thought of passing around a token between them. All the authorization concerns are handled internally by the platform itself. A microservice only needs to know who a user is.

As far as I know, id-tokens always should remain at the client and not be passed around. The access token is used for authorization and should be passed to the API of my platform but should not be used for authorization.

How can I handle this?

​

BR and much thanks!! :)

https://redd.it/10rtfcn
@r_devops

In your company, which monitoring system do you use?

.

https://redd.it/10sicvz
@r_devops

What is the best redis client for mac out there?

I want to stop using redis-cli but can't find a client out there that's free or at least not too expensive. Would appreciate it if you know one

https://redd.it/10sjjih
@r_devops

EKS - managing SSL/routing through annotations and controller vs managing routing outside of EKS on the ALB/LB

I have a client, who is stubborn on trying to manage rules and routing and TLS on the ALB instead of letting the controller do the work through annotations. I am trying to convince them that this would be an anti pattern for Kubernetes deployments.

points for controller + cert manager etc.

\- reduce blast radius to the pod that annotated

\- let developers test faster by deploying and not worrying about rules getting added for something to work

\- all the neat features the LB controller brings

\- group workloads to use a certain LB

\- dynamically provision the LB

\- scaling will be easy as adding new pods and annotations

\- industry standard??

points for putting the SSL cert on the ALB and then managing target groups/rules either manually or through terraform.

\- will need to manage the target groups and rules outside of EKS

\- increases the blast radius to all the pods in the cluster

\- scaling will become tedius

\- ips on pods can change, so before every apply or change, will need to ensure the target groups point to the right ips etc

What other points might i have missed which can tip in favor of one or the other?

https://redd.it/10sl20e
@r_devops

Puppet Bolt: multilevel inventory file

Hey all,
We've been using puppet bolt for a while now. For our new deploymwnt strategy we would like to use a multilevel inventory file, however I can't find any examples of the file or the code to call the file correctly in a bolt command.

Can anyone help me and provide these examples?

https://redd.it/10rrzab
@r_devops

Why no dabases mentionned in Road to devops

I noticed that the Road to devops site that is very practical does not have any databases.

I would expect it to have at least Postgres and MySQL mentionned.

Any thoughts? Some DB knowledge seems like something important to acquire IMHO

https://redd.it/10ry1db
@r_devops

GitLab CI: How to run only after deployment?

Hi Reddit,

I am putting database deployment into the CI pipeline, and thanks to Liquibase, that is made simple by only having to run a single command.

The problem is every time a merge request is created in GitLab, the pipeline will run before the merge happens. I would like it to run after the merge has been completed, and not before.

I tried using $CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" but it still runs at the merge request creation.

I do have GitLab premium and enabled the merge result pipeline, but I just couldn't figure out how to use it.

Does anyone know how to accomplish that?

https://redd.it/10spomx
@r_devops

From IT to DevOps

After 9 year of working as an IT Support finally got this first junior DevOps job offer after trying so hard to make this transition.

Learned for almost a year , i did so many courses and paid labs , also plan to take kubernetes and SAA exam soon and .
BUT i dont have any prior hands on experience in production with Kubernetes, Docker , CICD ,EKS .

Can someone give me some tips , advice or any self well documented step by step procedures on how to deploy apps in EKS etc, best practices and anything that could help in this difficult beginning period.

Thanks

https://redd.it/10sqxir
@r_devops

Should I accept the position?

Current job - working as contractor ( my employer has managed services contract with them) to a large govt bank in UAE. I was only supposed to manage Vmware vsphere and nsx. At this point had learned bit of Azure, terraform and Kubernetes by side projects and certifications. Since NSX is comes under networks so I ended up working for my current boss (Irish) who is really supportive and overall great leader to work for. I saw a gap where network team did not have azure and infra as code expertise and after discussing with my boss I started working in those areas and have become the go to guy for any azure related projects and using terraform to automate lot of deployment flows. This has brought lot of praise to my boss as his team was lagging behind on those areas and he now relies heavily on me also he is very vocal in recognising my effort when ever he gets chance. I am not early riser and come in late then every one else but he told not to worry about it. I do work late to compensate but already don’t have to wake up early.

After completing a year, I asked him if I can get hired directly as they have great pay and benefits ( company takes care of housing and kids eduction) and he said it might happen but not any time soon.

I wanted to test waters and got a job offer. Where pay is same as I am already at higher pay band but they have much better benefits-

36 days leave comparing to 22 days now.

Yearly bonus, equal to 2-3 months of pay.

Direct hire position with chance of pay rise in next year. My current employer has no notion of promotion or pay hikes. They just send you client where you work as long as client is happy.

Kids eduction allowance

I will have more things to learn based on job description and not limited to network devops. Like an overall Azure cloud engineer/devops + Kubernetes.


should I take this offer to my current boss and find out if he really wants to hire me direct and if he does not then just accept new job and move on? On other hand I am thinking to stay put as I am comfortable here and have great supportive boss, i still have few interesting projects in pipeline. Help me out guys what would you do in my shoes?
Also there is always the unknown factor of how your new colleagues or manager will be as that can cause unnecessary stress trying to settle in a new job where you have prove your self.

https://redd.it/10ssmon
@r_devops

Why add Varnish caching between CDN and origin

From my understanding, a CDN already does the caching, so why would you want to add a Varnish cache between the CDN and the origin. I'm sure there is a good explanation for it it's just not clear to me.

https://redd.it/10ss3hs
@r_devops

Kubernetes secrets management on autopilot

An article I wrote that discusses how we can automate secrets management within Kubernetes effortlessly, check it out!

https://maidul.medium.com/kubernetes-secrets-management-on-autopilot-36e0c6373024

https://redd.it/10svsdp
@r_devops

Higher ups want to move me into middle management (dev manager)

I'm an engineer with 8 years experience and am currently crushing those sweet sweet service mesh tickets. Had a normally scheduled meeting with a manager and they said that the company is changing and see me as having the skills to be a dev manager for the platform since they are making changes in the coming months. Im confused. I was on track for principal dev and now they want to me to switch. Is this a sign that the company will be laying people off and they like me so want me to move? Do they think I'm a bad engineer (forever learning but def not bad)? Or am i looking into it too much and should get back to work?

https://redd.it/10sx5cp
@r_devops

Don't be afraid of the "Transition"

Hi all,

I am writing this post here as I see most of the people whom I have talked about transitioning their career, have second thoughts about it. Just sharing my journey here.

If you want to move into a different role, position, or different industry. Just do it.

I have transitioned from a role in "Telecom" Radio Engineering to "DevOps" based role as an Infrastructure engineer. I had previously worked in the same domain & industry for the past 12+ years, growing vertically from a Systems Engineer to a Solution Architect.

However, I was not enjoying the domain and really wanted to move to Cloud/DevOps and thus I decided to make the switch.

I started applying to various open roles, with very few interview invites.

I was rejected by many organizations in the interview including Facebook, AWS & RedHat.

But after several iterations and several "feedbacks," I landed a role.

During this time, I constantly kept on working towards improving my skills, through self-learning and got certified myself in AWS SAA & CKA. Personally, for me, the biggest challenge was the massive gap in what is being taught and what is being implemented in the industry.

It has been a learning journey after landing in this role.

Had I would have not been determined to move to this role, I would not have been where I am.

I have been a silent lurker in this community as well and I just wanted to say a big "Thanks" to everyone here.

​

TL;DR- Don't give up.

https://redd.it/10sylqo
@r_devops

How can I transition from a traditional role to devops?

I am a network security engineer with little over one year experience. While looking into possible jobs to try in future, I figured devops is something I would like.

Working in netsec, I would say I've a strong foundation in networking and I've constantly been a part of configuring interfaces, creating changes w.r.t firewalls, switches and troubleshooting if there's any errors with proxy. Will it be useful to include this in my resume?

Pros:
So far I've fiddled with Git/GitLab(using yaml to deploy pipelines in 2 seperate environments using aws.. which i pretty much larger watching Freecodecamp video on YouTube) read on basics of devops regarding CI/CD, Version control and concepts and I've done across the devops roadmap on this sub which was very useful, I understand I've a lot to learn and catch up on like automation/scripting etc(btw I've done my bachelor's in CS, and few projects. So I've worked with python, C etc just never to automate infra)

Cons:
I've been little experience in building applications. If I'm asked to use react, API calls or work with code in general I would say I'm pretty much a novice, except the basic background I've had in programming since high school.

I'd prefer to work fully remote from home.

So my questions would be:

1. Is it feasible to think I can transition to devops directly?
2. If yes, and if I am going to dedicate my time more seriously to learn (let's say few hours everyday) how long will it be before i can realistically start looking for jobs?
3. Should I try a sysadmin/cloud engineer role before I try to work in devops?
4. I'm lost in trying to learn what's important for the job vs what's feasible, for example k8s is very important for interviews but i should probably prioritize Cloud, terraform or Jenkins(I've work experience with logs so i havent really checked out grafana etc)?

This has turned out to be alot longer than I expected but this has been on my mind for few weeks now and felt like just saying it in words would help. If I'm 10000th guy posting the same question here, then well.. even a link to op would be very helpful

https://redd.it/10syqen
@r_devops

how to migrate to helm

So we have a configuration repository written in typescript that generates yamls for our kubernetes. I think it would be a better practice to migrate it all to helm for easier deployment with terraform.

How / where do I start this migration considering I have only high level helm understand and small installation experience

https://redd.it/10spkwn
@r_devops

Unpopular opinion: CI/CD engines are an awful idea

I need scripts to run on github webhooks just as bad as you do. But this one-size-fits-all "lets make scripts but in yaml" shit has got to go.

Every CI/CD engine has a million different 'input' layers - environment variables, repo variables, workflows from various branches, maybe the git commit object. Oftentimes your scripts will have the ability to jam more variables into the 'input' layer for later scripts - such as in the case of the github environment variables. UNFORTUNATELY, it's never clear WHICH input layers are available to WHICH parts of the script. "Oh, no, you can't use outputs from this script to as arguments for something in our yaml, because the yaml is calculated first." Well thanks, Mr Nadella, I'll just torch my entire pipeline and start over.

Every CI/CD engine is trying to have some sort of rudimentary 'secrets management', but that's not its core competency. There are always rules - "only 100 secrets", or "Secrets can't be passed from job to job," or "We allow secrets but you can only put them in via our GUI" or "We use hashicorp vault as a secrets backend but we didn't study the docs so we're just using their 'approle' backend as a username/password." Looking at you, Mr Tabib.

Every CI/CD has obscure, insane 'defaults' left and right. "Oh, you ran a job that was called by another job called by another job, and everything succeeded, but one of the if statements didn't run? Our 100% proprietary fancy-pants no-source observer calls that a failure!"

FOR GOD'S SAKE, YOU ARE A GLORIFIED SCRIPT RUNNER.

I'm dying over here, you guys. What happened to the unix principle? What happened to each thing doing one thing really, really well and stringing it all together? Why does each CI/CD runner have a GUI and a CLI and two APIs and secrets management and slack integration and built in graphs and dashboards and runs shell scripts against my cappucino maker?!

Like, SO MUCH of our CI/CD in every place I've ever been could be done with docker containers and python. Github pull request? Send a webhook to the container, run the scripts, fuck off. Am I missing a GUI I admit it might be a little irritating to build my own GUI, but everything else? Emailing people from a script? Slacking people from a script? Correctly gathering secrets from my vault instance from a script? CORRECTLY PASSING RETURN VALUES BETWEEN SCRIPTS?!

But instead, I spend my days becoming a goddamn little cackling wizard on someone's particular dumbass piece of software with features we'll never use, hidden 'sane defaults' that get in the way, security that protects someone else, translating yaml to basic-bitch bash.

ugh. Thanks guys. rant over.

https://redd.it/10t0xqj
@r_devops

Helm CD vs GitOps

I'm trying to understand the advantages of implementing GitOps in our DevOps department.

Currently, we have microservices running on an EKS cluster, with each microservice having an associated Helm chart stored in a Git repository that describes its deployment.

Our CD process runs the Helm upgrade command after CI is completed. The desired “state” of our applications is stored in the Git repository, and changes to the desired “state” trigger an update to the actual state through the CI/CD pipeline. However, I'm considering the possibility that adding GitOps may not bring significant advantages and could instead introduce additional complexity to our pipelines.

Do you know what I may be missing and why GitOps could bring benefits over our current setup?

https://redd.it/10ssbpp
@r_devops

How do you fix an official docker container?

Trivy finds 1,000 issues even in official docker containers, so how do you learn to fix them? I am wondering how, because some of the errors seem to require fixing the operating system itself.

https://redd.it/10t4h8b
@r_devops

Specify at least 1 User command in Dockerfile with 1 non-root user as an arg

Why is this necessary, and why is it considered as a security issue? I never ever need to create a user to my Dockerfile, so I am trying to understand what this warning is about.

https://redd.it/10t5ib1
@r_devops