Don't open holes in the firewall to use Prometheus. Use a zero trust overlay network and keep your firewalls closed

Prometheus wants to reach out and scrape targets. In this series of posts I outlined how to deploy Prometheus on a zero trust overlay network based on OpenZiti to keep firewalls closed.

The post is long but hopefully easy to read. Part1 is setup, part 2 is the specifics and part 3 is the 'payoff' where you can see Prometheus scraping targets from one Kubernetes cluster to another (and vice versa).

Part 1 is here and links to parts 2 and 3:
https://openziti.github.io/articles/zitification/prometheus/part1.html

I forked the latest Prometheus released and embedded the Golang OpenZiti SDK into the server to allow listening and dialing over the overlay network. If interested you can find that code at https://github.com/openziti-test-kitchen/prometheus/tree/feature/zitify-federation.

https://redd.it/uxwup9
@r_devops

Don't open holes in the firewall to use Prometheus. Use a zero trust overlay network and keep your firewalls closed

Prometheus wants to reach out and scrape targets. In this series of posts I outlined how to deploy Prometheus on a zero trust overlay network based on OpenZiti to keep firewalls closed.

The post is long but hopefully easy to read. Part1 is setup, part 2 is the specifics and part 3 is the 'payoff' where you can see Prometheus scraping targets from one Kubernetes cluster to another (and vice versa).

Part 1 is here and links to parts 2 and 3:
https://openziti.github.io/articles/zitification/prometheus/part1.html

I forked the latest Prometheus released and embedded the Golang OpenZiti SDK into the server to allow listening and dialing over the overlay network. If interested you can find that code at https://github.com/openziti-test-kitchen/prometheus/tree/feature/zitify-federation.

https://redd.it/uxwup9
@r_devops

Am I naive

Hi everyone. First time posting on this sub, gonna try to keep it as short as I can.

Earlier this week I accepted an internal promotion as a DevOps engineer in my very large, well known news/media company. I am ALREADY feeling intense imposter syndrome and am starting to second guess things.

I am coming from a support role supporting the company's primary web application and a few other smaller apps, which I did for about 8 months. In that time, I was able to take over the current CI/CD process, which is basically manually merging GitHub PRs into different branches and watching the existing automation work magic.

Prior to that role, I've had a few other IT admin roles where I supported on-premise Windows infrastructure, with my only AWS experience being with a service that is not generally well known (AppStream). I've also been taking coding classes, right now primarily in JavaScript, but am hitting python soon. My goal is to eventually get a dev job, ideally in front-end, but my thought was to use the new role to get my CI/CD skills up for that move.

I have little to no knowledge of Jenkins, GitHub Actions, and/or Terraform. And now, my boss (who was my boss before the move) is throwing workflows and docs at me left and right that make no sense to me.

During my interviews, I believe I was pretty honest about my skill level and I actually thought I wasn't gonna get it. However, the director and VP (above my boss) who hired me said they were impressed with a mediocre monitoring solution I pulled out of thin air (google script automation to pull in rss feeds and send an email alert if there was an incident).

They also told me that while they work to backfill my old role, I would be "eased in" to this new role so I had time to learn, since they were aware that I'm relatively new to this.

Am I naive for believing them? Thanks for reading.

https://redd.it/uyi4pl
@r_devops

Monorepo for unrelated services, or setting up micro services for each?

Hi all,

I'm responsible for a repository at work, which basically handles all of the miscellaneous services that don't quite fit anywhere else in our ecosystem.

the thing that makes me want to go the micro services route is that right now, we have to schedule monthly releases for this repo, and if one thing breaks in production, everything needs to be rolled back; including the changes that were successful but unfortunately broke because of an unrelated service. And then often times we'll need to cherry pick some changes into a release and leave some out because a release needs to happen even if something else in the repo isn't working. I would also feel more comfortable developing a more agile approach and releasing frequently if I knew the micro services were self contained and small to roll back.

but at the same time, we have dependencies which are shared by all these services.In other words, there would be more overhead; having to set up these dependencies for each micro service and deploy them all.. also, I would need to take time breaking up the mono repo..

I've mostly been a pure software developer up until now, but I suppose my boss is giving me more devops responsibilities, just want to make sure I'm making the right decision here.

thanks!

https://redd.it/uyk4dk
@r_devops

Need advice on becoming a DevOps Engineer.

I am a Senior System Engineer. I mostly am writing python scripts, scraping bots,doing automation. Linux has been my OS since 2016. I manage multiple linux vps. I also work on Laravel as a Backend. So I know php too but Python has always been my favorite.

I know about Networking, a basic understanding of penetration testing and preventions from a few common attacks.

Wanted to get into DevOps, so went ahead and learned Docker (skipped Docker Swarm), working with YAML and Dockerfile. Tried Jenkins once. Have worked on Github and GitLab many times.

What should be my next step? I would love some advices. I think Kubernetes, as I skipped Docker Swarm. I just need to know what to do next, I'll find a way to learn. Thank you.

https://redd.it/uyytta
@r_devops

Need advice on becoming a DevOps Engineer.

I am a Senior System Engineer. I mostly am writing python scripts, scraping bots,doing automation. Linux has been my OS since 2016. I manage multiple linux vps. I also work on Laravel as a Backend. So I know php too but Python has always been my favorite.

I know about Networking, a basic understanding of penetration testing and preventions from a few common attacks.

Wanted to get into DevOps, so went ahead and learned Docker (skipped Docker Swarm), working with YAML and Dockerfile. Tried Jenkins once. Have worked on Github and GitLab many times.

What should be my next step? I would love some advices. I think Kubernetes, as I skipped Docker Swarm. I just need to know what to do next, I'll find a way to learn. Thank you.

https://redd.it/uyytta
@r_devops

Storing env vars in .git/config?

I just realized you can effectively use git config as a place to store and reference environmental variables.

For example, you can write:

git config env.private charlie
git config env.public bob

and in .git/config you will see:

env
private = charlie
public = bob

You can just read these variables too:

git config env.public
bob

---

What I am wondering is, why not just use this in place of the typical .env file usage where you just keep some KEY=VALUES in plaintext and don't commit it?

It seems far harder to accidentally commit your git config, and it even ships with one-level-deep yaml-esque organization!

https://redd.it/v02aoi
@r_devops

Do you use nano/vim?

Do i need to learn any of these type of text editor?
Or i will always have vscode or something like that?

How often do you use these?

Which is the "better"?

https://redd.it/v06r4y
@r_devops

Salary Sharing Thread May 2022

This thread is for sharing recent offers you've gotten or current salaries.

Please only post an offer if you're including hard numbers, but feel free to use a throwaway account if you're concerned about anonymity.

Education:

Prior Experience:

Company/Industry:

Title:

Tenure length:

Location:

Base Pay

Relocation/Signing Bonus:

Stock and/or recurring bonuses:

Total comp

Last thread was a huge success so bringing it back on popular demand

https://redd.it/v0h3z7
@r_devops

DevOps Bulletin Newsletter - Issue 53

Hey folks,
My weekly DevOps newsletter aka DevOps Bulletin -  Digest #53 is out. Check out a sneak peek of the topics covered on this weekly issue:

* 🏗️ "**Building a Frontend Testing Pipeline**" - This hands-on tutorial will walk you through implementing a testing pipeline from scratch.
* 🧠 "**Think like Git**" - This article is for people who already know how to use git day-to-day, but want a deeper understand of the why of git to do a better job reasoning about what should or should not be possible rather than just memorizing incantations.
* 🧾 "Software bill of materials: What it is and why you need one" - **Learn what a software bill of materials is and why it is important for DevOps**
* 🤯 "**Over 380k+ open Kubernetes API servers**" - Damn! If you are notified of an instance that is accessible, please consider implementing authorization for access or block at the firewall level to reduce your exposed attack surface.
* 🌎 "**Lessons learned from running Apache Airflow at scale**" - Shopify shares some of the lessons learned and solutions they built in order to run Airflow at scale.
* 🔒 "**RBAC explained with examples**" - Kubernetes RBAC tutorial with two examples, using ServiceAccounts and OpenSSL to create separate contexts for users.
* 📦 "**Robust Terraform setup with workspaces**" - Snowflakes as code is an anti-pattern where separate instances of infrastructure code are maintained for multiple instances of infrastructure that are intended to be essentially the same.
* 📹 Video of the week goes to a talk given by David Flanagan where he shares the key methods, tools and **takeaways from fixing over 50 Kubernetes clusters live**.
* 🛠 Project of the week goes to "OWASP WrongSecrets p0wnable app" - an **open-source app packed with various ways of how to not store your secrets**. These can help you to realize whether your secret management is ok. The challenge is to find all the different secrets by means of various tools and techniques.

Complete issue: [https://www.devopsbulletin.com/issues/kubernetes-toolkits](https://www.devopsbulletin.com/issues/kubernetes-toolkits)

Feedback is welcome :)

https://redd.it/v10h7u
@r_devops

Build flexible abstraction for any Kubernetes Resources with CUE and KubeVela

https://kubevela.io/blog/2022/05/30/abstraction-kubevela

https://redd.it/v0wts4
@r_devops

What do you guys think about Thalès ?

I have the opportunity to do a work-study program in devOps at Thalès. I know it's a large group that touches on cutting-edge technologies, on exciting subjects such as defense and aerospace, and I tell myself that it can be very interesting and educational.

For those who know the company, what do you think? Do you consider it a gateway to well-paid and interesting jobs?

https://redd.it/v0wi5g
@r_devops

Why you shouldn't consider using Oracle Cloud

Worst practices you can imagine like deleting your account without any warning.

And if you will go to support you are going to get a standard answer: we are not going to tell you the reason bye.

They don't even bother to let you move your data.

And the reason is that i am holding Russian passport. But i am not in Russia, nor i am anyhow connected to. But the real point is that Oracle got a lot of corrupted money from Putin regime for years and i have been witnessing that since i was working on a civilian government company in the past.

​

Company that i am currently working for migrating away from OCI and the reason for that is vendor-locking practices. Oracle tries to lock you on their databases for example.

​

OCI is the worst choice for personal and business needs. Just facts.

https://redd.it/v1rcac
@r_devops

Devops Exercises

https://github.com/sottlmarek/DevSecOps

Complete exercises and scripts + Question Answers +Test etc

https://redd.it/v1qqfh
@r_devops

Interaction between Docker, AMI and Ansible

I am wondering if anyone can set me on a path to understand how these different technologies interact or compliment each other in the devops world:

- Using Ansible (or Terraform or similar tools) for infrastructure as code

- Building and deploying Docker containers

- Saving/loading machine images like Amazon AMIs

Some of my confusion comes from the fact that all these seem to overlap a bit.

For instance, is it best to put most low level config and build in a Dockerfile and then just use Ansible for glue and deployment? Or could you build everything using Ansible to setup a VM which you would then save as an AMI?

I know a lot of different setups are possible but I am wondering about best practices especially as it pertains to interactions between tools like Ansible and Docker (Ansible has Docker bindings but is that all you would use it for in a Docker-centric infrastructure?)

https://redd.it/v25jqb
@r_devops

devops is not about receiving the recipes and not doing the effort by yourself

I did appreciate this subreddit year back but now, i am really getting fed up by this trend of asking all and everything without any effort. Even with such amount of resource you can find on google.

Stupid questions like: Should I learn this, should I learn that?

The fact that you ask such question shows that you don't even try or were not interested in first place, you just want/think to get quick money.

https://redd.it/v2aoh2
@r_devops

Stop Messing With Kubernetes Finalizers

Hi /r/DevOps,

Today I published an article titled "Stop Messing with Kubernetes Finalizers", where I explain why it's a bad idea to force-delete Kubernetes resources, what are the consequences and how to delete "hanging/stuck" resources the right way.

Here's the link: https://betterprogramming.pub/stop-messing-with-kubernetes-finalizers-b849511b2329

Feedback is very much appreciated!

https://redd.it/v2mllf
@r_devops

Which alternatives to Azure DevOps do you prefer and why?

Curious about the community's experience with Azure DevOps vs others.

https://redd.it/v2kswh
@r_devops

Kubernetes Challenge: 500USD to give away

I’ve maliciously broken two Kubernetes clusters and I’ll be jumping on my YouTube channel live tonight to give people the chance to fix them.

20USD Amazon voucher to anyone that comes on the stream and tries to fix one.

200USD voucher if you manage to actually fix one.

Who’s feeling confident?

https://youtu.be/bQsubShHE94

https://redd.it/v3831e
@r_devops

Two of my senior devops were transfered to another projects, I'm at lost because I don't know what I'm doing.

Currently helping customer setup CI/CD pipelines with Jenkins and Gitlab and deploy into Rancher. I'm still new to this and I don't really know what I'm doing. I have to report to my PM everyday what am I doing but I don't know what to tell her becuase I'm at lost without my seniors to guide me. I'm trying to learn as much as I can but I don't think I can do this job by myself. I told my manager that if they can't find someone to help me I will quit because I'm really stressed out.

edit: I've only been in DevOps role for four months from a sys admin backgroud. I did learn a lot but not enough to troubleshoot or work on this complex project alone.

https://redd.it/v38kff
@r_devops

Monthly 'Shameless Self Promotion' thread - 2022/06

Feel free to post your personal projects here. Just keep it to one project per comment thread.

https://redd.it/v3gvey
@r_devops