Help me review if this workflow is possible

This is the workflow I devised that I want for my devs to make their work more easy.

I will set up a k8s cluster and configure everything prior to this so this can be achieved:

1 - dev requests a new URL to start a project;
2 - I create the repository for them with three branches: dev, stg, prod, populated with the basic files they'll need (laravel, wordpress, whatnot)
3 - I deploy that basic structure from dev branch and hand over the repo to them
4 - they clone the repo, work on it and push
5 - github actions builds the image tagged latest for us and restarts container
6 - container with imagepullpolicy set to always pulls new image from registry
7 - there, container is updated with latest code

Then I can use code review from github to manage pushes to stg and prod branches. Use github actions to build the images for stg and prod branches too and I can launch staging and production URLs when they hit those milestones.

Is this entirely possible with vanilla k8s? Do I need something else? Does this look too crude/uneducated and I should read on topics X and Y (please state them so I can educate myself)?

Thanks in advance, appreciate the help!

https://redd.it/uw4eli
@r_devops

Infra mapping

Hey guys, how do you keep track of which app is connecting to which db? Is there a tool out there that could do the mapping for you or is it a manual process?

something like this

App A and App B connect to DB A

https://redd.it/uw1ubm
@r_devops

How to get access to Google Ads Standard API

I'm planning to build a keyword explorer tool for bloggers but can't get access to Google ads API. Any other not so costly Keywords analysis API that can analyse a bulk keywords - Volume & CPC?

https://redd.it/uw5f4t
@r_devops

Get the pushId in release pipeline

I would like to identify all of the files that were changed in push that triggered an Azure DevOps release pipeline.

I figured that I would get the commits, then the changes in each commit.

[Get Push Commits](https://docs.microsoft.com/en-us/rest/api/azure/devops/git/commits/get-push-commits?view=azure-devops-server-rest-5.0)
Get Changes

Unfortunately, I can't seem to find the pushID in the pipeline's environment variables. Am I not seeing it? If it isn't present, I will need to find another approach.

https://redd.it/uw8nmn
@r_devops

Looking for multi-cloud monitoring tool

Hi, I’m looking for a tool which can monitor resource and cost across multiple cloud. I checked newrelic and datadog which will provide resource monitoring but not cost I believe (Correct me if am wrong). If anyone knows any tool which can fulfill both requirements, Please suggest.

https://redd.it/uwa33j
@r_devops

Get the pushId in release pipeline

I would like to identify all of the files that were changed in push that triggered an Azure DevOps release pipeline.

I figured that I would get the commits, then the changes in each commit.

[Get Push Commits](https://docs.microsoft.com/en-us/rest/api/azure/devops/git/commits/get-push-commits?view=azure-devops-server-rest-5.0)
Get Changes

Unfortunately, I can't seem to find the pushID in the pipeline's environment variables. Am I not seeing it? If it isn't present, I will need to find another approach.

https://redd.it/uw8nmn
@r_devops

Am I being set up for failure?

My supervisor left a week and a half ago, after giving a weeks notice. I inherited a few servers: Jira, Bitbucket, Jama, Confluence, Bamboo, Fisheye/Crucible, SVN, a build server that's never been used, a few C# applications that integrate with our servers. There are around 300 daily users across systems.

I had already owned almost all of the internal Jira administration & processes, internal Confluence administration, some duties related to the other Atlassian systems, a set of ISO 27001 and ISO 9001 processes and our software development processes serving around ~150 developers.

Our org has a good amount of technical debt.. 40 years of adhoc infrastructure.. In addition to the servers above our engineering developers have a Gitlab instance, standalone PCs for building linux OS's, and a drupal site serving built software that are hosted on a separate inaccessible subnet, another VLAN with a few ubuntu boxes, a polyspace server, a series of internal test servers.... etc..

Last December our team was split off to work under my supervisor as our organization's first DevOps team serving all of our engineering developers. We've had long term plans to merge our disparate systems to correct old technical debt (Bitbucket, Gitlab, SVN) onto Bitbucket, which is integrated into our Bamboo, so that we can automate testing and building. We were a team of 3 before my supervisor left: My supervisor, me, and a technical writer who does software releases. I'm now the lead and only developer on our team working on build stuff and the devops administration, having been here for 3.5 years (This is my first tech job out of college).

In December we started to work towards setting up build boxes to allow our developers to build remotely. We're supporting somewhere between 500-1000+ codebases written in C#, C, C++, VCL, some other languages, scattered across a dozen working groups and multiple engineering departments.

To kick this off, my supervisor asked me to provide a proposal that would allow us to build a build system and I built one using IAC principles to allow us to trigger builds to run remotely and containerize the builds so that the result is consistent; I was rejected and told my solution was too complicated, and our director (who is managing our group directly) wanted us to do one-off solutions for our current projects and not address existing technical debt. We started to stand up the architectural resources we needed for this from our IT department but we kept needing one more server or tool and the IT department would drag their heels on requests, so we ended up with a few months trying to line up resources (while fixing the normal fires and fulfilling user requests for our system).

As I had said before - my supervisor quit about a week and a half ago and left to seek greener pastures. I had about a week of turnover for everything he had been working on the previous 7-8 years, and now I'm owning all of his stuff + my stuff. I didn't have any project plans or anything left for me and my Director (who is not managing me directly) has been asking me for a list of what we need to get the build boxes up and running since my supervisor left; I am trying to put together the list, based on the slough of turnover documents for various systems, the knowledge of our current setup, the tasks I can find in Jira (while trying to keep everything in our systems running as best as I can)... In the middle of this I was tasked with assisting on the project management for a large project that is running several months overdue which needs infrastructural support and DevOps support (a vendor project which was dumped on our group suddenly to be supported in production).

I just got out of a one on one where my Director indicated that he is upset I do not have the list of items to get the build box up and running, and is upset I'm still working on it. He said we need to have (~10) projects (~3-4 different architectures) with built build boxes, due in about a month and a half since he has an ISO 27001

nonconformity related to it. (A deadline that I was never made aware of). He implied I haven't been doing anything the last couple of weeks because I'm not done with this list of tasks and I feel like I'm being set up to be thrown under the bus in this upcoming audit. I was in and out of meetings and providing trainings most of last week on our architectural requirements for the new system (for the project I was thrown at) and the previous week was trying to get access to and assess what turnover items were on my plate, having been turned over by my supervisor that left.

Is this a normal DevOps situation? Does it just feel like I have a lot on my plate or is this normal for DevOps? We are a company with 2 billion a year profit and our team has been begging for resources since long before my supervisor left, but we haven't even been allowed to hire anyone. I'm hoping for some input from people who have wider industry experience on how my situation looks from the outside, and recommendations on what I should do to get a handle on things..

https://redd.it/uw724p
@r_devops

What are your thoughts on Cisco as a company in 2022?

I'm considering becoming a contractor there for a dirt cheap hourly rate because I can't find anything better and need to pay the bills. They want me to help migrate their on-prem infrastructure to AWS. What do you all think of them as a company in 2022? Dinosaur company trying desparately to stay relevant, or something else?

https://redd.it/uwk3ip
@r_devops

What are your thoughts on Cisco as a company in 2022?

I'm considering becoming a contractor there for a dirt cheap hourly rate because I can't find anything better and need to pay the bills. They want me to help migrate their on-prem infrastructure to AWS. What do you all think of them as a company in 2022? Dinosaur company trying desparately to stay relevant, or something else?

https://redd.it/uwk3ip
@r_devops

What cloud provider do you enjoy most working with?

I have been heavily using AWS, Azure and GCP lately and have noticed how different the developer experience is for all of them.

AWS is usually the easiest to get something running but troubleshooting is really a nightmare. Azure on the other hand usually gets me annoyed just even getting something up and running, or maybe it's their terraform provider. So many weird issues, vm sizes that don't exist in specific regions or api just stops working for some reason..

Learning GCP now and enjoying the way they do things.

Which one do you prefer and why?

https://redd.it/uwn7lg
@r_devops

Alternative to Datadog?

I've been using Datadog for my apps and am fed up with their predatory billing practices. They don't let you modify your plan easily; you need to contact their (slow) support team any time you want to make a change to your plan.

I've been overcharged and ignored. I'm ready to move on to something more user friendly.

Any recommendations?

https://redd.it/uwv5zg
@r_devops

How Terraform Provider Iterative Helps ML Teams Save Resources By Managing Cloud Resources Properly

The founder's interview explains how the product helps machine learning teams manage their computing resources more efficiently. It offers full lifecycle management of computing resources (including GPUs and respawning spot instances) from several cloud vendors (AWS, Azure, GCP, K8s) without needing to be a cloud expert: Terraform Provider Iterative Helps Machine Learning Teams Save Resources And Money

https://redd.it/uwx5ak
@r_devops

Jenkinsfile shared library - refactor duplicate code

Hi guys!

I have four different pipelines that only differentiate with two variables, they all take the same parameters that I want and only default differently by adding the two internal variables. Else it is the same.

They then call a shared pipeline using these parameters. But the code is very duplicated and they ask for the same things. Could anyone recommend a read to solve this problem, making them share parameters or just how to structure it to avoid duplicated & messy code?

I tried to build a Groovy script to solve it but cannot seem to take parameters and pass them to the other "final-shared-Jenkins-pipeline".

I hope the question is not too confusing (I am pretty new). I would love any help. Got stuck for hours.

​

Thanks!!!

https://redd.it/ux06gv
@r_devops

New to DevOps - Kompose to Deployment Manifest generates too many files

I have a mismatch between my docker-compose version of superset (BI Dashboard) and what I have deployed on Kubernetes, which was generated with the help of a DevOps guy.

I try to close this mismatch I need to either try to use helm from Superset's repo, or Kompose to convert my docker-compose file. I tried the latter and it's creating a file for each service, how do I just create one manifest? Is it literally a concatenation of all of them?

https://redd.it/uwzhyl
@r_devops

Do you write templates and pipelines?

We're running a 2 man team supporting about 12 Devs in 3 sprint teams.

We're at a point where they come to us for standing up new resources where we will write IAC and pipelines for them.

With the imbalance on numbers we often hold up the teams while we do this work. Often they are late coming to us with the requirements also.

Wondering what other teams here are doing if you are working in a small team. Is it possible to shift towards getting Devs to self serve?

https://redd.it/uxdthj
@r_devops

What's the best cloud provider for me to mess around in and learn k8s without accidentally getting charged a lot of dollar?

I was thinking digital ocean?

I'm after fast set-up and transparent charges

https://redd.it/uxokyc
@r_devops

Kubernetes The IaC Way - Or how to install Kubernetes The Hard Way in one command using Infrastructure as Code

Wanting to better understand Kubernetes, I took on the well known "Kubernetes The Hard Way" project, on which I spent several days understanding and deploying.

After completing Mumshad's version of kubernetes-the-hard-way, I challenged myself to do it in one command using Infrastructure as Code.

As a result, Kubernetes installation is now fully automated and takes 10 minutes instead of several hours over several days.

Here is the link to the Github page: https://github.com/ydamni/kubernetes-the-iac-way

Currently, the project allows to install Kubernetes on two master nodes, two worker nodes and one load balancer node; in the 192.168.42.0/24 subnet.

In the future, the code will allow to generate as many nodes as desired in any subnet by editing the Vagrantfile.

I am open to any feedback, and would love to hear from you.

P.S. Before using my repository, I strongly recommend that those who have never deployed "Kubernetes The Hard Way” give it a try to better understand how Kubernetes works. You won't regret it.

https://redd.it/uy74lz
@r_devops

Don't open holes in the firewall to use Prometheus. Use a zero trust overlay network and keep your firewalls closed

Prometheus wants to reach out and scrape targets. In this series of posts I outlined how to deploy Prometheus on a zero trust overlay network based on OpenZiti to keep firewalls closed.

The post is long but hopefully easy to read. Part1 is setup, part 2 is the specifics and part 3 is the 'payoff' where you can see Prometheus scraping targets from one Kubernetes cluster to another (and vice versa).

Part 1 is here and links to parts 2 and 3:
https://openziti.github.io/articles/zitification/prometheus/part1.html

I forked the latest Prometheus released and embedded the Golang OpenZiti SDK into the server to allow listening and dialing over the overlay network. If interested you can find that code at https://github.com/openziti-test-kitchen/prometheus/tree/feature/zitify-federation.

https://redd.it/uxwup9
@r_devops

Don't open holes in the firewall to use Prometheus. Use a zero trust overlay network and keep your firewalls closed

Prometheus wants to reach out and scrape targets. In this series of posts I outlined how to deploy Prometheus on a zero trust overlay network based on OpenZiti to keep firewalls closed.

The post is long but hopefully easy to read. Part1 is setup, part 2 is the specifics and part 3 is the 'payoff' where you can see Prometheus scraping targets from one Kubernetes cluster to another (and vice versa).

Part 1 is here and links to parts 2 and 3:
https://openziti.github.io/articles/zitification/prometheus/part1.html

I forked the latest Prometheus released and embedded the Golang OpenZiti SDK into the server to allow listening and dialing over the overlay network. If interested you can find that code at https://github.com/openziti-test-kitchen/prometheus/tree/feature/zitify-federation.

https://redd.it/uxwup9
@r_devops

Am I naive

Hi everyone. First time posting on this sub, gonna try to keep it as short as I can.

Earlier this week I accepted an internal promotion as a DevOps engineer in my very large, well known news/media company. I am ALREADY feeling intense imposter syndrome and am starting to second guess things.

I am coming from a support role supporting the company's primary web application and a few other smaller apps, which I did for about 8 months. In that time, I was able to take over the current CI/CD process, which is basically manually merging GitHub PRs into different branches and watching the existing automation work magic.

Prior to that role, I've had a few other IT admin roles where I supported on-premise Windows infrastructure, with my only AWS experience being with a service that is not generally well known (AppStream). I've also been taking coding classes, right now primarily in JavaScript, but am hitting python soon. My goal is to eventually get a dev job, ideally in front-end, but my thought was to use the new role to get my CI/CD skills up for that move.

I have little to no knowledge of Jenkins, GitHub Actions, and/or Terraform. And now, my boss (who was my boss before the move) is throwing workflows and docs at me left and right that make no sense to me.

During my interviews, I believe I was pretty honest about my skill level and I actually thought I wasn't gonna get it. However, the director and VP (above my boss) who hired me said they were impressed with a mediocre monitoring solution I pulled out of thin air (google script automation to pull in rss feeds and send an email alert if there was an incident).

They also told me that while they work to backfill my old role, I would be "eased in" to this new role so I had time to learn, since they were aware that I'm relatively new to this.

Am I naive for believing them? Thanks for reading.

https://redd.it/uyi4pl
@r_devops