Optimizing CI CD

Hi all,

I'm trying to move my team to an optimized CI/CD that can release more frequent than 2-3 weeks. While we've built in some good quality gates in our release pipelines, one of our bottlenecks is before pushing to production we have a bottleneck of manual approval such as manual security team code reviews, architect review/sign-off, etc. How are you handling this in your workflows to ensure fast, but secure and quality releases. Where/when in the process is this happening so that it's not causing a slowdown to the fluid movement of code through to Production?

Thanks!

https://redd.it/fiilox
@r_devops

Chef -- Cookbook not executing properly?

Hi everyone,

Bare with me still pretty new to Chef! Right now, I have the following:

**Chef Server**: Hyper-V Ubuntu VM

**Chef Workstation**: Hyper-V Win Server 2019 VM

**2 Nodes**:

**buildagent01**: Win Server 19 with Chef Client installed as well as Jenkins Build Agent

**buildagent02**: Win Server 19 with Chef Client installed as well as Jenkins Build Agent

​

Right now on my Chef Workstation I have a Jenkins freestyle job running the following:

$SecurePassword = 'Password01' | ConvertTo-SecureString -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential -ArgumentList '.\Administrator', $SecurePassword
Invoke-Command -ComputerName buildagent_01 -Credential $cred -ScriptBlock {chef-client}

Simple little PowerShell script that is being run using the **Jenkins slave buildagent\_02** and the script is obviously **executing 'chef-client' on buildagent\_01**. Now this works perfectly fine when I was running one cookbook in the run list for buildagent\_01, which is simply this:

file 'C:\\tmp\\something.txt' do
action :create
content "this worked!"
end

Now, I have created a second cookbook which is running the following recipe:

windows_package 'dotnet-sdk-3.1.102-win-x64' do
source 'https://download.visualstudio.microsoft.com/download/pr/5aad9c2c-7bb6-45b1-97e7-98f12cb5b63b/6f6d7944c81b043bdb9a7241529a5504/dotnet-sdk-3.1.102-win-x64.exe'
installer_type :custom
action :install
end

Now, the the Jenkins job still runs, and the first cookbook works no problem, create the something.txt file and contents within, good to go! But, what's weird is that the second cookbook starts up, but the Jenkins job just runs forever?

If I run 'chef-client' locally on buildagent\_01 from PowerShell, no problem! First cookbook does its thing, and then, boom the .Net installer pops up right away. Not sure why this isnt working when running the Jenkins job? Any help would be much appreciated! Thanks!

https://redd.it/fip1tv
@r_devops

what will next 2 years bring in the DevOps ?

Hi guys, what do you think will be the trend in DevOps in the next year and 2 years?
Please feel free to discuss...im interested in what do you think...Thanks :)

https://redd.it/fih3qw
@r_devops

DevOps metric based governance / analytics

Hey guys, seems like a lot of teams have somewhat nailed DevOps down. One of the so called 'pillars' of a DevOps teams are metric based governance. Next to monitoring the stack, talks from example GoTo or Google bring up the subject of collecting DevOps metrics about for example lead time of stories, escaped bugs, number of deployments or pipeline duration. The idea would then be that based on these indicators, since your team is responsible for steering itself, you have the data to measure the effect of your decisions and adjust if necessary.

When looking up automation for this however, there is not a lot around it seems, and not a lot of teams are yet actively busy with this it seems. There are loads of dashboarding tools, but aggregating this kind of data or generating it does seem hard to come by. Do you have any leads here, experience with or suggestions for DevOps metric based team governance?

https://redd.it/fiqxem
@r_devops

DevOps interview Preparation

How are DevOps interview compared to SDE? Should I focus on Data Structures and Algorithms? Any study guide that anyone can share? Any helpful resources will be appreciated. Thanks!

I am looking for resources to help with prepping for Technical Screenings for DevOps/SRE roles.

* Data Structures and Algorithms. (I havent done prep in a while so would like to start from the basics).

* System Design

* Linux

* Networking

* Troubleshooting issues

https://redd.it/fip52w
@r_devops

Packaging Software with ‘fpm’

> *fpm (eFfing Package Management) enables you to build all sorts of Linux packages with great ease and sanity.*

Basically, ‘fpm’ allows you to deploy any software via OS packages, from an installation tree on disk or from already built artifacts specific to the chosen implementation language.

The main advantage over native tooling is you do not need to know about every minute detail of the involved commands and metafile formats for every platform. In case of Debian, that is at least the “control” and “rules” files, and tools like “buildpackage” and “debhelpers”.

[Read more…](https://dev.to/jhermann/packaging-software-with-fpm-58k3)

https://redd.it/fimmsz
@r_devops

Spark vs Pub/Sub Consumers

Hey,

I am new to spark and looking for realtime stream processing solution.

I didn't really get what advantage does spark streaming provide over simple rabbitmq/kafka consumers.


My current usecase is to get realtime stock data stream and run models to produce near realtime predictions.

Does anyone have experience with stream processing in production? What tools are you using and why?

Thanks!

https://redd.it/fitpn9
@r_devops

How well does your job title describe your daily work?

What is your job title? How would you describe your actual duties and responsibilities? I want to know if there are other, related jobs titles I can be on the look out for. As well as to know what I could be getting myself into.

https://redd.it/filizu
@r_devops

Looking for someone with 20+ github followers.

As the title says, if you have 20+, please open a chat or message me. I'll pay you $20 USD for about 5 minutes of your time, and you can keep your account.

https://redd.it/fix492
@r_devops

Making something good out of this: what can we learn during these two weeks of isolation?

Title basically. What would you advice one could learn during these two weeks? And what resources would you suggest for it?

https://redd.it/fj0ft3
@r_devops

Consul Connect multi-DC service mesh

Check out an article that describes the construction of PoC multi-DC service mesh with Consul Connect.

[https://medium.com/@oleksiypylypenko/consul-connect-multi-dc-mesh-part-1-43d1c35322f7](https://medium.com/@oleksiypylypenko/consul-connect-multi-dc-mesh-part-1-43d1c35322f7)

Part of series of articles that drive through complexities of multi-DC setup.

https://redd.it/fj0jbi
@r_devops

Building a tool from scratch that may exist already... am I reinventing the wheel? Does this already exist?

Hey everyone, happy Sunday!

So I have been working on a personal project over the past few years off and on that essentially does the following:

- Manages and scales a set of Docker instances hosting a web API
- Manages incoming requests to the api. These requests are then routed to the Docker instance that my tool deems fit, and then the Docker instance "owns" it
- Maintains state for all of the Docker instances, which requests they are hosting, and their metrics.

But long story short, I'm beginning to worry that this could be done with modern tools that already exist, in a better fashion. For example: Use some sort of reverse proxy like apigee to handle the incoming api requests, and then use something else to manage the state and load of what these instances are hosting. Then use traditional k8/docker swarm to use these metrics to tell when to scale the solution out.

Does this all sound like something that exists currently, and when is it a good idea to maintain an internal tool with a high LOE for changes over a set of existing platforms in combination?

https://redd.it/fj7gre
@r_devops

How to install eclipse che on kubernetes cluster that is set with kubeadm ?

I have set a kubernetes cluster with kubeadm on 3 VMs and i want to enable eclipse che on it but according to the official documentation there is no way to set on kubeadm, can anyone please suggest me a way to install it ?

https://redd.it/fj6vfh
@r_devops

PoP: Moving Back To The Monolith

Practical Operations Podcast Episode 86 is now available for download! For this show we look at a couple of articles that highlight some of the trends around micro-services. Especially, why a traditional monolith application may best suite your needs. Is the future of micro-services really the monolith? Tune in and find out.

[Practical Operations Podcast Episode 86: Moving Back to the Monolith](https://link.chtbl.com/Zl65STuX)

We're very open to feedback and comments. Please leave your feedback below. You can also write a comment on our website. [https://operations.fm](https://operations.fm)

https://redd.it/fj8zk7
@r_devops

Know of any good sample resources for automated testing?

I'm setting up some training material for Web UI automated testing as well as API Web service automated testing.

Do you know of any good sample web applications and sample web services that I can pull down from GitHub (or wherever) and deploy to write these 2 types of automation against?

Ideally a sample e-commerce site and a REST web service with a few endpoints to cover the basic GET, POST DELETE would suffice.

The idea is to focus less on the deployment and setting up of the website & web service and more on making a good test suite for whichever someone may encounter. It doesn't matter to me which language these are and I can deploy to either Linux or Windows--no preference there either.

Thanks for your help!

https://redd.it/fja9t7
@r_devops

I'm a software engineer and working from home for the foreseeable future, and I want to learn more DevOps

I just purchased a used Dell T1700 machine with a Xeon E3-1245 V3 and 8gb of ram of ebay purely to experiment with and break stuff. What crazy stuff can I do with it?

https://redd.it/fjb5ns
@r_devops

Lost my job because of the virus. Anyone else in the same boat?

I've been a remote DevOps worker for the last two years. Our industry is in event hosting, naturally with all the closures work came to a halt as profit was down to zero.


I'm just a contractor without a contingency plan. I thought I would be safe because my work was remote, but I was on the first chopping block. Now I have no idea where to start looking for something else.

https://redd.it/fjby5d
@r_devops

Is there a good self-serve tool for infrastructure deployments?

The company I am in has been growing beyond the scale of our DevOps team and I was wondering if there is a tool that allows for sane self-serve deployments of cloud resources like Kubernetes clusters, network policies, etc...

My main requirements are:
- IAC (Can be rebuilt from scratch with code)
- Simple setup
- Low maintenance
- Preferably open source

Things I looked at:

- Rundeck(Not IAC)
- Ansible Tower (unreliable)
- Writing my own( don’t wanna reinvent the wheel if there is one)

Thanks

https://redd.it/fj0xwd
@r_devops

Your own Kubernetes controller - Laying out the work

It’s hard nowadays to ignore Kubernetes. It has become the ubiquitous platform of choice to deploy containerized applications.

>Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

In a few years, Kubernetes has entrenched itself deeply in the DevOps landscape under the tutelage of the Cloud Native Computing Foundation. One could speculate about the reasons. IMHO, one very compelling argument is that it allows users to be independent of the API of a single cloud provider. If you’ve been living under the monopoly of Microsoft on the desktop in the 2000’s, you probably know what I mean.

Another reason for Kubernetes to be so widespread is that it’s relatively easy to extend it. In fact, it’s so easy a lot of software providers who offer a Docker image also provide one more operators.

In this 3-parts post, I’ll describe how to start implementing your own controller in a language other than Go, assuming zero knowledge of the subject but Kubernetes itself.

https://blog.frankel.ch/your-own-kubernetes-controller/1/

https://redd.it/fj4agk
@r_devops

What is your preferred cloud service to manage a postres db?

I'm most experienced with mongo and atlas, but right now I'm working on a side project and for sure want to go with postgres, but there are a lot of cloud service options for that and I'm kind of overwhelmed by them and in rush. So what's your favorite? Preferably it'd be low cost.

This project has 35 million+ users and that's going up fast so it'll be useful to have a redis cache as well to reduce db calls.

While the number of reads will be really high, the number of total documents and writes will be relatively small to begin with.

https://redd.it/fixw6v
@r_devops

Tomorrow I will be telling story tales for engineers

Hello everyone.

I am convinced that if you explaining something to a group of people who are engineers like you, you do not need to go deep into details, because I assume that tech people can understand the idea even without knowing to many details. But tomorrow I am going to explain what is devops to such group of engineers. Suddenly I come up with idea not to used tech language but try to make up imaginary situation or even story which have to draw the picture in their minds. Short story long. What do think? Is it good story to tell?

Imagine you are pirates. Our company is a ship, dev teams are ship crew. The ship crushed and when storm died down we found out that most of us are on island, somewhere in the ocean. Our main goal is to survive because nobody knows when we will be rescued. After stand-up meeting we decided to start building house and make the island our temporary home. We have to act quickly. The thing need to be done first, is to gather remainings of the ship, and everything that could help us survive. When we were done. It turned out that one group of people collected all the tools an instruments, another group, all the food we had on the ship. Of course there were people who collected nothing but they had experience which is nevertheless very helpful in such situation. Every time when group of people have something valuable they will convince the others that something they have is the most important thing and everybody should obey them. No doubts without tools we can not build houses and we can not get by without food. What if group of people which owns tools starts limiting the access to these tools? What if the group of people which owns food starts using food for manipulating the others? Probably we end up in fighting and arguing. Instead of finish building in 7 days we will have finished in 20 days if we will be lucky. As you probably have understood, group of people is the team, things they possess, is silos, house is the application. In order to finish building house on time we have to work together as one team. Our responsibility is not to keep safe the stuff we own, instead we are responsible for final result, for building house, for building application.

https://redd.it/fj6iua
@r_devops