Lifecycle management: versioning and vulnerability tracking of your tools, applications, containers and more

Every project has tools, applications, Docker containers and more that are used. All of these need to be regularly updated for feature completeness or security and compliance reasons. Some teams use paid systems to track this information but more than often this is checked manually now and then and in the worst-case versions are checked barely. The reason why this is such a tedious task is that most of the time manual labor is required.

To help with this I have created an open-source project that can track automatically current versions that are being used in your project. But it can also tell you if there are new versions and if your containers are vulnerable. If you are interested please have a look at [https://github.com/arminc/k8s-platform-lcm](https://github.com/arminc/k8s-platform-lcm)

I hope this tool can save you time and give you faster insights in what needs to be updated.

https://redd.it/ebjp6d
@r_devops

Using Node.js to Write Safer Bash Scripts

Hey everybody,

This is an article I've written about how and why we're wrapping our Bash CI/CD scripts with a more modern language. In our case, since the entire company programs in JS, we use Node.JS. You could just as well use Python or Java if so inclined.

https://medium.com/getvim/using-node-js-to-write-safer-bash-scripts-ad6a523a5324

https://redd.it/ebcjyn
@r_devops

Microsegmentation --> 0 trust

Interesting (albeit loaded) read.

I never thought of microsegmentation and 0 trust to go hand in hand, but it seems if leveraged correctly they can. The article says doing so may have some technical challenges, curious as to what they are.

https://redd.it/ebfnxn
@r_devops

Big Data learning curve too big? New Job

Background: I'm a Linux/Unix admin with over 5 years of experience. Been stuying a lot about DevOps metodology and its tools, such as: Docker, Kubernetes, Ansible, etc.

I've been offered a job opportunity that involves managing Cassandra, Spark and Kafka(Big Data I believe).

I don't have much knowledge about these tools(And thats what I told the company). However, they seem to really like my profile and told me that if I promise to take some courses, I'm pretty much in. I'll be studying while on the job.

I really think this could be a good opportunity to learn more about DevOps and getting some experience.

​

Do you think the learning curve is too big? This is not an entry level job. The position is ITops Architect. If no, where should I start stuying, Cassandra, Spark or Kafka. Or the three of them at the same time.

https://redd.it/eblyfk
@r_devops

OSS Scanning for Conda and RStudio/Cran

Anyone working with data scientist out there and have a need to scan R and Cran packages? As well as Conda?

I’ve been looking into Tidelift and Blackduck. Both do 75% of what we are looking for but the lack of Cran package scanning is apparent in all OSS scanning utilities.

Anyone else have any insight?

https://redd.it/eblpaq
@r_devops

Where in process to perform static analysis?

Hey /r/devops, I am in the process of implementing SonarQube in our environment to do some automated quality checking on our projects. Our release process consists of using 3 git branches, one for raw development, one for preparation of release candidates, and then a release-only branch. Would you recommend having the quality scans run for every development build? Or would it be better to scan release candidates and do remediation as part of the release process? I'm as much interested in specific advice as I am in generating discussion in this thread!

https://redd.it/ebley4
@r_devops

Short summaries of main concepts of DevOps

Tomorrow is my first interview on a DevOps position. Maybe exist short summaries and concepts of devops, CI/CD, and so on, to read, and reprat main ideas and main moments? Or frequent questions on interviews?

https://redd.it/ebhxr5
@r_devops

Reducing risk by deploying clusters with different configurations

Hey all,

We are currently engaged in an effort to increase the reliability and resiliency of our kubernetes clusters. We currently ensure high availability by deploying 2 identical EKS clusters in 2 separete AWS regions (both configured for multi-AZ), backing them up using Velero and monitoring them extensively with Prometheus and other similar tools.

We are currently toying around with the idea of deploying one of the clusters with a different configuration to ensure a bug in either configuration doesn't bring down our entire production environment. The first idea that popped up is using kops for one cluster and EKS for another.

The pros of this approach as we see it is reducing the blast radius of any bug that might hit either configuration, retaining full control on the cluster we manage and keeping the current body of knowledge we've accumulated running our own clusters up to date (as we've been managing our own clusters for 2 years before moving to EKS a few months ago)
The cons are the increased effort required to maintain 2 sets of clusters, being limited only to the features available for both configuration sets and lack of proficiency in either configuration.

My question is - have any of you encountered use-cases of companies deploying multiple sets of infrastructure in order to reduce risk?

P.S I'm well aware of companies choosing to deploy multi cloud workloads, but I was under the impression that even when choosing such an approach the goal is to try and abstract these changes as much as possible to try and minimize the price of these multiple configurations, or choose specific solutions that are only available on certain clouds.

https://redd.it/ebj1ym
@r_devops

What do you think of Kubernete's documentation?

I find it a PITA. Even AWS documents is better than this one. Maybe that's just me though.

https://redd.it/ebhiqf
@r_devops

Can I build a Jenkins Multi-branch project in scripted pipeline?

So, far I have not found any example of a Multi-branch project written in scripted pipeline and I wonder if it is supported.

Why am I asking this?
I'm trying setup a production Jenkins environment where Jenkins master runs the both CI and CD on a remote docker host, in docker container, and there seems to be no support for that in declarative pipeline.

https://redd.it/ebdk0s
@r_devops

Concourse CI Multi-branch Pipelines

Is it possible to support multi-branch builds/testing in Concourse CI? I don't mind auto-creating new pipelines and grouping them, but I do need to figure out how to support testing, building and deploying dynamically against different branches from a git repository.

Currently, what's the best way to go about this? Or is this fundamentally opposed to Concourse CI's philosophy that it isn't possible? I've also read that spatial resources are in the roadmap - do I just need to wait on those?

https://redd.it/ebqul3
@r_devops

Azure Devops pipeline - queue jobs?

Hi,

I have some pipelines as code in Azure Devops - how do I allow an invocation of the pipeline per commit to a branch, but to queue that invocation until a previous run on different branch is finished? `batch : true` seems to only work within a given branch, allowing runs to cancel so the new changes can be used - I need it to queue per branch.

Cheers!

https://redd.it/ebsorv
@r_devops

Feeling mentally drained after work and Lack of motivation to go workout

As Developers sometimes fitness feels not very congruent with our lifestyle, work. I have recently decided to help developers with their fitness and start my consulting company and I'd like to learn more about their problems. One of the issues that people report me again and again is the lack of motivation. They tell me that this is due to being sedentary and working long hours (sometimes coupled with commute.) Looking at the screen for hours definitely effects mental energy and it is often common for me to find myself feeling drained, without energy to workout or to cook healthy etc.

What do you guys think about this issue, are you facing the same challenge when it comes to fitness that is lack of motivation ? Knowing that you should do something about it but lacking motivation. Or do you have some other issues you are dealing with, please go ahead and share, I'd like your feedback.

PS: I said developers because I'm a developer myself and I can relate to them better as I have the same lifestyle but any other profession is welcome to contribute. Thank you for reading/replying!

https://redd.it/ebrpi1
@r_devops

A question on the container image lifecycle

I've had this question on my mind for a while. We currently build (and re-build) container images from the same source in each of our different environments. Feature and Dev build all the time when a target branch is updated, Testing gets built when we have a targeted release, then staging gets built when testing passes, then prod gets built when staging passes. Each image is tagged `X.Y-env`

The images from Testing up to Prod are essentially identical since they're built from the same source. I've wondered if this was massively wasteful since we could just push a new tag on the same image/digest as the environments pass.

On the other hand, it's a security concern that passing an image that a tester had their hands on to production effectively elevates that tester's access to production. In our case, we have to be in compliance with several security standards since we have some public sector clients.

What's the "best practice" way to handle images per environment? Is there any way to reconcile this inefficiency with a security-conscience approach?

https://redd.it/ebqu7q
@r_devops

Bitbucket --> Jenkins trigger – Infrastructure As Code

IAC – Infrastructure As Code.

I'd like to create Bitbucket hooks to trigger Jenkins jobs. But all the ones I've seen can only be created through the UI.

Does this exist? If not, would anyone be interested in developing such a thing?

https://redd.it/ebno75
@r_devops

Stop telling me to log less to Splunk and that Splunk can't scale.

Splunk costs too much. We log 4TB a day we can't scale it. The logs should be better curated, stop logging so much data. We should switch to a cheaper log management system. It's all horseshit. Splunk is amazing, you should log everything. Info level events, metrics, stack traces, E.V.E.R.Y.T.H.I.N.G when you can correlate all of your data points across an organization in one place you have the ultimate tool. How people can't see that boggles my mind.

How to Scale in an Enterprise environment?

1) Get everything into "A" Splunk indexer. Put a Splunk forwarder on it, sylog it and index syslog. Whatever you have to do. There isn't a system data that you can't get into Splunk one way or another.
2) Give different parts of the organization their own Splunk Indexers and Search heads for the applications or infrastructure they maintain. Let them do whatever they want with it. Write tools for them to generate alarms and ticketing from their servers that integrate with your ticketing systems. Rinse, Repeat.
3) Teach developers to write code that takes advantage of KV Pairs in their logging, and encourage them to write MORE logs with timing and metrics tied to events. Encourage everyone to create curated data FROM all the data that is being collected.
4) Forward content that is relevant across orgs to a higher tier of indexers/search heads. This is where the real magic happens.

When a person in application team A can correlate their apps performance with metrics and relevant data from anywhere in their entire stack of dependencies, you win. Granted the people who are capable of doing that are too few and far between but... that is the problem, Not too much data.

https://redd.it/ebqs08
@r_devops

What do you think is the best automated code review integration for Github and OS projects?

I wanted to seek the communities opinion on what you think is the best automated code review integration in Github. Typically I code in either Python or Node, and have tried SonarQube and LGTM. They both have their pros and cons, but I simply dont like the non reviews for LGTM on pushes. It just feels clunky to me. It is also slow as sin.

I know where are a bunch of other ones out there that I found via [https://github.com/joho/awesome-code-review](https://github.com/joho/awesome-code-review), but wanted to see what others experiences are.

​

Crucible Atlassian's on-premise code review tool.

Gerrit Open source git code review tool originating out of Google.

GitHub Git hosting and pioneer of the "Pull Request".

Gitpod Code review pull requests in a full IDE within your browser.

GitRise Slack Reminders for GitHub pull requests

LGTM Automated Git code review for GitHub and Bitbucket pull requests for finding security vulnerabilities and code quality issues.

Phabricator Open source git/mercurial/svn code review tool originating out of Facebook.

PullRequest Code review as a service for GitHub pull requests.

Pull Reminders Automated Slack reminders and metrics for GitHub pull requests.

Reviewable Code review tool built on top of GitHub pull requests.

Review Board Open source review tool that is SCM/platform neutral.

Rubberduck Browser extension to adds code-aware navigation to GitHub pull requests.

Sider Automated code review service for GitHub.

Softagram Automated code change visualization (and dependency analytics) for pull requests, merge requests (GitLab) and patch sets (Gerrit).

SonarCloud Detect code smells, bugs and vulnerabilities in Azure DevOps, Bitbucket and GitHub repositories.

Upsource JetBrain's on-premise git/mercurial/perforce/svn code review tool.

https://redd.it/ebx9rw
@r_devops

Is Chef.io already outdated?

With cloud providers offering both Serverless functions and Container Orchestration, is there still a need for software like Chef? What am I missing?

https://redd.it/eby66u
@r_devops

Good summary of the last 15 of cloud computing

[video](https://www.youtube.com/watch?v=1pBuwKwaHp0)

https://redd.it/ec0z7g
@r_devops

Linux Foundation Announces a Free Online Course: Introduction to DevOps and SRE

https://www.linuxfoundation.org/press-release/2019/12/linux-foundation-training-announces-a-free-online-course-introduction-to-site-reliability-engineering-and-devops/

https://www.edx.org/course/introduction-to-site-reliability-engineering-and-devops

https://redd.it/ec5xaq
@r_devops