DevOps career stalled.

I have worked in Devops / System administration for 10 years. I make an above average salary at a midsized company. We do not use k8s or have a strong CI pipeline. The release processes is an over complicated mess the infastructure has largely been developer driven until last year when the CEO decided to build a "devops" team.

​

I am unsure what to do to make things better the company is very sucessful and growing but a release processes that seems like it should be very simple ( we have a monolith and some workers) has become increasingly complex eg. some of a terraform code is generated by python ( which is fine it just does not seem necessary)

​

Not sure what to ask to be honest . I am finally a little frustrated at not making any major improvements after a year and not seeing a path forward.

https://redd.it/unsp8r
@r_devops

Cheapest managed Kubernetes ?

Hi, I'm looking for a cheap managed Kubernetes cluster at any Cloud Provider. I want to host 2 websites with very low traffic, the Grafana/Prometheus/Loki stack to train monitoring, and maybe working later with telemetry and tracing tools. I'll also add a LoadBalancer with nginx-ingress. What are the options for a cheap cluster ? DigitalOcean starts at ~25$/month, can it be cheaper ?

https://redd.it/unlgco
@r_devops

Introduction to Test Driven Development course

I've just released another course: Introduction to Test Driven Development on Coursera in addition to my Introduction to DevOps and Introduction to Agile Development and Scrum courses. This course is an adaptation of part of the graduate class that I teach at NYU on DevOps and Agile Methodologies. The focus is on the practice of Test Driven Development (TDD). That is, writing tests first for the "code you wish you had" and then writing the code to make the tests pass. TDD keeps you focused on the behavior of the code and ensures that your code is always working properly.

Throughout this course, I'll teach the workflows and techniques that I use every day as a software engineer. You will learn how to write test assertions to check the behavior of your code. How to use test fixtures to establish an initial state so that tests run in isolation and you get repeatable results. I will teach you how to use factories and fakes to generate test data, and how to use mocking to make sure that your tests are isolated from external systems, and to simulate error conditions to test your exception handlers. Finally. I will take you through the red, green, refactor workflow with hands-on sessions.

We tied something different in this course. I recorded live demonstrations of me implementing the testing concepts that I just explained in the video lecture. So you get to watch me code "live" and then you get to try your hand at it in the hands-on labs. The idea is to present a concept, demonstrate the concept, and then have you perform the concept in the lab to reinforce your learning.

I'm looking for feedback on how well you think this works and if I should continue with this approach in future courses. Here is a link to the course introduction video of me giving a course overview: https://www.coursera.org/lecture/test-and-behavior-driven-development-tdd-bdd/course-introduction-0yZqX

https://redd.it/uo3ser
@r_devops

High Steal Percentage on One VM

Hi All,

I'm running into an issue where one of my VMs has around a 40-50 steal percentage. I checked the rest of the VMs on the host and no others are really hitting the CPUs that hard, one is almost at 75%. Also none of the other VMs have any steal percentage. The one thing is that due to licensing we have to run several of these on the same CPU eg. 0-3 and 25-28. I checked the host and both CPUs are showing. Does anyone else know what else I could check?

I am using OVirt as the Hypervisor.

Thanks,

https://redd.it/uo1xce
@r_devops

Secret detection needs to be free, even for private repositories

My name is Nir. I am one of the three co-founders of arnica.io.

I’ve worn many hats in cyber security over the years – sys admin, pen-tester, security architect, and Chief Information Security Officer (CISO). What really gets me excited about my work is making security easy and effective for developers and ops teams! In my last role, at one of the top 3 FinTechs, following the attack against Solarwinds, the CEO asked me what we are doing to secure our software supply chain. I met with 15+ vendors, did a few POCs, but unfortunately each solution either increased operational cost or was too narrow in scope. I really wanted to buy a solution instead building it, but even the ones that hit the short list were rejected by my team (thank you guys!).

I also found that many fellow CISOs faced the same problem. This is when I joined forces with my incredible co-founders - Diko and Eran. They were seeing the same pain in their worlds (engineering and ops) too! As a starting point for Arnica, we researched every software supply chain attack since 2018, and based on our research, we found two primary root causes:

1. Improper access management to developer tools
2. Inability to identify abnormal identity and code behavior

We studied the anatomy of each supply chain attack and designed a product to effectively secure developer tool stacks with a DevOps-first approach.

We decided to release a couple of features for free:

1. Identify excessive permissions to source code starting with GitHub and Azure DevOps repos
2. Automatically generate & modify a CODEOWNERS file via pull request, based on the contextual behavior of the pull request reviewers
3. Secret detection and validation without modifications of the build pipelines for all repositories, public and private without any user-count limitations
4. Map GitHub users to your SAML/SSO provider. Also free forever.

Why are we giving away so much functionality for free? I believe Arnica can do well by doing good in the DevSecOps community. Our mission is to be the easy button for DevOps security, and we are going to deliver it.

More info about why we released unlimited secrets scanning for free: https://www.arnica.io/blog/secret-detection-needs-to-be-free-even-for-private-repositories

https://redd.it/uo6jdk
@r_devops

Where are you finding new job postings these days?

I'm a build and release engineer with about 3 years of experience (and 2 more doing tech support before that). I manage releases at my company and build and support our CI/testing pipelines and related tools. Basically, most of the "dev" part of devops. Not as much ops, which is part of why I want to move (in addition to generally plateauing in my role). I'm in the Bay Area.

I've been applying to DevOps and other build/release engineer positions. I've been hearing that the market is hot right now but so far that hasn't been my experience. In fact, I've been getting less traction than I was getting in 2019 applying to very similar roles as I am now, with three years less actual experience.

I'm not ruling out that something about me is the problem, but given the variance from last time (and the fact I don't think I look terrible on paper), I want to address the approach first. Last time around I used google's job search aggregator to find postings to apply to. This time, I'm doing the same but the tool is just turning up fewer suitable reqs and I wind up applying to only 5-10 ones per week, of which maybe around 1 or 2 respond.

I know it's a numbers game but I'm not hitting sufficient volume at the moment to matter. My question is, for those of you who are looking for jobs, what are the resources you've had the most success with? And is there anything obvious I'm doing wrong?

https://redd.it/uo7t1h
@r_devops

Source of truth for sensitive kv pairs

I want to migrate away from storing our sensitive KV pairs in environment variables for task definitions and lambda functions, and instead grab them from SSM Parameter Store.

Normally, I set the environment variables as part of the terraform configuration for deploying the infrastructure (so Sensitive TF Var in TF Cloud, used in the resource definition which then sets the environment variable in the lambda function etc), but obviously if I want to pull these values from SSM, it doesn't necessarily make sense to seed them from the same terraform config in the first place (especially if the credentials are used in multiple configs, such as API keys for observation platforms)

How do I get away from manually defining secrets in some kind of secure "variable" system (like TF Cloud), and move into a more structured source of truth (like a file where I define all of my secrets that then runs a job to add them to SSM when I update the file).

My first thought is to host a JSON or YML file in a git repo, and then have a post-commit job that updates SSM, but we use Github pretty exclusively and I don't feel like that's a very good idea.

Are there any tools for doing this? (industry standard single sources of truth for KV pairs that are the definition, not the store itself)

https://redd.it/unwa2q
@r_devops

How to share infrastructure details with application Helm charts

Not sure if I worded this question in a way that makes sense, but our situation is as such: we create AWS resources such as security groups, ACMs, IAM resources etc. which we need to reference in application helm charts, for example ACM certificate ARN in the ALB annotations. The infrastructure is created via Terraform. Making manual changes to the values.yaml file of a Helm chart seems like a tedious and error prone practice. Deploying Helm charts via Terraform in the same state as the infra doesn’t seem like a proper practice as it couples the infra to the app too tightly.

What should be the approach here?

https://redd.it/uoaqii
@r_devops

Microservice Governance - Resilience Patterns

Microservice Governance - Resilience Patterns

https://redd.it/uoapus
@r_devops

Looking for contributors for a K8s related open-source project

Hi, I'm building a community driven K8s related open source project. We need help, lots of help, not only with coding. Thank you.

Project: [Kubevious](https://github.com/kubevious/kubevious)
Many ways you can help: Requirements, Coding, K8s Experience, Docs, Walkthroughs
Where do we want to be: [Roadmap](https://github.com/kubevious/kubevious/blob/master/ROADMAP.md)
How to contact: Slack or [email protected]

https://redd.it/uo6oep
@r_devops

Branch Deployments with IssueOps and GitHub Actions 🚀

# Branch Deployments with IssueOps and GitHub Actions 🚀

I developed my first ever GitHub Action and wanted to share it with the community as it is open source!

## What is this?

A fully package GitHub Action that can be used in any repo on GitHub with just a few lines of code

## What does it do?

Enables branch deployments so you can get far far away from the "merge -> deploy" model of the past

## Links please!

- Here is a Medium writeup about the project: link
- Here is a link to the full source code, Action, and documentation: link

♥ open source

https://redd.it/uoesik
@r_devops

SystemD Unit not Found error when trying to start service with an argument

I'm trying to pass environment information to a service on start so it can set environment variable for the application it starts (test, prod, etc).

According to posts like https://superuser.com/questions/728951/systemd-giving-my-service-multiple-arguments I should be able to start the service with

systemctl start [email protected]

but when I try that (or any variation of it (test@app, app@"test", etc)) systemd just says Unit not found. If I start the service without the @parameter part it starts up fine.

Does anyone know why this way of passing parameters isn't working? It is the most convenient way for me to pass a single parameter when starting the service.

https://redd.it/uoemqn
@r_devops

Build List of Companies for DevOps Opportunities

I am a DevOps Engineer. I am looking for DevOps community members that have job-hopped around or have friends at decent companies to weigh-in and help me build this table of DevOps/Software companies collaboratively. Curious to compile a list of A, B, C, and D-Tier companies to continue skill-building on the job. I think this would help people interested in building a solid list of exit opportunities. The idea is to rank companies based on internal talent, performance, and current tech. stacks utilization. Please help me build this thing!

Example Table for DevOps prospects (excluding FAANG/MAANG):

|Tier|Non-FAANG Companies|
|:-|:-|
|A|CompanyZ, CompanyY|
|B|CompanyX|
|C|CompanyW|
|D|CompanyT, CompanyU, CompanyV|

TLDR;

Use DevOps community build a ranking system of non-FAANG/MAANG companies.

https://redd.it/uohin7
@r_devops

This is an interesting list published by Devoteam’s TechRadar on tech to Adopt/Trial or Assess in 2022

Keeping up with the pace of the rapidly changing tech landscape isn’t easy and making tech-related decisions for your organization can feel overwhelming. Devoteam’s TechRadar is the opinionated tool that aims to help you decide on which technologies you need to hold out on, assess, start a trial with, or adopt completely.

You can see the 'Distributed Cloud' list of tools here.

Or download the full guide (note - you need to provide your dets to get the full guide).
https://www.devoteam.com/techradar/

https://redd.it/uoih94
@r_devops

Hello, I am new to programming and I have been coding in C++ for the past 6 months. I want to delve into an aspect in coding/programming and DevOps looks cool to me. How many months/ years will it take me to know DevOps to a point I can work for a company as an intern? .

How long will it take to land an internship in DevOps

https://redd.it/uo2ydd
@r_devops

Hello Worlds in 22 Langs on Kraken CI

There is a new, quite funny workflow example for Kraken CI.
It shows how to compile and run Hello World programs
in various programming languages, 22 languages.

https://kraken.ci/blog/hello-worlds-in-22-langs-on-kraken-ci

What do you think about that?

https://redd.it/uoka2o
@r_devops

How can I transform a variable in my json file, when variable if not a string in the json?

How can I transform a variable in my json file, when variable if not a string in the json?

The case is that I cannot convert the value to a string "true", it has to be true.

https://redd.it/uom6k5
@r_devops

Free kubernetes course launched this morning

Hi All

I'v released a Free Kubernetes course this morning. It's a community give back. Hope you all enjoy and please pass on to any colleagues who may need this before Kubecon next week - https://www.youtube.com/watch?v=n4zxKk2an3U

https://redd.it/uooey8
@r_devops

What excites you about being a DevOps engineer?

I just made a career switch to DevOps and am curious to know what my fellow engineers get excited about in their day to day work.

https://redd.it/uombzv
@r_devops

3 Months to get ready for global lead position

Hi all,


So the good news is I managed to successfully apply for and get a Global lead devops position in a large enough company. It's a massive step up for me and the fear is starting to kick in .


There's definitely some elements of imposter syndrome at play but I'm honest enough as well to understand how limited technically I am , and realistically out of my depth I might be.


I've be a devops lead for 2 years but in my current company/project I've camouflaged my weaknesses technically with soft skills. The project I was working on essentially had no CI/CD in place and really bad infrastructure as code. So I am comfortable with Terraform and designed the projects CI pipelines giving them the option of CD if they ever choose to do it ( They're a bank so likely won't ) .
But If i was to describe the technologies I'm comfortable with it would be


Terraform
Gitlab CI
helm charts/ helmfile
K8s

And that's kinda it ! Which is why I'm panicking massively. I got referred to the CTO by someone on the project who the CTO respects but otherwise I'm really not qualified for the role

\- If someone asked me to implement terraform dynamic credentials with vault I wouldn't even begin to know where to start.

\- My Observability knowledge is useless. I have such foundational knowledge of grafana/loki etc that it's borderline criminal.

\- I've never used anything other than helmfile to deploy applications.


So really the only thing I'm confident with is my people management skills ... But I have 3 months to fix this until I start. I'm definitely willing to put in the time and have access to a udemy account if I need to access training materials etc


Hopefully the rant isn't too long but what would you suggest I work on improving in terms of priority , just right now I feel like there's so much I'm missing I don't know where to start

https://redd.it/uov0i6
@r_devops

All you need to know about DORA Metrics!

The DORA metrics are a set of metrics that measure the quality and velocity of software delivery of an engineering organization. By measuring and continuously iterating on these metrics, engineering teams can deliver better software to their customers faster, and achieve significantly better business outcomes.

## Where did the DORA metrics come from?

The DORA metrics were put forth by the **DevOps Research and Assessment (DORA) organization** that synthesized several years of research studying engineering teams and their DevOps processes. The group publishes a yearly report called the State of DevOps Report, and was acquired by Google in 2018. In 2018 the group also published a widely acclaimed book called Accelerate on building and scaling high performing technology organizations.

## Why are the DORA metrics interesting?

The DORA metrics are especially interesting because they correlate with actual business outcomes and employee satisfaction. In addition, they finally give the software engineering world a set of industry standards to benchmark against. It’s not an overwhelming set of indicators either. Turns out, just 4 key metrics are sufficient to distinguish truly elite engineering teams from mediocre ones.

# So what are the DORA metrics exactly?

The DORA metrics were inspired by lean manufacturing principles. The first two metrics are measures of software delivery velocity. They are:

1. Deployment frequency: *“How often an organization successfully releases to production”*This metric measures the frequency at which an organization successfully releases code to production. There is some latitude in how “production” is defined, depending on a team’s individual business requirements. But in essence, smaller, more frequent releases incur less risk and indicate a more predictable, consistent delivery of value to customers. Elite teams are able to deploy on-demand, typically several times a day, while lower performing teams make more big-bang releases once every several months.

2. Lead Time: *“The amount of time it takes for changes to get deployed to production”*This metric measures how long it takes on average for committed code to reach production. The metric is thus a measure of the efficiency of the DevOps tool chain and processes in an organization. Quicker deployments mean faster value delivery to customers. For elite teams, it typically takes less than an hour from when code gets checked in to when it gets deployed in production.

The next two metrics are measures of quality and stability in software delivery. They are:

3. Change Failure Rate: *“The percentage of deployments that cause a failure in production”*This metric measures the quality and stability of the code that a team is shipping. It is calculated as the percentage of deployments that result in severe service degradation and require immediate remediation such as a rollback or a hotfix. For elite engineering teams, no more than 15% of their deployments result in degraded services.

4. Time to Restoration: *“How long it takes an organization to recover from a failure in production”*And finally, unplanned outages always happen. This last metric measures how quickly organizations can recover from them and restore service availability for the end user. Elite teams typically take less than an hour to restore degraded services.The table below taken from the **State of DevOps Report 2021** summarizes four distinct performance profiles for engineering teams, with statistically significant differences in measures among them.

## How can you measure your DORA metrics?

Measuring and monitoring an organization’s DORA metrics can be difficult because the underlying data needed to compute them often comes from many different systems and isn’t always easy to correlate. For instance, in order to measure the average lead time for changes, you need to be able to compute