Kubernetes and Istio service mesh workshop materials

Hi,

I have open-sourced the materials (exercises) I delivered as part of my Kubernetes and Istio service mesh workshop.

**Kubernetes basics**

1. Installing Kubernetes and the CLI (kubectl)
2. Running containers
3. Accessing services
4. Scaling up/down
5. Config maps and secrets
6. Health checks: liveness, readiness and startup probes
7. Resource quotas

**Installing Istio** **Traffic Management**

1. Accessing services through a gateway
2. Splitting traffic
3. Using a service entry

**Service Resiliency**

1. Dashboards (Grafana, Jaeger, Kiali)
2. Slowing down the services
3. Breaking the services

**Security with Istio**

1. Using JWT for authentication
2. Enabling RBAC
3. Service accounts


Check it out here: [https://www.learncloudnative.com/blog/2020-02-04-kubernetes\_and\_istio\_workshop\_materials/](https://www.learncloudnative.com/blog/2020-02-04-kubernetes_and_istio_workshop_materials/)

https://redd.it/ezg7os
@r_devops

How do you migrate existing infrastructure from Docker to Kubernetes?

I had two servers running various Docker Compose projects (well, technically they are all Portainer powered). Now I had some success story with Kubernetes (specifically, it's k3s), I've got ingress & inlets working, now I want to move my current infrastructure to k8s so that I can have that nice horizontal scaling.

However, I had no idea how to migrate it, that there is no way to upload Docker volume as a persistent volume, the rest could be easily handled with some modification of outputs of Kompose. At least hidr them secrets please.

I used to migrate my old SonarQube compose file. That "works" until I got database error. Now database migration is less PITA as I can at least dump SQL files into different places and rerun them on my own. What's youe story of dealing those situation otherwise?

https://redd.it/ezedk0
@r_devops

Automatically generate terraform code by scanning your cloud

Hey everyone, I hope many find these useful.

I found this CLI tool called [terraformer](https://github.com/GoogleCloudPlatform/terraformer) which scans your cloud and generates terraform code. The auto generated code looks a bit weird so I created another CLI tool called [terrafactor](https://github.com/Ghost---Shadow/terrafactor). It refactors the terraform code to make it a bit more readable. Finally, I also created a [visualizer](https://github.com/Ghost---Shadow/terraform-board) to make sure it did its job.


C&C please. Thanks

https://redd.it/ezbbus
@r_devops

Overcoming YAML Pain in Continuous Integration

Hi everyone, this is my first post in /r/devops. I’m very passionate about build systems and continuous integration. A few years ago, I started developing my own build system, with the goal to solve problems of existing solutions, but also to be more able to research on my own ideas (usually that doesn’t work well with other peoples code :)).

Anyway, my post is titled “Overcoming YAML Hell In DevOps”. Please don’t scream at me! I’m a .NET developer, hence, the title is a little broad for this sub, but in essence, it’s about continuous integration and authoring pipelines. More concrete _generating pipelines_ from an existing C# implementation of the build. I’ve received good feedback from many folks in the .NET ecosystem, yet I want to spread this idea to other communities (or check if this has already been done), and also get feedback what you think could be improved. And now finally the link! https://ithrowexceptions.com/2020/02/04/yaml-hell-in-devops.html

https://redd.it/ezi8fg
@r_devops

If you are re-creating whole Kubernetes cluster on deployment, how do you deal with persistent storage?

Hi, I remember seeing someone mentioning that they would do deployments by re-creating whole kubernetes clusters. As much as I like they idea, I can't figure out how to deal with persistent storage in that pattern.
Does it mean you're using completely separate infrastructure for that, such as RDS, or is there another way? I mainly would like to ask if there are any options to use this pattern while having some persistent data on the cluster being re-created - thank you!

https://redd.it/ezj7qz
@r_devops

Question out of curiosity, is there anything out there that will map out a kubernetes application and automatically draw a diagram of how everything connects? Seems like it could be a useful tool for keeping documentation up to date.



https://redd.it/ezba19
@r_devops

Worst or Best DevOps automation

Generally intrigued... I find people are fairly proud of automations that don’t really help much and largely ignore really beneficial automations... case in point:

My (once) proudest moment:

I once wrote a python script that watched syslog for configuration change notifications and auto generated change tickets in the ticketing system. The purpose was not to document work but to annoy my my director who had decided anytime we type config-t a change must be created. His approval of the change took more work than the changes I was making (which were hundreds a day)... shitty I know.

However, TIL that a vbscript I wrote 12 years ago in excel to ping a hostname or IP in a cell in excel is used by several large consulting firms and was asked to update it to work with 64-bit excel on windows 10.

So any good (or bad ones) out there?

https://redd.it/ezn0o3
@r_devops

Recommendation for web monitoring/alerting

I know there are posts already asking for this info, but many of the suggestions are cloud based.

I'm looking for an on-premise solution for web monitoring/alerting. I need to monitor internally developed websites to ensure they are up and working (IIS) and alert if issues are found. No cloud, only on-prem. Thanks.

https://redd.it/ezkcof
@r_devops

Any feedback on concourse?

https://concourse-ci.org/

https://redd.it/ezn3wr
@r_devops

I built a Google assistant integration to tell you about the latest AWS product announcements

Simply say "OK Google, talk to cloud computing news"

Should work on both iOS and Android.

It will give you an itemized list with hyperlinks about the latest product enhancements from AWS.

App details:
https://assistant.google.com/services/a/uid/0000006c6dc51de5

Source code:
https://github.com/circa10a/google-home-aws-news

https://redd.it/ezlj4w
@r_devops

[Prometheus] 'Maintenance' mode for alertmanager

Our problem arises with tons of alerts firing during monthly patching on our windows hosts. We want a way to intelligently (or not) silence them during patch time.

"offline" alerts are only raised if scrapes fail on a host for 3-5 minutes but we usually have some that dont apply patches and reboot as fast as others so about half the fleet raises alerts and we end up with a ton.

Any ideas here?

https://redd.it/ezflas
@r_devops

is ansible_memtotal_mb in megabytes or bytes?

I want to know that is ansible\_memtotal\_mb in bytes or megabytes. Is that for all that has \_mb at the end?

https://redd.it/ezev0y
@r_devops

How to manage a fixed infrastructure?

I work for a company that serve software for gas stations, our job is to provide a software that manage all functionalities, including financial, vendors, attendants, prices, etc.

For this we use many softwares in background, including Postgres database and some self-made python and javascript softwares, where each client have their own database and suit of applications (in many different versions, its a nightmare).

​

I'm looking for a way to automate implantation, deployment and management of new versions, in a robust and trustworthy way, with high avalability because our clients cannot stop.

​

I'm studying docker and swarm, but I dont think that make each client a node in a giant cluster is the ideal solution, because each one need to have their own database, and deployment need to be in each specific client.

​

What is the most robust way of doing this? Is this possible of doing with docker or ansible?

https://redd.it/ez9f6k
@r_devops

[idea] yet another devops website

I have an idea to create website/newsletter with personalized content related only to the technology stack you use (like linux, aws, gitlab ci, docker, ansible).

I've tried some newsletters, but unsubscribed since most of the content was unrelevant to me.

Now I'm using feedly pro, but still it's far from perfect (althouth using self-wrote command line client is great).

What do you think? (I've also set up simple typeform to validate the idea: [https://allaboutdevops.typeform.com/to/tmmfLU](https://allaboutdevops.typeform.com/to/tmmfLU) ).

I hope this post is not against the fourth point of the rules, since it's not a project (yet).

https://redd.it/ezad6a
@r_devops

Kubernetes debugger pod



Hey guys, I wrote a small container (that you can run in a pod) named KDBG packed with tools to help debugging kubernetes.

It runs on Alpine Linux and its pretty lightweight \~ 26mb.

Project link: [https://github.com/nvucinic/kdbg](https://github.com/nvucinic/kdbg)

In addition, I really need advices and feedbacks from you guys! Feel free to post a question or PM me if you get in trouble when trying the project.

Criticism is always welcome, too.

https://redd.it/ez90r0
@r_devops

Creating AMI using packer

I wanted to create AMI from my existing EC2 base instance using packer. And I found chroot builder which create AMI from existing instance. But I don’t understand how to use it to create AMI from existing instance. I read packer documentation but it is not helping. I am going to use this AMI in my auto-scaling group which I deploy using terraform.
If I use terraform to create AMI then every time I make some change in my base instance and apply terraform, terraform will delete my old AMI and create new one which I don’t want. So I am using Packer to create AMI. Need some help in this problem ?

https://redd.it/ez86mc
@r_devops

Statistics for DevOps

Can anyone recommend a good book or course in Statistics for Devops Engineers? I'm looking for one that will aid me in creating (and understanding) advanced visualisations in grafana / prometheus and refresh my knowledge on fundamental concepts of statistics and probability besides covering concepts like linear regression etc. Thanks.

https://redd.it/ez5ebx
@r_devops

Redis Observability

Greetings!


I'll try to keep this as succinct as possible. Essentially, I'm looking for advice or tooling to help us get more insight and observability into our Redis nodes in AWS. The basic metrics like cache misses, etc are not enough.


We are having an issue with our production redis cluster right now that's causing occasional downtime on our site and we need more insight into the inner workings of what's happening on the node. The problem usually presents itself as a sudden spike in connections on the node. We have flow logs setup to see what's connected to it, but that's it. It's just telling us what we already know, that our API nodes are contacting the redis node. The question is, why is this sudden spike happening?


We are aware of the typical redis commands like CLIENT LIST, but we were hoping for something that didn't require us to be manually on the box WHILE an issue was happening to figure out what's going on.


Thank you!

https://redd.it/ezuk2j
@r_devops

Managing SSH user access to AWS instances across multiple accounts.

Hi guys,

I am curious to hear how others are currently achieving this for your teams and/or users?

I recently worked a on a project where I was required to use SSM for all access to instances and at first I really didn't like it. I can safely say now that I'm absolutely an advocate! Our team is now using SSM where possible and taking advantage of no bastions, using AWS IAM for auth and not needing SSH keys stored locally or on servers.

I thought I'd share this small tool I created recently for anyone interested. It's not a completely new concept but I was more so interested in being able to git-manage and centralise config for our team, making sure we're in sync and up-to-date. Obviously it still requires that you actually keep the configs up-to-date or find a way to automate as you provision new instances. I now simply symlink to our git-managed SSH config.

[GitHub - elpy1/ssh-over-ssm: SSH over AWS SSM.](https://github.com/elpy1/ssh-over-ssm)

Any feedback or suggestions would be greatly appreciated!

https://redd.it/ezskxe
@r_devops

Gandalf: An Intelligent, End-To-End Analytics Service for Safe Deployment in Cloud-Scale Infrastructure

This paper describes a system that uses anomaly detection in deployments (PDF): [https://www.usenix.org/system/files/nsdi20spring\_li\_prepub.pdf](https://www.usenix.org/system/files/nsdi20spring_li_prepub.pdf)

https://redd.it/ezuj03
@r_devops