If you are re-creating whole Kubernetes cluster on deployment, how do you deal with persistent storage?

Hi, I remember seeing someone mentioning that they would do deployments by re-creating whole kubernetes clusters. As much as I like they idea, I can't figure out how to deal with persistent storage in that pattern.
Does it mean you're using completely separate infrastructure for that, such as RDS, or is there another way? I mainly would like to ask if there are any options to use this pattern while having some persistent data on the cluster being re-created - thank you!

https://redd.it/ezj7qz
@r_devops

Question out of curiosity, is there anything out there that will map out a kubernetes application and automatically draw a diagram of how everything connects? Seems like it could be a useful tool for keeping documentation up to date.



https://redd.it/ezba19
@r_devops

Worst or Best DevOps automation

Generally intrigued... I find people are fairly proud of automations that don’t really help much and largely ignore really beneficial automations... case in point:

My (once) proudest moment:

I once wrote a python script that watched syslog for configuration change notifications and auto generated change tickets in the ticketing system. The purpose was not to document work but to annoy my my director who had decided anytime we type config-t a change must be created. His approval of the change took more work than the changes I was making (which were hundreds a day)... shitty I know.

However, TIL that a vbscript I wrote 12 years ago in excel to ping a hostname or IP in a cell in excel is used by several large consulting firms and was asked to update it to work with 64-bit excel on windows 10.

So any good (or bad ones) out there?

https://redd.it/ezn0o3
@r_devops

Recommendation for web monitoring/alerting

I know there are posts already asking for this info, but many of the suggestions are cloud based.

I'm looking for an on-premise solution for web monitoring/alerting. I need to monitor internally developed websites to ensure they are up and working (IIS) and alert if issues are found. No cloud, only on-prem. Thanks.

https://redd.it/ezkcof
@r_devops

Any feedback on concourse?

https://concourse-ci.org/

https://redd.it/ezn3wr
@r_devops

I built a Google assistant integration to tell you about the latest AWS product announcements

Simply say "OK Google, talk to cloud computing news"

Should work on both iOS and Android.

It will give you an itemized list with hyperlinks about the latest product enhancements from AWS.

App details:
https://assistant.google.com/services/a/uid/0000006c6dc51de5

Source code:
https://github.com/circa10a/google-home-aws-news

https://redd.it/ezlj4w
@r_devops

[Prometheus] 'Maintenance' mode for alertmanager

Our problem arises with tons of alerts firing during monthly patching on our windows hosts. We want a way to intelligently (or not) silence them during patch time.

"offline" alerts are only raised if scrapes fail on a host for 3-5 minutes but we usually have some that dont apply patches and reboot as fast as others so about half the fleet raises alerts and we end up with a ton.

Any ideas here?

https://redd.it/ezflas
@r_devops

is ansible_memtotal_mb in megabytes or bytes?

I want to know that is ansible\_memtotal\_mb in bytes or megabytes. Is that for all that has \_mb at the end?

https://redd.it/ezev0y
@r_devops

How to manage a fixed infrastructure?

I work for a company that serve software for gas stations, our job is to provide a software that manage all functionalities, including financial, vendors, attendants, prices, etc.

For this we use many softwares in background, including Postgres database and some self-made python and javascript softwares, where each client have their own database and suit of applications (in many different versions, its a nightmare).

​

I'm looking for a way to automate implantation, deployment and management of new versions, in a robust and trustworthy way, with high avalability because our clients cannot stop.

​

I'm studying docker and swarm, but I dont think that make each client a node in a giant cluster is the ideal solution, because each one need to have their own database, and deployment need to be in each specific client.

​

What is the most robust way of doing this? Is this possible of doing with docker or ansible?

https://redd.it/ez9f6k
@r_devops

[idea] yet another devops website

I have an idea to create website/newsletter with personalized content related only to the technology stack you use (like linux, aws, gitlab ci, docker, ansible).

I've tried some newsletters, but unsubscribed since most of the content was unrelevant to me.

Now I'm using feedly pro, but still it's far from perfect (althouth using self-wrote command line client is great).

What do you think? (I've also set up simple typeform to validate the idea: [https://allaboutdevops.typeform.com/to/tmmfLU](https://allaboutdevops.typeform.com/to/tmmfLU) ).

I hope this post is not against the fourth point of the rules, since it's not a project (yet).

https://redd.it/ezad6a
@r_devops

Kubernetes debugger pod



Hey guys, I wrote a small container (that you can run in a pod) named KDBG packed with tools to help debugging kubernetes.

It runs on Alpine Linux and its pretty lightweight \~ 26mb.

Project link: [https://github.com/nvucinic/kdbg](https://github.com/nvucinic/kdbg)

In addition, I really need advices and feedbacks from you guys! Feel free to post a question or PM me if you get in trouble when trying the project.

Criticism is always welcome, too.

https://redd.it/ez90r0
@r_devops

Creating AMI using packer

I wanted to create AMI from my existing EC2 base instance using packer. And I found chroot builder which create AMI from existing instance. But I don’t understand how to use it to create AMI from existing instance. I read packer documentation but it is not helping. I am going to use this AMI in my auto-scaling group which I deploy using terraform.
If I use terraform to create AMI then every time I make some change in my base instance and apply terraform, terraform will delete my old AMI and create new one which I don’t want. So I am using Packer to create AMI. Need some help in this problem ?

https://redd.it/ez86mc
@r_devops

Statistics for DevOps

Can anyone recommend a good book or course in Statistics for Devops Engineers? I'm looking for one that will aid me in creating (and understanding) advanced visualisations in grafana / prometheus and refresh my knowledge on fundamental concepts of statistics and probability besides covering concepts like linear regression etc. Thanks.

https://redd.it/ez5ebx
@r_devops

Redis Observability

Greetings!


I'll try to keep this as succinct as possible. Essentially, I'm looking for advice or tooling to help us get more insight and observability into our Redis nodes in AWS. The basic metrics like cache misses, etc are not enough.


We are having an issue with our production redis cluster right now that's causing occasional downtime on our site and we need more insight into the inner workings of what's happening on the node. The problem usually presents itself as a sudden spike in connections on the node. We have flow logs setup to see what's connected to it, but that's it. It's just telling us what we already know, that our API nodes are contacting the redis node. The question is, why is this sudden spike happening?


We are aware of the typical redis commands like CLIENT LIST, but we were hoping for something that didn't require us to be manually on the box WHILE an issue was happening to figure out what's going on.


Thank you!

https://redd.it/ezuk2j
@r_devops

Managing SSH user access to AWS instances across multiple accounts.

Hi guys,

I am curious to hear how others are currently achieving this for your teams and/or users?

I recently worked a on a project where I was required to use SSM for all access to instances and at first I really didn't like it. I can safely say now that I'm absolutely an advocate! Our team is now using SSM where possible and taking advantage of no bastions, using AWS IAM for auth and not needing SSH keys stored locally or on servers.

I thought I'd share this small tool I created recently for anyone interested. It's not a completely new concept but I was more so interested in being able to git-manage and centralise config for our team, making sure we're in sync and up-to-date. Obviously it still requires that you actually keep the configs up-to-date or find a way to automate as you provision new instances. I now simply symlink to our git-managed SSH config.

[GitHub - elpy1/ssh-over-ssm: SSH over AWS SSM.](https://github.com/elpy1/ssh-over-ssm)

Any feedback or suggestions would be greatly appreciated!

https://redd.it/ezskxe
@r_devops

Gandalf: An Intelligent, End-To-End Analytics Service for Safe Deployment in Cloud-Scale Infrastructure

This paper describes a system that uses anomaly detection in deployments (PDF): [https://www.usenix.org/system/files/nsdi20spring\_li\_prepub.pdf](https://www.usenix.org/system/files/nsdi20spring_li_prepub.pdf)

https://redd.it/ezuj03
@r_devops

Looking for a reality check with using Docker

I run the tech side of a small web development shop. We're going through some changes with our hosting infrastructure, so I thought this would be a good opportunity to introduce Docker to the mix. However, I haven't ever used Docker or the like, so even after watching a whole bunch of explainer videos, I'm still not totally clear on the specifics of how it will fit into the puzzle.

All of the websites we host are run off the same environment. They're fairly complex sites that utilize a number of different applications: nginx, php-fpm, mysql, elasticsearch, redis, openoffice, node.

My understanding of Docker is that we can configure these applications to run in containers so that we can potentially deploy a secondary hosting environment, or quickly set up a new developer's local environment to mimic production. But what I don't quite understand is how the site-specific configuration gets into place.

So - we've got the applications running. What about the user accounts that each house their specific website files and under which specific php-fpm pools are run? What about the server monitoring tools that keep the applications running? What about the SSL certificates and cron tasks? Where does Docker end and some other tool begin?

https://redd.it/ezx34r
@r_devops

Gitlab vs many integrated tools?

I'm new to DevOps and am currently training. Is there any benefit to choosing Jenkins plus other tools over a complete offering like Gitlab? My understanding may be a bit off so bare with me haha. But gitlab seems like a nobrainer to my untrained self.

https://redd.it/ezrjy5
@r_devops

PHP not Dynamical rendering Select List

Hi,

I am trying to get this PHP Page working, I am trying to populate a Drop Down list based on the Selection of the First List, The First List is population, but unable to get the Second list populated:

I have one Table called Test with four Columns:

`<?php`
`function load_datacenter()`
`{`
`$servername = "localhost";`
`$username = "dev";`
`$password = "/";`
`$dbname="vsphere";`
`$connect = new mysqli($servername, $username, $password, $dbname);`
`$sql = "Select DISTINCT Datacenter from vsphere.test";`
`$result = mysqli_query($connect, $sql);`
`while($datacenter = mysqli_fetch_array($result))`
    `{`
`echo "<option value= $datacenter[Datacenter]>$datacenter[Datacenter]</option>";`
    `}`   
`}`
`?>`
`<html>`
`<head>`
`<title>`
`</title>`
`</head>`
`<body>`
`<p>Select DataCenter`
`<select name="datacenter" id="datacenter">`
`<option value=" ">Select Data Center</option>`
`<?php echo load_datacenter(); ?>`
`</select></p>`
`<p>Select vCenter`
`<select name="vcenter" id="vcenter">`
`<option value="">Select vCenter</option>`
`</select></p>`
`</body>`
`</html>`
`<script>`
`$(document).ready(function(){`
`$('#datacenter').change(function(){`
`var datacenter = $("#datacenter").val();`
`$.ajax({`
            `url:"fetch_vcenter.php",`
            `type:"post",`
            `data:"datacenter="+datacenter,`
`success:function(data){`
`$('#vcenter').html(data);`
            `}`
        `})`
    `});`
`});`
`</script>`

&#x200B;

The Fetch\_vCenter.php

`<?php`
`$servername = "localhost";`
`$username = "dev";`
`$password = "";`
`$dbname="vsphere";`
`$connect = new mysqli($servername, $username, $password, $dbname);`
`$Datacenter=$_POST["Datacenter"];`
`$sql = "select DISTINCT Vcenter FROM vsphere.test WHERE Datacenter='$Datacenter'";`
`$result = mysqli_query($connect, $sql);`
`while($row = mysqli_fetch_array($result))`
    `{`
`echo"<option value=$row[Vcenter]> $row[Vcenter]</option>";`
    `}`   
`?>`

https://redd.it/ezz80k
@r_devops

CoreOS end of Life

https://coreos.com/os/eol/

https://redd.it/ezpbvj
@r_devops