What do you monitor/alert on your Kubernetes cluster(s)?

I'm setting up some production kubernetes clusters and looking to see what others monitor/alert on their clusters to give me some inital ideas past the very basic CPU/MEM/Network

​

I'm using Prometheus and Alertmanagerwith Grafana for visualization if that makes much difference.

​

PS, I'm thinking of this as more discussion than a direct ask for help so forgive a lack of specific details here.

https://redd.it/ek4an3
@r_devops

Ephemeral Environments with Gitlab and Microservices?

I've been requested to bring ephemeral environments to my development team (they're dividing up into a few separate teams) , and we're currently looking into using Gitlab. I'm supporting a single Springboot application with a dozen microservices and a single database. Is Gitlab going to be the best solution to deploy all of the microservices at once into ephemeral environments?


We're dividing the teams up into a backend developer, a frontend developer, and a QA resource. The overall goal is that the backend and frontend developer can work on a single feature at a time and then have the QA person test afterwords. I'm new to Gitlab and I'm having a little trouble understanding how/if this can be easily done. Has anyone done anything similar to this using Gitlab?

https://redd.it/ek91mp
@r_devops

Hi DevOps,

Hi DevOps,

Here i created playlist of my videos.

One place to learn Linux,Python,Ruby,Git,Github,Jenkins,Teamcity,Chef,Ansible,Docker,Kubernetes,Aws,Azure,Nagios.

real-time hands on videos. Nearly we have 100 videos .

subscribe my channel for more upcoming videos

Devops video Tutorials

\----------------------

Linux Videos Play list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76ILhSIu6m9p\_Cr4-SeFosD-y](https://www.youtube.com/playlist?list=PLsgnv1SN76ILhSIu6m9p_Cr4-SeFosD-y)

Python videos Play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76IKsK24E1MHQ9aT9OpvGl5NR](https://www.youtube.com/playlist?list=PLsgnv1SN76IKsK24E1MHQ9aT9OpvGl5NR)

Ruby Videos-Playlist:

[https://www.youtube.com/playlist?list=PLsgnv1SN76IJb7KCj98rSz7FSjYtTn6Yg](https://www.youtube.com/playlist?list=PLsgnv1SN76IJb7KCj98rSz7FSjYtTn6Yg)

Git videos Play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76IL13nvGWcrJoUWOfxLQB9XA](https://www.youtube.com/playlist?list=PLsgnv1SN76IL13nvGWcrJoUWOfxLQB9XA)

Jenkins videos Play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76IJXVCrAxk-LK5Iy9jt-SGm6](https://www.youtube.com/playlist?list=PLsgnv1SN76IJXVCrAxk-LK5Iy9jt-SGm6)

Teamcity Videos Play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76ILDYbKJdKs2sbgRAZfhl4\_F](https://www.youtube.com/playlist?list=PLsgnv1SN76ILDYbKJdKs2sbgRAZfhl4_F)

Chef Videos Play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76ILtD3TnVtXpX1hmwjyY9OuT](https://www.youtube.com/playlist?list=PLsgnv1SN76ILtD3TnVtXpX1hmwjyY9OuT)

Ansible video:

[https://www.youtube.com/watch?v=kmE\_WKCdbdA](https://www.youtube.com/watch?v=kmE_WKCdbdA)

Docker videos play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76IIIhgEJDZfrF8m1o85-J7O8](https://www.youtube.com/playlist?list=PLsgnv1SN76IIIhgEJDZfrF8m1o85-J7O8)

Kubernetes videos play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76ILS5Z\_VYJWm8KIoicAjFjCX](https://www.youtube.com/playlist?list=PLsgnv1SN76ILS5Z_VYJWm8KIoicAjFjCX)

Aws Videos Play-list :

[https://www.youtube.com/playlist?list=PLsgnv1SN76IJU7JGlAMcIRBGpu279\_lud](https://www.youtube.com/playlist?list=PLsgnv1SN76IJU7JGlAMcIRBGpu279_lud)

Azure Videos Playlist:

[https://www.youtube.com/playlist?list=PLsgnv1SN76ILiWjq0mMW32wggNASNsp3P](https://www.youtube.com/playlist?list=PLsgnv1SN76ILiWjq0mMW32wggNASNsp3P)

Packer Videos Play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76IK78KZTqyGaws2q17vM7HAp](https://www.youtube.com/playlist?list=PLsgnv1SN76IK78KZTqyGaws2q17vM7HAp)

Terraform Videos Play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76IIrgVnNCi-URqEkPFRN4LWh](https://www.youtube.com/playlist?list=PLsgnv1SN76IIrgVnNCi-URqEkPFRN4LWh)

Nagios Videos Play-list:

[https://www.youtube.com/playlist?list=PLsgnv1SN76ILOMxlUmTTCoxFUq74mps51](https://www.youtube.com/playlist?list=PLsgnv1SN76ILOMxlUmTTCoxFUq74mps51)

https://redd.it/ek8jwa
@r_devops

What tools are you using the keep track of ECR vulnerability scan reports?

Originally, scan reports can be observed manually via Console for each repository, which is not really great in case you have 100+ of them. AWS also mentions two "ECR" native solutions to tackle this problem:

\- One involving a bunch of Lambda functions (calling the Lambdas via bash and channel them into an ATOM feed

\- The other involves Athena to query scan findings (I find it a bit of an overkill)

As for me, I have created two Lambda functions which enables me to posts daily vulnerability scan findings digested into Slack.

I am curious if it is a problem for anyone else and which tools are you using to tackle this problem?

https://redd.it/ekhsnj
@r_devops

Certs in DevOps

Hey All,

Wondering about the communities perception on this. Is anyone else feeling like needing certs is dwindling down? It seems like more and more people that reach out about jobs or that post job descriptions are not even asking for certifications anymore. Is anyone seeing the same thing?

https://redd.it/ekfiwj
@r_devops

Configure Jenkins as code with jcasc plugin

Here are steps what I used to configure Jenkins with jcasc plugin that can help to setup Jenkins to fully configure with job or pipeline you want.
[GitHub](https://github.com/Rishabh-Tamrakar/Jenkins-Configuration-as-Code-jcasc/blob/master/README.md)

https://redd.it/ekau4f
@r_devops

I created an app to show how to use Ansible, along with Ansible-Vault to create and configure a simple webserver on AWS.

I mainly use Terraform so I wanted to see what it was like to both create and configure a server using Ansible.

I created a demo app to show how to use Ansible, along with Ansible-Vault to store AWS credentials, to create and configure a simple webserver on AWS.

It works, though I think I'll stick with Terraform for this kind of thing. Since Ansible doesn't store any state, it seems to get off track easily enough when making changes to instances or security groups

[https://github.com/gordonmurray/ansible\_webserver\_aws](https://github.com/gordonmurray/ansible_webserver_aws)

https://redd.it/ekbh5o
@r_devops

Randomly started having an issue with let's encrypt and SSL certs

For the past month or so I've been building / destroying a Kuberentes cluster about once a day sometimes twice but not often. I have been using scripts this whole time which haven't changed except I added some echo statements to them. I am using cert-manger as my method of getting a cert from lets encrypt.

​

The issue I am having is it wil not issue a cert and let me see in chrome that it has a valid ssl cert just uses the fake cert

This is as far as it gets

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Requested 28m cert-manager Created new CertificateRequest resource "letsencrypt-dev-1423799144"

​

Below are my scripts and my ingress/cert resources.

​

kops create cluster --name=k8s.<domain>.com --master-zones us-east-1a \
--zones us-east-1a,us-east-1b,us-east-1c --node-count=1 --node-size=t3.medium \
--master-size=t3.small --dns-zone=k8s.<domain>.com --vpc vpc-<VPC-ID> \
--topology public --networking flannel --ssh-public-key "~/.ssh/id_rsa.pub"

kops update cluster --name k8s.<domain>.com --yes

cd ~/Projects/k8/namespaces
kubectl apply -f .

helm repo add jetstack https://charts.jetstack.io
helm repo update

kubectl apply --validate=false -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.12/deploy/manifests/00-crds.yaml

helm install \
cert-manager-dev \
--namespace cert-manager-dev \
--version v0.12.0 \
jetstack/cert-manager

helm install ingress stable/nginx-ingress -n development

cd ~/Projects/k8/ingress
kubectl apply -f dev.yaml

Then I wait and get the LB name and put that in my DNS provider (cloudflare) and wait about an hour then I check my local to see if I can connect to the api server via postman and I can so I go ahead and apply the lets encrypt file

kubectl apply -f letsencrypt-dev.yaml

&#x200B;

dev.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
# add an annotation indicating the issuer to use.
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-dev"
# needed to allow the front end to talk to the back end
nginx.ingress.kubernetes.io/cors-allow-origin: "https://app.<domain>.com"
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, PUT, POST, DELETE, PATCH, OPTIONS"
name: dev-ingress
namespace: development
spec:
rules:
- host: api.<domain>.com
http:
paths:
- backend:
serviceName: api
servicePort: 8090
path: /
tls: # < placing a host in the TLS config will indicate a certificate should be created
- hosts:
- api.<domain>.com
secretName: dev-ingress-cert # < cert-manager will store the created certificate in this secre

letsencrypt-dev.yaml

apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: letsencrypt-dev
namespace: cert-manager-dev
spec:
commonName: me@<domain>.com
secretName: letsencrypt-dev-tls
issuerRef:
name: letsencrypt-dev
---
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-dev
namespace: cert-manager
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
# server: https://acme-staging-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: [email protected]
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-dev
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:

class: nginx

https://redd.it/eklfek
@r_devops

Help with travis CI

I have linked by github repo with Travis CI. It is configured to upload files to AWS S3 and perform an invalidation on AWS CloudFront. It is uploading some files like [README.md](https://README.md) which I DON'T want in S3.

Here is the log file snippet from travis showing it is uploading [README.md](https://README.md):

[33mDeploying application [0m
uploading "assets/lex.png" with {:content_type=>"image/png"}
uploading "package-lock.json" with {:content_type=>"application/json"}
uploading "js/aws-sdk-2.585.0.min.js" with {:content_type=>"application/javascript"}
uploading "package.json" with {:content_type=>"application/json"}
uploading "dist/main.js" with {:content_type=>"application/javascript"}
uploading "index.html" with {:content_type=>"text/html"}
uploading "README.md" with {:content_type=>"text/markdown"} <====================
uploading "css/styles.css" with {:content_type=>"text/css"}

Here is the full log:

[https://api.travis-ci.com/v3/job/272569969/log.txt](https://api.travis-ci.com/v3/job/272569969/log.txt)

This is my .travis.yml config:

language: node_js
node_js:
- '10'
before_deploy:
- yarn global add travis-ci-cloudfront-invalidation
deploy:
provider: s3
access_key_id: $AWS_ACCESS
secret_access_key: $AWS_SECRET
bucket: $S3_BUCKET
region: $AWS_REGION
skip_cleanup: true
file_glob: true
file:
- "index.html"
- assets/*
- dist/*
- js/*

on:
branch: master
after_deploy:
- travis-ci-cloudfront-invalidation -a $AWS_ACCESS -s $AWS_SECRET -c $CLOUDFRONT_ID -i '/*' -b $TRAVIS_BRANCH -p $TRAVIS_PULL_REQUEST

It looks like I configured travis to pull certain files. As I'm new to travis not sure if I have a configuration error. Note that build itself succeeds.

https://redd.it/eklf47
@r_devops

Understanding member base of DevOps SE

Howdy!

A couple of years ago, the founder of DevOps Stack Exchange Q&A community asked for participation.

Today, that SE community counts about 16K members, while r/devops has 128K memebers.

How would you explain this large difference?

https://redd.it/ekkhsd
@r_devops

What's the point of doing tests on a build server (e.g. Jenkins, Circle CI, GitHub Actions) when you can just do pre-commit/push hooks and test locally?

I'm still new to DevOps, so I'm finding it hard to understand. Why can't I just have some pre-push hook that will run some automated tests locally on my machine? (This is with Docker). Why do I need a build server that does the exact same thing, but in the cloud?

Is there a point in doing the same thing twice? Is there something that you can run with Jenkins that you can't do locally?

https://redd.it/ekc9b7
@r_devops

Deployment tools

Hey guys, I'm trying to find a solution for the deployment of our code to our servers with an option to roll back versions, currently, we are not working in a docker environment so I'm trying to find other solutions please suggest me one

Thanks!

https://redd.it/ek9zl2
@r_devops

How do you guys measure Python proficiency?

Hard to measure so it's hard to track my progression and knowing if I'm a candidate for some roles...thanks in advance.

https://redd.it/ekqt3c
@r_devops

I have a docker image that runs Python interpreter. How do I handle this?

* Newbie to docker
* Currently you submit code in a json format like \[{code: "print(1+2)"}\] and it returns response in a similar format.
* I am planning to let users on the website compile python code
* Should I have one docker image per user/per session/ or do I use the same image for everyone
* Also when people do stuff like import numpy , how do I handle the libraries? Should I download and keep every single library or is there a better solution?
* Thanks for the suggestion

https://redd.it/ekqwfi
@r_devops

On-Prem Email Queue

Hi All,

Looking for an assist, I need to create a mail queue for our custom apps BEFORE they are sent to our regular SMTP server. Our IT department is constantly having issues with their SMTP Servers and we often lose emails, so what we're trying to do is have all our app emails funneled through our own queue which stores and then sends to our internal IT-provided SMTP server. If there is an error then it waits and then resends - ideally we'd have some sort of logging and see the contents of the email as well. Any help would be appreciated.

I guess this would be some sort of Mail Transfer Agent with a GUI.

https://redd.it/el0an5
@r_devops

Network engineer trying to go devops. Before I spend a fortune on the udemy sale what classes can I start free elsewhere?

Or better to do cbtnuggets or cloud guru

Edit: thank you all and please keep the comments coming. I’m choked up what a great community reddit can be for those willing to ask for help.

https://redd.it/el03t2
@r_devops

Can you simplify for me one thing about Varnish Cache?

The documentation says:

>Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents

Its pretty straightforward, but the case whenever I can use it seems to be "it depends".

On example if I have a site that with the help of backend language (ie. PHP or Python) generates the HTML output containing current epoch timestamp then it's not cache'able because every page refresh would return different second on the page.

However, if instead of the epoch timestamp I would print current day, then Varnish could suite better since the cache could be invalidated once per 24 hours... oh wait, no, it depends where you live since if you live in USA the date may be different than ie. in Europe...

If I start googling for a Varnish Cache in front of popular ecommerce shops (a very dynamic content), then there are some configuration examples so this has to somehow work, or be helpful at least.

I cannot understand how this work to bring more help rather than pain

https://redd.it/ekzxx2
@r_devops

Update excel data through ansible

I have data in my excel about 140 lines. I want to add a next line through ansible, Instead of adding manually. My next line 141 should come through ansible. Existing file to update new line from ansible.

https://redd.it/ekyqoq
@r_devops

Unit Tests coverage integration with Datadog

My team has finally decided to integrate the coverage for unit tests (created by devs) with datadog. If anyone has experience with datadog or another tool, how was the experience? I've been told DD can get pretty expensive. It is being used for mostly UI tests currently.

https://redd.it/ekyq3t
@r_devops