Advice Request Infrastructure + Configuration + Deployment Management Stack

Hi all,

I am strating to automate the infrastructure and the configuration for a personal project. And I am using the following tools:

Terraform: AWS EC2 instances, RDS and other resources management (creation, destruction etc..)
Ansible: Configuration management (installing all the required packages)
K8s: Apps deployment
Jenkins: CI/CD with Gitlab

I want to know what do you guys think about this stack? Are better ways to do what I want to acheive? I am not a devops expert and I want to have a reference from more experienced folks.

Thanks in advance

https://redd.it/lkenwy
@r_devops

How to improve?

Hi!
How do you guys improve yourselves for your job? does your company give you any tools/resources to do so? how did you guys manage or managed when you started working?

https://redd.it/lkea0p
@r_devops

The role of "Devops Engineer" in 2021?

Hi all! I'm going to be interviewed for a position with title Devops Engineer, which is a bit controversial since devops should be a practice/way of working instead of separate role or position. Based on the requirements on job ad, it looks more like a cloud infrastucture engineer since technologies like AWS, Kubernetes, Terraform etc. were mentioned alongside CI/CD, Golang, ELK etc.


The question is, what would you expect from a role like this in 2021 and what questions would you ask to clarify the job responsibilities? I have a background in backend development with lots of cloudy+devopsy stuff which I like, but I like coding a lot too and wouldn't want to drop it completely.

https://redd.it/lkcakv
@r_devops

Modernizing Workflow

Hello everyone,

at work I am part of a two-man IT/development/sysadmin/... team supporting a team of academics. Two main tasks I am tackling are:

1. development of small, one-shot web apps that need to be hosted indefinitely
2. Continuous development of larger web apps

Both are hosted on-premise, my publish workflow currently consists of:

1. Marking a new release via a git tag
2. Have Gitlab-CI create docker images for me
3. Use SaltStack to manually update which container version runs on a single Docker host. Some HA is achieved by the hypervisor which runs the Docker host.
4. Manually update ports/hostname in the Apache reverse proxy.

While this works ok, I feel I need some advice into modernizing this workflow. All the manual pushing/updating is very error-prone and slow while I am finding it hard to monitor all these apps, especially the ones that are out of development.

Additionally, I find myself needing to run batch jobs (on schedule or triggered by data transfers) that are either Python or Fortran code with a complex chain of dependencies. I thought about making my app start Docker containers with pre-prepared images containing required libraries but I am not sure how to approach this.

Will using something like Nomad/Consul or Kubernetes make all this easier to work with? Especially the deployment and monitoring workflow. Most of this stuff is not critical (production-lite) as of now so it's easy to try various stuff.

https://redd.it/lk9vhm
@r_devops

Managing Azure DevOps Service Principals

I have written up a simple blog post on how to reconfigure Service Principals so they are clearer and have better security within Azure DevOps. Hope this helps some people

https://jamescook.dev/cleanup-azure-devops-service-principals

https://redd.it/lkb59p
@r_devops

biggest struggles as a software developer (all levels)

Hi,

which are your main struggles as software developers?

https://redd.it/lke7v2
@r_devops

Stderr monitoring in k8s

Hello guys, i was talking with few developers from my company today and they asked me is there a way to get alerts via slack when there is an error in deployment, all of the things i can think of atm are metrics only, do you have any idea what would be the best way to handle something like that? I would like to avoid having some 3rd party apps like slack bots having access to my namespaces and pods, I also deployed ELK a few weeks ago, so is there maybe some elk alerting service that would come in handy?

Thank you!

https://redd.it/lkogzq
@r_devops

Curated GitHub repository on how organisations around the world practice SRE and DevOps

I’m curating engineering blog posts, videos and other cool resources on Site Reliability Engineering ans DevOps in a GitHub repository

https://github.com/upgundecha/howtheysre

https://redd.it/lkr952
@r_devops

What are some must know technologies for devops?

I am learning devops after being a developer and I want to know what are the must know technologies and which ones I should start with learning first and are the most essential and which are the least essential and anything in between. I am currently learning docker and jenkins. I have heard some about kuberneties and terraform but not sure how important it is to have each in a server.

https://redd.it/lksygo
@r_devops

CircleCI vs Jenkins

Just wanted to know if CircleCI was offering something new that Jenkins hasn't offered before??

https://redd.it/lklsrn
@r_devops

Estimate cost per user for cloud infrastructure

Lets say that I have a product which allows users to sign up and sell personal services (think craigslist..etc). How do I estimate cost per user, cost per month..etc.

Basically, I am trying to figure out, how much should I be charging user so that I am atleast breaking even, and I want this trend to scale with users.

Is there a formula or other tools that allow to do this end to end cost estimation and growth simulation ?

https://redd.it/lko5ym
@r_devops

K8s & Consul - Avoid cross-DC/AZ traffic

I have huge app in Kubernetes with about 90 containers on 5 k8s worker nodes. Nodes are spread between Availability Zones in AWS, like that:

node1 - AZ-A
node2 - AZ-A
node3 - AZ-B
node4 - AZ-B
node5 - AZ-C

Also, I got many small apps which use API from that huge app. All apps use external/public host of that app, so actually all traffic between apps goes through ELB.

I can use internal Kubernetes hostnames (now we got coreDNS), but I would like to avoid cross-zone traffic, so how can I implement consul to be AZ aware and prefer traffic between pods in the same AZ, e.g.

Diagram

Something similar but with HAProxy and Redis:

https://medium.com/helpshift-engineering/saving-inter-zone-transfer-costs-in-ec2-with-haproxy-c4217ac2ca04

https://redd.it/lkkeat
@r_devops

Where to get help?

Where to get help if you get stuck using or learning devops solutions? For example: Ansible, Jenkins...etc

The reason I am asking is for the past 2 days, Ansible wouldn't run playbooks for me. It took me 2 days to figure out the problem Which is that i installed the needed libraries for python3 while ansible was running on python2.

https://redd.it/lkkdtd
@r_devops

Azure DevOps Demo Generator

Just starting poking around here in this subreddit and I suspect I may be spending a lot of time digging around in here the next few months. :) I just stumbled into this Azure DevOps Demo Generator article from a few days ago and thought I'd share it in case anyone else finds it helpful.

The Azure DevOps Demo Generator | Azure DevOps Blog (microsoft.com)

Also just a side note, I'm aware Microsoft's unfortunate naming of "Azure DevOps" the tool is confusing along side general "DevOps" concepts of technologies LOL...but if you're someone in Microsoft's bubble, it's like a tool you'll likely end up running into.

https://redd.it/lky7hg
@r_devops

KubeHelper - simplify many daily Kubernetes tasks through a web interface

KubeHelper \- simplifies many daily cluster tasks through a web interface. You can easily do search, analysis, run commands, cron jobs, reports, filters, git synchronization and many more.

KubeHelper is not yet another attempt to display the Kubernetes API in a GUI. Not an attempt to replace Lens, the official Dashboard, or other similar products. This is my humble contribution to the Kubernetes Open Source community. KubeHelper does not have any highly specialized direction and contains quite a few different functions that will be useful in daily work with Kubernetes.

Read full article on Medium

GitHub

https://redd.it/lkk90p
@r_devops

Polling an API or MYSQL query to do alerting and monitoring?

Is anybody using polling of an API or MySQL query to make sure everything is normal?

Like alerting if the number of failed transactions in a time window is greater than the threshold, or if the API response has a field with a value beyond acceptable limits.

What tools do you use to do such alerting and monitoring?

https://redd.it/lkzkiv
@r_devops

Does anyone else have experience running spinnaker as their main CD strategy?

Does anyone else have experience running spinnaker as their main CD strategy?

https://redd.it/lkjje7
@r_devops

Create signed certificates

My CTO sent me a zip containing a .crt, .csr, .key and a .ca-bundle.

I was under the impression that I could use them to created signed certificates for use in nginx (or similar) for using https on our various subdomains.

I'm unsuccessfully trying something like the following

ROOTCAKEY=.mydomain.com2020.key
ROOTCA=.mydomain.com2020.crt
ROOTCSR=.mydomain.com2020.csr

openssl genrsa -out auth-proxy.api.mydomain.com.key 2048

openssl req -new -sha256 -key auth-proxy.api.mydomain.com.key -subj "/C=US/ST=CA/O=MyOrg, Inc./CN=auth-proxy.api.mydomain.com" -out auth-proxy.api.mydomain.com.key.csr

openssl x509 -signkey $ROOTCAKEY -in auth-proxy.api.mydomain.com.key.csr -req -days 365 -out auth-proxy.api.mydomain.com.crt

#openssl x509 -req -in auth-proxy.api.mydomain.com.key.csr -CA $ROOTCA -CAkey $ROOTCAKEY -CAcreateserial -out auth-proxy.api.mydomain.com.crt -days 500 -sha256
#echo crt=$(base64 -w 0 auth-proxy.api.mydomain.com.crt)
#echo
#echo key=$(base64 -w 0 auth-proxy.api.mydomain.com.key)

Can anyone here point me in the right direction?

https://redd.it/lkgvll
@r_devops

Kubernetes README

https://kubernetesreadme.com/

https://redd.it/ll7t5p
@r_devops

Multi tenant platform for running docker containers

Guys, i'm looking for open source platform for my users to sign up, request pool of resources and then run containers in each own pool. I have a cluster of virtual machines i want to split between my users (even multiple users on same VM).

I tried kubernetes+dashboard with RBAC/namespace resource allocation. While it perfectly limits cpu/memory usage, but couldn't limit disk space usage per namespace. Even with small persistent volume limits i could open container and create 10 Gb files inside without any restriction. Besides some malefactor can possible mess up cluster for every1 else.

My other idea was to run cloudstack to spin up small VMs for each user, but my CPUs don't have virtualization enabled.

I like how Loft is using virtual k3s clusters, but was hoping to find free alternative.

Lxd and kata containers looks interesting. I hope to find open source service to manage cluster preferably with web-ui. Or any other solution that comes to mind.

https://redd.it/ll7cz0
@r_devops

Need help understanding the concept of stateful serverless.

Hi all,


I'm trying to understand how tools, like cloudstate, from an architecture meaning.
I get it why FAAS are no good for working with data because you need to call the database every time that the function runs and it's gonna take some time.
But I'm struggling to understand how this tool is working that you can have both serverless architecture and stateful functions?


Thanks!

https://redd.it/llb6gg
@r_devops