Cargo culting...? [discussion]

It seems like the entire industry is following one "right" approach and whenever something shifts towards some "new" technology everyone just follows without actually thinking about it (ehem, Kubernetes). What is your opinion about cargo culting in the DevOps world and IT in general? Why does it happen - is it lack of education, time constraints... something else? Is it a cause of us reusing too much and trying not to reinvent the wheel?

https://redd.it/dktipc
@r_devops

Kubernetes Patterns : The Service Discovery Pattern

**Folks,**


The client is responsible for determining which service instance it should connect to. A load-balancing layer exists in front of the service instances. The client connects to the well-defined URL of the load balancer and the latter determines which backend service it shall route the request too.

Here's this week article Kubernetes Patterns : The Service Discovery Pattern.

**What's inside:**

* Why do we need Service Discovery?
* inter-app communication Scenario.
* connecting to an outside resource Scenario.
* Why use a Service to connect to an external resource?
* Accepting outside connections Scenario.

[https://www.magalix.com/blog/kubernetes-patterns-the-service-discovery-pattern](https://www.magalix.com/blog/kubernetes-patterns-the-service-discovery-pattern)

https://redd.it/dl0j1j
@r_devops

Policy as a code services

Hi guys,

Any recommendations/reviews on Policy/Compliance as a code services?

[Sentinal](https://www.hashicorp.com/sentinel/) by Hashicorp looks interesting.

Thanks!

https://redd.it/dl18gd
@r_devops

When can someone call him/herself a DevOps Guy?

I'm just getting into things as it's my internship going on.

Was just curious about when can someone call him/herself a devops engineer? i.e. what skills/knowledge/achievements are the ones that are very very necessary? And experiences too?

I am just a beginner so I don't know much about it.

https://redd.it/dl51v1
@r_devops

Ideas for maintaining SOX compliance with feature toggles and frequent deployment?

Feature toggles (or feature flags) enable me to deploy unfinished features without comprehensive manual testing of features toggled off. We can toggle the feature on for certain users, try it out, and then choose to toggle it off or on for all users. This enables more frequent integration as suggested by version control strategies such as [Trunk-Based Development](https://trunkbaseddevelopment.com/), and more frequent deployment of the trunk (master) branch into user's hands.

Sarbanes-Oxley rules require that all features that are in user's hands be fully tested.

So if I have a SOX-compliant system, how do I proceed with using feature toggles without having to do full manual testing of the feature toggled ON? It makes sense to do regression testing with the feature toggled OFF, but the most common need is to integrate and deploy good code but not incur the manual testing cost for a feature toggled off.

**So any suggestions on ways to have SOX-compliant feature toggles without comprehensive manual testing on every release?**

https://redd.it/dl60t4
@r_devops

Open source tool to track software release for various products

Looking for a simple way to track software releases across various products. Ideally, I’d be able to notate what releases an organization should stay away from due to known vulnerabilities or bugs.

Any recommendations for something to use?

https://redd.it/dlaq8c
@r_devops

How to install Traefik 2.x on a Docker Swarm

Here's a little article that I've wrote about the installation of Traefik on a Docker Swarm. This will be the first article on a series on swarm. May I have some reviews?

Hope it helps someone :)

[https://creekorful.me/how-to-install-traefik-2-docker-swarm](https://creekorful.me/how-to-install-traefik-2-docker-swarm/)

https://redd.it/dl7j28
@r_devops

What are the best things I can do as a Linux Admin to move towards an official devops role?

Hi All, as the title mentions I am a Linux Admin at an MSP. After debate over the last few years between going the route of cyber security, devops, or developer, I've settled on Devops.

I am curious, what are the best things I can do right now to get there? For a bit of background, I have started learning puppet and python. I'm fairly decent with bash scripting, and I would like to learn Ansible. Some of it is hard, as it's all in my free time, since things it isn't directly applicable to my current work. For Linux I should hopefully have my RHCE by the end of the year. I have a homelab that I work on to try and further my own knowledge. I don't have a degree, and I'm fairly young. I like to believe, I am a fast learner.

Would working on something like the AWS Solutions Architect be the best? Are there broader concepts I should understand? Networking is a weaker area and I've been trying to improve upon that. Any guidance you have I would love to hear!

​

Edit: Forgot to add - I am aware of the the sticky on the front page, which has been a massive help! Just curious if there is any more granular feedback/insights for my case

https://redd.it/dlcevp
@r_devops

Is it time for /r/learningDevOps?



https://redd.it/dlgrp9
@r_devops

What are the basic Kubectl commands to know?

I often get the same questions about Kubectl, so I wrote [an article about the basic Kubectl commands](https://www.padok.fr/en/blog/kubectl-cluster-kubernetes). Check it out and give me your opinion. Do you see a command I should add?

https://redd.it/dljee8
@r_devops

Ansible vs Airflow

Hi,

I know both tools are different but when it comes to a use case where you need to execute a series of tasks (i.e move this file from a to b, edit this file, then move again) in a specific order and idempotently why to use Airflow and not stick with the good ol Ansible?

https://redd.it/dlj2v1
@r_devops

Vault + Consult + EKS?

Hi everyone!

Have worked quite a bit with ECS in the past, and I'm trying to spec out a new POC for EKS + Vault. My knowledge of Vault is pretty limited so far, so I'm trying to tie everything together. Was hoping you all could help.

We're using Terraform, so it seems that creating secrets in Vault could be somewhat trivial.

The area that I'm kind of grey on is, how are other folks injecting/providing the secrets to their application deployments, without implementing Vault at the application-code level?

For example, we're using Kong and it needs certain environment variables set to connect to the DB. How would you guys go about making that happen?

Thanks in advance!

https://redd.it/dlixqh
@r_devops

google/schedviz: A tool for gathering and visualizing kernel scheduling traces on Linux machines. Built with Go
https://github.com/google/schedviz

https://redd.it/dlltza
@r_devops

Good pairing between Terraform and Docker, for the non-ops person?

I'm not an ops by trade. I do however have a few side projects that require servers, domains, etc.

I was going to deploy it by hand, and my project is small enough in scale that doing it by hand would be *totally fine*. With that said, I often find that over long periods of time "by hand" results in a bunch of servers that I forget exactly how I setup. So, I'm looking to not have it be by hand.

My original plan was to manually deploy Docker on a machine to handle configuring the application environment; as I have a good amount of experience with Docker, and am not a big fan of Ansible _(though I've never tried Chef/etc)_. So my goal is to stick with Docker.

Terraform seems like a good tool for provisioning so I plan on using that to manage creation of my infra. However that leaves a bit of a gap; what would I use to configure the machines that Terraform provisions?

I'm in a bit of a weird place. I do not have a private Docker registry, nor did I plan to make one. So to bridge the gap between Terraform and my application running in a container, will I still need to use something like Ansible to configure the machines and deploy new containers?

Just seeking some advice from seasoned folks. My goal is simple. Docker, to me is simple. Terraform looks elegant, and simple. In between is where I'm a bit stuck at what will be simple; and not have me confused when I come back to it in 6 months. Advice appreciated, thank you.

https://redd.it/dl8jrv
@r_devops

Is "infrastructure" a two-pizza team?

Dear Father Bezos,

I recently converted to the Religion of Microservices, and I endeavor to follow all the commandments: microservices-shall-own-their-data, microservices-shall-own-end-to-end-delivery-of-their-business-domain, "You build it; you run it," etc. As a Believer, I know it's bad to create a "database team" and an "application team" because then I've structured the two-pizza teams based on function rather than business domain. The teams ought to include DevOps/Operations pros who know how to deploy and operate the service, just as they include developers and testers, which in other religions work in separate departments.

Selah.

Monitoring, CI/CD, Kubernetes, DB admin... as I add up everything these teams must do, I find it increasingly difficult to see how these small teams can do everything we need a production service to do. Terraform makes Infra something developers can incorporate, but getting all the routing and permissions and CI/CD integration right... I can see why some have left the Faith and keep a dedicated Operations team for these things. At what point do those things become a business domain in themselves, and how do I avoid that becoming a functional team all over again? To take just one example: Kubernetes isn't easy; won't the teams have a hard time managing it themselves?

When I ask these questions, I hear a lot of, "there's no right answer," and "it totally depends on the size and shape of the company." I suppose that's true. I guess what I'm asking, Father, is how should I go about drawing these team+service boundaries? Specifically when it comes to things like infrastructure, Kubernetes, load balancing, logging, monitoring, CI/CD, how can each team be expected to do all those things well?

Tell the Saints I said, "hi."

Sincerely,
A True Believer

https://redd.it/dl4ass
@r_devops

Vanilla Kubernetes is completely over my head...

Anyone else feel the same way? Like, I can use EKS or GCE all day long, but vanilla kubernetes built more of less from scratch? Just so fucking lost.

I'm honestly strongly considering switching careers. I started out as a Linux admin where the most difficult thing was trying to fix a package that failed to install, and I've failed upwards to the point that I've "risen to incompetence" as they say.

Might be time to go to culinary school.

https://redd.it/dlnlo4
@r_devops

Anyone here worked on Apach Nifi or zookeeper

I have a question
Can zookeeper fail over when there are only 2 nodes?

Leader and worker

To test failovers, I close down the Server that has been selected as Cluster Coordinator

In this case, zookeeper should automatically elect the remaining one server as leader. But it keeps failing and goes into continuous loop of trying to connect to the first server

Zookeeper Logs in Server 2 when leader (Server 1) went down:


-----—------------------
2019-10-22 18:44:01,135 [myid:2] - WARN [NIOWorkerThread-2:NIOServerCnxn@370] - Exception causing close of session 0x0: ZooKeeperServer not running
2019-10-22 18:44:02,925 [myid:2] - WARN [NIOWorkerThread-3:NIOServerCnxn@370] - Exception causing close of session 0x0: ZooKeeperServer not running
2019-10-22 18:44:03,320 [myid:2] - WARN [QuorumPeer[myid=2](plain=/0:0:0:0:0:0:0:0:2181)(secure=disabled):QuorumCnxManager@677] -
Cannot open channel to 1 at election address ec2-server-1.compute-1.amazonaws.com/172.xx.x.x:3888
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

-----—------------------


Server 2 Config files:

zoo.cfg


-----—------------------
tickTime=2000
initLimit=5
syncLimit=2
dataDir=/home/ec2-user/zookeeper
clientPort=2181
server.1=ec2-server-1.compute-1.amazonaws.com:2888:3888
server.2=0.0.0.0:2888:3888

nifi.properties

nifi.cluster.is.node=true
nifi.cluster.node.address=ec2-server-2.compute-1.amazonaws.com
nifi.cluster.node.protocol.port=8082
nifi.cluster.flow.election.max.wait.time=2 mins
nifi.cluster.flow.election.max.candidates=1

# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=localhost:2181
nifi.zookeeper.root.node=/nifi

-----—------------------

Server 1 Config files:

zoo.cfg

-----—------------------

tickTime=2000
initLimit=5
syncLimit=2
dataDir=/home/ec2-user/zookeeper
clientPort=2181
server.1=0.0.0.0:2888:3888
server.2=ec2-server-2.compute-1.amazonaws.com:2888:3888

nifi.properties

nifi.cluster.is.node=true
nifi.cluster.node.address=ec2-server-1.compute-1.amazonaws.com
nifi.cluster.node.protocol.port=8082
nifi.cluster.flow.election.max.wait.time=2 mins
nifi.cluster.flow.election.max.candidates=1

# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=localhost:2181
nifi.zookeeper.root.node=/nifi

-----—------------------


What am I doing wrong?

https://redd.it/dlmptz
@r_devops

How to Build a Platform Team

Hello all,

I'd like to share this [blog](https://lightstep.com/blog/how-to-build-a-platform-team/) (that's part of a series) on what I've seen work and not work when trying to move to "platform" based devops. There are some very different parts of the lifecycle of building a team and ways that I've seen things go wrong.

Does this match others' experience?

What might be missing?

What tradeoffs do you all see in platform vs less centralized approaches?

Best,

1mentat

https://redd.it/dlmj7w
@r_devops

Looking for some advice!

I have recently changed job and moved into a DevOps role. The role itself is fine and it’s everything that I expected. However, this new company has a separate department for networking (NOC), Part of application sits on top of their network which we have no control over. For context the application is VOIP related so it’s very dependent on a stable network. It is also has issues with NAT, else I would simply move the whole stack into the cloud to resolve the issue but that’s simply not possible at this time.

The problem is they continue to have issues with packet loss and outages but our team seems to get most of the blame (because all regular people see is the application failing, despite it normally being the networks not being set up correctly).

They don’t seem to care about the trouble it causes our department. They also outright refuse to move with the times. A lot of there work is repeatable and could be easily done with ansible but they would rather do everything manually but then complain they never have time setup monitoring for their equipment etc...

Just wondering how you deal with this sort of situation and if anyone has experienced this sort thing before?

https://redd.it/dlvgx6
@r_devops

Starting with terraform

Hi a Jr Deployment engineer here, i am starting to use terraform with aws and some gcp for IaaC can you give me some advices or materials to start and get the right knowledge in little time? Thanks.

https://redd.it/dljdvt
@r_devops

terraform vs ansible question: ssh key management for a large number of EC2 servers

hi guys,
I'm new to ansible and terraform.
we have to frequently manage a large number of EC2 servers for ssh - each server may have a different set of users. I know what is the current set of users i want there (meaning - i may have had a set of users there yesterday...but today there's a different set of users. so delete the ones that are no longer needed).
In a lot of ways it seems like what Terraform was designed for, but i cant find any information about users/ssh creation through terraform. Everyone says this is for terraform.

But I cant seem to find any terraform documentation around "desired final state" for users/ssh.

Any help/advice/sample code would be much appreciated

https://redd.it/dlzxvu
@r_devops