How to install Traefik 2.x on a Docker Swarm

Here's a little article that I've wrote about the installation of Traefik on a Docker Swarm. This will be the first article on a series on swarm. May I have some reviews?

Hope it helps someone :)

[https://creekorful.me/how-to-install-traefik-2-docker-swarm](https://creekorful.me/how-to-install-traefik-2-docker-swarm/)

https://redd.it/dl7j28
@r_devops

What are the best things I can do as a Linux Admin to move towards an official devops role?

Hi All, as the title mentions I am a Linux Admin at an MSP. After debate over the last few years between going the route of cyber security, devops, or developer, I've settled on Devops.

I am curious, what are the best things I can do right now to get there? For a bit of background, I have started learning puppet and python. I'm fairly decent with bash scripting, and I would like to learn Ansible. Some of it is hard, as it's all in my free time, since things it isn't directly applicable to my current work. For Linux I should hopefully have my RHCE by the end of the year. I have a homelab that I work on to try and further my own knowledge. I don't have a degree, and I'm fairly young. I like to believe, I am a fast learner.

Would working on something like the AWS Solutions Architect be the best? Are there broader concepts I should understand? Networking is a weaker area and I've been trying to improve upon that. Any guidance you have I would love to hear!

​

Edit: Forgot to add - I am aware of the the sticky on the front page, which has been a massive help! Just curious if there is any more granular feedback/insights for my case

https://redd.it/dlcevp
@r_devops

Is it time for /r/learningDevOps?



https://redd.it/dlgrp9
@r_devops

What are the basic Kubectl commands to know?

I often get the same questions about Kubectl, so I wrote [an article about the basic Kubectl commands](https://www.padok.fr/en/blog/kubectl-cluster-kubernetes). Check it out and give me your opinion. Do you see a command I should add?

https://redd.it/dljee8
@r_devops

Ansible vs Airflow

Hi,

I know both tools are different but when it comes to a use case where you need to execute a series of tasks (i.e move this file from a to b, edit this file, then move again) in a specific order and idempotently why to use Airflow and not stick with the good ol Ansible?

https://redd.it/dlj2v1
@r_devops

Vault + Consult + EKS?

Hi everyone!

Have worked quite a bit with ECS in the past, and I'm trying to spec out a new POC for EKS + Vault. My knowledge of Vault is pretty limited so far, so I'm trying to tie everything together. Was hoping you all could help.

We're using Terraform, so it seems that creating secrets in Vault could be somewhat trivial.

The area that I'm kind of grey on is, how are other folks injecting/providing the secrets to their application deployments, without implementing Vault at the application-code level?

For example, we're using Kong and it needs certain environment variables set to connect to the DB. How would you guys go about making that happen?

Thanks in advance!

https://redd.it/dlixqh
@r_devops

google/schedviz: A tool for gathering and visualizing kernel scheduling traces on Linux machines. Built with Go
https://github.com/google/schedviz

https://redd.it/dlltza
@r_devops

Good pairing between Terraform and Docker, for the non-ops person?

I'm not an ops by trade. I do however have a few side projects that require servers, domains, etc.

I was going to deploy it by hand, and my project is small enough in scale that doing it by hand would be *totally fine*. With that said, I often find that over long periods of time "by hand" results in a bunch of servers that I forget exactly how I setup. So, I'm looking to not have it be by hand.

My original plan was to manually deploy Docker on a machine to handle configuring the application environment; as I have a good amount of experience with Docker, and am not a big fan of Ansible _(though I've never tried Chef/etc)_. So my goal is to stick with Docker.

Terraform seems like a good tool for provisioning so I plan on using that to manage creation of my infra. However that leaves a bit of a gap; what would I use to configure the machines that Terraform provisions?

I'm in a bit of a weird place. I do not have a private Docker registry, nor did I plan to make one. So to bridge the gap between Terraform and my application running in a container, will I still need to use something like Ansible to configure the machines and deploy new containers?

Just seeking some advice from seasoned folks. My goal is simple. Docker, to me is simple. Terraform looks elegant, and simple. In between is where I'm a bit stuck at what will be simple; and not have me confused when I come back to it in 6 months. Advice appreciated, thank you.

https://redd.it/dl8jrv
@r_devops

Is "infrastructure" a two-pizza team?

Dear Father Bezos,

I recently converted to the Religion of Microservices, and I endeavor to follow all the commandments: microservices-shall-own-their-data, microservices-shall-own-end-to-end-delivery-of-their-business-domain, "You build it; you run it," etc. As a Believer, I know it's bad to create a "database team" and an "application team" because then I've structured the two-pizza teams based on function rather than business domain. The teams ought to include DevOps/Operations pros who know how to deploy and operate the service, just as they include developers and testers, which in other religions work in separate departments.

Selah.

Monitoring, CI/CD, Kubernetes, DB admin... as I add up everything these teams must do, I find it increasingly difficult to see how these small teams can do everything we need a production service to do. Terraform makes Infra something developers can incorporate, but getting all the routing and permissions and CI/CD integration right... I can see why some have left the Faith and keep a dedicated Operations team for these things. At what point do those things become a business domain in themselves, and how do I avoid that becoming a functional team all over again? To take just one example: Kubernetes isn't easy; won't the teams have a hard time managing it themselves?

When I ask these questions, I hear a lot of, "there's no right answer," and "it totally depends on the size and shape of the company." I suppose that's true. I guess what I'm asking, Father, is how should I go about drawing these team+service boundaries? Specifically when it comes to things like infrastructure, Kubernetes, load balancing, logging, monitoring, CI/CD, how can each team be expected to do all those things well?

Tell the Saints I said, "hi."

Sincerely,
A True Believer

https://redd.it/dl4ass
@r_devops

Vanilla Kubernetes is completely over my head...

Anyone else feel the same way? Like, I can use EKS or GCE all day long, but vanilla kubernetes built more of less from scratch? Just so fucking lost.

I'm honestly strongly considering switching careers. I started out as a Linux admin where the most difficult thing was trying to fix a package that failed to install, and I've failed upwards to the point that I've "risen to incompetence" as they say.

Might be time to go to culinary school.

https://redd.it/dlnlo4
@r_devops

Anyone here worked on Apach Nifi or zookeeper

I have a question
Can zookeeper fail over when there are only 2 nodes?

Leader and worker

To test failovers, I close down the Server that has been selected as Cluster Coordinator

In this case, zookeeper should automatically elect the remaining one server as leader. But it keeps failing and goes into continuous loop of trying to connect to the first server

Zookeeper Logs in Server 2 when leader (Server 1) went down:


-----—------------------
2019-10-22 18:44:01,135 [myid:2] - WARN [NIOWorkerThread-2:NIOServerCnxn@370] - Exception causing close of session 0x0: ZooKeeperServer not running
2019-10-22 18:44:02,925 [myid:2] - WARN [NIOWorkerThread-3:NIOServerCnxn@370] - Exception causing close of session 0x0: ZooKeeperServer not running
2019-10-22 18:44:03,320 [myid:2] - WARN [QuorumPeer[myid=2](plain=/0:0:0:0:0:0:0:0:2181)(secure=disabled):QuorumCnxManager@677] -
Cannot open channel to 1 at election address ec2-server-1.compute-1.amazonaws.com/172.xx.x.x:3888
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

-----—------------------


Server 2 Config files:

zoo.cfg


-----—------------------
tickTime=2000
initLimit=5
syncLimit=2
dataDir=/home/ec2-user/zookeeper
clientPort=2181
server.1=ec2-server-1.compute-1.amazonaws.com:2888:3888
server.2=0.0.0.0:2888:3888

nifi.properties

nifi.cluster.is.node=true
nifi.cluster.node.address=ec2-server-2.compute-1.amazonaws.com
nifi.cluster.node.protocol.port=8082
nifi.cluster.flow.election.max.wait.time=2 mins
nifi.cluster.flow.election.max.candidates=1

# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=localhost:2181
nifi.zookeeper.root.node=/nifi

-----—------------------

Server 1 Config files:

zoo.cfg

-----—------------------

tickTime=2000
initLimit=5
syncLimit=2
dataDir=/home/ec2-user/zookeeper
clientPort=2181
server.1=0.0.0.0:2888:3888
server.2=ec2-server-2.compute-1.amazonaws.com:2888:3888

nifi.properties

nifi.cluster.is.node=true
nifi.cluster.node.address=ec2-server-1.compute-1.amazonaws.com
nifi.cluster.node.protocol.port=8082
nifi.cluster.flow.election.max.wait.time=2 mins
nifi.cluster.flow.election.max.candidates=1

# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=localhost:2181
nifi.zookeeper.root.node=/nifi

-----—------------------


What am I doing wrong?

https://redd.it/dlmptz
@r_devops

How to Build a Platform Team

Hello all,

I'd like to share this [blog](https://lightstep.com/blog/how-to-build-a-platform-team/) (that's part of a series) on what I've seen work and not work when trying to move to "platform" based devops. There are some very different parts of the lifecycle of building a team and ways that I've seen things go wrong.

Does this match others' experience?

What might be missing?

What tradeoffs do you all see in platform vs less centralized approaches?

Best,

1mentat

https://redd.it/dlmj7w
@r_devops

Looking for some advice!

I have recently changed job and moved into a DevOps role. The role itself is fine and it’s everything that I expected. However, this new company has a separate department for networking (NOC), Part of application sits on top of their network which we have no control over. For context the application is VOIP related so it’s very dependent on a stable network. It is also has issues with NAT, else I would simply move the whole stack into the cloud to resolve the issue but that’s simply not possible at this time.

The problem is they continue to have issues with packet loss and outages but our team seems to get most of the blame (because all regular people see is the application failing, despite it normally being the networks not being set up correctly).

They don’t seem to care about the trouble it causes our department. They also outright refuse to move with the times. A lot of there work is repeatable and could be easily done with ansible but they would rather do everything manually but then complain they never have time setup monitoring for their equipment etc...

Just wondering how you deal with this sort of situation and if anyone has experienced this sort thing before?

https://redd.it/dlvgx6
@r_devops

Starting with terraform

Hi a Jr Deployment engineer here, i am starting to use terraform with aws and some gcp for IaaC can you give me some advices or materials to start and get the right knowledge in little time? Thanks.

https://redd.it/dljdvt
@r_devops

terraform vs ansible question: ssh key management for a large number of EC2 servers

hi guys,
I'm new to ansible and terraform.
we have to frequently manage a large number of EC2 servers for ssh - each server may have a different set of users. I know what is the current set of users i want there (meaning - i may have had a set of users there yesterday...but today there's a different set of users. so delete the ones that are no longer needed).
In a lot of ways it seems like what Terraform was designed for, but i cant find any information about users/ssh creation through terraform. Everyone says this is for terraform.

But I cant seem to find any terraform documentation around "desired final state" for users/ssh.

Any help/advice/sample code would be much appreciated

https://redd.it/dlzxvu
@r_devops

InSpec profile organization with Ansible

I work at a company whose software is primarily deployed on-premise by our customers. We are transitioning to a cloud offering that will be deployed and managed by our team. Our software consists of a number of components installed on Windows compute instances. The number of machines in each customer deployment vary greatly from customer to customer.

To enable deployment of our software in the cloud, we have created a number of Ansible roles. These roles are then collected into a couple playbooks and run against a custom inventory file for each customer. We are really happy with how this is working out for us.

The next step in our cloud evolution is to audit these deployments with InSpec. We are struggling with deciding how to organize the InSpec profiles. One option we played with was to place each InSpec profile in the tests directory of the corresponding Ansible role. We came across an Ansible role that would actually run the InSpec profile. The thought was to have a task at the end of each role that runs its own InSpec profile for validation.

This approach would work well as the software is being deployed, but I'm not sure how we would re-run these profiles after (we want to regularly run the InSpec profiles for compliance and auditing purposes). I can't just re-run all the Ansible automation as its not all idempotent, and I don't believe it can all easily be made idempotent.

Any thoughts on this problem? How do you organize your InSpec profiles?

https://redd.it/dlkoeh
@r_devops

They're hiring. But do they want me, or just my code? The big issue with DevOps hiring.

This is the second time this has happened to me.

For the purposes of confidentiality, I will refer to this as CompanyX. I will also iterate that this has happened twice now.

As part of a DevOps position, I am being asked to complete a technical exercise.

That sounds sensible to me, but let me cut straight to the point - I believe that the gravity and sheer volume of the exercise leads me to believe that the DevOps role is being used as a pawn for CompanyX to obtain a technical deployment solution for their technology. Therefore, I believe that they are not actually hiring, they’re looking for code solutions.

In more depth, the technical exercise is divided into 3 main sections of requirements with around 15 sub tasks. In the shortest summary possible this involves:

* a self-contained modularisable docker image build: obfuscated, cyphonized, minified Flask-App for \*TWO\* generic GitHub Python Flask examples which is then uploaded.
* a docker-compose solution for multiple containers, exposing relevant ports for connectivity, with best practices that include volume exposure,
* a deployment of the generated image as a load balanced cluster with workers. (I would use K8), with a provisioned port. Handling extensive load of requests and automatically spinning up and down containers.
* A continuous integration pipeline with Travis to support the docker image, suitable for deployment on AWS.
* Other python scripts for optimising the workflow.
* Testing the correct balancing on a simulated load with an entire testing phase (python scripts to check)

Im not looking for solutions or correction on this, so don’t dive into what was asked of me.

But I can tell you for sure, as a seasoned DevOps engineer with a masters in computer science, the above technical exercise is not something you can just stroll through in a few hours or days. It requires a lot of requirements engineering, designing and optimisation. It also requires a lot of questions too. I’ve worked clients before on a contractual basis and full-time in a role where this sort of task would take between 4-6 months to do properly, and I would charge around £10k+ for reasons like it collides so many different technologies together and needs to be scalable for environments like the cloud. Not trivial. And before you say that DevOps roles require more than just this - maybe, but not a huge amount more than maintenance and optimisation.

Sure, as a company you should ask technical questions like this. But not on this scale. This is essentially completing the job, before you started the job. And doing it for free. If we take the perspective of CompanyX, its actually quite smart to think of a list of requirements for your tech stack, and put it out as a job interview exercise, get the solution back, integrate their solution, and then reject the candidate. But its highly immoral.

I want to know from you guys - any relevant experiences with companies’ technical exercises as such?

Does anyone have a similar story?

Does anyone have a response for something like this?

How would you tackle this?

Thankssss a lot in advance everyone :)

https://redd.it/dlysgj
@r_devops

Security tools for DevOps ecosystem

In company I work for we have multiple devops teams which work using different tool sets. Trying to ensure security of such environment was horror so I have decided to create something like hub for security tools(scanners and audit tools) which is ment to be easily accessible for any kind of CICD.
Functionalities so far:
- service discovery (integration with openstack and GCE)
- SAST testing ( integration with micro focus fortify and checkmarx)
- DAST testing (integration with acunetix and w3af)
- network scanners ( integration with Nessus 6, openvas, nexpose)
- CIS compliance (for docker and k8s)

Standard input:
- possibility to run any kind of test via API - doesn’t matter which tools are being used by security, API is the same, ( for example running SAST on every merge request or DAST during integration testing)
Standard output:
- security quality gateway, deploy or no deploy dependent on the result of provided tests.

I am really interested if anyone else would be interested in such tool, or what else should it implement so You would be interested?

https://redd.it/dm2ekm
@r_devops

A task for aspiring devops engineers

You got a developer who needs to create, build and deploy an application to an infrastructure, but doesn't want to deal with systems (no time, other priorities, should never really touch systems without experience with them).

Your job is to automate deployment of infrastructure required for creating, building and deploying the project, let's start:

This is your developer source code:

https://github.com/spring-projects/spring-petclinic

What do you do with it? Just build locally and copy binaries to some fresh VM? Let's do it the right way:

----

First - let's create a simple infrastructure:

We will need a set of Ansible playbooks for following components (don't use Ansible Galaxy directly, it's bad practice, only as inspiration):

* Gitlab

* Jenkins (from which you have to authenticate to Gitlab)

* Grafana and Prometheus

* One application dev server

* Two application production servers

* Load balancer

Obviously we are missing secrets management, authentication services, gathering logs and so on, but it was suppose to be simple.

Goal is to deploy whole thing using a single playbook. Prometheus is suppose to grab metrics from all VMs, load balancer and our application.

Now that we have our infrastructure, let's get into the meat:

* Create Gitlab project and clone Spring Petclinic repository to it
* Create two new branches: dev and prod (usually we would have staging too, but we are keeping it simple af)
* Create two Jenkins pipelines which do the following on each commit - fetches code from `dev` or `prod` branches, builds our code, runs tests, deploys to relevant dev or prod servers

When a developer pushes changes to branch `dev`, Jenkins should automatically build, test and deploy to `dev`where developer can further test the application before merging the code into `prod` from which same process takes place, but results in a production deployment.

----

This is super simplified and you can choose different technologies, but that's basically what devops is about - glueing systems and development world into something coherent and automated.

P.S. Feel free to post ideas for this task or entire new tasks, but let's keep it simple.

https://redd.it/dm3mdj
@r_devops

Managing configuration files inside of a Kubernetes cluster

First, I'm sorry if it sounds confusing, wrong or really simple, I'm still at the beginning of my devops role.

​

I'm starting to work with Kubernetes to migrate a Java application to a pod (as a first step).

My application:

Currently this application obtain it's configurations from a couple of files: a application.properties (1 per app) and client-name.properties (1 or more per app). These are simple files that have multiple configurations such as which address/port should a client use or SMTP configurations, all of the are key/value configs.

From what I saw there is configuration management such as ConfigMaps and Secrets to change simple configs inside of a pod in a key/value fashion. My problem is that I can have n clients per app and n\*(client-name.properties) files, therefore it seems that using a ConfigMap ends up as a hassle (if it's even possible).

​

Is there a (better) way to manage this multiple configurations that my app uses today?

​

Any help and comments are appreciated.

Thanks.

https://redd.it/dm4ozl
@r_devops