Full Infrastructure as Code with Terraform, Ansible and Docker Swarm

It's been 2 years that my team use Docker Swarm in production and for development. Throughout these years, I used Ansible and bash scripts to automate development environment creation and production provisioning and deployment.
I am really sad about the recent Docker news which is a synonym of death for Swarm :( I think it's a really good choice for small deployments and thanks to its shallow learning curve, I think it's a better candidate to spread the DevOps mindset in a team.

Anyway, I wanted to open-source our setup because it's a good DevOps and Infrastructure as Code example, and I hope it will be useful or inspiring for some teams.

[https://github.com/Thomvaill/tads-boilerplate](https://github.com/Thomvaill/tads-boilerplate)

It's my first open-source project, so your feedback will be really appreciated!!

https://redd.it/e1frg5
@r_devops

Project Layout: Import existing AWS infra into terraform project.

This is one of those questions that i'm sure will bring some highly different, for good reasons, thorough, though not incorrect responses and that is what I'm looking for.

I have an existing AWS account with a production, staging, and demo VPCs. I had built all of these servers in scala using the aws-jdk, now i'd like to move to terraform. There are only about 40 ec2 instances across all vpcs. My question is how do you prefer to layout a project like this. There are actually two more vpc's for failover but trying to keep this question simple. I used [terraforming](https://github.com/dtan4/terraforming) to export all aws resources. I was thinking about this sort of structure ([https://aws.amazon.com/blogs/apn/terraform-beyond-the-basics-with-aws/](https://aws.amazon.com/blogs/apn/terraform-beyond-the-basics-with-aws/)) but with vpc based data at a level higher.

I've read quite a bit about terraform, see its elegance, tested a few simple build outs, but migration i would like to get this right the first (or 4th) time. If anyone has any good reading that you may like and/or an ideal/preferred structure, to get me rolling, i would greatly appreciate it.

The software/network installs are all managed via saltstack. With native apps running and some in docker, but don't think that should matter for this.

Thanks in advance!

https://redd.it/e1nr6a
@r_devops

Shall I create elastic search index with timestamp ?

How do you specify elasticsearch index name ?

I am adding timestamp in index name in logstash.conf . Is this correct index naming convention ?

output {
hosts => [https://localhost:9200]
index => "application-logs-%{+YYYY.MM.dd}"
}

I am adding {+YYYY.MM.dd} in index name.

Is it correct way of writing index name ?

https://redd.it/e1kcal
@r_devops

How does nginx authentication work?

I have Nginx + Kerberos authentication working well. When a user hits the endpoint I can determine remote\_user

proxy\_set\_header REMOTE\_USER $remote\_user;

​

But, I am not sure how to setup authentication in my app.

​

Say, I have allowed users as ("Bob","Tom","Bryan"). How can I check in nginx if $remote\_user is in that set?

Is that how it works?

https://redd.it/e1r0r1
@r_devops

The Why-What-How of AWS EC2 Instance Metadata Service update adding defence in depth

About the recently released update to IMDS and how to enable it - [https://blog.appsecco.com/getting-started-with-version-2-of-aws-ec2-instance-metadata-service-imdsv2-2ad03a1f3650](https://blog.appsecco.com/getting-started-with-version-2-of-aws-ec2-instance-metadata-service-imdsv2-2ad03a1f3650)

https://redd.it/e1rx8t
@r_devops

Additional Classes running automatically in Git PR

Hello ! When I made a Pull Request in Git , some of Additional classes are running automatically which are not part of commit - And Pull Request is getting failed

https://redd.it/e1swzl
@r_devops

Docker Swarm vs. k8s-SingleHost Implementations - is this something worthwile?

Since docker swarm seems to be getting smaller/on the down/used less, i wanted to know, what you think as single host replacement is worthy for production.
Is minikube or microk8s or k3s a viable alternative if i have only ONE host and want to manage that instead of docker swarm?

https://redd.it/e1v8jr
@r_devops

Storage options for a kubernetes home server
https://www.reddit.com/r/homelab/comments/e1pi8h/storage_options_for_a_kubernetes_home_server/

https://redd.it/e1uud7
@r_devops

Managing Resources of multiple team on aws via Terraform

Hey Guys,

Wanted to know if anyone has implemented a gitops like workflow for terraform with aws provider.

What I want to achieve is:

1. Each team has access to a single gitrepo which has common modules and
then team specific folders with team specific modules.
2. Person from team A is supposed to make changes only to his project specific folder and
then create a PR
3. Jenkins will run terraform plan check for error and mark the PR successful.
4. Finally sumone needs to approve the merge request and jenkins will run terraform apply
and create resources on aws.

Not able to figure out how to go abt the hierarchy within the gitrepo.

PS: using opensource version of terraform and jenkins.

https://redd.it/e1yncs
@r_devops

UDemy Recommendations that are on sale/

Any recommendations for UDemy courses that are on sale now?

https://redd.it/e1n5zi
@r_devops

Best practices for sharing helm value files

Hi all,

Let me start by saying that I am probably missing something obvious.

I've been playing with helm for 9 months or so and I've built out a helm chart for my companies application. What I am struggling with is enabling the team to collaborate on making changes to deployments for updating image versions and such. We deploy this chart to many environments. Many per cluster, and on multiple clusters. Where I struggle is that we have secrets in the value files and commiting secrets into Git repo's is a no no. I've considered putting the original value files in something like vault, or maybe in a secret in the namespace of the deployment. I've even looked into encrypting and then commiting to Git, but it feels overly complicated. What are other teams doing?

I'll be honest, I haven't done much testing of upgrades to understand the process better, partially just too busy and partially because I've been assuming I need the full original value file to avoid wiping out the original secrets that get deployed.

Thanks in advance for your feedback.

https://redd.it/e1l426
@r_devops

HPE SSD's just die after 32,768 hours of operation

https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-a00092491en_us

https://redd.it/e20edo
@r_devops

What is the CI/CD tool at Tesla/SpaceX?

I vaguely remember that GitLab had Tesla listed as one of their customers a few months ago but it's not there anymore. In general I'd like to know what CI/CD tools are being used in a place where there are a lot of embedded devices/development.

https://redd.it/e20c9p
@r_devops

Help with IBM functions

Hey guys,

I have a problem with the return values of IBM Functions using the "Enable as Web Action" HTTP link. There forum is dead and so is their reddit. Was hoping one of you might be able to help me with this problem.

I have an very simple **Python** function that just raises an Exception:

import sys
def main(dict):
raise Exception("Why you do this IBM?")
return { 'message': 'Hello world' }

When invoking the function using the button on the website this is whats shown under "Results": (looks good)

{
"error": "Why you do this IBM?"
}

But when calling the function via HTTP request (using Postman) this is whats returned:

{
"code": "909f88a14c771bc0dd5592d12456f9ec",
"error": "Response is not valid 'application/json'."
}

Second Example using **Node.js** (again throwing a exception):

function main(params) {
throw "Whyyyy IBM?"
return { message: 'Hello World' };
}

When invoking the function using the button on the website this is the result: (looks good):

{
"error": "An error has occurred: Whyyyy IBM?"
}

And again when calling the function via HTTP request (using Postman) this is whats retuned:

{
"code": "8ad0376f32e811a02aa1db14e72bed72",
"error": "There was an error processing your request."
}

How can i get the same return value as when pressing the button using HTTP requests? Thank you

https://redd.it/e21f2h
@r_devops

Do you guys have any feedback on this postmortems article?

Hi everyone! My team worked with Steve McGhee from Google on this article about crafting excellent postmortems. Do you have any extra tips/feedback on the article? If you like it, please feel free to share :)

[https://www.blameless.com/improve-postmortem-with-sre-steve-mcghee/](https://www.blameless.com/improve-postmortem-with-sre-steve-mcghee/)

https://redd.it/e24ps3
@r_devops

Question About Microservices Login Auth and Microservice DB schema

I recently just started and have been playing around with microservices. and there's a few things that confuses me.

first off for context, i want to create a separate MC for Auth and User and have this db schema for User

​

user table

\---------------

id

name

email

etc

\---------------

​

user\_pass table

\---------------

id

user\_id

pass

salt

alg

etc

\---------------

​

The question i have is the login flow authentication and i have narrowed down 3 options

​

1. a browser request gets sent to -> APIGateway -> User MC for pass details then message queues -> Auth MC -> validate --> send back token

​

2. a browser request gets sent to -> APIGateway -> Auth MC -> gets details straight from db -> validate -> send back token

​

3. a browser request gets sent to -> APIGateway -> User MC -> validate with small auth code -> send back token

​

I'm planning on just doing no.2 but i've read that it's bad to share db between MCs, so is it maybe the db schema that's the problem?

Should i just make user\_pass table a table specific for Auth MC's db and have it query from there? (UserDB.user\_pass --->>> AuthDB.user\_auth)

​

I'm also planning on just making the Auth MC as a ticket checker for JWT in every request or maybe filter that in the Gateway with the exception of the login flow no.2

but that's not any of my concerns right now as i know it could be very broad, my main concern right now is the login flow.

​

So please let me know what i'm doing wrong or if something is not clear. That would be very much appreciated. Thanks in advance.

https://redd.it/e23mbl
@r_devops

These are the steps that I've taken to SSH into a remote server in a Jenkinsfile but failed. Need help!

Hello, I am new to Jenkins, and I am building a pipeline where I have to ssh into a remote server to perform some tasks. I am finding a way to ssh without typing the password. In a general environment (no Jenkins), this would be quite simple. I would have to generate a key (\`ssh-keygen\`) and put the public-key into the remote server's \`authorized\_keys\`.

However, since my goal in Jenkins is to \`ssh\` into the server, I am not able to do that. What I tried was using \`ssh-agent\`. Reading some tutorials, I was told to do something like (this is with \` withCredentials(\[sshUserPrivateKey...\`)

eval $(ssh-agent -s)
chmod 600 ./key_key2.key
ssh-add ./key_key2.key
ssh root@<remote_server> docker ps -v

where \`key\_key2.key\` is the private\_key of the remote server.

Is this the correct way to do it?

I feel like using ssh-agent is skipping the part where I add the public\_key to the remote-server's authorized\_keys, but I thought to skip that part was one of the points of using the public\_key.

&#x200B;

Any help, please?

https://redd.it/e27bcj
@r_devops

introducing devops-pipeline

I'm writing a tool to coordinate other tools traditionally used in a devops environment. It's configured as a DOT graph file. It started out as trying to represent an environment in totality by all the running of different tools. With devops-pipeline, you can represent your entire environment as the pipeline as code that is needed to deploy it fully.

[https://devops-pipeline.com](https://devops-pipeline.com)

Currently I have an example infrastructure that provisions two worker nodes that run builds via SSH, provisions 4 machines in AWS : HashiCorp Vault, Bastion, Prometheus&Grafana, Java machine. Uses an AMI pipeline to bake images.

This is the environment configuration - [https://github.com/samsquire/devops-pipeline-starter/blob/master/architecture.png](https://github.com/samsquire/devops-pipeline-starter/blob/master/architecture.png) devops-pipeline runs against the underlying graph of this picture. @ means runs on local machine, \* means manually triggered. I call attention to packer/source-ami, packer/authenticated-ami, packer/ubuntu-java

This is the pipeline code - [https://github.com/samsquire/devops-pipeline-starter/blob/master/architecture.dot](https://github.com/samsquire/devops-pipeline-starter/blob/master/architecture.dot)

This is the example infrastructure code project: [https://github.com/samsquire/devops-pipeline-starter](https://github.com/samsquire/devops-pipeline-starter)

https://redd.it/e26g03
@r_devops

Datadog Alternatives

Good recommendations for a Datadog alternative?
Need something to:

1. Monitor AWS infrastructure

2. Application Performance Monitor (APM) for Java Applications (spring boot)


Reason: DD is too expensive $(12k) a month.

My thoughts turn to Prometheus, Alertmanager and Grafana....

I am not sure about APM...


I’m interested in what the community has implemented .. or is Datadog the recommended way to go?

https://redd.it/e1zcsr
@r_devops

Ran a free 12 week DevOps Bootcamp on Twitch! Here are the numbers!

Hey everyone! I posted on here a few months back, but I ran a free 12 week DevOps bootcamp on Twitch from Aug-Oct. I absolutely consider it to have been a success and Just wanted to come share the results! These are the averages and totals from the 23 streams:


* Avg. Viewers 38
* Avg. Max Viewers 51
* Total Minutes Watched 115,350
* Total Chat Messages 5,949
* Total Minutes Streamed 3,131(52.2 hours)

Ill be making some tweaks and re-running the bootcamp in starting the second week of January. I will also be running a 12 week Python Bootcamp at the same time if anyone was looking to pick up Pyhton. The updated curriculum as well as dates will go live in the next couple of days on [https://academy.mastermnd.io](https://academy.mastermnd.io), or you can head straight over to [Twitch](https://twitch.tv/mastermndio). Thanks for making the first Bootcamp successful and I look forward to the next!

&#x200B;

P.S. If you missed the first one, All of the streams can be found at this YouTube playlist. They will only stay up until the next bootcamp, then I will replace them with the new recordings.

[https://www.youtube.com/playlist?list=PLleOCN2eBn8IBa07mO-mfUffUIz9LK6B5](https://www.youtube.com/playlist?list=PLleOCN2eBn8IBa07mO-mfUffUIz9LK6B5)

https://redd.it/e2ags6
@r_devops