infrastructure-as-code: yaml/hcl vs general purpose programming framework

Hi Devops!

As the title suggest what are your preferences and thoughts regarding this? Pro's and con's? Would be interesting to hear your thoughts.

I honestly havn't made up my mind what the best approach is atm. I've been using Terraform and Cloudformation for quite some time (strongly favours Terraform).

As great as Terraform is there are always times when I wished that I had general purpose programming constructs to work with, like if/else statements, loops and what not. Terraform have added some features in this regards however does not feel 100% natural, often feels like I'm fighting the dsl.

Recently Pulumi and aws cdk has popped up, where instead of a dsl (yaml/hcl) you write in javascript or your favourite programming language to provision your infra. From my understanding you get state and resource dependency graphs (the thing that makes an IaC tool worthwhile).

https://redd.it/gbhv65
@r_devops

Why I got rid of our dev, test, staging and prod environment

Hi Reddit, I wanted to share a process/concept I introduced where I work for how we manage our environments.


I'm sure many of you are all aware of the usual dev, test, staging and prod environments and application changes move through these stages to finally get released to the end user. A problem me and my team had was environment bottleneck where for example devs would finish a feature but couldn't move it to the next stage because QA were still testing the previous feature in the next environment. Developers would also develop locally but if they wanted to test on the more closer to production like dev environment they risked wiping out a another devs current changes so there were constant slack messages along the lines of "Can I deploy X to Y" and you hoped someone would reply before you overwrote something you shouldn't have.


We are already a team that embraces infrastructure as code and our environment were brought up in an automated consistent matter. The problem was there was a 1 to many relationship between our environment stages and team members.


So since we can bring up an environment with code, why fix ourselves to 4? I called the concept color environments (but really you can use anything that has an an essentially infinite pool of options to choose from). Now when we work on a feature we deploy to a random color that isn't already in use and our stack gets a domain to access it based on that i.e. "cyan.example.com".


We've been doing this for half a year now and it has drastically changed our development and deployment process for the better.


* Developers can spin up their feature without waiting for an env to be available
* QA can test against a devs color or re-create a new color on their branch
* Our product owner can be given a color env with a feature to review it for as long as they need
* We can do user research and AB testing between colors
* Environment drift is not an issue as colors dont stay up very long and we always create an env from scratch
* Our deployment to prod is just bring up a new color and do a blue green DNS flip


Theres a few hurdles we had to overcome so here's a few of the main ones:


Spinning up infinite of environments can be costly. We're on AWS so took advantage of services like Lambda and other serverless services to keep costs right down. Our environments are also ephemeral by default and after a few days of being brought up they destroy themselves unless configured otherwise (such as prod envs or features that are taking longer to develop).


We gained extra flexibility with our environments but that also came with extra complexity and time spent waiting for an environment to be available. The application stack we did this on was fairly small and we found the sweet spot for time to getting a new env up from scratch ~15 minutes. Enough time to grab a coffee and not too long, updates to an env after are much quicker once its up. For that reason I don't recommend this for large application stacks, maybe this could work for a part of a stack such as a micro-service that is part of a bigger monolith.


Databases and blue green flips can be a bit tricky. Luckily since blue/green deploys are not a new thing there were a few resources out there to help us with this.


Anyways thats a quick rundown of the concept, hope it's something interesting. Has anyone else done something similar? Also if you have any questions about the concept/process let me know :)

https://redd.it/gbhtk0
@r_devops

Does Devops and Windows mix?

Honest question, does anyone practicing DevOps actually enjoy working with Windows? Besides using MS centric languages & frameworks is there any benefit to running over linux?

It seems like more of a hinderance due to the lack of tooling support (Windows brings more costs/licensing so dev resources usually go to *nix first, meaning less feature parity or at least more buggy) and lack of flexibility. My Windows knowledge is limited and I've avoided supporting MS software like the plague for a while now so I'm probably naive to the way things are today.

https://redd.it/gblq1y
@r_devops

Using Docker and Terraform for hermetic AWS Lambda CI/CD

Hi,

I was looking for a good way to make Lambda fit with our existing CI/CD workflows around Terraform and CircleCI, when I ran into another problem of the Lambda being a bit more complicated than just the python files involved. I wrote up our solution and was curious how other people are making these all play together well. Using docker files for lambda like this gave me hermetic builds with very few lines of code.

[https://medium.com/@cep21/using-docker-and-terraform-for-hermetic-aws-lambda-ci-cd-b57a77dcaaf6](https://medium.com/@cep21/using-docker-and-terraform-for-hermetic-aws-lambda-ci-cd-b57a77dcaaf6)

https://redd.it/gbl44l
@r_devops

Question; What are some simple DevOp rules all Web Apps and Websites should follow?

I.e.

1. never editing the production server directly
2. using certain GIT branches for X, Y and Z
3. etc.


I'd love to learn more about this stuff.

https://redd.it/gbn212
@r_devops

Swarmlet - A self-hosted, open-source Platform as a Service based on git and Docker Swarm

Hi r/devops ! I wrote a Heroku/Dokku-like tool for easy app deployment and Docker container orchestration when working with a personal server cluster (it also works fine on a single server).

[https://swarmlet.dev](https://swarmlet.dev/)

[https://github.com/swarmlet/swarmlet](https://github.com/swarmlet/swarmlet)

Swarmlet is a thin wrapper around [Docker Compose](https://docs.docker.com/compose/) and [Docker Swarm mode](https://docs.docker.com/engine/swarm/).
[Traefik](https://github.com/containous/traefik), [Consul](https://www.consul.io/), [Let's Encrypt](https://letsencrypt.org/), [Matamo](https://matomo.org/), [Swarmpit](https://swarmpit.io/) and [Swarmprom](https://github.com/stefanprodan/swarmprom) are included by default.
Swarmlet uses these to provide automatic SSL, load balancing, analytics and various metrics dashboards.

The project is WIP, please let me know if you have any comments or feedback!
Don't hesitate to contact me, this is a **learning project** (a few weeks ago I knew nothing about Docker Swarm mode).
I'm definitely no expert yet, so lots of things to improve.
If you're interested, I'd love to collaborate.

https://redd.it/gbfask
@r_devops

Looking to become a devops team lead

Hello everyone,



I'm thinking of becoming a devops team lead. What are your suggestions and how did you make the jump?



Thank you

https://redd.it/gbft2f
@r_devops

Getting started with Docker Compose - Video Demo and Companion Repo

Have a look at the video tutorial and clone the companion repo to follow along! This is aimed at beginners to Docker and Compose but those with some experience might find some tips in there as well.

https://www.youtube.com/watch?v=\_EV5jLtWX8k

https://redd.it/gbecsq
@r_devops

User Input in order to configure different configuration files for multiple containers

I was successfully able to create a `docker-compose.yml` for the Telegraf-InfluxDB-Grafana-Mosquitto Broker stack.

I need some understanding as to how can I request User input before spinning the containers.

Within the `telegraf.conf` which is a TOML file I need the user to map one-to-one information as follows:

```
sensor_name_1 = meta-data_1
sensor_name_2 = meta-data_2
```
This is part of the [enum processor for telegraf](https://github.com/influxdata/telegraf/tree/master/plugins/processors/enum)

I looked into Jsonnet, which currently has a PR regarding TOML unmerged. But I am not sure where will Jsonnet fit in this scenario?

What are other options that I can look into?

https://redd.it/gbeabo
@r_devops

Saving Your Linux Machine From Certain Death

Hi, /r/devops

Today I published my new article on how to troubleshoot and fix some common problems with Linux systems, like recovering root password or fixing unmountable filesystems and I think it might be useful for some of you here.

Here is a link:

https://medium.com/better-programming/save-your-linux-machine-from-certain-death-24ced335d969

https://redd.it/gbcwvs
@r_devops

Terraform AWS FIPS provider

Hey guys, this was a royal pain in the ass to type up and I figured you guys may find it helpful. I've had some compliance requirements and FIPS 140-2 validated encryption is a requirement. I went through the AWS docs and got every AWS FIPS endpoint into the AWS provider.

You can maybe modularize something like this but I haven't ever tried setting up a provider in a module so I'm not sure if that's possible.

Here's a blog link: https://blog.kwnetapps.com/terraform-aws-fips-provider/

Here's a link to the github repo: https://github.com/Kaydub00/terraform-aws-fips

Now, there's probably more to meeting these requirements for your org, but if you need to meet these requirements and you're using TF and AWS you'll need this. Granted I've never been asked by an auditor to see this stuff, you may get an auditor who knows their stuff.

https://redd.it/gc9lie
@r_devops

Can't link Git repo on Jenkins

Hello guys,

I'm trying to link my git repo to jenkins, and getting the following error.

**Failed to connect to repository : Error performing git command: C:\\Program Files\\Git ls-remote -h https://github.com/xxxxxxxxxxxxxx/xxxxxxxxx.git HEAD**

What am I missing here? I started leaning DevOps recently and still practicing few things. Please excuse me if it's just a silly doubt.

​

Namaste!

https://redd.it/gcacoy
@r_devops

Container environment variable

I'm running an ECS Fargate Task.

This is my Entrypoint script in the Dockerfile

startapplication.sh :

#!/bin/bash
set -e
java -jar test.jar --spring.profiles.active=${envparam}

and then I go to AWS ECS Fargate web console > Task Definition > Container definition > declare environment variable "envparam" and value "dev" there.


But ${envparam} in shell script is not resolving when container launches. What is the issue ?

https://redd.it/gc93ir
@r_devops

What are all the alternatives to Jenkins?

Current Technical stack: Python, Java, Scala and NodeJs
Please suggest a commercial and Easy to setup/maintain kind of alternatives.

https://redd.it/gc6ylv
@r_devops

Good tool for monitoring .net app/infrastucture on Azure

I've been using Zabbix in Linux for years and I'm now tasked with adding monitoring to a web app with a .net backend that is hosted in Azure. What are people using to monitor their .net/Azure apps? Ideally it would be opensource or not too expensive and support the standard system metrics such as CPU, mem, storage, I/O and then custom app metrics.

Thanks ahead of time

https://redd.it/gc71gh
@r_devops

Creating a premade work environment similar to Docker

Sorry if I am posting this in the wrong thread. So I just learned about Dockers and started getting them setup for my work projects and everything is working great. After learning about them I was wondering if there was something similar that could be implemented for our development team. So the issue we run into whenever we hire a new developer or get a computer upgrade is that we manually have to go through and install everything needed for these computers to be setup. Github, Putty, SSH Keys, Virtualenv, etc. Is there a way to do the same thing that Docker but in a regular format? By that I mean no need for Docker itself with containers since all our work computers are Windows based.

In simplest terms, I am looking to do something like Ninite but not as fancy as a UI just a script. I know I can do it through a bash script just wanted to check if there were other ways before I take that approach.

https://redd.it/gca7sy
@r_devops

recommendations for a good cloud service that can analyze data posted to it?

first of all, I'm just a programmer, not a dev op - so please keep that in mind, I don't really even know if this is the place to ask such a question so please point me to a better place if there is one.

I'm looking to send a web request possibly POST or GET, not really sure which one as possibly only one of them might work...

which will contain some text data, maybe json maybe xml - doesn't really matter, I can send either.

I then want to use some professionally hosted service that I just create an account with and they give me a url and I send the POST or GET to it and it then can parse this data and store it and, crucially, be able to make charts from it.. being able to send notifications based on the data would be nice too.. ability to use formulas as conditions would be great.

so I would then have a webpage I could navigate to to look at the chart of the data that is constantly being submitted

Can anyone give me some good suggestions? thanks

https://redd.it/gc7ztg
@r_devops

Do you help architect new applications?

What level of input do you have when it comes to influencing the architecture of new applications?

My team have little to no input on how new systems are designed.

This feels quite frustrating because the decisions taken by each application team have an obvious impact on my team when it comes to running them in production.

In addition to this I feel that some of the members of my team have a significantly better understanding and experience of architecture best practices than those who are actually making the decisions. Which to me seems like a waste of talent.

I guess if we were doing this right someone in Ops would be included in the early part of each new project? The reality seems to be we don’t know much about it until someone wants it deployed to a hosting environment.

Having a particular application team “hold the pager” isn’t really an option unfortunately.

Is this a common experience for anyone else here?

I’d be open to suggestions on how to change this but my hunch is that a large part of the problem is down to politics at a leadership level. It’s late and I need some sleep so I’ll leave it at that.

https://redd.it/gc4rc3
@r_devops

[Shifting to Devops]

Hi All I have a small question for you all DevOps here

I have now 6+ years experience in systems administrator and I'm looking to shift my career path definitively to DevOps.

I have experience with servers, load balancing, SaaS, python, PowerShell, bash.

​

The problem is that when I look for junior DevOps jobs they always ask for 1 or 2 years of experience with orchestration software (like ansible terraform, dockers etc.).

What's the natural way to gain that "experience" if you are applying for a junior position?

I think 1 or 2 years of experience should not be junior at that point anymore, should junior be an entry point for someone that has a solid of sysadmin/programming base and willing to learn?

​

Thank you for your time and help.

https://redd.it/gc4h5o
@r_devops

Switching from Software dev to dev-ops in Covid-19 World

I am a former application software developer with 20 years of experience looking to start a new career in DevOps. It's a tough world we live in now, but I am very eager to see how I can start as a junior DevOps guy as a former software dev. I have all the Udemy courses and have started my DevOps journey with learning CentOs Linux (come from Windows/.Net dev background) and would love to know in today's COvid-19 world how a newbie to DevOps could put his foot in the door; there are so many technologies its a bit overwhelming.

- Learn Linux /CentOS
- Learn Python well enough to write scripts
- Docker
- Kubernetes (Most my courses are Docker + Kubernetes, why?)
- AWS (or major Cloud platform of choice, I am choosing AWS , try to get DevOps cert)
- Learn Git in Depth

Any info about Ansible, Terraform, Jenkins or other stuff?

Looking for insight in what to learn and how to break in.. Thanks.

https://redd.it/gc2lo5
@r_devops

Packaging builds as RPMS as a deployment mechanism

More than a decade ago I setup an rpm build server just using scripts and pulling from subversion. Since then I have not had to chance to revisit using RPMS as a way to deploy my builds and have done what most folks do to publish artifacts via ssh and other ways using Jenkins.

Are there still folks here who go through the RPM way? I.e. build the final product as an RPM and deploy it that way and just copy the rpm to the repo? If yes I would love to hear your use cases.

https://redd.it/gc4e3h
@r_devops