Given the tools, how do you incentivize your developers to make great dashboards/increase system observability?

A system dashboard is really valuable for stakeholders, new teammates and on-call personnel - but I realize that developers themselves don't have any real incentives to make great dashboards.. They often do not need an abstract level of overview - they already feel and navigate in more natural, low level metrics like logging, error handling software, health checks or simply just by looking at your system input and outcome with a certain level of cultural know-how.

I really like Google's approach with SLOs, which tries to align the incentives more between devs and ops in regards to reliability. Are you aware of any similar dynamics when it comes to observability?

https://redd.it/eak0a6
@r_devops

Given the tools, how do you incentivize your developers to make great dashboards/increase system observability?

A system dashboard is really valuable for stakeholders, new teammates and on-call personnel - but I realize that developers themselves don't have any real incentives to make great dashboards.. They often do not need an abstract level of overview - they already feel and navigate in more natural, low level metrics like logging, error handling software, health checks or simply just by looking at your system input and outcome with a certain level of cultural know-how.

I really like Google's approach with SLOs, which tries to align the incentives more between devs and ops in regards to reliability. Are you aware of any similar dynamics when it comes to observability?

https://redd.it/eak0a6
@r_devops

I'm confused - Build Stage for CI/CD with Php (not compiled language?)

Can someone please explain how a language that is not compiled (PHP) have a build stage in CI/CD?

[https://thecodingmachine.io/a-practical-guide-to-continuous-integration-in-php](https://thecodingmachine.io/a-practical-guide-to-continuous-integration-in-php)

https://redd.it/eawehk
@r_devops

Advice wanted for working abroad

Hi r/devops. I'm a fairly junior DevOps engineer who's been working in tech for just over two years and doing DevOps role for about a year. I'm from and currently live in the UK. The prospect of moving to another country with lots of opportunities in tech is something that really excites me and am seriously considering. Canada is somewhere I'm finding especially desirable. However I have some worries, especially in regards to experience.

To give some context, I'll just talk about what I've been through and where I'm currently at. I finished my Masters and started off as an entry-level developer in October 2017. I started to get more interested in DevOps work when as part of that role, I was tasked with writing Terraform and helping to solutionize some AWS infrastructure challenges. It really seemed to come to me very naturally and there was something about it that made me really attracted to this kind of work - never really figured out why.

It took some time for my company to move me into a DevOps role, but eventually I was moved onto another project and was fortunate enough to work with one of our most senior DevOps engineers. During my time on this project and under this guys tutelage, I have used Openshift to deploy our services, I've created and maintained many of our CI/CD jobs in Atlassian Bamboo (often leveraging Docker as part of that), I've written lots of bash scripts with a bit of Python thrown in, I've become more familiar with Linux and I've set up monitoring alerts in New Relic.

I have had very good peer feedback since I've been doing this role. Recently the senior DevOps guy rolled off, so I was tasked with being the lead (and indeed only) DevOps engineer on the project. There's more high pressure situations and it was certainly a bit scary. But again, the feedback I've had has been very good and I'm pleased with how I've settled into the role, and I'm pleased to know that I'm trusted well enough to take on this responsibility.

My main frustration is that there's a lot of decent tech that I haven't had the opportunity to use for a while now. To give some examples, I haven't done any cloud platform work since my previous project (over a year). I haven't written any SQL since I was at Uni or indeed deployed a SQL service, as we are lumped with using a really awful database on my current project. I've also not had the opportunity to work with Jenkins for CI/CD. I've also only ever worked with JavaScript/NodeJS applications as well, so I wouldn't know much at all about Maven/Gradle builds with Java for example.

So sorry that was a bit me-me-me but hopefully it gives some context! :) Can anyone offer any advice for someone in this position? This would be the first time I've ever at looked at living in another country as well, so applying for jobs abroad, applying for work permits etc. is also all new territory for me. So any advice on that area as well would also be really useful.

Thanks for reading :)

https://redd.it/eajbav
@r_devops

Convert Any Cloud Linux Instance To PostgreSQL Server.

Please checkout my blog on converting any cloud instance to psql server.

Link: [https://medium.com/@sumitkukade/convert-any-cloud-linux-instance-to-postgresql-server-ffc9faecbeae](https://medium.com/@sumitkukade/convert-any-cloud-linux-instance-to-postgresql-server-ffc9faecbeae)

https://redd.it/eaxuc6
@r_devops

[Question] How to deploy Angular App to Azure Web Apps?

My code is on Github and want to deploy via CI/CD, triggered on the master branch.

I have followed multiple how-tos and have found so far, that there are two ways:

1. Use Kudu to automatically deploy and run the app. Not my goal. I want to run the Angular App serverless, i.e. the dist folder that is generated via ng build [—prod].

2. Run a build+release pipeline. This seems more like what I intend, but I keep failing at the deployment stage.

All tutorials I have found online seem to be outdated or straight up didn’t work for me.

Am I missing anything? The only guidelines that are not frankensteined seem to be guided towards server-ful node apps.

https://redd.it/eaykks
@r_devops

Need some advice regarding Terraform

This might turn into a wall of text, but hear me out.


First, some background info for clarity:
We use terraform extensively at my current job, along with Azure for our cloud provider. We use the open source version and use a storage account in Azure for the back end/state file storage and have automated the entire plan/apply (with approval) using Jenkins. Everything works pretty well for the most part and there are few issues overall.


Now, onto the issue:
The way we use terraform currently is on a per-subscription basis (i.e. nonprod subscription has a got repo with all of the Infrastructure for that subscription in it, same for prod). They don't use workspaces. There are literally hundreds, if not more, resources defined in each of these state files.

Now I am relatively new to both the team and to terraform, but this struck me as a problem for a number of reasons. My main concerns were security (giving someone access to create Infrastructure means essentially giving them access to ALL the infrastructure in a given environment), lack of scalability, and blast radius (if someone makes a mistake, it could potentially affect all the infrastructure in a given environment). I expressed my concerns with the team and got some pushback. I suspected there may have been a lack of understanding as to what workspaces are and how they work, so I put together a poc to help clarify my points.

This is where I am confused and maybe I'm just not understanding or am way off base here. The team thinks that we don't need to use workspaces because we aren't "big enough" to justify their use. They feel that a single state file that contains all the infrastructure is the right way to proceed because as you build, it's always revalidating the entire infrastructure (with terraform plan) and if someone changes something manually in the portal, we find out right away. They have also argue that it would take way too much effort to rewrite everything to accommodate workspaces.

So, we are in the process now of upgrading from terraform 0.11 to 0.12 and are in a position where we are going to have to rewrite a lot of our terraform code as part of this effort. I suggested again that we consider splitting the code our into separate repos (even putting the terraform code in the same repo as the application that it supports) and using workspaces. Again, I was met with the same resistance. I even offered a compromise: rewrite the old code using their old method (since that was considered easier by the team) and any new infrastructure could be done using my proposal. They also disagreed with that and instead want to push on with their method and "revisit" workspaces after they finish upgrading to 0.12.


So my question to you all is this: am I not thinking about terraform the right way? I have read the documentation from hashicorps site and thought I understood their best practices, but my team seems to think that their documentation regarding best practices is "too vague" or that it doesn't apply to us because we aren't at that scale yet. I believe we should build a solution that can scale with the needs of the business, but I feel less and less certain after each of these discussions. We also want to get other teams to accept terraform as the primary method of creating infrastructure, so I'm not sure if I am approaching this the right way or not.

Sorry for the long read but really wanted to get this off my chest. Any advice is appreciated!

https://redd.it/eaxzki
@r_devops

Do you still run a reverse proxy behind an ALB?

We’re migrating from ELB to ALB. Now that we have some basic intelligent routing and we know it has strong connection management, do you still run a reverse proxy behind an ALB and why?

For reference, I’m managing ELB -> Apache -> JVM today. While rewrites are nice, it’s also easy for us to move / correct rewrites back up stream in the service. We keep domain (and https) redirects outside this setup. All that is left is proxy / connection management.

https://redd.it/eah1eg
@r_devops

Would you guys be interested in a web app to convert docker-compose files into kubernetes files ?

Hey guys, everything is in the title. Just an idea for a side project I had and I was wondering if anyone else besides me would find this interesting.

Thanks.

https://redd.it/eb0y8v
@r_devops

Vault Token Creation Query

Hello.

I've taken responsibility for vault at my company, trying to tidy it up and bring it in line in terms of security etc. In the old world, the root token was everywhere and everyone knew it. We may as well have kept all our passwords in an excel spreadsheet!

For users who required temporary vault access, I'd create them a token with the 32 days TTL. Then keep creating new tokens for them when that expires.

However, this is incredibly manual and involves someone (me), logging into the vault as root to create a new token for them.

I'm seeking some inspiration, I have some ideas but curious about what others are doing.

Is there a way to automate this, as I want our engineers to not have access to the root token, and just have what's needed to do their job. But also be able to create their own tokens when needed.

But quite simply, the mission here is for me to not have to ever login as root again to renew all these tokens. Or, at the very least do I increase the max TTL in config to say 1 year?

Cheers

https://redd.it/eb15hz
@r_devops

Pros and cons of using azure pipelines over travis?

I am thinking of shifting my C++ project from Travis to Azure Pipelines because of the unlimited build times in Azure Pipelines. I am trying to make a comparison chart to figure out if the shift is worth the effort and also if there is any other benefit of moving to azure pipelines. I would love to hear your thoughts on this.

https://redd.it/eb0khp
@r_devops

Molecule systemd doesn't work

I'm having trouble getting systemd to work when I do molecule test. I can get it to work when I start the docker container manually on the command line. Is there some special configuration necessary for molecule that I am missing? I've followed the documentation and I'm using centos/systemd which as I've said works outside of molecule. With molecule I get a failed dbus error or it simply fails to even create the container and says that it doesn't exist. Also, is there configuration needed to use oci-systemd-hook or is it used automatically?

https://redd.it/eb1plk
@r_devops

What is the best approach to serve high bandwidth through NAT Gateway

I want design AWS architecture like this, but not sure how to handle high bandwidth (>100GB) traffic.

A kubernetes cluster with lots of microservices , both frontend and backend. An LB in front of the worker nodes. K8s replica can scale high bandwidth traffic.

My question is where should I create the Kubernetes cluster? I know there is no bandwidth constraints in Public subnet, but AWS NAT Gateway has bandwidth constraints. What is the approach by big companies to serve high bandwidth through NAT Gateway. Or should I put my K8s cluster in public subnet itself.?

Any help is appreciated .Thanks

https://redd.it/eaa7ea
@r_devops

Fargate and EC2 for ECS: Trying to understand their best use cases.

Hey there,

I am trying to figure out which of the two launch types is best, taking into account that I already have experience with managing an EC2 container cluster:

- There was a significant [price reduction](https://aws.amazon.com/blogs/compute/aws-fargate-price-reduction-up-to-50/) to Fargate pricing earlier this year.
- Amazon [has just released](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cluster-auto-scaling.html#asg-capacity-providers) improved ECS autoscaling, where they provide cluster scale-in, scale-out and instance draining automation
- Recently they also launched [saving plans](https://aws.amazon.com/blogs/aws/aws-ecs-cluster-auto-scaling-is-now-generally-available/) that offer more flexibility than reserved instances (can also be used by Fargate)
- Fargate spot could also provide some additional savings.
- In Fargate, you only pay for the CPU and Memory that you define. But as you can't predict how it will consume CPU and Memory exactly, there will still be unused capacity, that you would have to pay in the end, similar to having EC2 container instances. Does it still make sense to go for Fargate, comparing the same average reservation rate?
- There's this nice [comparison](https://www.trek10.com/blog/fargate-pricing-vs-ec2/) which suggests that Fargate's pricing for an average reservation of 70% is similarly priced with an EC2 based cluster. Has anyone tried to do a similar calculation? Does it sound right according to your experience? It doesn't seem to take into account, other charges such as Data transfer charges related to cluster computing




Any input greatly appreciated :)

https://redd.it/eb7pnm
@r_devops

How are you guys building your containers for your code?

Hey guys.

I’m migrating an app into a container to allow me to be a little more flexible. I am curious how are you build the container with the code in your pipeline? Do you copy it in from a system path? Or do you have a run command for docker to clone the repo?

Just kind of trying different ways to see what is the best for me and would like to hear your opinions.

https://redd.it/ea53by
@r_devops

[My thoughts] Cloud-Native is not about containers, nor a synonym for Microservices.

Cloud-Native really started to be sort of a thing in about 2010 by a couple of industry thought leaders, and one of them is Paul Fremantle, who wrote about it in his blog. He doesn't talk about the tools and technologies in his blog; instead, he states that for systems to behave well on the cloud, they need to be written for the cloud & this is where the cloud-native approach began.

Containers & microservices give a boost to the cloud-native model, but that's not everything.

The tools and technologies empower today's application deployment; Cloud-native computing takes advantage of many modern methods, including PaaS, multi-cloud, microservices, agile methodology, containers, CI/CD, and DevOps. Now, we could even see its own foundation: the Cloud Native Computing Foundation (CNCF), launched in 2015 by the Linux Foundation.

According to a recent survey by CNCF, it is found that the use of Cloud Native technologies in production has grown over 200%.

https://redd.it/ea0reb
@r_devops

What is the best approach to serve high bandwidth traffic with AWS NAT Gateway

I want design AWS architecture like this, but not sure how to handle high bandwidth (>100GB) traffic.

A kubernetes cluster with lots of microservices , both frontend and backend. An LB in front of the worker nodes. K8s replica can scale high bandwidth traffic.

My question is where should I create the Kubernetes cluster? I know there is no bandwidth constraints in Public subnet, but AWS NAT Gateway has bandwidth constraints. What is the approach by big companies to serve high bandwidth through NAT Gateway. Or should I put my K8s cluster in public subnet itself.?

Any help is appreciated .Thanks

https://redd.it/eaacua
@r_devops

Does AWS come with a load balancer or do I have to build one myself?

I was wondering whether if AWS comes with a load balancer. If not, I was thinking about using NGINX but also I would like your opinions on which do you think is cheaper and easier to set up an maintain?. Opinions are welcomed.

https://redd.it/eaelgv
@r_devops

[blog] Shadow requesting for great good

[https://medium.com/carwow-product-engineering/shadow-requesting-for-great-good-92cde331363a](https://medium.com/carwow-product-engineering/shadow-requesting-for-great-good-92cde331363a)

https://redd.it/ebdaky
@r_devops

How to use multiple Docker containers to set up Jenkins agent in Jenkins pipeline?

The following snippet is an example provided by Cypress, a Javascript testing framework that I'm using. Here is the [link](https://github.com/cypress-io/cypress-example-kitchensink/blob/master/Jenkinsfile) to the Github page.

pipeline {
agent {
// this image provides everything needed to run Cypress
docker {
image 'cypress/base:10'
}
}

stages {
// first stage installs node dependencies and Cypress binary
stage('build') {
steps {
// there a few default environment variables on Jenkins
// on local Jenkins machine (assuming port 8080) see
// https://localhost:8080/pipeline-syntax/globals#env
echo "Running build ${env.BUILD_ID} on ${env.JENKINS_URL}"
sh 'npm ci'
sh 'npm run cy:verify'
}
}

stage('start local server') {
steps {
// start local server in the background
// we will shut it down in "post" command block
sh 'nohup npm run start:ci &'
}
}

// this stage runs end-to-end tests, and each agent uses the workspace
// from the previous stage
stage('cypress parallel tests') {
environment {
// we will be recording test results and video on Cypress dashboard
// to record we need to set an environment variable
// we can load the record key variable from credentials store
// see https://jenkins.io/doc/book/using/using-credentials/
CYPRESS_RECORD_KEY = credentials('cypress-example-kitchensink-record-key')
// because parallel steps share the workspace they might race to delete
// screenshots and videos folders. Tell Cypress not to delete these folders
CYPRESS_trashAssetsBeforeRuns = 'false'
}

// https://jenkins.io/doc/book/pipeline/syntax/#parallel
parallel {
// start several test jobs in parallel, and they all
// will use Cypress Dashboard to load balance any found spec files
stage('tester A') {
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run e2e:record:parallel"
}
}

// second tester runs the same command
stage('tester B') {
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run e2e:record:parallel"
}
}
}

}
}

post {
// shutdown the server running in the background
always {
echo 'Stopping local server'
sh 'pkill -f http-server'
}
}
}

My goal is to have a Jenkinsfile that is very similar to the above because I want to have parallel Cypress testing as shown in the above snippet. In the example above, the Jenkins agent is simply the official Cypress Docker image `cypress/base:10`.

agent {
// this image provides everything needed to run Cypress
docker {
image 'cypress/base:10'
}
}

However, for me to run all my tests with my own database, I need to spin up two separate Docker containers. One container contains the front-end portion of my web app and the other container contains the back-end portion of my web app.

Below is the Dockerfile for my front-end container, which is located in `my-app/docker/combined/Dockerfile`.

FROM cypress/included:3.4.1

WORKDIR /usr/src/app

COPY package*.json ./

RUN npm install

COPY . .

EXPOSE 5000

RUN npm install -g history-server nodemon

RUN npm run build-test

EXPOSE 8080

Below is the Dockerfile for my back-end container, which is located in `my-app/docker/db/Dockerfile`. All it is doing is copying some local data into the Docker container and then initialising my MongoDB dat

abase with this data.

FROM mongo:3.6.14-xenial

COPY ./dump/ /tmp/dump/

COPY mongo_restore.sh /docker-entrypoint-initdb.d/

RUN chmod 777 /docker-entrypoint-initdb.d/mongo_restore.sh

Usually, I would use `docker-compose` and the following `docker-compose.yml` file to spin up these two containers. As you can see, the front-end container called "combined" is dependent on the back-end container called "db".

version: '3'
services:
db:
build:
context: .
dockerfile: ./docker/db/Dockerfile
container_name: b-db
restart: unless-stopped
volumes:
- dbdata:/data/db
ports:
- "27017:27017"
networks:
- app-network

combined:
build:
context: .
dockerfile: ./docker/combined/Dockerfile
container_name: b-combined
restart: unless-stopped
env_file: .env
ports:
- "5000:5000"
- "8080:8080"
networks:
- app-network
depends_on:
- db

Below is the docker-compose command I would use.

docker-compose up --build

**I would like my Jenkins agent to be the** `combined` **container; however, I need the** `combined` **container to connect to my** `db` **container, which needs to be spun up. My question is, how do I achieve this in Jenkins pipelines? I've read** [this documentation](https://jenkins.io/doc/book/pipeline/syntax/)**; however, it doesn't mention anything about using multiple Dockerfiles to create a Jenkins agent. Is something like this possible and could someone please show me what my Jenkinsfile should look like in order to achieve my goal?**

https://redd.it/ebeu8d
@r_devops