2020 Database DevOps Survey

Every year DBmaestro holds a survey among DevOps professionals and teams to try and learn about the state of DevOps as well as the state of Database DevOps. We'd appreciate it if you could take 5 minutes to help us and take the survey.

[https://www.surveymonkey.com/r/2020\_Database\_DevOps\_Survey](https://www.surveymonkey.com/r/2020_Database_DevOps_Survey)

Thanks in advance!

https://redd.it/epkmm1
@r_devops

For those getting into DevOps from OPs background, how would you rank the learning of these DevOps tools in order, with 1 being first priority, 2 being second, etc: (tools in the description, feel free to add tools I may have missed) Structure it in a way to get up-to speed the fastest.

AWS
Terraform
RabbitMQ
Apache Kafka
Python
Go
RBDMS/NoSQL
Docker
Kubernetes
Splunk
Prometheus
Jenkins
Git
Ansible
Selenium

https://redd.it/ephrgb
@r_devops

How long would it take to generate the following cloud formation template to build this infrastructure from scratch? Where to begin? cloud designer? or an easier way?

\*two vpcs, one on only private subnet, one on another

\*one vps instance in private VPC, one in public

\*the VPC with private subnet should have internet connectivity.

\*instance in public subnet must belong to securitygroup, as a webserver, including ssh ability by ip given as a parameter,

\*vpsinstance in private has securitygroup enabled for some randoom port, that the vpsinstance in public vpc can access.

https://redd.it/epvgnk
@r_devops

In general, who are more successful at DevOps, those who come from strong operational backgrounds, or those who come from strong development/software engineering backgrounds? And who has a greater learning curve when transitioning to DevOps from their respective fields?



https://redd.it/eph83l
@r_devops

How to inventory microservices and their endpoints/versions etc? Need help with ideas.

Hello everybody,

I am working as an architect and received a request from my organization to think about a solution to document microservices- & API's.

Essentially, what they want is a list that they can share with developers and also import to a ticketing tool. So people can report an incident on a microservice/API. The list should contain some basic information like:

​

* API Name
* API Location
* Endpoints (create, update, delete etc)
* Versions
* Other suggestions?

​

I am not working with DevOps normally, and I'm not very well versatile in it. So this is quite out of my comfort zone. So forgive my somewhat simplified view.. But how I see it, isn't it just a matter of exporting this information via an integration (REST) that puts it into a database?

Any tips on how to consolidate this and do it as automatic as possible? Where would the functionality of "registering" a new API in the DB actually be put? That is what I'm struggling with most to understand. Maybe a script which always run for each API (upon creation) that registers it in the database?

There are of course many more questions that needs to be solved. Such as how do you maintain it up to date when an API changes (version, endpoints etc) without too much manual intervention.. Relationships between different API's etc. But that comes later.

Any nudge in the right direction would be very much appreciated.

https://redd.it/epijt8
@r_devops

question on containers and queues

should I put the queue process manager in the same container as the main app?

for example if I have a laravel app with supervisord as the process manager, should I run supervisord in the same container with the app + nginx?

https://redd.it/ephd28
@r_devops

Looking for a system for sending out system status and planned maintenance by email

Hi,

I'm looking for a system to easily send out notifications to our customers when a service degrades/downtime on a service, and optional for planned maintenance.

We have a system where we have several middleware servers, and several customers with frontend-services on top of this middlewareservers.

I looking for a service for quickly sending out notifications to the affected customers. I.e customerA should be notified when we detect errors on customerA-frontend, and a predefined set of middleware-servers that affects customerA.

I have looked at [sorrypage.app](https://sorrypage.app) et al, and I like it's structure with components etc, but I don't want a public statuspage, but only notification by mail.

The closest I have found is using mailchimp and tags, i.e tag a customer contact info with "customerA", and for example "middlewareC" and "middlewareE" - but it really like a tools special made for this, with nice, (predefined) templates.

​

Any tips?

https://redd.it/epges2
@r_devops

ISPconfig for docker ?

I have multiple web servers running remotely, I’d like something to manage them just like ISPconfig (chroot dir, setup ftp users, manage configs) is there anything on the market for this ?

My only alternative right now is to stop using dockerized setups in favor of the old fashioned setup nginx and php-fpm on each server so I can use ISPconfig but I really like being able to tinker with upgrading my containers to the latest php etc without hassle

What are you guys doing or using to make life easier ?

https://redd.it/epg2m1
@r_devops

In looking for work, the most common and difficult interview question is how I would diagnose performance issues between multiple tiers and/or services, cloud or bare metal. What's your answer to this?

The problem is in the subject.

I have a fair understanding of this and don't wash out, but I feel like my response is sub-par and not on point.

How do I respond to this question as a generalist being interviewed?

https://redd.it/epfl2i
@r_devops

Is sentry.io only monitoring errors or can I use is as a cross-project centralized logs platform?

I know that it supports many cool frameworks which could help me a lot during development of some apps, but I am wondering whenever it can gather rather simple, not error related logs from the projects?

If there is anything else you can suggest as an alternative then please, share.

I intend to self-host a [sentry.io](https://sentry.io) since they provide clean docker-compose file

Especially I am looking into a possibility of live-tracking logs messages from frontend app and backend API (at one screen) which could possibly help me with debugging async apps.

https://redd.it/epdtxp
@r_devops

Secure Kubernetes on Hetzner Cloud with a node driver for Rancher

Hi! I wrote a post on how to deploy Kubernetes to Hetzner Cloud (a cheap and very good cloud provider from Germany) using a node driver for Rancher. The node driver is very convenient, but requires that a firewall is configured separately, which is what I show in this post. Any feedback is much appreciated!

https://vitobotta.com/2020/01/17/secure-kubernetes-hetzner-cloud-rancher-node-driver/

https://redd.it/eq1bbw
@r_devops

Hi r/devops, we recently open sourced a command-line cloud automation tool and would love to get feedback on it. We've been using a version of this tool internally for many years and decided to clean it up and open source it.

Disclaimer: I'm co-CTO at Perfect Sense, the company that built Gyro, and the original author of the tool.

With that out of the way, we'd love to get feedback (good or bad) on Gyro. If you've ever used Terraform or Cloudformation you'll feel right at home with Gyro. It allows you to automate any cloud provider with an API. Currently we have good (but not yet 100% of resources) support for AWS and Azure. We're working to add Google Cloud support.

Gyro evolved from an internal tool we built starting way back in 2014. The initial goal of that tool was to automate creating and updating our infrastructure. As time went on we integrated various parts of our infrastructure into this tool. For example, we used it to execute Chef Solo on our hosts, ssh into hosts, interact with our service discovery, and deploy our code using with blue/green deployment model. We found this "one tool for most of your day-to-day ops activities" to be extremely valuable. This is why we decided to refactor the code, make it more flexible, and open source it so others could do the same.

Why the name Gyro? It's short for Gyroscope which is an essential device that allows airplanes to navigate in the clouds. I love aviation. Also, if you read "gyro" and thought of a greek sandwich, you're not the first, definitely won't be the last, haha. That's ok though, gyro sandwiches are super yummy. :)

There are a few things that make Gyro different from similar tools. I'll try to highlight those here but encourage you to read either the developer documentation site ([https://gyro.dev](https://gyro.dev/)) or if you want a higher level quick hits page read [https://getgyro.io/introducing-gyro](https://getgyro.io/introducing-gyro).

**1) Simple: Configuration Syntax (aka the Gyro Configuration Language)**

I know, I know, another Yet Another DSL. Originally we wrote this using YAML but we wanted clean (and limited) logic in our configuration and YAML didn't really fit the bill. We tried a few different language based internal DSLs such as Kotlin, Groovy, and even TCL (don't hate) but the language always bled through and didn't feel right.

Ultimately we decided to design our own simplified, but powerful, language. This allowed us to have greater control over scoping rules, control structures, and runtime execution. Building a tool that generates an internal graph of resources is extremely complex and not having complete control over what is happening during execution makes it much more complex.

[https://gyro.dev/guides/language/](https://gyro.dev/guides/language/)

**2) Logic: Control Structures**

We're well aware of the ongoing debate of whether allowing logic (control structures) in a configuration is a good thing or not. We believe it is as long as you provide reasonable limits. We tried to strike a balance between no logic and a full programming language. To start with we've implemented two control structures we think are most important for configuration logic, "if" and "for".

Control structures are actually an extension of Gyro rather than baked into the language parser.

[https://gyro.dev/guides/language/control-structures.html](https://gyro.dev/guides/language/control-structures.html)

**3) Workflows**

We think this is huge. What we mean when we say workflows is the ability to define transition stages for complex cloud infrastructure updates. Blue/green deployments are a good example of this. With Gyro you can define a stage to create a new load balancer and new virtual machines with your updated code. After this stage executes you can either prompt the user to continue allowing them to validate the new deployment or you can automate it with simple logic. The next stage you define can either drop those new machines into the load balancer taking traffic or you can flip DNS depending on how you like to do blue/green. If at any poi

nt things don't look right Gyro can roll back to a previous stage.

This functionality has been extremely important for us to be able to allow people to do deployments and to quickly roll back should anything go wrong.

[https://gyro.dev/guides/workflows/](https://gyro.dev/guides/workflows/)

**4) Extensibility**

Aside from workflows, extensibility is what I'm most excited about in Gyro. We've included a number of ways you can extend Gyro with plugins.

* Add new commands ([https://gyro.dev/extending/commands/](https://gyro.dev/extending/commands/), [https://github.com/perfectsense/gyro-ssh-plugin](https://github.com/perfectsense/gyro-ssh-plugin))
* Add new language functionality, aka Directives ([https://gyro.dev/extending/directive/](https://gyro.dev/extending/directive/))
* Add custom variable resolvers ([https://gyro.dev/extending/resolver/](https://gyro.dev/extending/resolver/))
* Add custom event hooks (undocumented) to trigger custom logic when various things happen such as a resource is created or updated

TL;DR Check out [https://gyro.dev](https://gyro.dev/) or [https://github.com/perfectsense/gyro](https://github.com/perfectsense/gyro), this the developer/user documentation for Gyro and the main repo. It should give you a good idea of what Gyro is and how it works!

https://redd.it/eq023h
@r_devops

Run pre-deployment policy checks on Terraform using Open Policy Agent with Regula

Hi everyone. We just open sourced Regula, a tool that evaluates Terraform infrastructure-as-code for policy prior to deployment. Regula includes a library of rules written in Rego, the policy language used by the Open Policy Agent project. Where relevant, we’ve mapped Regula policies to the CIS AWS Foundations Benchmark, and we'll be adding more rules as we go.

There are two parts to Regula. The first is a shell script that generates a Terraform plan in JSON format, ready for consumption by OPA.

The second part is a Rego framework that:

* Merges resource info from planned\_values
and configuration in the terraform plan into a more conveniently accessible format.
* Looks for rules and executes them.
* Creates a report with the results of all rules and a control mapping in the output.

We hope you give it a try and would love to hear what you think. [https://github.com/fugue/regula](https://github.com/fugue/regula)

https://redd.it/eq2s6m
@r_devops

20 years since Y2K

Thought this article might be of interest--it looks back over a couple of big tech incidents and discusses what we've learned: [https://www.transposit.com/blog/2020.01.15-left-pad-heartbleed-and-devops/](https://www.transposit.com/blog/2020.01.15-left-pad-heartbleed-and-devops/)

(full disclosure, a co-worker wrote it.)

https://redd.it/eq3ot9
@r_devops

Deployments sharing helm charts best practices

What are the best practices as far as deployments sharing helm charts?

The company I started working for is using a single huge, monolithic chart for all their deployments (100's) and that strikes me as very wrong.

Equally, having a helm chart for every deployment/repo has the downside of creating difficulties when you want to change the deployment for a subset of microservices that all share a common pattern, which makes using a shared chart attractive.

So in general, what should be done?

https://redd.it/eq0ecx
@r_devops

Spin up temporary site for each pull request

I have been looking for a way to create a temporary environment (azure app service) for my application (aspnetcore + react) for each and every pull request that is made against master, so developers can review the code and testers can test the features before they are actually merged into master and Azure Devops sends the changes towards production.

The reasoning behind this is to make sure we are always able to deploy to production (preferrably automatically) without risking untested features are blocking tested features. I would also like us to be able to use only one pull request, and not first a PR for each feature against dev, then against test and then against master when it is ready for production.

Am I insane?

I can't seem to find anyone that spins up new app services on azure from their ci/cd system. We are using Azure DevOps for build and release pipelines now, everything is hosted on Azure. Any suggestions are appreciated.

https://redd.it/epz3bm
@r_devops

sonar-cxx: flags to prevent a rule to be triggered

Hi.

I was searching for a possibility to deactivate some rules for a particular piece of code when using [sonar-cxx](https://github.com/SonarOpenCommunity/sonar-cxx).

Here is an example. The `cxx:TooLongLine` is a controversial rule that triggers when a line's length is > 160 characters, but I try to abide by it whenever possible. But in some rare cases, it can be troublesome.

In my latest project, I have to compose web pages using pieces of HTML, CSS and SVG pictures in particular, all of this using C++. You guessed it: SVG pictures produce very long `const char *` strings, and there are a few of them.

So yes, one of my header files has plenty of `cxx:TooLongLine` violations, and I'm searching for a way to deactivate the rule, *one line at a time*.

I remember, back when I was coding in Java, that there was a inline flag that allowed to deactivate a rule for a specific piece of code, for example (might be wrong):

```
//@deactivate(java:TheRuleToDeactivate)
```

So my question is: is there a way to do something similar with sonar-cxx?

Thanks a lot for your help.

https://redd.it/eq1a0b
@r_devops

So, how are you guys releasing your artifacts?

Similar to another enquiry I've made in the past about what Artifactory software you are using, but a more general information gathering one this time - how are you releasing artifacts and getting them to your customers?

Our shop isn't a full SaaS (yet), so when we do CI/CD we push changes to our own live/customer environments where we can develop against. Our artifacts strategy at the moment is to release the files on our Jira Board in the announcements page to our respective customers. I want to try and improve this as it's been there since god knows when and it feels like a better process should exist.

Our customers take the artifacts we release and give them to there own QA team who do there own CD process after they've quality checked and raised any tickets we might have missed, followed by pushing them to there own live environment we don't control.

We have a container registry where I've been trying to push changes into our docker image and have our environments refresh that everytime there's a change to our image, but it's just experimental and I want to know how everyone else is doing it.

https://redd.it/epxmjt
@r_devops

Scaling docker container horizontally benefits if on same host

What are the benefits of scaling a service horizontally when they run on the same host? Say two instances of a microservice instead of one.

Wouldn't the resources that could have been used for the single microservice just get split up and used for the second, giving no real benefit. Like instead of having one car go 100km/h now there are two cars each going 50km/h ?

https://redd.it/eq8n66
@r_devops

How to automatically ensure pipeline compliance

So, let's say that we want to ensure that all software pipelines put the code through SAST testing (or any other important step), how do we as DevOps engineers ensure that the pipelines for 100 or 1000+ all confirm to business requirements? What's to stop one developer removing a crucial step in the pipeline and another approving this and it therefore not going noticed for a very long time? or, possibly more likely, what's to stop a developer taking a template pipeline and simply removing a step on the initial pipeline commit?

For context we are currently using gitlab CI but are looking at potentially moving to cloud build, I hope the answer to this is platform agnostic (but assume it won't be)

https://redd.it/epyi34
@r_devops