Appside secrets management.. How are y'all doing it these days?

Hey folks,

Got a design question.

My company is improving the way it looks at cybersecurity, and one thing that came up was the need to both version control configuration data but to prevent secrets from being leaked through that same vector.

Presently, our apps pull config data from environment variables, which are populated on deployment via an ansible push mechanism.

The threat model has the following vectors:

* A developer virtual machine image being leaked
* A developer virtual machine getting pwned via a network level attack
* Somebody getting access to our git remotes
* Disgruntled employee syndrome
* Somebody getting access (any access) in which they can expose Pam env vars on one of our service instances

Presently, we're transitioning to using Ansible. Ansible Vault has been looked at, but then there's still the issue of storing the AES key somewhere.

We've considered using AWS secrets manager to store our AES key for ansible vault but are additionally concerned about secrets being located in environment variables. I've considered writing an on-demand retrieval mechanism that statelessly pulls the AES keys from AWS Secrets Manager, but then there's the issue of round trip time to actually fetch the key each time. The application runs on a mix of node and PHP, so caching the key natively in application memory isn't really an option.

Reddit, wat do

https://redd.it/dp67io
@r_devops

How to wait for Postgres before running API services through docker-compose when psql can't be installed?

I have a docker-compose.yml file with a database and multiple APIs that I'd like to launch through a simple "docker-compose up" in the terminal. However, I'm running into the issue of my APIs starting up before the Postgres server becomes available.

I saw a solution for using a command statement in the docker-compose.yml file that runs a shell script with "psql" to test the Postgres connection before starting an API service, but the issue there is that my API images only have the API service on them (*no psql, python, ping, etc.*) and I'm pulling those images from AWS ECR so I don't have the option to install those.

To complicate things further, any command statement within docker-compose.yml gets in the way of the API image's command statement from the Dockerfile, which would ordinarily have it run its own docker shell script to start up the service. The Dockerfile that was used to build the image has a command to execute an external script to start the service.

Does anyone know of a solution for this issue? I'm a newbie and lost as to how to proceed.

https://redd.it/dpawwz
@r_devops

how do I add more disk space to my google cloud instance?
https://www.reddit.com/r/googlecloud/comments/dpkfc5/how_do_i_add_more_disk_space_to_my_google_cloud/

https://redd.it/dpknd6
@r_devops

Developers are increasingly interested in and adopting DevOps and Mini apps. Interest and adoption of cryptocurrency and computer vision is also increasing but to a lesser extent.

[https://www.developereconomics.com/resources/reports/state-of-the-developer-nation-17th-q2-2019](https://www.developereconomics.com/resources/reports/state-of-the-developer-nation-17th-q2-2019)

https://redd.it/dpm2dc
@r_devops

Building and deploying lots of microservices using werf and GitLab CI

[Here](https://medium.com/flant-com/building-and-deploying-lots-of-microservices-using-werf-and-gitlab-ci-3ce2b7d19450) is how we build & deploy to Kubernetes (via GitLab CI) many similar microservices (i.e. an application represented by multiple repositories). Our goal is to avoid an obvious pain of copying pipelines/infrastructure configs. We assume that deployments are described by Helm charts and werf is in use.

https://redd.it/dpmev2
@r_devops

What I learned at SREcon EMEA 2019

I recently went to SREcon EMEA 2019, and decided to write up my notes as a blog post. I tried to extract the common themes and trends I noticed rather than just the raw notes from the talks. It's very much my take!

[https://making.pusher.com/hot-sre-trends-in-2019/](https://making.pusher.com/hot-sre-trends-in-2019/)

Please let me know if you have any questions or things you disagree with.

https://redd.it/dpnv4z
@r_devops

Lesson from Deploying an ML Model with Kubernetes to GCP

Article mostly focuses on overcoming certain pain points in deploying a machine learning model and web application to GCP. Uses Kubernetes to deploy a containerized Flask API with a persistent volume for hosting the large model assets (.pkl files). Also, includes a short overview on machine type / pricing optimization for handling the high-memory utilization of the model.

[https://medium.com/@pat\_migliaccio/lessons-from-deploying-a-machine-learning-model-with-kubernetes-to-google-cloud-platform-ab84f71ab311](https://medium.com/@pat_migliaccio/lessons-from-deploying-a-machine-learning-model-with-kubernetes-to-google-cloud-platform-ab84f71ab311)

https://redd.it/dpp0iv
@r_devops

DevOps Projects / Practice?

Is there a service or resource that provides example projects for developing devops related skills? Something like LeetCode except geared towards providing example projects/problems focused around devops?

As an external learner (not currently in the field), I'm itching to test drive some of the knowledge I've gathered over the last few months but am struggling to find ways to practice it. Open to any suggestions and ideas!

https://redd.it/dpvhmv
@r_devops

I'm using ansible tower to provision VMs, blow away, and then recreate. What other tools to integrate or use to make my life easier?

New to automation, CI/CD, pipelining, etc. but am making good progress learning (i think). I create a bunch of VMs with specific IP addresses from a VMWare template using vmware\_guest module in ansible. I configure the new VMs with some additional playbooks, do some work, and then blow them away. Some times I want to tweak the VM names and IP addresses and start over. How do I get ansible to watch my inventory file for changes and automatically run my playbook? Do I need something like jenkins, or is that overkill?

https://redd.it/dptch6
@r_devops

Looking for best service such as sendgrid, ses, Postmark etc for outgoing mail from multiple apps with suppression list

Hi all,

I have decided to use a service to help the delivery of emails from all my apps and websites.

Can someone recommend a service that has a good API, has dedicated IPs ( that is not ridiculously priced ) that will work with office365, and others that mark new IPs as spam, to receive our emails. Has the ability to add email addresses to suppression lists that get bounced automatically. The suppression list would be great if configurable. such as any hard bounce will add it to the list for 30 days. If another bounce happens after 30 days it will add it again for 60 days, and so on.

we dont send many emails, maybe 300 a day at most. But we do require them to get delivered and past spam filters.

https://redd.it/dpxsct
@r_devops

Tutorial: Deploy a Docker Swarm Cluster on DigitalOcean (with Traefik, Let's Encrypt and Swarmpit)

I put together a lengthy tutorial on how to deploy a Docker Swarm cluster on DigitalOcean, together with Traefik (and Let's Encrypt certificates for services) and Swarmpit as a web interface. I realize Docker Swarm might be out of fashion in these Kubernetes times, but still!

I learned a lot by writing it, so see it as a learning exercise rather than a blueprint for production deployments. :)

Let me know if I can improve the tutorial! I really appreciate feedback! :)

[https://lunar.computer/posts/docker-swarm-digitalocean/](https://lunar.computer/posts/docker-swarm-digitalocean/)

https://redd.it/dq2hdv
@r_devops

Hey SREs: What are your favorite metrics to monitor?

I'm going out for an interview as an SRE, and I was thinking about which metrics were most important and why. Which ones do you find the most helpful in preventing and identifying issues?

* Response time, especially for database queries
* CPU and memory utilization per container, VM, and host (as appropriate) to spawn additional nodes
* Minimum available disk space
* Number of (GET) requests to monitor for spikes in traffic and DDOS attacks

Assuming a decent monitoring infrastructure, what else am I missing?

https://redd.it/dq54ur
@r_devops

Monthly 'Getting into DevOps' thread - 2019/11

**What is DevOps?**

* [AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

**Books to Read**

* [The Phoenix Project](https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/1942788290) - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
* [The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
* [Google's Site Reliability Engineering](https://landing.google.com/sre/books/) - Google engineers explain how they build, deploy, monitor, and maintain their systems.
* [The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book

**What Should I Learn?**

* [Emily Wood's essay](https://crate.io/a/infrastructure-as-code-part-one/) - why infrastructure as code is so important into today's world.
* [2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
* [This comment by /u/mdaffin](https://www.reddit.com/r/devops/comments/abcyl2/sorry_having_a_midlife_tech_crisis/eczhsu1/) - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
* [This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

**Previous Threads**

https://www.reddit.com/r/devops/comments/dbusbr/monthly_getting_into_devops_thread_201910/

https://www.reddit.com/r/devops/comments/cydrpv/monthly_getting_into_devops_thread_201909/

https://www.reddit.com/r/devops/comments/ckqdpv/monthly_getting_into_devops_thread_201908/

https://www.reddit.com/r/devops/comments/c7ti5p/monthly_getting_into_devops_thread_201907/

https://www.reddit.com/r/devops/comments/bvqyrw/monthly_getting_into_devops_thread_201906/

https://www.reddit.com/r/devops/comments/blu4oh/monthly_getting_into_devops_thread_201905/

https://www.reddit.com/r/devops/comments/b7yj4m/monthly_getting_into_devops_thread_201904/

https://www.reddit.com/r/devops/comments/axcebk/monthly_getting_into_devops_thread/

**Please keep this on topic (as a reference for those new to devops).**

https://redd.it/dq6nrc
@r_devops

Joel Test for Devops or "How to find out if you want to work for a company"

Hopefully you've heard of the Joel Test for programmers. In case you haven't, they go like this:


1) Do you use source control?

2) Can you make a build in one step?

3) Do you make daily builds?

4) Do you have a bug database?

5) Do you fix bugs before writing new code?

6) Do you have an up-to-date schedule?

7) Do you have a spec?

8) Do programmers have quiet working conditions?

9) Do you use the best tools money can buy?

10) Do you have testers?

11) Do new candidates write code during their interview?

12) Do you do hallway usability testing?



I think that's a pretty good start, but I wanted to start a conversation about the equivalent for DevOps, maybe call it the Humble Test or something lol. Clearly a healthy dev environment will make for a healthier DevOps experience. What do you think? What's your 12 questions a company should answer about their DevOps culture before you should work there?

https://redd.it/dq67ob
@r_devops

Setting up Netflix Bless for your enterprise

I just spent a lot of hours setting up the Netflix Bless for my enterprise using terraform and ansible.

[https://medium.com/@ikshitijsharma/blessing-away-ssh-worries-ultimate-practical-guide-to-setup-netflix-bless-in-under-60-minutes-d6880f2a8e23](https://medium.com/@ikshitijsharma/blessing-away-ssh-worries-ultimate-practical-guide-to-setup-netflix-bless-in-under-60-minutes-d6880f2a8e23)

Thought this will help to setup Bless in less than an hour for anyone :)

https://redd.it/dq28vd
@r_devops

Cheapest cloud compute resources with GPU

I'm looking for the cheapest possible cloud compute instance with an NVIDIA GPU. My only other requirements are minimum 2GB RAM and some storage. I need a complete VM (or bare metal server), not the ability to run playbooks, etc.

What provider should I go with?

https://redd.it/dqh09n
@r_devops

Adding Static Image Scanning to the DevOps Pipeline
https://www.reddit.com/user/weighanchore/comments/dq8ux3/adding_static_image_scanning_to_the_devops/

https://redd.it/dq8x0k
@r_devops

Where I can find some projects about applications of devOps / Kubernetes / Docker?

I want to know some project ideas which includes devops concepts. Basically a final year project on the concept. what can we do to contribute the community ....stuff like that ....

https://redd.it/dqbe4j
@r_devops

How to open a reverse tunnel into any pod or container with tcpserver and netcat

I wrote an article on how to use open-source tools to open a TCP tunnel into any container or pod that can have netcat installed on it.

[The article is here](https://layerci.com/blog/container-tcp-tunnel/)

[Inspired by discussion in this post in /r/kubernetes](https://www.reddit.com/r/kubernetes/comments/dq4st1/github_omrikieiktunnel_a_cli_that_exposes_your/)

https://redd.it/dq897q
@r_devops

MinIO image suited for Development

Hello Folks,

I'd like to share a simple project that supports minIo as an S3 Server/implementation with an initial bucket and permission configuration using environment variables:

[https://github.com/gcavalcante8808/minio-dev](https://github.com/gcavalcante8808/minio-dev)

I use it mainly to reduce drift on dev teams that relies on s3, retiring that approach of 'tests that use filesystem on dev phase' but the CI or production relies on S3 to serve files.

Suggests and PRs welcome.

https://redd.it/dq7pwd
@r_devops