Monthly 'Getting into DevOps' thread - 2019/11

**What is DevOps?**

* [AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

**Books to Read**

* [The Phoenix Project](https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/1942788290) - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
* [The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
* [Google's Site Reliability Engineering](https://landing.google.com/sre/books/) - Google engineers explain how they build, deploy, monitor, and maintain their systems.
* [The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book

**What Should I Learn?**

* [Emily Wood's essay](https://crate.io/a/infrastructure-as-code-part-one/) - why infrastructure as code is so important into today's world.
* [2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
* [This comment by /u/mdaffin](https://www.reddit.com/r/devops/comments/abcyl2/sorry_having_a_midlife_tech_crisis/eczhsu1/) - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
* [This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

**Previous Threads**

https://www.reddit.com/r/devops/comments/dbusbr/monthly_getting_into_devops_thread_201910/

https://www.reddit.com/r/devops/comments/cydrpv/monthly_getting_into_devops_thread_201909/

https://www.reddit.com/r/devops/comments/ckqdpv/monthly_getting_into_devops_thread_201908/

https://www.reddit.com/r/devops/comments/c7ti5p/monthly_getting_into_devops_thread_201907/

https://www.reddit.com/r/devops/comments/bvqyrw/monthly_getting_into_devops_thread_201906/

https://www.reddit.com/r/devops/comments/blu4oh/monthly_getting_into_devops_thread_201905/

https://www.reddit.com/r/devops/comments/b7yj4m/monthly_getting_into_devops_thread_201904/

https://www.reddit.com/r/devops/comments/axcebk/monthly_getting_into_devops_thread/

**Please keep this on topic (as a reference for those new to devops).**

https://redd.it/dq6nrc
@r_devops

Joel Test for Devops or "How to find out if you want to work for a company"

Hopefully you've heard of the Joel Test for programmers. In case you haven't, they go like this:


1) Do you use source control?

2) Can you make a build in one step?

3) Do you make daily builds?

4) Do you have a bug database?

5) Do you fix bugs before writing new code?

6) Do you have an up-to-date schedule?

7) Do you have a spec?

8) Do programmers have quiet working conditions?

9) Do you use the best tools money can buy?

10) Do you have testers?

11) Do new candidates write code during their interview?

12) Do you do hallway usability testing?



I think that's a pretty good start, but I wanted to start a conversation about the equivalent for DevOps, maybe call it the Humble Test or something lol. Clearly a healthy dev environment will make for a healthier DevOps experience. What do you think? What's your 12 questions a company should answer about their DevOps culture before you should work there?

https://redd.it/dq67ob
@r_devops

Setting up Netflix Bless for your enterprise

I just spent a lot of hours setting up the Netflix Bless for my enterprise using terraform and ansible.

[https://medium.com/@ikshitijsharma/blessing-away-ssh-worries-ultimate-practical-guide-to-setup-netflix-bless-in-under-60-minutes-d6880f2a8e23](https://medium.com/@ikshitijsharma/blessing-away-ssh-worries-ultimate-practical-guide-to-setup-netflix-bless-in-under-60-minutes-d6880f2a8e23)

Thought this will help to setup Bless in less than an hour for anyone :)

https://redd.it/dq28vd
@r_devops

Cheapest cloud compute resources with GPU

I'm looking for the cheapest possible cloud compute instance with an NVIDIA GPU. My only other requirements are minimum 2GB RAM and some storage. I need a complete VM (or bare metal server), not the ability to run playbooks, etc.

What provider should I go with?

https://redd.it/dqh09n
@r_devops

Adding Static Image Scanning to the DevOps Pipeline
https://www.reddit.com/user/weighanchore/comments/dq8ux3/adding_static_image_scanning_to_the_devops/

https://redd.it/dq8x0k
@r_devops

Where I can find some projects about applications of devOps / Kubernetes / Docker?

I want to know some project ideas which includes devops concepts. Basically a final year project on the concept. what can we do to contribute the community ....stuff like that ....

https://redd.it/dqbe4j
@r_devops

How to open a reverse tunnel into any pod or container with tcpserver and netcat

I wrote an article on how to use open-source tools to open a TCP tunnel into any container or pod that can have netcat installed on it.

[The article is here](https://layerci.com/blog/container-tcp-tunnel/)

[Inspired by discussion in this post in /r/kubernetes](https://www.reddit.com/r/kubernetes/comments/dq4st1/github_omrikieiktunnel_a_cli_that_exposes_your/)

https://redd.it/dq897q
@r_devops

MinIO image suited for Development

Hello Folks,

I'd like to share a simple project that supports minIo as an S3 Server/implementation with an initial bucket and permission configuration using environment variables:

[https://github.com/gcavalcante8808/minio-dev](https://github.com/gcavalcante8808/minio-dev)

I use it mainly to reduce drift on dev teams that relies on s3, retiring that approach of 'tests that use filesystem on dev phase' but the CI or production relies on S3 to serve files.

Suggests and PRs welcome.

https://redd.it/dq7pwd
@r_devops

Creating a CLI in Python vs Go

Hi all, I need to build a CLI and expect this to be used significantly throughout my company. I'm an experience python dev and absolutely love the language. At the same time, I do understand that the python package management is a nightmare.

For my CLI, I would like to:

* Install on mac through brew (and whatever the equivalent is on windows/linux)
* Should not need a venv
* Should not be affected by other software running on the users system

Personally, I've always built my python applications in isolation using a venv but since this new CLI would be used by 1000+ users, I'm second guessing if python is the right language for this.

I don't know Go (yet) but have been eager to get my hands dirty and this seems like a good place to start - at the same time writing a large tool in a new language is a recipe for disaster.

Could someone provide their experience with large CLI tools written in Python? As far as I can tell awscli and azure cli are also written in Python so maybe i'm just overthinking this and should stick to what I know.

Thoughts?

https://redd.it/dq6kiy
@r_devops

How to do ssh forwarding in compass(mongodb) client.

Folks, I need some help/advice on who to connect document db via Compass client. using ssh forwarding.

My scenario is I have an ec2-A(public) instance which uses ec2-B(private)instance to connect to MongoDB. I know how to do it on Windows via putty but how to do it on a **Mac**??

Using this to connect jump server

ssh -i ~/B.pem -o "ProxyCommand ssh -W %h:%p -i ~/A.pem [email protected]" [email protected]

and then mongo connection command.

`mongo --ssl --host <mongodb_endpoint>:27017 --sslCAFile rds-combined-ca-bundle.pem --username usernam --password password`

&#x200B;

I read this link: [https://docs.mongodb.com/compass/master/connect/](https://docs.mongodb.com/compass/master/connect/)

but it was not helpfull.

https://redd.it/dq6ay0
@r_devops

Nameing conventions to seperate tools QA and production and application environment

Our company is developing a CI/CD solution. We are having trouble differentiating between our tools environments from our application environments to management. I am having a lot of trouble coming up with a naming scheme that clearly separates them without needing a few sentences to explain.

So, we Have our Git Repo on-prem, Build Server (CI), Deploy Server/tool(CD). All of those tools have a QA environment, QA meaning we test patches and new configurations there before we push to the production ones that are in active use.

&#x200B;

We also have a normal setup of Acceptance, Staging, and Production application development environments.

The issue is everyone associates QA with the application staging environment.

&#x200B;

So my question is, what would be a clear way to separate these two different ideas?

&#x200B;

Thanks

https://redd.it/dq3kue
@r_devops

Advice needed on an open-source cloud devops tool.

Hey,

I am currently planning to develop a Cloud DevOps open-source tool. But I am not sure whether it is gonna be helpful and how helpful it is. So want to hear from tech pros first.

Here is a short description of the tool idea.

&#x200B;

>*A Cloud DevOps Orchestration tool that enables you to build and manage Cloud DevOps solutions, including infra templates, code pipeline YAML, SRE tasks and etc, in a single package stored in Git. The tool could connect with required DevOps tools to provision resources and run the tasks.*
>
>*For example, you could put CloudFormation templates in the package, and write some Ops Tasks to do some operational work on the resources provisioned by CFN. With the tool, it calls AWS-CFN to start provision and runs the tasks on your command.*
>
> *The tool tries to automate Cloud DevOps itself. It simplifies the toolchain integration, provides a way to organize your DevOps work, automates the provisioning and runs the Ops tasks in a consistent and centralized way.*

My description may not be quite clear. Please let me know if I need to clarify.

Just want to know:

1. Is it helpful?
2. If NO, a little on why and any improvement room?
3. If YES, how do you plan to use it?

Your feedback and advice are much appreciated. Any random thoughts may help here.

&#x200B;

Thanks a lot!

https://redd.it/dqmw13
@r_devops

CI/CD for containers? Kubernetes/version control?

What is the typical process behind deploying containers, and also deploying containers in kubernetes, including how the source code is managed, source code/container security, etc?

https://redd.it/dqnzbe
@r_devops

I inherited management of a junior/entry level DevOps engineer, how to train them?

So, another manager in my company was building an ops team, but then left the company and that team was folded into my DevOps/SRE/platform team.

We've always had high-level hiring standards (apart from me, lol), so everybody on my team is a heavy hitter. This one guy coming from the ops team, had just transitioned to that area from doing Windows desktop support. He's a smart guy, hardworking, really eager to learn... but he knows nothing about the tools or technology we use. I've gotten where I am because people gave me chances, so I want to give him the same.

However, I've never really trained/mentored anybody that junior before. I'm thinking that he needs to focus on fundamentals:

* Linux OS concepts
* TCP/IP networking stack
* Bash scripting
* Python programming
* Git

He's eager to get stuck in to working with AWS and k8s, but I think it's more important to understand the basics before getting too deep into the specifics of a cloud provider.

Does anybody have any advice on how to help train somebody in DevOps? Are there any areas that it seems I might be missing out on?

https://redd.it/dqmmza
@r_devops

Why is MTTR (mean time to restore) rather than availability considered a key metric?

This is something that's confused me about the [four key metrics](https://www.thoughtworks.com/radar/techniques/four-key-metrics) promoted by Forsgren, Humble and Kim in the Accelerate book and reports.

If it only takes ten minutes to restore service that sounds pretty good, but it wouldn't be great if it breaks every twenty minutes.

I get that MTTR is probably a better metric than meant time between failures, (MTBF), but I would expect dividing one by the other would give availability, which would seem like an even more important thing to optimise.

https://redd.it/dqnqxy
@r_devops

Vettery? Hired? Worthwhile (as candidate) or not?

Earlier this year I accepted a position with a new organisation. Whilst some things are very good, some things are not, and I am looking to move. I am wondering if it is worthwhile to sign up with Vettery or Hired.

For context I have almost 500 connections on LinkedIn and several people contact me there each business day with possible positions (although I am not saying that all these positions are attractive or that I would be suitable for all of them). I also attend various meetups and expect to be speaking at two in the next quarter plus a paid conference next year (if I can get a worthwhile project between now and then to speak about!).

The reasons that I am considering Hired/Vettery are that:

* Most of my contacts via LinkedIn are from third party recruiters, where there is always a lot of additional effort required on my part compared to, and the 'yield' is much less than, in-house recruitment teams.
* A more senior person I worked with previously found a position where I was then working via Hired and seemed to think well of them.

Whilst I do try to screen recruiters, it is still tiresome going over the same questions over and over again. Unless you are a hot pick for a specific position that that specific recruiter is trying to fill right now, then no, they aren't interested and on the tiny number of occasions when they come back with something else down the line all context has been lost (and invariably changed). Trying to get feedback from a recruiter is nearly impossible- they either aren't interested or they just want to know what would persuade you to take a position, or they have no info.

The obvious alternative would be for me to either wait for the right opportunity to pop-up on LinkedIn, or to apply for advertised positions myself. Sadly I have had a poor enough experience with recruitment processes on many occasions that I am loath to do the latter. I also feel that it puts me at a disadvantage.

I am wondering if anyone here can speak from personal experience of being a candidate with either Vettery or Hired as to whether or not they are worthwhile in my sort of situation. I am a mid/senior DevOps/Cloud/Microservices etc type engineer.

https://redd.it/dqnmwm
@r_devops

So excited! Just got an offer for a devops position yesterday!

I've worked in the past few years in software engineering roles and in IT operations roles, and had decided at some point I would want to get into devops.

Two weeks ago, I got pinged about a role for a devops engineer position involving building CI/CD pipelines for serverless microservices. Not something I've had the opportunity to do yet, but I know the concepts and general workflows, so I decide to respond.

Had a pretty good phone interview directly with the hiring manager last Friday. We talk about my experience, he tells me more about the role, and it seems pretty sweet. I get a follow up Monday saying "hey we want to bring you on-site to talk to you more, more details to come."

Wednesday I get an email from them. They want to do a 3 hour interview. With a full itinerary. With multiple directors, multiple senior staff, a VP, and they want me to start by giving a presentation about how I would migrate a monolithic on-prem application to serverless microservices with minimal downtime and high resiliency, with full CI/CD pipelines. Queue me freaking out a little.

That night I sketch up some notes and diagrams on my Surface and share with a friend who has some knowledge on devops to make sure it looks okay. He gives me a thumbs up, so I wait for the interview.

I don't sleep at all the night before, so I play video games to take my mind off things. When it is time to get ready, I get showered and changed, and then go get an energy drink to perk myself up.

I get to the interview, and I give my presentation, scribbling all over their whiteboard. For the most part, they like it, but they have several questions about it. I can answer some of them, but some I can't and I tell them as much.

The rest of the interview goes well enough, with them asking my background, education and all that. I explained that my degree program was horrible and I pretend it doesn't exist, and that everything I have done career-wise is 100% self-taught or learned on the job, and they asked how I would approach various problems. They also ask me about containers, OOP, unit testing, infrastructure as code and several other things.

I really got put through the meat grinder and was left feeling pretty uneasy about it all when I left. I knew I gave it the best I could, but was still worried. Around 3:30 though, I got a call that they wanted to extend an offer. They told me my skill set specifically within cloud technologies was a little light, but I know enough about everything else and I know the high level concepts well enough that they want to bring me on anyway.

I'm so excited! It went better than expected!

https://redd.it/dqjf5n
@r_devops

Deploy Prometheus exporter together with Grafana dashboard and Prom alerts

Hi I need advice from gurus :)

Currently we need to have like a package for monitoring. It means that once we install for example Kafka exporter, also should be installed Grafana dashboards and Prom alerts which are related to that exporter.

If it’s exporters which were developed by me - I can put dashboard and alerts inside of the exporter. But if it’s official exporter - it’s problematic to do.

So, please share how do you install exporters with dashboard and alerts and how do you change thresholds dynamically (if you have multiple clusters with the same exporters but thresholds should be different)

Thank you for your answer

https://redd.it/dqjkpm
@r_devops

CI pipelines and README.md updates

What are you all using to update your README.md documentation? It would be nice not to have to manually update badges and Docker tags in examples on each version bump.

https://redd.it/dqfg9b
@r_devops

Ideal Automated IIS Deployments

Suppose that a single machine hosts numerous (25+) IIS Websites, App Pools, virtual directories, and Web Applications.

From a clean slate (i.e., only the Default Site and Application pool are created), what is the ideal way to automate the deployment of the above components?

Should the Default site just be disabled, or deleted? Do we create all App Pools first, and then the Web Sites and other components? How about the point at which the source code is extracted into the relevant folders? How about additional third party applications (e.g., nServiceBus), or registry modifications?

Any ideas are appreciated!

https://redd.it/dqe631
@r_devops