First DevOps Interview

Hey guys. I'm having my first on site interview for a devops engineer position.

I'm surprised to have made it this far, because my experience with jenkins is basically none, and quite basic experience with Ansible. My main strengths are in programming, docker, and I guess having the mental urge to automate tasks and try new things.

That being said, what are some of the things I should look out for, or have knowledge of before hand?

I'm not expecting to get the job, however I am extremely excited to get experience and learn more through the interview.

https://redd.it/dpue2z
@r_devops

How to get listed in CNCF?

Does an early stage startup have any chance to get listed to CNCF landscape? Does it have to be a sponsor? Any recommendations on the process? Thanks.

https://redd.it/dpt697
@r_devops

generating adhoc plots

When I do system benchmarking I create stats for each time I run the benchmark/test. Is there a tool which I can post the data to to generate graphs? I prefer web based. At the moment I use python matplotlib but not all machines have particular version of Python and especially Matplot lib and its dependencies.

Ideally, Grafana would be awesome for this but I don't think I can create adhoc graphs. There is a JSON plugin but I couldn't get it to work. Are there any similar setups like that?

https://redd.it/drykec
@r_devops

Am I Doing it Wrong?

Just one month ago, I joined a team of 4 developers starting my DevOps engineer job.

Since then, I've been in a daily conflict with one of the developers, let's can call him Brett.

Brett is highly motivated freshly graduated engineer that writes a functional but garbage-quality code. I'm fine with it though, not really my current major concern as I'm still new to the project.

What really bothers me is the changes he does to infrastructure files, projects structure (monorepos here), and other stuff that don't contribute to the project functionality. He not only generate technical dept on my shoulders (writing a two lines of awk, duplicating it through the make file, and inventing a new files-based way to lock system packages in docker to solve a problem we don't have), but prevent me from cleaning the old stuff (he's not ok with removing a binary that counts lines of code from repository, and don't really have a problem with running migration automatically every time a container starts).

I know devops is mainly about collaboration and closing the gap between operations and development, that's what I'm trying to do when I automate the devs manual tasks into a simple make command, but I believe Brett is trying to limit my contributions, and forces bad practices that will inevitably lead to confusion for our next recruits, and he writes scripts that don't really complicate simple tasks and hard to maintain.

We talked, we agree with the business owner on a decision making process, but today after we argued about changes he made, he was very stubborn, another colleague agreed with him and approved the changes, then Brett merged his work.

While I'm still working on my own branch, he copies some of my changes to his work and commit it, and till now I've yet to merge my branch due to constant arguments he made against my work and conflict his copying results that add additional rebasing work I have to do.

Feeling disrespected, paralyzed, and not fit with the team, I offered that I resign, and the business owner replied by scheduling a call tomorrow on which I'll resign unless Brett limits his contribution to the core project. But I guess I'm going against the devops way here.

What am I doing wrong as a devops engineer? is it my approach that causes the conflict? any suggestions? Have you find yourself in a conflict with other developers when trying to bring better practices to the team and how did you deal with it?

https://redd.it/dpsafj
@r_devops

How to avoid the 403: FORBIDDEN due to the IAM service?

Account, users, role, resource, policy... when you are using IAM for the first time, it is not easy to cope with all these new denominations and concepts. I wrote an article that explains [what is IAM and how it works](https://www.padok.fr/en/blog/aws-iam), with concrete examples. Check it out and give me your opinion.

https://redd.it/dpp6iw
@r_devops

Easy way to create internal web tools

Hello, I do some devops work for my company, mostly involving various powershell and SQL scripts that I run to do some of our internal tasks or stat gathering. In order to present these stats I have been using an internal website that I have made completely manually with php. Since I do not want to focus on the web development side of things and would rather work on other things, I was wondering what a good tool would be that I can use to easily develop some fairly simple sites for things like presenting stats from in a table that connects to sql, triggering JSON file creations etc.

​

Basically, I just want an easier way of doing the web side of things so that I don't have to re-learn and mess around with PHP every time I want to add something to our internal tools, since I am not a "web developer" per se. What are some decent options that I can learn to unload the web dev side onto a tool?

https://redd.it/ds1bvv
@r_devops

Given a blank slate, tool recommendations?

Hi Guys, so I have 8 months of DevOps experience previously working as a desktop engineer for two and a half years.Starting a new role in the education sector, recently they brought their IT department which previously worked in different projects/faculties and now they have all been brought together.So this is where I come in to help dictate the application development process by designing and maintaining the infrastructure to build an efficient route to live regardless of the project. So this is essentially a blank slate.

As a very high level overview

So my ideal setup would be creating a production environment for whatever project, replicating the stack and using that as UAT. This would be done through Terraform and Ansible. The Dev estate I am not too sure how this is going to be setup yet but I was thinking about learning docker and k8s to bring up a dev estate.

The way I would look into setting up a CI/CD pipeline using Jenknnis and actioned through Azure DevOps as this is what they use, I would ensure code pushed from the DEV branch is deployed to UAT and tested then deployed to replicate production servers and once tested DNS is switched over to the new servers and the old ones are brought down.

The way how the code is deployed is through building the codebase on the same server Jenkins would be setup on, sending a tar'd file of the code to AWS CodeDeploy to be taken to the next environment. 

Through caveats include no changes or access will be allowed on production or UAT servers, any requested changes will be done through terraform and ansible to keep things consistent.

Personally with my limited experience I have used Terraform and ansible and I have felt this provides alot of the work to have that infrastructure as code system in place. At the same time I understand there are tools like chef, puppet and salt which would probably make the job easier but that is where my question is what would you guys want and how would you want to implement it. I should finally mention we seem to be using Azure DevOps as the main tool to setup and manage the pipelines.

https://redd.it/ds2fhn
@r_devops

Testing Pulumi infrastructure as code written in Python

Most of you are already aware of [Pulumi](https://www.pulumi.com/), an open source tool that allows writing infrastructure as code using high-level programming languages (JavaScript, Python, Go, and soon C#) instead of a domain-specific language.

Pulumi has an official integration testing library written in Go: https://godoc.org/github.com/pulumi/pulumi/pkg/testing/integration. However, if are developing in Python, you probably want to write your tests in Python as well. To that end, I'd like to share a Python integration testing framework that I've been working on: https://github.com/bincyber/pitfall

If you find this project useful, I would love to get your feedback on usability, any bugs encountered, or missing features that should be added.

https://redd.it/ds46nh
@r_devops

Best way to upgrade Kuberenetes without breaking anything? (EKS)

We have jenkins on EKS which needs to upgraded b/c AWS is not supporting our version anymore 1.11. Upgrading will probably break several pipelines. Has anyone had any experience with this or recommend a way to tackle it?

https://redd.it/ds3sb3
@r_devops

Why collaboration technology is critical for GitOps

“Collaboration is key to this entire GitOps process,” . “Infrastructure teams, development teams, even management, project management, security, and business stakeholders, all need to collaborate together to produce this code in a fast and efficient manner.”

[https://about.gitlab.com/blog/2019/11/04/gitlab-for-gitops-prt-1/](https://about.gitlab.com/blog/2019/11/04/gitlab-for-gitops-prt-1/)

Code: [https://gitlab.com/gitops-demo](https://gitlab.com/gitops-demo)

https://redd.it/ds3de2
@r_devops

Tutorial: Install vagrant-libvirt on macOS to Manage Remote Libvirt Hosts

I've written a tutorial on how to get [vagrant-libvirt](https://github.com/vagrant-libvirt/vagrant-libvirt) running on macOS to manage remote libvirt based virtualization hosts using Vagrant running on my Mac. It's mostly based on work done in [an issue](https://github.com/vagrant-libvirt/vagrant-libvirt/issues/497) over at the `vagrant-libvirt` repository.

Maybe someone will find it as useful as I have. :) Any feedback will be greatly appreciated!

https://lunar.computer/posts/vagrant-libvirt-macos/

https://redd.it/ds6zl2
@r_devops

Running Containers in the Cloud - What's Your Solution of Choice?

If you were starting a greenfield project today and planned to utilize containers, what would be your solution of choice? Assume a microservices architecture with multiple services (yes I know it is 'bad' to start with microservices), all stateless applications.

These are some of the options I can think of off the top of my head.

All the managed kubernetes offerings (aks, eks, gke).

Cloud vendor specific solutions like Azure Container Instances, AWS ECS (fargate or ec2 backed), GCP App Engine Flexible Environment.

Roll your own kubernetes cluster either manually or via a tool like rancher.

Run containers on vms with container runtime installed (gross).

I ask this because we are starting a project soon and a lot of the team is familiar with AWS. But I'm not personally a huge fan of ECS (and I'm not sure it is still relevant), and I'm not sure I trust the AWS EKS offering. Interested to hear peoples thoughts!

​

I think I would lean toward GKE as kubernetes and containers have kind of been their baby. I also prefer the GCP platform, I feel like they have a much better developer experience than AWS or Azure. Things just seem simpler. AWS and Azure have too many relics from the days of on prem data centers. GCP abstracts a lot of that complication away.

https://redd.it/drz57o
@r_devops

Anyone else feel like this subreddit has really gone downhill? Is there anything we can do?

Maybe I'm just a major asshole. I try not to be. But I feel the same way about here as I do about /r/sysadmin anymore. Every time I come here anymore I feel like it's just an echo chamber of "Terraform, Kubernetes, and Azure DevOps". And everything is a nail for those three hammers. And no one is doing anything other than really vanilla infrastructure as code. Like that's the end-all and be-all of DevOps.


I also feel like the vast vast vast majority of conversations going on here are about greenfield new stuff. And a bunch of people that aren't all that close to production are like "yeah, Azure DevOps is great. go with that. we got our little hello world app up in like an hour". Like, I know people can't just wave a wand and choose where they work. It's not anyone's fault if they work in an old school slower moving shop. But maybe don't evangelize stuff if you're not using it in an extremely high velocity production scenario? (ie dozens of devs deploying to production many times a day)


Like I said, maybe I'm just an asshole. But it would be cool to talk about stuff like solving real business use cases with creative solutions. And not just "oh you want to automate creating servers? use this tool". Maybe there needs to be another subreddit for (what I, as a piece of shit call) "real" DevOps? Like, no company is making it to their IPO because an engineer moved their shit to Kubernetes. So can we just talk about tools a little less? Or, really, a *lot* fucking less?

https://redd.it/ds7vbc
@r_devops

Grafana Annotations in pie chart

Hi Everyone

We were exploring grafana annotations published by Jenkins, and they are showing up in graphical charts. But annotation markings were invisible when we changed the graph to pie mode.

Is it possible to have annotations on top of pie charts?

Regards

https://redd.it/ds9gas
@r_devops

[aws beanstalk] is it possible to configure application load balancer to return 403 based on path?

Hi,

I am aware that I can configure load balancer --> listeners --> rule and use path matching + action.

However, I am wondering if it is possible to achieve the same using beanstalk configuration?

I was reading

[https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-cfg-alb.html](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-cfg-alb.html)

and in the end of the link it gives us a example:

aws:elbv2:listenerrule:admin:

PathPatterns: /admin/\*

Priority: 1

Process: admin

​

From the code and also the beanstalk - configuration - load balancer UI page, it looks like it cannot return 403 and all action must tight to a "Process" (target group).

Am I missing something?

Thanks!

Jay

https://redd.it/ds8rud
@r_devops

What does your local development environment/setup look like?

I'm curious about how other companies and organizations setup their local development environment.

\- Do you perform most if not all of your development locally? Or on a remote host or virtual machine? e.g. AWS Workspaces/AppStream/Windows Remote Desktop

\- Are you calling your Cloud Provider's APIs using a full-fledged Admin role? Or a role with least privileges? Are you able to stay authenticated to a single role to perform all of your job duties/functions? Or do you switch IAM roles based on the services you are invoking/working with e.g. VPC, EC2, Lambda?

\- Do your developers write code locally? How are they invoking your Cloud Provider's APIs? Do they have their own isolated sandbox/dev accounts? Or do they all share the same dev account?

https://redd.it/ds3ud3
@r_devops

Building a CI/CD pipeline, suggestions?

Hi Guys, so I have 8 months of DevOps experience previously working as a desktop engineer for two and a half years.Starting a new role in the education sector, recently they brought their IT department which previously worked in different projects/faculties and now they have all been brought together.So this is where I come in to help dictate the application development process by designing and maintaining the infrastructure to build an efficient route to live regardless of the project. So this is essentially a blank slate.As a very high level overviewSo my ideal setup would be creating a production environment for whatever project, replicating the stack and using that as UAT. This would be done through Terraform and Ansible. The Dev estate I am not too sure how this is going to be setup yet but I was thinking abouta learning docker and k8s to bring up a dev estate.The way I would look into setting up a CI/CD pipeline using Jenkins and acted through Azure DevOps as this is what they use, I would ensure code pushed from the DEV branch is deployed to UAT and tested then deployed to replicate production servers and once tested DNS is switched over to the new servers and the old ones are brought down.The way how the code is deployed is through building the codebase on the same server Jenkins would be setup on, sending a zip'd file of the code to AzureDevops to be taken to the next environment. Through caveats include no changes or access will be allowed on production or UAT servers, any requested changes will be done through terraform and ansible to keep things consistent.Personally with my limited experience I have used Terraform and ansible and I have felt this provides alot of the work to have that infrastructure as code system in place. At the same time I understand there are tools like chef, puppet and salt which would probably make the job easier but that is where my question is what would you guys want and how would you want to implement it. I should finally mention we seem to be using Azure DevOps as the main tool to setup and manage the pipelines.

Would like some opinions on what could be changed and as mentioned how I could use the other configuratoin tools such as salt, chef and puppet to help manage the infrastructure better.

https://redd.it/dsdymp
@r_devops

Just need some clarity.

I’ve been asked to help setup a free ci/cd solution for a friend. At work we use Azure DevOps and we have on prem servers which we build, then install an agent on and the code gets deployed there etc. I started looking at azure pipelines as it’s free however I’m a tad confused. They offer CI/CD for free however I’m confused by the terminology. Do they also supply something for the code to sit on or would I need to setup a cloud solution with say Azure and build a server etc or am I getting confused because my brain is so baked with onPrem? If that’s the case what free solutions are out there so I don’t need to have a server. It’s c# code if that matters

Thanks

https://redd.it/dse80g
@r_devops

Migrating your app to Kubernetes: what to do with files?

While [almost?] everyone knows about 12factor app, it's still quite often that we have to explain our customers what we expect from their apps to migrate them to Kubernetes. [Here](https://medium.com/flant-com/stateful-app-files-in-kubernetes-d015311e5e6b) is our list of common workarounds for dealing with application's files (which we don't really recommend to use) and more practical solutions for that.

https://redd.it/dsfec3
@r_devops

Free event: All Day DevOps 2019 happening right now

Just want to let you guys know there's a 2 day DevOps conference going on and it's possible to watch for free:

https://www.alldaydevops.com/2019-schedule

https://redd.it/dsgfip
@r_devops

Deploying docker container to production - how to do versioning and have production pull new image? Using docker compose

I am creating a CI for production deployment using docker and circleci.


On production I am using docker-compose with:


services:

sp5: # Tomcat 6.0.39 /w 6 JRE w/o default apps

image: foo-registry/sp5-web-app-front:5

container\_name: 'sp5\_application'

I will have CircleCI build the Java application, build a docker image containing it, and push it to my docker registry. Then I will have CircleCI let production host know so it can pull down the docker image.


Question - how do I handle production pulling down the image? Meaning.... it does not seem efficient to use code to modify the docker-compose.yml file to change the version number in \` image: foo-registry/sp5-web-app-front:5\`. I could use 'latest' instead, but I think versioning would be better. Also, I have had issues with docker refreshing the image when it's 'latest'.... it's like it works off of cache and does not know latest changed.

https://redd.it/dshwq3
@r_devops