[mini-rant] Doing "DevOps" work, but not sure if others value it

It feels like sometimes it's more about "hey, just make DevOps do that so we don't have to".

I keep thinking to myself that if they truly valued and prioritized DevOps principles, that my job wouldn't exist, i'd just be another dev or sysadmin contributing towards sprint goals. instead, they've just created another silo to bury work that other people don't want to do, despite the fact that it's there to make their jobs and lives easier.

/rant

https://redd.it/dshfnx
@r_devops

Need help setting up Gitlab CI/CD to build docker images on new push to master, push these images to gitlab docker registry and then trigger a redeploy on an AWS EC2 instance with the new images.

I am new to devops. I need some help with understanding how I can set up my Gitlab CI/CD, is what I want even possible? I am thinking I need somekind of mix between using Gitlabs hosted shared runners and a runner installed and registered from my EC2 instance. Is that right? Where should I start?

https://redd.it/dsix8k
@r_devops

Artifact repository and other things....

My company and me are using .NET and npm most of the time. Actually we store our releases in a git repository in bitbucket (we also use Jira and Bamboo).

Now we are searching for an artifact repository where we can store our releases and in the future our container images. So, we need a piece of software which fits our requirements and is easy to use. We also want a feature called "package promotion".

At the moment we have ProGet installed and i'm testing Artifactory right now. But i'm not happy with any of these. I just found Azure DevOps, but i'm also not sure about it because it doesn't provide a container image registry. It has only a few package providers (artifacts). Because Azure DevOps provides pipelines we would also use them because Bamboo isn't that good :P

Maybe anyone has already the same experience and know which software is good? Maybe only a good artfact repository with atlassian integrations or a completely new DevOps Platform (sadly gitlab has bad support for package /artifact providers)

https://redd.it/dsi4p7
@r_devops

Seeking Advice with Separate Monitoring Stacks; Splunk vs Prometheus + ELK

My organization is trying to improve our monitoring and alerting capability across all of our new and existing applications. Across the board we have over 200+ applications and over 300+ VM’s we need monitored. Currently we have 85% of our infrastructure on primarily Windows machines in VMware and vSphere. This includes a large series of DotNet applications along with some of our message brokering systems like rabbitMQ and our Kafka Clusters.

The monitoring for these applications (which we consider legacy) includes Nagios and Splunk via WMI on the Windows VMs. These metrics are all exported to our Enterprise Splunk Cloud server. We’re importing about 50GB a logs a day (I know that’s next to nothing to many of you) and we expect that to grow to roughly 150GB as we enabled more application logging. Considering that we already have the license for Splunk Cloud, Splunk seems like the right solution when it comes to being able to just dump logs in any format and search through them.

This presents some challenges for us however. Right now our indexing strategy is garbage, where nearly all of these logs, OS metrics and other metadata for completely separate services and events are all sent to only a handful of indexes, making our 1000+ alerts on them all the more exhaustive. In reality I believe we need a more structured set of data being input into the system so that we can create simpler, less resource intensive queries to application metrics as well as infrastructure metrics (such as CPU, Memory over weeks for some applications).

To throw a wrench in the machine we’ve also been facing a lot of pressure to monitor the remaining (but growing) 15% of our tech stack, which is in kubernetes. I came from a background of monitoring kubernetes clusters using open source tools like prometheus, grafana and logging/monitoring solutions like ELK. I am trying to set a new standard for monitoring moving forward by using these solutions in our kubernetes environment but I’ve begun to face some pushback from others who are concerned about having all these metrics, dashboards and alerts all in one area, and that area they’re suggesting is Splunk. I totally understand the value of being able to centralize this information as it provides a lot of added value, but I’m wary as I don’t really think Splunk is the wisest choice for things moving forward considering how many of these open sources tools are being built around kubernetes while the kubernetes to Splunk exporter is fairly lack luster.

Have you guys ever worked in an organization that successfully had two entirely separate monitoring solutions for legacy vs present/future applications or for different tech stacks?

https://redd.it/dshg7i
@r_devops

New to Terraform: is there a way to get the object values of a existing resources in Azure. In powershell I can just query the Resource and get all its properties. It easier to get an ideas what property values a resource or an object pipes out.

Apologies for being naive..

https://redd.it/dsqx42
@r_devops

Any idea about the Cloud/DevOps market in China?

For the most years, we know the only leading name, 'Alibaba' when it comes to China's cloud market. But do you have any idea how it exactly is doing DevOps in China? Who are the players, and are Chinese organizations use Docker and Kubernetes, or they have something else that is employed to do the same job?

https://redd.it/dst1ba
@r_devops

What are you doing for load balancing Hashicorp Vault (HA cluster, open source license)?

I've been running a Hashicorp Vault 0.10.x HA cluster in AWS with an ALB in front of it for a couple of years. It works fine but I've decided to set up a new cluster using a new version of Vault, and start migrating to it.

The new cluster would replace Consul with DynamoDB for storage, which will simplify setup significantly. I'm also adding TLS termination at the node level so data is encrypted at rest and in transit.

At this point I'm wondering what to do for load balancing. I know Vault elects a cluster leader, and requests that you send to non-leader nodes are forwarded to the leader. I know this works fine with an ALB but am not sure if I need the ALB.

The individual nodes will have static IPs so I could do "load balancing" by adding the static IPs to a DNS "A" record and access Vault through that record.

I can also keep using an AWS load balancer, though since I'll be terminating TLS at the nodes, I could switch to NLB instead of ALB.

The latter option is what seems to be Hashicorp's recommendation in their [reference architecture docs](https://learn.hashicorp.com/vault/operations/ops-reference-architecture#load-balancing) (it doesn't mention AWS NLB specifically, just an "external load balancer", but the HAProxy example is similar to (if simpler than) the way NLB works.

Do you have something of the sort running and how did you architect it?

Bonus question: do you have any thoughts about using DynamoDB as the storage backend? I'm also open to MySQL or Postgres. Basically just want to get away from Consul.

https://redd.it/dsk736
@r_devops

What business outcomes have you made possible recently?

There's a lot of focus on the tools we use day-to-day, and technical details around that.

Let's ignore that part, and talk about the positive change, we as a profession make possible in the world. What outcomes has your work made possible that you're proud of? What was the impact on the business and the people involved (aka stakeholders)?

Don't mention the "how" in the top comment, only the results achieved and the why you are proud to have contributed to them. Looking forward to reading your stories!

https://redd.it/dsdjqt
@r_devops

Monitoring wall reccomendations

We have 6 monitors on our monitoring wall all of which have chrome open and revolve through tabs for New Relic and some for Grafana. How do others automate the opening of these?

Currently were using Windows for the PC powering the wall & Nircmd to open chrome and the tabs on the correct screen. but I'd prefer a GUI solution or something easier to manage.

https://redd.it/dsh4zv
@r_devops

Tutorial: Deploy a Kubernetes Cluster with Service Mesh on DigitalOcean Using K3s

I threw together a tutorial on how to deploy a Kubernetes cluster with a service mesh on DigitalOcean. It uses the [k3s](https://k3s.io/) Kubernetes distribution and [Rio](https://rio.io/). The cluster will be a single master, so no HA unfortunately. In the future I'll write a tutorial on how to deploy a k3s HA cluster as well, and how to deploy persistent storage.

I hope someone will find it useful or educational. :)

https://lunar.computer/posts/k3s-service-mesh-digitalocean/

https://redd.it/dsyfcz
@r_devops

Why choose Terraform over Chef, Puppet, Ansible, SaltStack and CloudFormation?

Hi all,

I have this post on [why choose Terraform](https://selleo.com/blog/why-choose-terraform-over-chef-puppet-ansible-saltstack-and-cloudformation) vs \*insert another solution from the title\* and I would really appreciate any comments / thoughts / experiences / possible improvements to it.

Which one, in your opinion, is the biggest Terraform competitor and why? Currently researching this as well.

Thanks in advance!

https://redd.it/dswxc0
@r_devops

Cloudformation not assigning ingress to security group

I have a ALB that I am trying to assign a security group and ingress to. I want 443/HTTPS open only. Currently it creates the SG and assigns it to the ALB but no luck on the ingress. I've tried the below as well as the ingress as it's own resource.

​

ALBSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: "Security Group for ALB"
SecurityGroupIngress:
IpProtocol: tcp
FromPort: 443
ToPort: 443
VpcId: <VPC ID>

https://redd.it/dsz52x
@r_devops

S3 error when creating MongoDB stack via quick start
https://www.reddit.com/r/aws/comments/dsr3x5/s3_error_when_creating_mongodb_stack_via_quick/

https://redd.it/dt4ang
@r_devops

What am I missing in my Devops interviews?

Hey all, not sure if this kinda post is allowed, if not please delete.

I am currently a devops engineer and have been for the last 3 years. I started going on interviews again and have been rejected for a couple jobs I feel I had a great opportunity to get. I know this is kinda an open ended question, but what do you all feel are key points to hit on in a devops interview other then Docker/containerization, Kubernetes, AWS (I am certifited), CI/CD, scripting with bash/python etc., etc? Just want to see if there something else I maybe should add to my resume to make myself look stronger or for those answering who are Devops leads, what are things your are looking for?

Thanks for the help!

https://redd.it/dt6oe3
@r_devops

SSL Cert Rotation with Runbook

[https://patrick.ble.si/ssl-cert-rotation-with-runbook](https://patrick.ble.si/ssl-cert-rotation-with-runbook)

&#x200B;

How do you manage SSL certificates? Do you have an automated process? A vendor solution?

https://redd.it/dt1weq
@r_devops

Unable to start openvpn server after install
https://www.reddit.com/r/OpenVPN/comments/dt4uaq/unable_to_start_openvpn_server_after_install/

https://redd.it/dt4y6g
@r_devops

Terraform for provisioning bare metals?

Just read a comment about people using Terraform on bare metals. I thought Terraform was for provisioning on aws and other cloud providers. I know you can write your own custom provider, but I don't see what you would do with Terraform on bare metals.

If you personally use Terraform on bare metal, what are your use cases?

https://redd.it/dtabmo
@r_devops

Kubernetes Multi-Operator Stacks and Advanced Pod Scheduling

Here are some thoughts on how to think about Pod scheduling while developing Kubernetes Operators for multi-Operator environments.

[https://medium.com/@cloudark/kubernetes-multi-operator-stacks-and-advanced-pod-scheduling-c3131dde3755](https://medium.com/@cloudark/kubernetes-multi-operator-stacks-and-advanced-pod-scheduling-c3131dde3755)

Are you using Kubernetes Operators? Have you faced above issues? Would be curious to understand your use-cases in the comments.

https://redd.it/dt21hz
@r_devops

Heroku CI vs GitHub Actions
https://www.reddit.com/r/github/comments/dtbh6t/heroku_ci_vs_github_actions/

https://redd.it/dtblkd
@r_devops