Locked to Windows. Is there a better way?

I manage the infrastructure for a sports bar chain. We have a few strict conditions, which keeps leading me back to using a full windows stack as the only logical solution. I made a post a little bit ago about this but have finally fully understood some upgrade paths we can take and what it would actually look like to implement them.

So I believe we are locked into windows and I want to explore changing that to save on hardware costs and allow for future space for an internal development team to build and deploy apps at the Edge. We don't currently have an app dev team, but the business is rapidly moving towards data analysis and utilizing that data. At the same time, we are looking to expand, so fast and automated builds are a must.

So why are we locked in to windows? here's why:

1. Computing MUST be hosted at the edge. Restaurants have spotty internet and can experience frequent outages. Having the POS always available regardless of internet is non-negotiable. A cellular backup is at each site, but the bandwidth wouldn't support the massive load.
2. Our POS system (Oracle Simphony) is most supported on windows clients, for both the servers and the clients. We host windows VMs running this software and tablets RDP into the VMs from iPads. This abstracts the POS away from the environmental factors of a bar, such as water damage, people messing with cables, and utilizing a cattle mentality towards our POS systems with fast replacement of VMs instead of fixing or upgrading.
3. Since we use Windows clients VM. We need a license for each one, so we use Windows Hyper-V for the datacenter license which allows us to license each of the 30-80 clients per restaurant using AVMA and saving on individual licenses.

What we are planning to implement to improve our systems considering that we are locked to windows:

1. Cheap, but power Mini servers with enterprise hardware (up to 64GB of RAM, single socket, U.2 drives) essentially running as blade servers (4-6 per site). This would allow the restaurant to keep running even if they encounter a server failure.
2. Each mini server is running Hyper-V on Datacenter Core with KVM over IP and API based management using redfish. They are all clustered together and have enough capacity to live migrate VMs in the case of a server going down.
3. Packer + Terraform + Ansible to build and deploy clusters, allowing fast deployment for the fast scaling of building new restaurants without onsite presence needed.

A potential path to Linux?

1. Oracle Simphony running oracle linux is supported, but support would be hard to get. It would lead to need to deeply understand a new operating system and fix it ourselves even though we pay for a support contract.
2. Oracle Linux containers are officially supported, so we can run them in an edge kubernetes cluster, with the same architecture as above utilizing mini servers but would require less computer power and save costs there. Downside would be managing our own clusters. Chic-Fil-A did it, but I'm not sure I want to. I could instead use proxmox or openshift, which sounds easier, but openshift is a bit expensive compared to 4 one time $100 datacenter licenses.
3. Using the method outlined in this video: https://youtu.be/L4nqky8qGm8?t=671 We can deploy the POS and a VNC endpoint that the iPads can connect to and interact with their client.


In the end, this is all optimization, so it feels riskier to rip out an entire system that we have learned to tune well with automations based in powershell and have built a robust monitoring system around it to remediate issues quickly.

So what do you think? Is there a better way, or should we stick to windows as it works well right now and is the best supported?

https://redd.it/1criu6h
@r_devops

Benchmarking two log management engines: Loki and Quickwit

Hi there,

I wrote a blog post comparing Loki and Quickwit on a simple log management use case. I'm working on Quickwit engine so I'm obviously biased but:

- I tried, as best as I can, to balance my view on focusing on the tradeoffs between the two engines: basically Loki is faster at ingestion and Quickwit is faster at query time.
- I met many DevOps who were not understanding the architecture/datastructures/tradeoffs of Loki and Quickwit, I think it's worth sharing the results at it will help users to make the right choice given their use case.

And I would love to get feedback on this, the benchmark code is available on github.

Blog post: https://quickwit.io/blog/benchmarking-quickwit-loki

Benchmark repo: https://github.com/quickwit-oss/benchmarks/blob/main/loki\_quickwit\_benchmark.md

https://redd.it/1crkj6y
@r_devops

Any freelance jobs for devops/cloud engineer

Hi,
I am looking for freelance opportunities into devops. I have 6+ years of professional experience in cicd, azure, terraform andbother devops tools.
Thanks.

https://redd.it/1crme51
@r_devops

Update Resume with Home Lab Experience

I wonder that if I could list my homelab experience in my resume. Apart from my working experience, I selfhost my lab with k3s with many services in it, setup nas, pi-hole,... I thought that was a hard and fun process, and when I have problem in my work, some experiences save me much time. So, anyone here list you homelab exp in your CV/resume? Please give me you thoughts, I'm really appreciated that.

https://redd.it/1crnk49
@r_devops

DevOps tasks from QA perspective

Hi guys,

I wanna learn devops, right now I m a QA Automation. What DevOps task can I take as a QA Automation?

Thanks!

https://redd.it/1cro930
@r_devops

Would you be interested in an online Terraform file editor and visualizer?

I've been building out a prototype of a Terraform file editor and wanted to gauge interest in the features below. Is this a product that would interest you and help you in your day to day life, if you work with Terraform files?

Key features:

* **Editor Interface**: A code editor embedded in the browser.
* **Syntax Highlighting and Autocompletion**: Supports custom syntax highlighting and autocompletion.
* **Version Control Integration**: Allow users to push and pull their `.tf` configurations to and from Git repositories.
* **Docker and Kubernetes integration**: For testing live updates of Terraform changes before applying to production.
* **Collaborative Editing**: Implement operational transformation or CRDTs to handle concurrent edits if real-time collaboration is a requirement.

https://redd.it/1crryl3
@r_devops

Help with certificates, dns, secrets, tokens.

Hello all.

I started working for a DevOps consultancy. I have no IT experience whatsoever. They were kind enough to offer me a student job. I learned and passed the CKA exam, some VMware stuff, am currently learning Terraform. However, I have a feeling that I am missing a lot of underlying knowledge. A lot of courses are mentioning certificates, dns, secrets, tokens etc. I know of a very high level what these are, but I couldn't find a course that deals mainly with these topics.

Any course you can recommend that deals with dns, certificates etc?

Thank you so much

https://redd.it/1crtey3
@r_devops

Another guide to Istio Authorization Policies and Request Authentication, but combined with IAM automation

When speaking to folks who have deployed Istio in production, I'm always surprised that only a few utilize anything more than mTLS. Sometimes they're not even aware that if they don't change the namespace defaults, they end up with the default service account attached to every pod, which means a single certificate is used for workload authentication—kind of defeating the purpose!

Anyway, this is my attempt at demystifying Authorization Policies, Request Authentication, and OIDC/JWT user authentication workflows. Additionally, what if you could automatically generate Authorization Policies by letting a Network Mapper analyze your actual application traffic and pull metrics from Envoy directly? This is a very cool open-source project. Check out the details in the guide:

https://otterize.com/blog/Istio-authz-and-ingress-authn

https://redd.it/1cruw5h
@r_devops

Looking for a problem to solve

Hey everyone

Intro - I’m just some guy working in tech (like you!).

I’m pretty bored and uninspired at work atm and am looking for a project idea I can build and work on. I’d really like to make something people find useful in some way. Who knows - maybe I can turn it into a little business some day!

My question to you all is: are you facing any problems during your day-to-day work that you can’t find a solution for? Or is there a piece of software you use that you find to be particularly shit?

If so, I’d love to hear about it.

https://redd.it/1crw54l
@r_devops

Ingress depending on ssl/non-ssl traffic?

Please help my smooth brain. I'm helping a dev bring an app to kubernetes. Ive setup running 2 deployments, one with deployment that listens for SSL traffic and one that listens for non SSL traffic for legacy stuff.

How I want it to work:

Request comes to https://app.nonprod.com/ -> hits ingress -> goes to http service -> http deployment/rs
Request come to https://app.nonprod.com/ -> hits ingress -> goes to https service -> https deployment/rs

This is a generic version of what I have:

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-nonprod-https-ingress
namespace: app
spec:
ingressClassName: "nginx"
tls:
- hosts:
- app.nonprod.com
secretName: app-tls
rules:
- host: app.nonprod.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: app-nonprod-https-svc
port:
number: 443

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-nonprod-http-ingress
namespace: app
spec:
ingressClassName: "nginx"
rules:
- host: app.nonprod.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: app-nonprod-http-svc
port:
number: 80

But this isnt going to work because the path are both /. Is there a simple way around this?

https://redd.it/1crvqb1
@r_devops

Azure devops pipelines for Ansible

Hello everybody. I'm currently working with pipelines for deploying infrastructure with terraform but now I'm facing the challenge to configure this infrastructure and want to use ansible, the thing is we are thinking about going with pipelines to run the playbooks but there is a team that want to use awx to manage all. I know little about awx but know you can manage the hosts, credentials and many more.

I don't know if it would be worth to install awx and if we would gain much more than using pipelines for the playbooks. If anyone could enlighten me, Thanks.

https://redd.it/1crykhj
@r_devops

Am I on a good path?

Guys I'm currently going to WGU for the software development program. While doing this program I am also learning Python on the side with a course on Udemy. I have some experience with programming but I am not super confident in my skill set with programming. The job that it would like to make it to is becoming a cloud engineer or dev ops engineer. I have been working within an I.T help desk role for 3 years and I really want to specialize. I feel that I've learned all I can with helpdesk. Am I on a good path to becoming a dev ops engineer?

https://redd.it/1cs0pnp
@r_devops

Docker Nginx Proxy with high availability and security

* v5
* With your project and its sole Dockerfile, Docker-Blue-Green-Runner manages the remainder of the Continuous Deployment (CD) process with [wait-for-it](https://github.com/vishnubob/wait-for-it), [consul-template](https://github.com/hashicorp/consul-template) and [Nginx](https://github.com/nginx/nginx).
* [https://github.com/Andrew-Kang-G/docker-blue-green-runner](https://github.com/Andrew-Kang-G/docker-blue-green-runner)

https://redd.it/1crv738
@r_devops

Oracle goes to OpenTofu

Besides the fact that it's oracle and they are hell, this is a big shift.

https://www.thestack.technology/oracle-dumps-terraform-for-opentofu/

https://redd.it/1cs7fbe
@r_devops

Aggregating docker logs and metrics

I'm currently working in a new job that is trying to kick off their observability efforts. Currently, not much to say about their stack. Application-level metrics and logs are both emitted, somewhat bizarrely, to stdout and stderr. There is no orchestration, just a bunch of docker containers and their associated compose files.

Recently, the ask came down to investigate a way to collect these application-level logs and metrics and ship them to influx and a kafka-esque sink. Additionally, container-level host metrics (i.e. docker stats) should also be shipped to that influx sink.

I'm a bit unfamiliar with the space, but I vectored in on Telegraf and Vector. I'd prefer to get away with just using one of them, but here's where I ended up:

Vector: Able to easily collect stdout/stderr from running docker containers, but non-trivial to collect docker metrics and sink those.
Telegraf: Able to easily collect docker metrics, but can't seem to figure out collecting stdout/stderr from running containers. It's purpose built as a metrics collector, so I guess this makes sense.

Can I get some feedback on this strategy? Seems a bit convoluted.

https://redd.it/1cs6pyk
@r_devops

I need help understanding Accept and Content-Type

Currently studying my Devnet associate and these 2 phrases are by far the most challenging about the entire exam.

I've honestly spent like around 5 hours trying to understand these 2 since studying. I do practice exams and get like 80%, but fail 3-4 questions because how these 2 are used and why is beyond me.

I got about 20 prompts deep into chat GPT really explaining it in the most baby terms possible, only to fail the questions again. At this point I'm thinking i just use the opposite of what i think I should use.

Example question: curl -i -k -u 'admin:admin' -X "GET" "https://RouterA:443:restconf/data/ietf-interfaces:interfaces/interface/gigabitethernet1/ietf-ip:ipv4/address"

What would be used here?

My logic is, okay we're asking for details about an interface on a router with a GET command. I, the client am requesting this from the server, so i will declare the preferred encoding type i prefer in this situation, XML. So i will say Accept: application/yang-data+xml. As in, I want to accept this data in this preferred format. WRONG.

I find i retain info best when making rules/ultimatums or jingles in my head. Some examples like:

NTP 123 (rhymes),
SSH is port 22 (SS and 22 are similar),
telnet is 23 (T E in telnet, 2 starts with a T and 3 looks like an E. TE = telnet),
I remember the requests api format by MUHD(and think of wet dirt in my head) method,url,headers,data. You get the picture.

Thanks in advanced for any commenters

https://redd.it/1csb4j2
@r_devops

What should my short term goals be if I want to transition into DevOps as a SOC analyst

Hey guys, I'm currently a SOC analyst that's worked in various level 1 roles (Help desk, NOC and now a SOC) over the past 5 years. I currently do not have any certifications, but mostly I'm realizing that I find Linux, the Cloud, containerization and development to be very interesting, so I'm currently exploring the idea of getting into DevOps.

I want to start off by saying that I know a little about a lot, the biggest hindrance I think are my programming skills which are very poor since I've only taken two classes in community college on Python (one I had to retake) and I haven't really used it much on the job. I'm trying to change that, so I started developing a script at work to automate some basic tasks. This is just a starting point and by no means is intended to be advanced.

Here are some of the ideas I had on what I could focus on in maybe the next year:

- Level up my Python skills so I'm much more comfortable creating scripts, and learn git so I have a good understanding of source control.
- Achieve my RHCSA (This is what I'm focusing on right now) so I have a good fundamental understanding of Linux and get a proper introduction to containerization. This should open up doors to more advanced projects down the line and achieving a CKA cert.

- Gain either the AZ-900 and start working towards the AZ-104, or achieve the AWS CCP and start working towards more advanced AWS certs from there.

- As I gain more knowledge in Linux, containerization, development and cloud services, I can start to integrate my learning into projects. I think the first one I'll try is containerizing a NGINX server.

- I use arch now btw

Any feedback on my year plan? And what should some of my long term goals be? I think the most difficult but likely beneficial would be achieiving either a Computer Science or Software Engineering degree. This ontop of my background, projects, and certs I hope to achieve should give me a good baseline and I can start looking to advance further down the path into DevOps.

https://redd.it/1csbz7t
@r_devops

Prometheus exporter for supervisord ?

Is there any exporter for supervisord so I can track processes and configure alerts ?

Basically I wanted to set alerts whenever a process restarts !!

https://redd.it/1csazvc
@r_devops

Implementing OpenTelemetry at Skyscanner

Hey everyone!

We had a great talk last week at the London Observability Engineering Meetup that I thought many of you would find interesting.

Dan shared some of his experiences leading an observability transformation at Skyscanner, from custom solutions to Otel standards.

You can find the talk here: https://youtu.be/bxNc45TMTsU?si=g\_68LLIKoL\_Vxt1w

There was some great questions during the Q&A so make sure to check those out:
28:18 Q&A | How to define and set up SLOs & SLAs
31:19 Q&A | Tail sampling best practices
33:52 Q&A | How to handle technical/organizational silos when implementing observability
35:16 Q&A | How effectively work with metric views in OpenTelemtry
43:38 Q&A | Standardizing semantic conventions with OpenTelemtry

Btw, if you have questions, you can drop them in the community Slack, and I'll try and get Dan and others to help out.

https://redd.it/1csgp6c
@r_devops

Questions about DevOps Cloud freelancing as a newcomer

I'm finishing a Cybersecurity, and I'll start an internship in a few weeks.

During the course, we saw a lot of different stuff such hardening, forensics, etc, being Pentesting and DevOps my favourite things, as I have professional experience as a web developer (both front and backend).

Initially, I was partial to Pentesting/Bug Bounty, as working from home in my own hours is my focus. But in the last week, I had to use Google Cloud and I liked it, and now I'm thinking about the possibility of working as a DevOps cloud engineer as a freelance.

So, my question is: If I like Pentesting, I can take the Bug Bounty route, working from home, on my own hours. There is something like that but for DevOps cloud engineer, where I can take jobs that will take me hours/days, or the classic 9-to-5 is the only way?

https://redd.it/1csi5ib
@r_devops

DevOps Engineer - What is the best way forward?

I come from the world of Sharepoint and .Net development, and I have studied and worked with devops.
I already have the AZ-900 certification and I thought about learning something about DevOps Engineer.

In my work I helped with processes with layout, states and rules, as well as permissions for users. In addition to creating integrations with other systems using an API with C#. Does this fall under devops engineer?

Analyzing the certification options I saw that there is Microsoft Certified: DevOps Engineer Expert - AZ-400 and it seems to make sense, but I don't know if what I did (manage processes and access, in addition to API consumption) fits here. There seems to be more, I don't know.

Looking at this AZ-400 exam, I see that there are 2 certifications indicated as prerequisites, with only 1 required to obtain this title of DevOps Engineer Expert.

Does it matter if you take the AZ-104 (Azure Administrator Associate) or the AZ-204 (Azure Developer Associate)? Does it really depend on your profile?

One last question would be about the Azure Devops Service that I researched and found layout things in addition to using the API. Is this while in the Devops Engineer or something else?

Thanks

https://redd.it/1csndzo
@r_devops