Easy K8S Connectivity for Local Utils

This blog post introduces a new operation mode for mirrord, where you don't need to have a target in your k8s cluster. You run a program locally on your machine, but with outgoing network connectivity as if they're running in your k8s cluster. This makes mirrord nicer to use with administrative and diagnostic tools. Do you think it can be useful for devops professionals? Does anyone here already use it that way?


https://metalbear.co/blog/easy-k8s-connectivity-for-local-utils/

https://redd.it/142al9z
@r_devops

Big Tech Digest #1: Exploring HTTP3, What is operational resilience, Improving performance with HTTP Streaming, Go Developer Survey Results, Building and deploying MySQL Raft and more!

https://bigtechdigest.substack.com/p/big-tech-digest-1

https://redd.it/1429f2i
@r_devops

Going into DevOps for the gaming industry

I've been working as a devops engineer for about two years now, and I'm working on developing specialties for a better job. I wanna explore the world of DevOps for the gaming industry but it sort of feels like an ethereal topic. Any advice on tools, pages, mindset and route to approach it? Any experienced comments or advices are welcomed!

https://redd.it/142eb1x
@r_devops

help - cloud based obsolete LAMP enviroment

Hi, hope not too newbie to be posted
got a php mysql site to be temporarly moved from the actual host (not my server) to an host handled by myself, until we re-develope a new version (1/2months)
Unfortunately this php site run over a php 5.3 and crash and burn when php8 or 7 is turned on.
I've personally tried upgrade the code, but... has been made completely from scratch from a self-thought dev with an huge amount of calls to unsupported functions.


I was thinking to set up on AWS a LAMP stack that suits those needs php 5.3 on a EC2 machine but, trying to create that stack on modern distro isn't straight forward as I thought.


Anyone could suggest me the low brainer solution to keep this site live at least for a couple of months without making a complete debug of the SW.


thanks

https://redd.it/142f0ar
@r_devops

Jaeger alternatives?

I’m getting tracing setup from OpenTelemetry and Jaeger is proving to be a little difficult. My backend is Elasticsearch 8 and after a lot of trouble it looks like Jaeger just doesn't support ES 8 at the moment. Not sure what the plans are here.

Anybody using any alternative for tracing?

https://redd.it/142hmde
@r_devops

I'm a Principal Architect (Cloud/Automation) at a large consulting firm, and I want out. Looking for advice on where to go where I can make a difference, instead of just making someone money. More details inside.

So, quick bullet points of my story for context.

* I'm 29
* I'm the Principal Architect for a region, for a large services provider.
* I'm pre-sales only, and specialize in cloud architecture (Azure and AWS both) and infra automation (Terraform, Ansible, Puppet, vRA/vRO, Nomad, Vault, Chef, etc)
* I'm very good at what I do - I am brought in for our most complex deals and highest value opps.
* I make about $300k/yr, work maybe 20~ hours a week most weeks, and work fully remotely, but I hate my job and this industry. I love the tech, and will happily chat about any of these technologies nonstop, and am genuinely passionate about them, but I hate my job and what the industry is doing to my mental health.
* I ran IT for a Dental Lab, then was an architect for a telecom, then was at two other services providers before this one.
* I'm gay, a huge theatre guy, a gamer, and do a ton of improv comedy. (relevant, I swear)

I just got back from The Improv Retreat, learning from some of the best improvisers in the world, and it
really hammered home how miserable I am in my current job. Tons of organizational political drama, restructuring, no review processes, bad leadership, etc. which is standard for these types of organizations. Beyond that though, I'm just so tired of being ground down every day by incompetent leadership and account reps who will wring you dry to make a sale. I want to do something with my skills that will help people, and this isn't it.

I'd love to do something with social justice and LGBTQ+ advocacy, and combine my IT skills with that, or use those skills with my love for theatre and comedy, or gaming. I just don't know what all is out there and where I can really go. The hard part of just "making a jump" is that I have my fiance and dogs to take care of; I make $300k/yr and he makes $70k/yr, and we can't live solely off his income. I could drop down to about $150k/yr and still be okay, but even that is *very* high when we look at pay in any of the industries I just mentioned.

Overall, I just want to bring joy and help people, and I can't do that where I'm at. I'm miserable and I've never been more depressed in my entire life.

Please don't get me wrong, I am fully aware that I am extremely lucky and privileged to be 29 years old making $300k/yr in a job where I don't really work much, but I can't help how I feel. I feel like I'm draining my soul in exchange for money. I'm just so lost. I'd love any advice y'all can provide, or any connections you might have that may have opportunities or advice for me.

Thank you.

https://redd.it/142iyj2
@r_devops

Sizing Thanos for large Prometheus installation

Hi guys, I am hoping someone who has built out a large scale Prometheus/Thanos setup can chime in here.


Currently we are running a set of fairly large sharded Prometheus clusters with each shard having 2 Prom instances for HA and use Promxy to aggregate the metrics.
Current Setup: 4 VPCs of various sizes

VPC1: 16 Prom shards producing 11 million metrics per second
VPC2: 8 Prom shards producing 5 million metrics per second
VPC3: 2 Prom shards producing 1 million metrics per second
VPC4: 2 Prom shards producing 2 million metrics per second

Initially I was looking into Mimir and Thanos for options, but with our scale Mimir setup appears to be too expensive as the ingester will need a crazy amount of resources to support all of this metrics.

Thanos seems like a better choice as the sidecar on each Prometheus shard will take care of writing the metrics to the object store.


There are 2 things I am not exactly clear on with Thanos setup and hope to get some clarity on.


1. From my understanding the Query and store gateway do not need to be sized to the number of metrics we produce but instead to the expected number of metrics we will be querying (If we only use 15% of the logged metrics in Grafana for example)

2. The only Thanos component that will need to be sided to the number of metrics generated is the Compactor. I have not been able to find any guides on sizing the Compactor (Mimir provides really good documentation on how to size their components based on the number of metrics)


If anyone has experience with this sort scale I would really appreciate to hear your experience on running long term storage for large Prometheus environments.

https://redd.it/142ku8i
@r_devops

Anybody running workload on OVH?

I know, I know, OVH doesn't have the best reputation. But every time they are mentionned, it turns in an immediate circle jerk of people with no first hand experience with them making jokes about datacenters on fire. Can we please skip that and talk this time?

This is a question for people with actual workload at OVH, on public cloud or dedicated servers. I'm looking at cheaper alternatives to AWS and GCP, and of all the underdogs (Digital Ocean, Linode, Vultr, OVH), OVH looks like the most interesting to me.

So, what's your take? How big is your workload? What kind of issues do you have?

https://redd.it/142m3hh
@r_devops

Simple Middleware for (3) REST API Softwares

I like to figure things out on my own, but am lost where to get started with this one, so I came to Reddit. I have (3) cloud based SaaS products with REST API's, and I'd like to get them to communicate to each other. Is there a simple/cheap middleware you can suggest for this?

https://redd.it/142n0bl
@r_devops

Providing AWS MSK Kafka Access To External Services

Faced a lot of challenges while trying to push Rudderstack events to Kafka destination during missing doc and inexperience with Kafka. Tried to document the process for setting up the public access to MSK.

https://nikhilsoni.me/2023/06/06/providing-aws-msk-kafka-access-to-external-services/

https://redd.it/142qfc0
@r_devops

What is the efficient Deployment Strategie you prefer?

Deployment Strategies
1.Rollout deployment
2.Canary Deployment
3.Blue green Deployment
4.Shadow deployment
5.Recreate Deployment
6.A/B testing
7.Big Bang
8.Feature toggle

I prefer Blue Green strategy
1.Efficient deployment and testing without disrupting live prod environment,
2.easy rollback
3.no maintenence and downtime
Once the green env passes tests, using load balancer switches traffic to green making it as live (blue)

https://redd.it/142r7t8
@r_devops

Um protesto sobre política de senhas

Recebi hoje a senha de um sistema importante aqui dizendo em letras garrafais:

​

>"Não grave essa senha em bloco de notas, nem em nenhum lugar, não anote, decore, sua senha é sua credencial na empresa"

A senha: zpWzqLL,_![

🙄🙄🙄

https://redd.it/142tq7t
@r_devops

What's your long term plan?

I'm in my 2nd year of platform/devops work, coming from 10 years of sysad roles. I'm curious about people's long term career plans. I'd like to hear what people plan to do and how you plan to get there.


\- Not in tech any more at all?


\- Management?


\- Architect/Principal Engineer?


\- Consulting?


\- Sales or pre sales engineer?


\- Self employed?

https://redd.it/142sgwu
@r_devops

When using helm charts, is there a way to write it down instead of just running the commands?

Running "helm chart ..." several times, doesnt really make sense so is there a way to make sure you don't have to run 10 commands if you use 10 helm charts for a project? Could you show an example?

https://redd.it/142vmi0
@r_devops

Confused with Grafana/Loki logs

I am writing a script to migrate our logs from Elasticsearch to Loki.

It seems to be working fine so far, it gets the logs from ES based on date and type, puts together the labels and json and sends them to Loki. Loki is set as a data source in grafana and if I go to the live stream I can see the incoming logs in real time.

The problem I have and where I need some help is because I can't query those logs in grafana using any of the labels I set up. I was thinking maybe the labels are not sent correctly but they don't show on the live stream.

This is my first time using grafana and loki, so I'm not sure where to go from here, any help would be greatly appreciated.

https://redd.it/142xase
@r_devops

pull request merging and sync runs Pipeline on source branch

On merging from development to staging branch as an example.

After submitting merge request, I have to run sync before approving and merging. Is it the right behaviour that it runs the pipeline as staging to development and then development deployment?

I'm following previous employee routine. Can I skip sync on merge? What is the proper way to do this?

https://redd.it/142ygkx
@r_devops

Where to start?

I just got an Associate SE degree and can get a Bachelor in 1.5y. did a 15 week SE internship at a startup where i built DB models and did backend dev with PHP but its not my thing. I wanna work more with Linux, open source, CLI tools, Automation, monitoring, security, .. I don't like making web applications and i'd like to have an impact on Developers/Org instead of Customers.


I feel like i'm not very good in one thing but rather decent in many things, I learned:

\- building webapp's backend / frontend with java
\-cleaning and reading data in Python (passed a pretty tough Data Science class)
\- linux / command line (learning nvim and going from windows to Fedora soon + did a Kali Linux class)


I have the option of working and not continueing school since i get some job offers like Data Engineering and SE. But i want to get a Bachelor i think and get better at the things i mentioned!



this summer (before i have to make that choice) I have a lot of free time to learn the topics i mentioned in the first paragraph. Any advice on where to start except for switching to Linux? What kind of thing do i start bulding in Go? I don't wanna keep watching videos forever.

https://redd.it/1430thq
@r_devops

What problems do you face while using multi cloud?

Hi all, I was curious to know what problems you face when you want to use services from different cloud providers. Please share your experience.

https://redd.it/1431rux
@r_devops

How would you deploy and update this Next.js app in AWS?

Hello guys. I am once again at your feet humbly asking for your sage advice.

I'm trying to create a deployment and CI/CD pipeline for this Chatbot UI project and I'm having a little bit of option fatigue after seeing all the choices. I'm using Bitbucket pipelines and my plan was to take the output from the build, upload it S3 (after adding the setup to host static websites), and invalidating the Cloudfront distribution to take the new changes.

After looking for similar pipelines I'm seeing everyone doing something completely different. Some people use AWS Amplify, some AWS Elastic Beanstalk, AWS ECS, and others even deploy directly on an EC2 instance.

How woud you guys recommend I do this?

https://redd.it/142wxwj
@r_devops

Versioning advice for a project that consists of multiple projects.

I am working on a small hobby project and we are getting around to do a public release soon, but one thing I was wanting to get around to was solidifying the versioning of the project.

For a background: the project consists of two sub-projects that perform different tasks: a Data Generator and a Data Interpreter. How it works is the Data Generator is hosted on a website and generates a data file, which can be downloaded and read by the Data Interpreter, which is a standalone application.

These projects are in separate Github repos and operate independently of each other. When combined, they make up the Main Project. An example of this in action is that Version 1.2 of the Data Generator and Version 1.5 of the Data Interpreter would combine to make Version 1.7 of the main project (these are just filler version numbers, but you get the point). The user doesn't need to know what version the Interpreter or Generator are on, only that the project is in Version 1.7

My big issue right now is how to establish the versioning in terms of marketing and future-proofing so that it doesn't get too complicated (or is at least maintainable). The issues I currently see coming up are some of the following:

* The Interpreter is a standalone application that has to be downloaded and ran locally to be used, whereas the Generator is hosted on a website, so getting an update pushed out to the user is much easier via the Generator than the Interpreter.
* The Interpreter cannot check for updates as it has no network connectivity, so the only way to get a new version of it is to manually download it from the website.
* If you increase the version of one of the sub-projects, then you have to increase the version of the entire project, which can be difficult to display to the user unless they re-download a new file from both projects.

We can probably merge the projects together into a single project where the Generator bundles the data file it produces with the Interpreter and provides a single download, however, the Interpreter currently is able to read in multiple Generator data files and we would like to retain that functionality if possible. If we bundle them together, then we lose that functionality.

​

I apologize if I missed anything and can clarify things as needed :)

https://redd.it/142svdr
@r_devops

What are some of the common technical debts in the DevOps profile?

Just wondering what are the Day to Day Technical Debts that a DevOps Profile handles or must know ?!

https://redd.it/14354kn
@r_devops