How to benchmark filesystem with multiple small files?

I'm currently experimenting with Longhorn on a k8s cluster as a storage provider, and now I need to measure the performance of Longhorn vs native k8s storage. My test cases include:

* Performance using a lot, but small files (let's say 100k files with 10k-15k size)
* One big file
* Latency

​

I'm using fio, but I want at least another tool to validate the test case with the small files. If there is no solution (that I'm highly in doubt), I want always create them and transfer them from one directory to another, but that would be my last resort.

https://redd.it/135nsek
@r_devops

Recommendations for Senior DevOps Lead in LA

Hello Reddit community,

I recently relocated to LA from the UK and am actively searching for new job opportunities. I have 16 years of experience in Azure and Kubernetes and am currently applying through LinkedIn, Dice, and Indeed. However, I would appreciate any recommendations or tips on where else to look for jobs in the LA area.

Additionally, I am curious about the current market rate for W2 vs. C2C positions for a senior DevOps lead in LA. I have been receiving calls, but the rates seem lower than what I was expecting. Any advice on how to negotiate for a better rate would also be greatly appreciated.

Thank you for your help!

https://redd.it/1360icd
@r_devops

How do you select good jobs as devops engineer?

Hello there,

I have a good opportunity as devops engineer but sometimes it looks that the job description looks more like operations and maintainance. What do you look for when you seek new opportunity in devops engineering path?


Thanks in advance!

https://redd.it/135kbql
@r_devops

Kubernetes and feeling defeated

I recently started experimenting with Kubernetes (k3s in my case) by first hosting a simple Wordpress instance locally, then migrating it to Docker services (Nginx local upstream to application and a separate DB instance).
Now I wanted to go full k3s with two servers and the connection between master and worker and all these shenanigans was a piece of cake.


Now I‘m at a point where I can‘t even make the Wordpress instance publically available because I can‘t wrap my head around the Ingress/Loadbalancer config.
My loadbalancer service „Public-IP“ stays in status „pending“.

What in the hell am I doing wrong? Is it a weird k3s specific quirk? Would it have been better to use minikube instead of k3s?


I feel stupid for getting my ass kicked by something seemingly so simple :(

https://redd.it/1362sgb
@r_devops

Azure zero downtime deployments with Terraform

I was just wondering if anyone has any strategies for zero downtime production deployments with Terraform.

Normally I would use the lifecycle hook “create before destroy” which spins up a new resource, moves any dependencies to that new resource, and then destroys the old resource. In Azure basically everything needs a unique name so the new resource and old resource cause a naming collision.

Any help would be appreciated.

https://redd.it/1363h3o
@r_devops

Converting from Pulumi to Terraform (rewriting all deployment code)

At my company we have a plan currently to rewrite all the frontend and backend deployment code from Pulumi to Terraform and we don't really have dev-ops people per se so I wanted to know what a specialist or someone with more experience thinks about this.

What spurred this on is we realized that we will need to switch to a paid pulumi plan for the cloud resource management (we are a startup and started on a free account). The issue is that we use a microservice structure with lots of resources. We worked out the monthly cost for 40,000+ active resources monthly and it is over $10,000 a month. Terraform cloud is $20 a month per person using the account and the only real advantage of sticking with pulumi seems to be the CDK since nobody knows HCL at my company. So we would be spending $120,000 a year just to avoid rewriting some deployment code.

Does this seems like a sane reason to rewrite our deployment code? Am I missing something with the pricing structure? The difference seems insane. Currently the deployment code and application code for the backend is sort of mixed together so it will take some work to untangle the files / dependencies, it would be nice if TF CDK was past 1.0 but we probably shouldn't use something in beta for production.

Also, I am relatively new and started at this company as a dev but have mostly worked on test automation, set up a selenium-grid with AWS fargate for browser testing. Have worked on some frontend deployment code and CI stuff. If I am able to complete this project would that give me a skillset that would allow me to apply for dev-ops jobs? I'm kind of concerned how little I've worked on the frontend/backend directly but if knowing AWS/CI/Terraform/Pulumi/Typescript and a little Docker is enough to apply for dev-ops roles in the future, I'll feel a little more secure. I also have some basic networking knowledge and linux just from personal experience on my home network. Really basic Docker stuff like setting up a media server with sonarr/radarr/jellyfin/jackett/qBittorrent/nzbGet/etc. I've only been in the industry a little over a year so I worry about my experience and ability to find a second job if I need to. We use typescript for the code currently which I am fairly proficient in.

https://redd.it/1363bux
@r_devops

Devops Days Conference?

I'm possibly interested in attending the DevOps Days conference in Austin, TX coming up this weekend. Coming from out of town, on the west coast?


Wondering if it is worth to attend, for the costs of travel and accommodation? About how many people attend the conference and what topics are covered? Is it a good opportunity for networking and also learning sessions?

https://redd.it/13632g5
@r_devops

Feeling hopeless in my pursuit of a career in DevOps/cloud engineering

I'm feeling down and out right now. Despite my passion for DevOps and trying to learn the necessary skills, I'm struggling to break into the industry. I've got a business background and no prior IT experience, but I took a chance on a 6-month DevOps bootcamp and poured into learning Linux, Scripting (Python/Shell), Networking, Terraform, etc. and earning three AWS certifications.

But despite all of this, I can't seem to get a single callback or even a response to my job applications. It's really discouraging, and I'm starting to feel like all of my efforts were for nothing. I know the job market is tough right now, but it's so frustrating to see my dreams slipping away despite the work I'm putting in.

If anyone out there has any advice or words of encouragement, I would really appreciate it. I'm trying to stay positive, but it's tough when it feels like the odds are against me. I know I'm fortunate to have a job right now, but it's hard not to feel discouraged when my goals feel out of reach.

https://redd.it/1365rp9
@r_devops

Any advice/tips will be appreciated

Im just here seeing if anyone has any tips or advice for me. I’m 20 turning 21 in September. I’m a comp sci major, learning c++ and binary. I’m trying to pursue a role in devops, Im learning Linux, bash scripting, starting to move into cloud now. Will also be doing Git, docker, ansible, terraform, kubernetes, Jenkins, maven, and gradle. Thoughts?

https://redd.it/136b01o
@r_devops

Kubiya Generative AI for DevOps

Hi all, I recently reviewed Kubiya a new tool that supports DevOps with the power of AI, I thought it may be interesting:
https://youtu.be/6GHbrQa9ANk

https://redd.it/136c70j
@r_devops

Integration of IBM WebSphere with Jenkins

Hello Everyone,
And I am working as Devops Trainee we have been assigned a task in which we have to deploy .war in IBM WebSphere thought pipeline script and I am unable to integrate Jenkins with WebSphere.I need help in integration and what steps we have to write in pipeline script.Our IBM WebSphere is installed in Windows Platform (localhost)

https://redd.it/136am41
@r_devops

Build Static Application Files Into Container or Download at Runtime?

Not sure what container building best practices are, I've always just been a consumer in the past.

Our current docker container qa build process for an asp.net application is that all of the app files are downloaded from a NAS in the ENTRYPOINT powershell script. There is a RUN powershell script that preps IIS but that's all. Everything else is done during ENTRYPOINT . The version grabbed from the NAS is set by environment variable and the container image is versioned separately from the application. All application files are stored in a volume and overwritten by another pull when the environment variable is changed.

In the past I've always used containers that are versioned the same the application and volumes for just the state/userdata.

Is there benefits to downloading files at runtime as opposed to building them into the image? I suppose it keeps the image smaller.

https://redd.it/1367hmj
@r_devops

How do i keep track for each of the step on the entire process

Basically i'm offering a service that would transform an asset (video, image, and audio). The entire process consists of a user uploading an asset to s3, this would then trigger a lambda function then trigger another lambda function then trigger another lambda function. I would like to keep track of what step is the user currently in and display it into their mobile screen real-time. Any leads on what should i look into?

​

https://imgur.com/a/51Zon7G

https://redd.it/136fkn8
@r_devops

1000 100% OFF coupons: DevOps course for small companies and individuals

Hello everyone,

I've made a DevOps course covering a lot of different technologies and applications, aimed at startups, small companies and individuals who want to self-host their infrastructure.
To get this out of the way - this course doesn't cover Kubernetes or similar - I'm of the opinion that for startups, small companies, and especially individuals, you probably don't need Kubernetes. Unless you have a whole DevOps team, it usually brings more problems than benefits, and unnecessary infrastructure bills buried a lot of startups before they got anywhere.

As for prerequisites, you can't be a complete beginner in the world of computers. If you've never even heard of Docker, if you don't know at least something about DNS, or if you don't have any experience with Linux, this course is probably not for you. That being said, I do explain the basics too, but probably not in enough detail for a complete beginner.

Here's a 100% OFF coupon if you want to check it out:

https://www.udemy.com/course/real-world-devops-project-from-start-to-finish/?couponCode=FREEDEVOPS2305KOQYV

Be sure to BUY the course for $0, and not sign up for Udemy's subscription plan. The Subscription plan is selected by default, but you want the BUY checkbox. If you see a price other than $0, chances are that all coupons have been used already.
You can try manually entering the coupon code because Udemy sometimes messes with the link.

The accompanying files for the course are at https://github.com/predmijat/realworlddevopscourse

I encourage you to watch "free preview" videos to get the sense of what will be covered, but here's the gist:

The goal of the course is to create an easily deployable and reproducible server which will have "everything" a startup or a small company will need - VPN, mail, Git, CI/CD, messaging, hosting websites and services, sharing files, calendar, etc. It can also be useful to individuals who want to self-host all of those - I ditched Google 99.9% and other than that being a good feeling, I'm not worried that some AI bug will lock my account with no one to talk to about resolving the issue.

Considering that it covers a wide variety of topics, it doesn't go in depth in any of those. Think of it as going down a highway towards the end destination, but on the way there I show you all the junctions where I think it's useful to do more research on the subject.

We'll deploy services inside Docker and LXC (Linux Containers). Those will include a mail server (iRedMail), Zulip (Slack and Microsoft Teams alternative), GitLab (with GitLab Runner and CI/CD), Nextcloud (file sharing, calendar, contacts, etc.), checkmk (monitoring solution), Pi-hole (ad blocking on DNS level), Traefik with Docker and file providers (a single HTTP/S entry point with automatic routing and TLS certificates).

We'll set up WireGuard, a modern and fast VPN solution for secure access to VPS' internal network, and I'll also show you how to get a wildcard TLS certificate with certbot and DNS provider.

To wrap it all up, we'll write a simple Python application that will compare a list of the desired backups with the list of finished backups, and send a result to a Zulip stream. We'll write the application, do a 'git push' to GitLab which will trigger a CI/CD pipeline that will build a Docker image, push it to a private registry, and then, with the help of the GitLab runner, run it on the VPS and post a result to a Zulip stream with a webhook.

When done, you'll be equipped to add additional services suited for your needs.

If this doesn't appeal to you, please leave the coupon for the next guy :)

I hope that you'll find it useful!


Happy learning,
Predrag

https://redd.it/136gc7d
@r_devops

Flyte - is it bad?

Just wondering if anybody has experience with Flyte and/or thoughts.

I've used other orchestration platforms before like Prefect and had a much easier time setting that up.

Flyte just feels unwieldy and auth is a clusterfuck. Also docs don't seem to have been updated for over half a year which is worrying. But maybe I'm just dumb.

Also, Lyft might be close to bankruptcy and I'm not sure if using their open source library is the right way forward as a result of that. Especially considering the company wants to offer this platform as a service to clients.

I've tried offering alternatives technologies to higher ups and they want Flyte. Obviously I should just deliver the deployment but I can just foresee how fucked I am in terms of maintaining this thing.

https://redd.it/136gncf
@r_devops

Transition to devops

Hello everyone,

I’m currently trying to transition from a data technician role to devops. I have 6 years experience as a telecommunications tech and 4 as a data tech. Currently my main responsibilities have been managing a site with 12 NVR’s (Dell servers). Have experience troubleshooting all types of networking problems. Also have experience with VM’s

I’m currently learning python, Linux as well as NGINX, AWS and plan to learn Docker, Kubernetes, Ansible, Jenkins,

Currently I have a A+ and AWS SAA cert and working on my AWS Sysops and Developer certs. I also took a couple courses on how backend Networking and engineering works when it comes to web apps.

The plan is to create a portfolio of projects using different technologies to showcase hands on experience and skills.

just want to know if my current track and experience is good enough to get a shot at a devops role.

https://redd.it/136is5f
@r_devops

I meshed Microsofts Bing AI technology into Instagram with Scripts

{ Sophie_Eloise.ai } Is the user name on Instagram

https://redd.it/136jnzs
@r_devops

fair price for up work contract

greetings

a client of mine asked me to set my price for the following task, I am new to up work and still trying to build up my profile, what do you think is the fair price for this job

We have 5 APIs that are connected. We want to deploy all of these into the k8s cluster in the most automated way possible.

​

Required Skills

Microservice architecture

Container orchestration

Kubernetes

Terraform

Docker

.Net

Linux

GitHub Actions

SQL Server

Cloud infrastructure

CI/CD pipelines

=====

Setup Docker images for all microservice apps

Setup Docker Registry

Setup Docker image for the database

Setting up database backup over FTP

Setup Kubernetes cluster

Setup Load balancers

Run the images using Kubernetes

create the necessary CI/CD pipelines

you will be asked to explain your code files by the end of the contract.

we will use Cloud Computing Services | VMs, Kubernetes, Storage | Linode for hosting

https://redd.it/136ks4m
@r_devops

How to open source code from a private monorepo

Hey r/devops!

I wanted to share an article I wrote about our experience open sourcing code from a private monorepo.

Check it out here!

https://redd.it/136pj21
@r_devops

The Hitchhiking Contributer’s Guide to Onboarding Docs in CI/CD

https://stateful.com/blog/guide-to-onboarding-docs-in-ci

It's all too commonplace in too many projects to obfuscate development tasks & workflows behind a complex Makefile or collect them in an extensive script list inside a package.json. How often are those files your entry point for understanding how to do anything inside a repo? Are you sure they always work?

Projects that initially invested in building out contributing guides eventually become outdated as contributors change workflows without updating the developer documentation accordingly. This drift of docs and code does not just slow us down; it often causes outright frustration and contributor churn.

A CI/CD pipeline is not much different than any code contributor to your software project. It requires some prerequisites to set up and an entry command to run a workflow like testing or deploying the app. What’s neat about pipelines is that they are a gateway for developers to ship code. If the project fails to build or tests fail, the CI/CD system turns red, preventing the code from floating under the radar and silently making it into main.

Wouldn't it be nice if we could teach our automated CI/CD pipelines to harness our developer docs and follow the same exact steps as any new employee or contributor?

You might think this could be an excellent job for chatGPT, but no! Plain vanilla automation to the rescue! We can run snippets of code from within markdown files with Runme anywhere, no AI (yet! 😉).

https://redd.it/136r2xl
@r_devops

A Guide to Privilege Escalation with AWS Identity Center (formerly known as AWS SSO)

https://www.cloudquery.io/blog/aws-priv-esc-identity-center

We did research on AWS's Identity Center (SSO) on paths to privilege escalation, the underlying IAM, and how to secure Identity Center. AWS Identity Center is AWS's recommended method of managing workforce access to AWS accounts and resources in an AWS Organization. (I'm the author).

https://redd.it/136nxx1
@r_devops