Azure storage account with containers

I have some troubles with creating storage account with containers with terraform.

Well, it kinda works, but not 100%. My storage account module creates storage account and then container in this storage account.

If I create storage account, then add configuration for container and run terraform again - it works just fine.

But if I add new storage account and container to input vars, it creates storage account just fine, but fails during creation of container. After it fails, no changes are needed, just run terraform again and containers are created just fine.

Its not issue with resources dependency, because containers are dependent on storage account resource and are created later. Its some kind of permissions missing, but I can't find out why.

Error:

Error: containers.Client#GetProperties: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailure" Message="This request is not authorized to perform this operation.\nRequestId:xxxx-xxxxx-xxxxxxx-xxx-xxx-xx...\nTime:2024-07-16T09:20:58.4531050Z"

It is AuthorizationFailure. However without any changes to permissions, running terraform just after it failed, works properly.


My theory: Is it possible, that terraform gets credentials at the beginning, when there is no such storage account (hence not having access to it), and then tries to use same credentials to create container in that new storage account (again without credentials yet)? Running terraform again would fix it by getting new credentials, with access to that storage account included (because it now exists on init/plan).

https://redd.it/1e4nvwl
@r_devops

DNS Knowledge | Private, Public DNS Zones, Auto-Registration

Hey,

Looking to expand my knowledge on DNS. Here's my journey so far, if anyone could critique where knowledge is wrong!:

So, I've been setting up my first vNet & while doing this I've learnt that you need a private DNS zone if using Azure Managed Postgres, because Azure managed postgres is only accessible via its FQDN so you need to resolve this with a DNS inside the vNet.

I set this up, and by default auto-registration is turned off. I assume since I am using TF to spin this up & down, turning this feature back on should be alright and save time. This should create the postgres server record set, and I don't have to do it manually.

In the Azure Portal, you can see the record sets for the Private DNS, which is cool and makes debugging a lot easier. nslookup command line utility also looks through these record sets.

Public DNS Zones, these are the name servers the internet relies on. These are for public-facing domains such as google.com and you can either leave these hosted on your register or move them to your cloud provider. Are there benefits of moving your public DNS zones to your cloud provider?


I set up my private DNS with two resources:

azurerm_private_dns_zone -> This was to actually create the private DNS ( Not sure why it's called a zone, multiple servers? )

azurerm_private_dns_zone_virtual_network_link -> This was to link the Postgres subnet to the DNS zone. By default the private DNS is not linked to any vNet.

Thanks, this is my knowledge so far! Other than the basic university knowledge of 'DNS makes domain names human friendly by mapping them to IPs'.

Any books/videos/blogs would also be amazing!


https://redd.it/1e4p6oa
@r_devops

NEW UPDATE: OneUptime - Open Source Datadog Alternative.

ABOUT ONEUPTIME: OneUptime (https://github.com/oneuptime/oneuptime) is the open-source alternative to DataDog + StausPage.io + UptimeRobot + Loggly + PagerDuty. It's 100% free and you can self-host it on your VM / server.

OneUptime has Uptime Monitoring, Logs Management, Status Pages, Tracing, On Call Software, Incident Management and more all under one platform.

New Update - Reliability Copilot:

OneUptime Copilot (currently PoC) is a tool that helps you improve your codebase automatically. Copilot can fix following issues automatically by scanning your codebase for issues and sending PR's automatically:

Performance Issues: Improve database queries, optimize code, reduce memory usage, decrease API response time, etc.
Security Issues: Fix security vulnerabilities, prevent SQL injection, XSS, CSRF, etc.
Code Quality Issues: Improve code readability, maintainability, and scalability. Improve comments, naming conventions, refactor code, etc.
Error Handling Issues: Improve error handling, exception handling, logging, etc.
Testing Issues: Improve test coverage, test quality, test performance, etc.
Documentation Issues: Improve documentation quality, comments, README, etc.

You can set this up by using an LLM Model of your choice. Here are the docs to read more:

https://oneuptime.com/docs/copilot/introduction

This integrates in your CI/CD pipeline and no code is sent to us.

Coming Soon (end of August, 2024):

Better Error Tracking Product:

You can track errors through traces, but we're working on a seperate error tracking view (something like Sentry), so you can replace senty.

Log Monitors, Metric Monitors:

We're working on Log / Metric monitors, so you can set up alerts and create incidents for logs, metrics or traces.

OPEN SOURCE COMMITMENT: OneUptime is open source and free under Apache 2 license and always will be.

REQUEST FOR FEEDBACK & FEATURES: This community has been kind to us. Thank you so much for all the feedback you've given us. This has helped make the softrware better. We're looking for more feedback as always. If you do have something in mind, please feel free to comment, talk to us, contribute. All of this goes a long way to make this software better for all of us to use.

https://redd.it/1e4r9uk
@r_devops

5 Advanced Kubernetes Operators Every DevOps Engineer Should Know About

Managing complex, distributed systems with Kubernetes can be challenging. That’s where Kubernetes Operators come in, automating and streamlining cluster management. But what exactly are operators, and why are advanced ones particularly useful?

https://piotrzan.medium.com/5-advanced-kubernetes-operators-every-devops-engineer-should-know-about-ab46bdc1c7d5

https://redd.it/1e4sah7
@r_devops

open source app with kibana, ES, and sqlite - ez deploy to fly

I made this Dockerfile:

https://github.com/andrewarrow/gnr/blob/main/Dockerfile

This fly.toml:

https://github.com/andrewarrow/gnr/blob/main/fly.toml

and this little reverse proxy logic for 5601 (kibana) and (9200) ES:

https://github.com/andrewarrow/gnr/blob/21e4e878ae3425f3792ddba263bfcc2d561a31c9/main.go#L76

My app runs on 8080 and that's all fly needs to know in fly.toml but go will look for request.URL.Path of kibana or esprefix and intercept.

https://g-n-r.fly.dev/kibana/login?next=%2Fkibana%2F

you can login with usename viewer and password testing123

https://github.com/andrewarrow/gnr

https://redd.it/1e4ub9u
@r_devops

Need advice. Under 20, didnt go Uni

Hey, need some advice.

I didnt go University and spammed certs and projects. Got CKA and CKAD, Terraform + 4 Azure certs including Az104 Administrator and Az400 DevOps Engineer Expert. Lots of high level projects using what ive learnt like Jenkins + Docker + AKS + DevSecOps with SonarQube/Trivy, Github Actions + GitOps on EKS with Prometheus and Grafana monitoring , Azure Landing Zone + Terraform and Azure DevOps etc and more.

Struggling to find anything at all, feel like ive done everything i can but cant find anything. Feels like no matter how long or hard i study, no matter how much upskilling i do its not enough. Just came out of school last year how can I have experience.

Currently 30% into studying for the CKS and then going to learn Golang.

Any advice moving forward?

https://redd.it/1e4vg27
@r_devops

Roast My Resume

Hey gang.

Been applying for jobs the past few weeks and I'm not even getting to an interview. In fact, all of my declines to date have been in under 24 hours, the rest just... ghost.

I've been aiming for roles which incorporate AWS and/or k8s. I also live in a tech black hole area and am Canadian so I'm stuck looking for remote work (which is getting thinner by the day).

Sent 51 applications in over the weekend, and as of today have 9 declines. Sigh.

https://imgur.com/a/roast-me-YoyPhND

Each resume and cover letter is customized for each job ad. As a result there can be a variance between them (illustrated in imgur link above; one was for platform engineering another was for finops), however, the general format still applies. As I've been applying for Intermediate and Senior roles, I've been focusing more on goals and outcomes over tech and tools, except for when the job ad focuses on tech and tools. I'd be happy to post most examples; this is just two out of hundreds.

My other problem is it seems like most positions posted are Senior-level. I see very few Junior and Intermediate-level roles. Until this experience, I believed I was Intermediate-to-Senior level, but at this point I'm hungry for success anywhere.

My last role was short... I was part of a round of mass layoffs. In fact, googling that employer still has those layoffs in the first page. Should I explain why the role was short, or will my explaining it validate an attribution bias a hiring person may possess against people with short tenures on their resume?

Also, I've paid for my CKA exam, just haven't taken it yet. I feel like I'd probably need to set aside my job hunt for 2-3 weeks to study. I used k8s a ton in my past role, and once I was laid off, I started a project called "KubeSprout" to really dive in... it basically stands up a k8s cluster using Taskfiles instead of kubeadm lol. If I were CKA certified, would the boost in candidacy justify the risk to income?

Is my experience consistent with what others are experiencing right now?

I still occasionally see posts from devops managers and other leadership-levels complaining about the quality of candidates. Like a candidate somehow making it to a final interview despite not knowing git, or finding out a new hire is actually 3 hotseating people in a strange country. I don't understand how this is even occurring considering my own experience does not seem to align with that. Ironically though, it does give me... some... hope? If the choice is down to me and some guy that doesn't know Git, well...

What am I even doing wrong. Jeez. I beseech the Internet for its help. My damn pocket has a hole in it, and my kids keep eating all my damn food. Ahhhhh!

I'm about 2 months away from buying a farm and containerizing goats instead of code. Hey, Pets vs. Cattle is transferable knowledge!

https://redd.it/1e4tkv2
@r_devops

Beginner in Git-flow. Need help.

Hi people

I'm working on a mobile app, and have been using Github(1 main branch and sub-branches originate from it. Also, all the sub-branches have been sequential until now) flow up until now. But, after applying for testing, when I thought about further app development, I was very confused.

After a long research, I up with a solution to use Git flow (https://nvie.com/posts/a-successful-git-branching-model/). But I have a lot of doubts regarding this workflow. Since most of you guys have tons of experience in this field, can you please help me.

Please visit the above link to understand my doubts.

1. First off, lets say I am simultaneously working on 2 features (f1, f2) on 2 different branches (b1, b2) originating from same develop branch. And remember that I have been pushing changes to the branches only after I had completed the objective/feature, but not frequently (I know its very bad, but I don't know how to make it right, plz help). So, during each feature development, I had to make changes in so many files (didn't sync the changes with one another). Now, after completion of f1, I pushed f1, and also merged with develop branch and pushed it. Now, if I want to merge f2, what is the process, so that all this wouldn't crash. (I'm also saying this, "I never faced conflicts while merging/pushing, since all my branches are from same parent branch and that too, all of them are sequential, and there are no side branches originating from these sub branches. Also, sorry for the wrong terminologies).
2. I don't understand the "git merge --no-ff" command, like how is it different from an ordinary "git merge"?

There might be even more as I dive through your comments, so please be patient. I tried learning using gemini, but it failed to make me understand.



https://redd.it/1e4uk3t
@r_devops

Doubts regarding semantic versioning system

Hey folks

Here's another one I'm really losing my mind over.

I'm developing an app, and just put it out of testing. And you know, not all updates given to an app are necessary to be downloaded and used by user. Lets say I'm using semantic versioning, MAJOR.MINOR.PATCH-BUILD, and I've written a piece of code that checks helps me redirect the user to Play-store to update the app if there's a must-be-updated update. Now, how do I maintain there semantic versions for that?

What I understood:

Major -> breaking changes, so must update.
Minor -> just a few changes/add-ons wrt UI/UX elements.
PATCH -> bug fixes.
BUILD -> to differentiate the usage (release, dev, etc)

What I want to know:

MAJOR -> I feel like I might make so many breaking changes, it might take the number so high up, which isn't very appealing.
MINOR -> Should I consider every UI/UX changes/add-ons a minor update and leave them as optional ones?
PATCH -> This is a tough one. Like, if a minor version has a bug, I should fix it and release it, so do I release it as a patch or a minor? I thought its a hierarchy (only Major needs updating and Minor doesn't. So, if Minor doesn't, why should a Patch need? But, if a patch helps resolve a Minor, it should right? But I don't this all patches requires updating, it would be a headache to users if there auto-update isn't enabled in the store.) How do I decide which patch requires updating and which don't?
BUILD -> I just came to know that there's this BUILD concept. Any suggestions about good usage?

https://redd.it/1e4uwcy
@r_devops

Sonarqube Custom Code

Anyone here uses Sonarqube with their tech stack? Do you think it is worth the trouble to create your custom rules in it? Or just go with what they have.. for reference we plan to use it with java code check

https://redd.it/1e4q3db
@r_devops

Keycloak v25 improving performances

Hello everyone, I've written this article analyzing the performance of Keycloak v25. The previous version suffered heavy losses at the CPU level to increase the level of security, due to the hash algorithm. The new version has corrected this performance loss while maintaining high security.

To find out more, click here: https://cloud-iam.com/rkc25devops

https://redd.it/1e4ox3j
@r_devops

Best cost effective way to run a lot of heavy tasks?

Hi everyone, to give some context i have a browser based game where every day at a certain time a the game matches are processed, this process takes about 5 seconds to run per match and there will be about 1000 matches to process, so running them sequentialy like i'm doing right now would take a lot of time to finish. Fyi i'm using Amazon SQS to queue the matches then i'm polling the messages from a Fargate ECS task with a cron job.

Thing is even processing a single match takes the container CPU to 99% (i'm using the smallest vcpu possible to reduce costs). I'm wondering what is the best course of action and architecture to handle this, i've read something about running several tasks in parallel but this is really not my area of expertise so i don't know.

Let me know if you need more info, i appreciate the guidance.

Thanks!

https://redd.it/1e584th
@r_devops

Private Equity Firm Acquires Flexagon, Signaling Continued Interest in Salesforce DevOps

https://salesforcedevops.net/index.php/2024/07/16/private-equity-firm-acquires-flexagon/

https://redd.it/1e5aoct
@r_devops

Please help me with professional knowledge

Hii people,

After reading all your answers to my previous questions, (first of all, I'm happy that you took your time to answer my dumb questions) I understood that there is still a long way ahead before I master git/versioning. Like I said, I'm a solo developer and not so familiar with Git/versioning, and have had zero guidance on that matters. I read docs, and a few other online resources and yet there are still a few concepts I don't understand. I just want to know how professionals and tech-workers/software engineers does these things, since I've been brute-forcing the flow up until now, and I'm at that point where that won't work anymore.

Also, how often do you guys PULL, ADD, COMMIT and PUSH changes?

I ADD+COMMIT+PUSH only once for a branch and that too when I'm done with that branch. And since its only me who works on the repo and that I only have a single branch originating from main/master at any given instance, I never used PULL, REBASE, RESET, MERGETOOL, DIFF. Can someone please guide me through the mistakes I've been making using Git.

https://redd.it/1e5cev0
@r_devops

Need to move from Frontend to DevOPS after 2 years?

Hello there,



Current status:
I am Front-end developer for last 2 years. Mostly I work with react and Vue along with typescript.

Motivation to DevOPS:

- Server management, writing Linux commands, setting Docker, using git, I enjoy them

- I'm very insecure about my career as being a fr dev.

- Need to move to dubai from south aisa country. I want to learn strong technology with less competition having interest too in it.

- HTML_css less work. I hate frontend, by mistake I am in.

- Exposure to new skills other than just development

- need to start own small scale hosting company, so it can be helpful.

Cons:
- it's way more than just basics I know already. Like Jenkins, ci/CS, pipelines, kubernetes. Etc

- 4 months I have, I need to learn new tech and moved to new place. Is it enough time while maintaining 9 to 5.

- again junior, after spending. By the way Dubai companies still consider junior if experience is not inside Dubai. Outside experience is just 0. I've heard this. Not confirmed.



https://redd.it/1e5dnv6
@r_devops

Sonarqube coverage

Is there any way to configure quality gates in sonarqube so that if the code coverage is less than before, the build pipeline should fail?

https://redd.it/1e5e0et
@r_devops

Should I find a tutor?

I'm currently learning devops. While it's going good, I feel that is too slow. Also on internet the information you find rarely matches complexity of an enterprise project. I've been a developer for more than 3 years now, working on huge projects. So when I learn something on the internet it always feels like scratching the surface.
Should I get a tutor with experience in the field?

https://redd.it/1e5f7c3
@r_devops

GitLab explores sale

https://www.reuters.com/markets/deals/google-backed-software-developer-gitlab-explores-sale-sources-say-2024-07-17/

https://redd.it/1e5i7pe
@r_devops

I have built a visual node system for workflows and pipelines

Hey everyone,

For a few months I've been working on a hobby project and thought I'd share it with you. I call it Actionforge, a visual node system to build workflows and pipelines.

A bit of a backstory, I’m a software engineer and over the years I've noticed that many of my technical colleagues (those who aren't programmers) have a bit of a love-hate relationship with the programming ecosystem. Some of them like Python, but overall could never fully commit to it. Since my background is in 3D and VFX, where intuitive node systems are very common, I decided to bring this intuitivity to DevOps.

https://www.actionforge.dev

In a nutshell, you can build a graph using an editor and then run it everywhere.

The runtime to execute these graphs can be downloaded from the webpage. The runtime is a single binary (\~10MB), no installation, has no telemetry, and no internet is required. It runs on all platforms and architectures. It supports concurrent node execution, group nodes, port groups, streaming files from and to the cloud, and so on.

On top, if you're more into GitHub Actions, I have built a full GitHub integration, meaning instead of writing YAML files for GitHub Action workflows, you can simply click together a workflow with actions from the marketplace.

Happy to share all the nitty-gritty if you're interested.

https://redd.it/1e5kcr0
@r_devops

Top skills in demand for devops in west coast??

Hello!
I will be graduating from my master's in information systems in December, and I also have experience in Devops of 6 years. I am looking for jobs and want to move to California Bay Area / West Coast mainly due to the nice weather and relatively chill lifestyle. I am also looking to situate myself in areas with lower living expenses since I will be dealing with student loans.

While I have my hands dipped in a bunch of different cloud providers, devops tools and have been working on my skills in IAC, Kubernetes, Observability, monitoring and other tools. I am looking to highlight skills which are more likely to be picked up by companies in the west coast? I am looking for recommendations such as - `this area has companies in robotics so they are more likely to use GCP` etc. Any other advice, suggestions or wisdom are extremely welcome!

Reposting this after editting since I mixed up my east and west and some commenter was more focused on teaching me my lefts and rights. I'm just looking for some kindness guys.

https://redd.it/1e5m5m2
@r_devops

Is this job really worth the salary?

I'm UK based so can't speak for the rest of the world.
In the UK, I am seeing many DevOps jobs going for around the 50k - 60k mark. Some positions are even lower than this!
I know there are positions which rise higher than this, and senior roles can even go a good bit higher. But, my question is, do you guys think it's worth it?

I am just shy of 2 years into a DevOps role. Worked as a dev for a year before hand - very luckily managed to upskill and became a DevOps engineer. However, my salary is 36k, move from 34k as a developer. But the stress I feel as a DevOps engineer is x10 worse than when I was a developer.

I work in a consulting firm, so maybe this is part of it. But I have non-stop customers messaging me all day. Things are faulty here and then, and tickets are raised all over the place. Constantly meeting with clients to discuss SOW and updates on projects, etc. I know my pay will bump up, but even at 50k+ it seems like a whole lot of responsibility and knowledge you ned to know!

Comparing myself to my friends that are still part of the developer team, by no means are their jobs relaxed, but I feel like it's soo much more peaceful that the constant bombardment I am under.
Granted I know a lot of this is to due to me having to learn a whole bunch of new things and be there to answer questions over a whole range of topics. But I feel like when you compare what knowledge is expected of you compared to other professions which can get to 50k... it is quite wild.

https://redd.it/1e5m1dy
@r_devops