What factors are important when wanting to downscale datadog-agents?

I'm working on reducing the number of datadog-agents we have running on our hosts. A lot of hosts I have visualized have below 5% CPU Utilization, low disk usage and not that much log ingestion. However, the downside of removing the agent is that you have no visibility other than accessing the instance itself or using cloud provider metrics. I wonder if those three points I mentioned are valid reasons to remove visibility on hosts. I could not find anything on the internet that answers something similiar.

https://redd.it/1d8pb7d
@r_devops

Found this DevOps burger illustration by Tech World Milan interesting.

I was browsing twitter and came across this pretty interesting illustration. Thought it's worth sharing.

https://x.com/milan\_milanovic/status/1797889065833898347

https://redd.it/1d8qepm
@r_devops

is anybody using dagger.io in production?

Hello,

I recently discovered https://dagger.io/ and I love the concept and after playing around with some scripts and examples, I have a semi working pipeline im starting to get happy with.

I am interested in looking into using this production (or at least some staging environments for now) but I have concerns, mostly a) the documentation is terrible, its constantly out of date and suggesting different approaches b) the API is constantly changing and there is no clear approach to solve problems. The best source of documentation has been the github issues list, which worries me.

That being said it seems like a fantastic tool to prevent our current YML soup and promote code reuse. Once I made some initial headway it has been refreshing.

Is anybody using it and having the same difficulties? Anybody know of similar tooling for producing build images? Interested to hear peoples thoughts.

https://redd.it/1d8sqk6
@r_devops

What make a monitoring tool (production ready) ?

There are numerous tools available for monitoring, ranging from open-source and freeware to premium options. Based on your experience, what makes a tool suitable for production use? What criteria/functinality do you consider when selecting a tool or stack for your environments?

In line with this discussion, here is my go to for discovering new open-source tools

https://ossinsight.io/collections/monitoring-tool/

https://redd.it/1d8uul5
@r_devops

Need Advice

After I graduated I got my first career job two years ago as an Integration Dev, I want to start learning new tech so I can find a DevOp job starting next year. Im looking at KoudeCloud tried there DevOp course on Udemy and its 90% of the things I do at work. Will it be hard finding a DevOp job with 3 years experience as I only do light work at my job and the 20+ years experience do the hard work. I want to learn more but everyone is busy at my Job and there is no salary increase for three years. Any courses, subjects I could learn that could help more to break into DevOps?
I got a BS in CS, work at IBM for two years now as an integration dev but barely any work as I had no experience at all.


https://redd.it/1d8wfoe
@r_devops

Web app deployment not pulling forked repository code

### Scenario
I'm attempting to deploy a sample .NET Hello World Webapp to Azure, referencing a forked GitHub repository. The deployment process indicates that it is referencing my forked repository, but the GitHub Actions workflow does not trigger, and the web app is not deployed from the forked repo. I've tinkered with various settings with the terraform config, to no avail.

### Issue
Despite configuring the repository and providing the GitHub authentication token, the GitHub Actions workflow is not triggered, and the web app is not deployed from the forked repository. I've also tinkered with setting the manual_integration variable of the source control slot to no avail.

### Question:
Is there an additional authentication step or configuration that I'm missing between GitHub and the Azure portal to ensure the deployment triggers and pulls the code from my forked repository?

I am being patient post-deployment and waiting 10 minutes for the site to show. And it does work when deployed manually in the Azure portal.

Any help would be greatly appreciated! Thank you!

## Terraform config:

### webapp.tf

resource "azurerm_service_plan" "srv_plan" {
  name                = "${local.prefix}service-plan"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  sku_name            = var.webapp_sku
  os_type             = var.webappos
  tags                = local.common_tags
}

resource "azurerm_windows_web_app" "dot_net_web_app" {
  name                          = "${local.prefix}dotnet-app"
  location                      = azurerm_resource_group.rg.location
  resource_group_name           = azurerm_resource_group.rg.name
  service_plan_id               = azurerm_service_plan.srv_plan.id
  https_only                    = true
  public_network_access_enabled = false
  enabled                       = true
  webdeploy_publish_basic_authentication_enabled = true

  site_config {
    always_on           = true
    minimum_tls_version = var.tls_version
    application_stack {
      current_stack  = var.stack
      dotnet_version = var.stack_version
    }
  }
}

resource "azurerm_source_control_token" "source_token" {
  type         = "GitHub"
  token        = var.github_auth_token
  token_secret = var.github_auth_token
}

resource "azurerm_windows_web_app_slot" "slot" {
  name           = "${local.prefix}app-slot"
  app_service_id = azurerm_windows_web_app.dot_net_web_app.id
  site_config {}
}

resource "azurerm_app_service_source_control_slot" "git_source" {
  slot_id                = azurerm_windows_web_app_slot.slot.id
  repo_url               = var.webapp_repo_url
  branch                 = var.webapp_repo_branch
  use_mercurial          = false
  use_manual_integration = false
  depends_on             = [azurerm_source_control_token.source_token]
  github_action_configuration {
    generate_workflow_file = false
  }
}

### variables.tf

variable "webapp_sku" {
  type    = string
  default = "P1v2"
}

variable "webappos" {
  type    = string
  default = "Windows"
}

variable "tls_version" {
  type    = string
  default = "1.2"
}

variable "stack" {
  type    = string
  default = "dotnet"
}

variable "stack_version" {
  type    = string
  default = "v7.0"
}

variable "subresource" {
  type    = list(string)
  default = ["sites"]
}

variable "webapp_repo_url" {
  type    = string
  default = "https://github.com/ZimCanIT/hello-world-webapp"
}

variable "webapp_repo_branch" {
  type    = string
  default = "main"
}

variable "github_auth_token" {
  type        = string
  sensitive   = true
  description = "Token for authorization"
}

https://redd.it/1d8wack
@r_devops

How does the logging platform of your company look like?

Would love to know the following,

Technology e.g. ELK, Splunk, or any other self managed OSS?
Data size
UX - SQL or free form text
Structured/Unstructured logs/Mix

https://redd.it/1d8zyzu
@r_devops

Devops Resume Review

I've been in DevOps for a little bit and I'm looking for another DevOps Position. I'm striking out on my hunt and I think its because of my resume.

Please either

- roast me
- tell me how to make my resume better

Thanks!

https://imgur.com/UNMF3Ar

https://redd.it/1d8yp6m
@r_devops

There are no good options for hosting medium-sized applications

I am a software engineer and have been actively developing and maintaining small to medium web applications for the last few years—for example, small CRMs, e-shops with thousands of users, etc.

There are great hosting options for small and large-size applications but no good options for medium ones.

For example, for free, a small-size website/web app can be hosted by a static website hosting provider such as Netlify/Vercel/Cloudflare pages or even by a PaaS such as Render or Digital Ocean Apps.

Large-size applications are complex, and there are plenty of options, such as Kubernetes.

Medium-size applications do not need the complexity of the large ones. They usually have one server for the web application, one queue system such as Celery or Laravel Horizon, one service for scheduling cron jobs, one cache service such as Redis, and one relational database.

If you host this in AWS using managed services such as AWS RDS, Elasticache, Fargate, etc, you might end up paying thousands of dollars for something that could host 50 dollars per month in a simple VPS in Hetzner. However, if you do it manually, then it is too much trouble to maintain lots of them without a full-time sysadmin/DevOps engineer allocated.

What are you using in these cases?

https://redd.it/1d938ub
@r_devops

How do I successfully start this open source project in github codespaces?

Here is a repo: https://github.com/pschlan/cron-job.org

Using a github codespaces instance with 16gb ram, I'm trying to figure out how to get the site working with docker compose, some instructions are in the readme. But the docker-compos.yml seems to be outdated.

I want to be able to access the local frontend, create an account, open the confirmation email, and then log in using my credentials. Also to be able to create a cron job and view it in the dashboard.

I will have to setup a SMTP server in docker-compose.yml for the site to work. I tried using mailserver container but it can't even send because the api container doesn't work.

I tried all this myself but got errors with mysql and the SMTP server not starting. Here is my troubleshooting so far: https://github.com/pschlan/cron-job.org/issues/242

Appreciate any help, thank you

https://redd.it/1d953ij
@r_devops

Oracle cloud wierd issue



I have 3 mongodb VMs.

I have NSG which allows both ingress and egress for the CIDR that all 3 VMs are part of.

If I allow all ports on source and destination on these NSGs then everything is fine.

But when I allow only mongodb ports, the communication stops

https://www.mongodb.com/docs/manual/reference/default-mongodb-port/

even netcat cannot reach another VM on the mongo ports only if I allow the mongo ports on the NSG.

I have tried this with both ingress and egress NSG with same result.

I also have a security list with mongo port 27017 for the mongodb VMs CIDR block. I did not touch that yet.

https://redd.it/1d96oxo
@r_devops

Subgroup of team selected for secret 6-12 months planning

Found out about it yesterday by mistake, same day as the meeting. 4 out of 10 in the team, and it's obviously been kept under wraps. We recently had an incident in which the incredibly incompetent scrum master screamed at some of our team members, yet for some reason both the PO and the PL not only keep the wanker on, but he's also in this meeting.


Meanwhile, I am known within the organization as the problemsolver and for working hard, and I sincerely believe that the entire team has input well worth taking into consideration.


Literally the only thing that they would have to do in order to achieve minimum effort would have been "we're doing this planning meeting, please put forth your opinions suggestions and recommendations". Instead, more than half of the people involved in this meeting are either bad or completely incompetent at their jobs.


I am honestly furious about this - I've worked very hard for the last ten months to get the platform into a usable state. I have a 1on1 with the PL tomorrow, and I'm feeling like I'm about to do a rage quit.


Would really appreciate some input on how to handle this - I really don't like to feel angry.

https://redd.it/1d9f0l7
@r_devops

PagerDuty

Hi guys, recently my manager tasked me with learning pager Duty as i have to implement it in our project. Any starting points that you guys can recommend? Any sites or Materials? Is it worth it to do a certification for jt?

https://redd.it/1d9g2vz
@r_devops

Help with my resume, is it that bad to not even get even a interview for a intern position?

Basically I am trying to get a entry level position for devops. So I've been applying to all the devops internship position, but still haven't even heard from one them till now. I do have experience with IT helpdesk, would that help in getting a devops role?

Link to me resume : https://imgur.com/xCD8Pe8

https://redd.it/1d9f5xc
@r_devops

Stormforge.io AI for automating kubernetes resource requests

Has anyone tried this out? I came across an ad for it on Reddit and it seems like it would solve a lot of the issues I'm dealing with in trying to administer a centralised runner solutions to thousands of end users but not allowed go too hard on limits because that will impact developers too much and so on.
I'd love to here if anyone has experience with this product and what their thoughts are?
It kinda seems a bit too good to be true ATM but I havent investigated the pricing models yet
https://www.stormforge.io/

https://redd.it/1d9i7ei
@r_devops

Orchestration solution advice

A project at work is in discovery and implementation round 1. Essentially it is in the "make it work" phase.

I am in a two-person SA team, and we will be the SA's on this new project.



It is going to be a distributed on-prem live service.

The system consists of a physical on-prem central server and an elastic number of on-prem worker hosts.

An example of the type of system being implemented is a self-checkout system in a supermarket - Imagine a server in the back room, and all the self checkouts running their own set of the same services independently.

Note: It is not possible to suggest cloud, as the industry is highly regulated and all site IT departments shutdown all outside lines.



The system is built using .NET and runs each service as a screen. I have been looking into building a case for migrating from screen to a container orchestration solution.

I want to be able to run the exact same set of services on each worker host (node) and be able to run local replicas of these services on each node to facilitate HA and next to zero downtime during software updates.



We are already familiar with Docker Swarm as one of our legacy products used parts it, however I'm not sure it fits the bill here. I toyed around with mode: Global however this replicates a single service instance on all the nodes, whereas I need each node to have it's own service instance and replicas of it.

The other option with Docker Swarm I have thought about is having each node run it's own Swarm with a service stack and a monitoring stack, and have a Portainer and Prometheus instance on the server scrape the Swarms.



I've been playing with a k3s cluster in spare time, and I think the closest thing I can find that fits the spec is a DaemonSet with nodeSelector labels, but I haven't been able to figure out replicas of each service in the set.

Am I on the right track? Is there a way to have local replicas of a DaemonSet on each node in a cluster?



Is there a different orchestration solution I don't know about that does this?



Thanks!

https://redd.it/1d9hczl
@r_devops

Managing deploys to multiple environments with Azure/GitHub Actions

Im looking for a starting point on what is probably a super easy problem to solve: if we have multiple sets of isolated infrastructure that are all considered a part of our overall "Production" environment, what are some good tools for managing the configurations of all these?

For example, we are storing info for each individual client's application instance as separate Github Environments, each with their own secrets/env variables. So, we have 'customerAproduction', 'customerAstaging', 'customerBproduction', 'customerBstaging', etc, each with their own secrets containing Azure credentials for deploy. When running a new release we manually trigger a build & deploy Action on each environment with Environment = customerXproduction and Branch = <releasebranch>. Repeat X times - this has worked fine until now.

It now would be really nice to just select Environment = Production, have the GitHub Environment 'Production' set up so it can connect to another service that contains the configurations for all our production environments (and any secrets associated with each of those) and just runs the same deploy action across them. Are there recommended tools for this sort of thing that it might be worth us looking into, or am I over complicating this and a JSON file / Azure KeyVault / some other centralized key-value store containing the deployment credentials for all of our apps might do the job just as well? We don't do containerized deploys currently.

I'll obviously weigh the pros/cons of all the options for our specific scale and situation, I'm mostly just looking for a few starting points for our research and to make sure we're in the right thought space. Thanks!

https://redd.it/1d9m1bt
@r_devops

How do you measure team performance?

Just curious so a vague question on purpose😊 pls tell me your opinions and experiences

https://redd.it/1d9lpmc
@r_devops

A Comprehensive Guide to API Endpoints


Hi hi! Thought this would be helpful for this group and those newer to API development. We have an educational deep dive on API endpoints including 10 Best Practices for designing them, an overview of the types of endpoints, how they work, the future of endpoints, and how to keep them secure. Enjoy!

And do let us know if you find it helpful or if we ought to do one with more advanced API endpoint techniques/if we missed anything major as it relates to endpoints! :)

https://www.getambassador.io/blog/guide-api-endpoints

https://redd.it/1d9lbid
@r_devops

Azure container apps price and alternatives

I have a few microservices that I decided to setup with azure container app. I created one container app that has nginx and this one is public so it can redirect the calls to the right container app depending on the path. I have 2 other container apps, these are private and are the actual services. Why did I do this? Because I want to have a single endpoint to call from the outside.

I now look at the prices and they say that I need to pay depending on number of requests. Which made me worried a little bit as you pay also for bad URLs. I can setup rate limiting in nginx but even this will not solve the problem of someone slowly making calls to bad paths which gets counted anyway towards the bill.

My questions are: any suggestion on how to handle bad http requests? Any other options apart from using azure container apps?

https://redd.it/1d9qoek
@r_devops

How does Russia block VPN services?

I am not sure if this is the right sub, but I didn't find other more appropriate.

I visited Russia yesterday and found out that the most paid VPN services I used - NordVPN, Surfshark are now banned. Both WireGuard and OpenVPN, even manual setup. I used to say that they wouldn't do it, because many businesses depend on private VPNs and it would break too much, but they seem to be doing it successfully. I had my job VPN working without issues the whole time. I was able to find some less known services in GooglePlay, but it was tricky.

Are they just banning IPs or there is something else? What if I set up my own VPN service and use it alone, is there a high chance that it may be banned too?

https://redd.it/1d9rrk8
@r_devops