How do you maintain a real time back up?

Just a small case study kinda question that crossed my mind, I'm a student be kind on me. So if there's a company, say Facebook. Now let's assume they have 100 new users joining every 1 hour. And each user addition is practically a completely new customer that they can't afford to lose . But what if the server goes down and they're forced to roll back to the most recent backup which might've been an hour ago? What happens to these new users between that remaining time frame, and what happens to all the content they posted? And Facebook could probably afford to back up every microsecond but what about smaller companies with 1000x lesser budgets ?

https://redd.it/13te3f7
@r_devops

What skills and concepts/topics are required to land dev ops straight out of college?

Hey just wanted to know the main important content fkr dev ops jobs. Also how hard is it to land a dev ops job straight out of college in comparison to swe jobs?

https://redd.it/13tfyxi
@r_devops

Has anyone tried using terragrunt for PR environments in GitHub actions pulling private repo?

Essentially trying to run terragrunt in a GHA workflow with the .hcl file sourcing from a private GitHub repo. Can’t get it to work even with a PAT token.

https://redd.it/13tfwcp
@r_devops

I don't get the point of KV/secrets manager (Vault, Azure secrets, AWS secrets...)

After reading some documentation, i still don't get the point of secrets manager services like Hashicorp Vault, AWS secret manager or Azure Key Vault secrets.I understand the need to not have secrets in env. files or Git. But i don't get how such services improve the situation.

Let say your app instance is authorized to get a token from the KV service to access another server or api. Now, if a malicious party takes control of the instance, what is preventing him to ask the vault service for the credentials or token. How is this improving the situation ?

Thank you.

https://redd.it/13qp391
@r_devops

Parllelx.sh - run chmod or chown command parallelly on huge amount of file

Hi, Just wrote simple bash script to run chown and chmod commands parallelly in background using screen command. hope his will be helpful if someone is running chmod/chown on large data sets.


Motivation:
After moving 650GB of jenkins data(/var/lib/jenkins) containing small small files from hard disk(AWS EBS) to NFS(AWS EFS) i had to change the ownership of files which was taking long time and after 3+ hours i just stopped the chown command. As Jenkins server had 8 CPU core i thought of using by going inside directories and running the chown command one by one with screen. which took around about 1 hour to complete but i had to do lots of stuff manually like getting the list of directory hierarchy and monitoring the status of previous running chown command before i can run another command. I thought of writing this script to fully automate those steps.

code : https://github.com/koolwithk/parallelx.sh

https://redd.it/13tk8rj
@r_devops

CPU vs vCPU

Hello, I’m about to purchase a server/switching from cloud hosting, I just wanted to ask (I’m not really good with tech at all), the cloud server I was on had 48vCPUs, what I’m moving to is a dedicated server 32 Cores/64 threads, in this case which is better? Again I apologize if this is a bad question in itself, I just want the basics of it

https://redd.it/13to58q
@r_devops

We summarized top 10 interview questions for DevOps engineer

Top 10 Interview Questions for DevOps Engineer

Can you comment with the questions you had before?

https://redd.it/13tp7hq
@r_devops

Resume review

Resume: https://imgur.com/a/pYFlVMb
Give your suggestions and advice. I will take them constructively. I am a sysadmin person but my role is called devops person looking to switch

https://redd.it/13tx4ik
@r_devops

No resources found in default namespace.

I ran this command to create deployment called mynginx.


kubectl run mynginx --image nginx:latest


But it created pods only but not deployment.

C:\Windows\system32>kubectl get pods
NAME READY STATUS RESTARTS AGE
mynginx 1/1 Running 0 3m31s


C:\Windows\system32>kubectl get deployment
No resources found in default namespace.

How do I fix this? I'm following a tutorial and in the tutorial, the deployment is being created.

https://redd.it/13ty9r1
@r_devops

Troubleshooting and Debugging Terraform Providers

Basic Steps - Terraform Provider Debugging

https://medium.com/illuminations-mirror/basic-terraform-provider-debugging-c20532268c3b

https://redd.it/13tztkl
@r_devops

A Manifesto for Cloud-Oriented Programming from the creator of the CDK

In this insightful article, Elad Ben-Israel, the mind behind the CDK, shares his love for the cloud, but also his frustrations with the complexity of building cloud applications. The challenges he identifies include:
1. Focus on non-functional mechanics: The need to understand and manage cloud platform mechanics instead of focusing on building valuable features for users.
2. Lack of independence: Developers often need to rely on others to handle parts of the deployment process or to resolve issues, interrupting their work flow.
3. Delayed feedback: The current iteration cycle in cloud development can take minutes or even longer, significantly slowing down the development process and making it harder for developers to stay in their flow state.

## It's not just a rant
Elad is not just ranting about cloud development. He proposes a solution in the form of a programming language for the cloud. This language would treat the entire cloud as its computer. The language compiler will be able to see the complete cloud application, unbound by the limits of individual machines. Such a compiler would be able to handle a significant portion of the application's non-functional aspects, enabling developers to operate at a more abstract level, thus reducing complexity and promoting autonomy. Moreover, it could expedite iteration cycles by allowing to compile applications to quick local simulators during the development process.

## The Winglang Project
Elad reveals that he's in the process of developing such an open-source, “cloud-oriented” language, dubbed Winglang. Wing aims to improve the developer experience of cloud applications by enabling developers to build distributed systems that leverage cloud services as first-class citizens. This is achieved by integrating infrastructure and application code in a secure, unified programming model. Wing programs can be executed locally via a fully-functional simulator or deployed to any cloud provider.

## My Interest in Winglang
I, together with a group of dedicated contributors, joined forces with Elad to develop Winglang.
While still in Alpha and not yet ready for production use, it's already possible to build some real applications.

Check out https://github.com/winglang/wing for more details.

https://redd.it/13u5zrj
@r_devops

Health Check Response Format for HTTP APIs

I’m continuing my journey on getting more familiar with HTTP APIs by reading related RFCs. This time, I read the Health Check Response Format for HTTP APIs on the suggestion of Stefano Fago. In this post, I’d like to summarize my reading.

Note that it’s a draft. Moreover, it has been dormant for nearly two years and, thus, has been automatically expired. However, it’s the closest to a specification on health checks and thus deserves some love.

Read more

https://redd.it/13u5cs2
@r_devops

Supplemental Productivity Software

I'm looking for some additional productivity software in addition to (or lack thereof) options that our company has. Granted, I've confirmed reimbursement, but as a small shop, most of it is "choose your own adventure." Our only two officially used platforms are GitLab and Slack. I think three categories can be broken down and used unless I'm overthinking them,

Wiki/Knowledge Base - GitBooks is what I'm leaning toward right now. Confluence is overkill, and options like Wiki.js and Bookstacks haven't been as seamless as I'd like. xTiles seems like a weird hybrid between all 3.

Workflow/Kanban - Like above, Trello/Jira are overkill; Monday requires three seats. So far, I'm enjoying Linear's free plan the most.

Notetaking - Notion, Craft, and Upnote all seem good, but this category also seems saturated with a ton of options.

Thoughts?

https://redd.it/13u5wkk
@r_devops

Exploring Datadog alternative

Hello everyone!

I work at a small startup and currently use Datadog for monitoring etc. Our stack is Kubernetes cluster on Azure, written in .Net. We also use DD for some SIEM purposes. Have a bunch of monitors, mostly about webservices uptime, error rate and response times. Some monitors around host level monitoring exist as well.

Not very happy with Datadog given their billing, what tools would you suggest we move to, for these purposes? How is Azure Monitor stack for these purposes?

Also, if someone has done this for their work and have time to consult/get it done for us, we can pay as well. Ideally looking for open source stack, but would love help from this group on getting started and initial setup!

https://redd.it/13ubihi
@r_devops

Has any Integrated RASP that can detect custom use cases ?

I am aware that RASP can protect against a host of known attacks inspite of detection signatures not being available. But is there any RASP solution whose rules / policies can be modified to detect application specific attacks that are more towards business logic and not just OWASP attacks ?

Thanks

https://redd.it/13uk983
@r_devops

Do You Know Devops - Crossword

https://www.funfordevs.com/crossword-devops.html


Credits to u/siddharthnibjiya on original post on **observability**...

https://www.reddit.com/r/sre/comments/13u7lhk/crossword_on_observability_any_enthusiasts_here/

https://redd.it/13ur7v0
@r_devops

step by step guide of lab scenario to test nodePort in kubernetes?

https://imgur.com/a/3IrFsgn

I want to achieve this scenario in a lab environment in kubernetes. How can I create one? I tried asking chatgpt but no help from there. I tried googling but no help from there either.

https://redd.it/13uv7jn
@r_devops

Advice needed on continuous integration

I'm part of a 2 people team developing a SaaS application. Our tech stack involves React.js for the web app, React Native for the mobile app, a web extension, and a desktop app with Electron. All these components are part of a monorepo.

We're currently in the process of setting up our CI/CD pipeline with GitHub Actions and are planning to use a trunk-based branching strategy with release branches for releases. We are also aiming to strike a balance between maintaining high quality software and keeping costs low, as we are a bootstrapped company. Therefore, we're trying to decide on an effective testing strategy that involves both local testing and automated testing with GitHub Actions.

We're contemplating on which types of tests (unit, integration, E2E, performance, visual regression) to run in which environment (local or GitHub Actions). Here are some considerations we've been thinking about:

Unit Tests: Given their speed and low cost, we're thinking about running these both locally before committing and on every PR with GitHub Actions.
Integration Tests: We're considering running a subset of these on every PR and the full suite on a schedule or when merging to the main branch.
E2E Tests: We're considering running a minimal set of these on every PR and a more comprehensive set on a schedule or when merging to the main branch due to the costs and time they take.

We would love to hear your insights and experiences on this. How have you decided which tests to run locally vs. in the cloud? How do you manage the balance between cost and quality in your testing strategy? Any advice or resources you could share would be greatly appreciated.

https://redd.it/13uxr6d
@r_devops

Interacting with Kubernetes Deployments and Services using Python SDK

Discover the power of Kubernetes combined with Python! Our newest guide delves into using the Python SDK for interacting with Kubernetes deployments and services. Harness the versatility of Python and the robustness of Kubernetes in one place. Check it out now! #Kubernetes #PythonSDK #Python #containers

https://www.faizanbashir.me/interacting-with-kubernetes-deployments-and-services-using-python-sdk

https://redd.it/13uz01n
@r_devops

My secret tricks on getting things done fast

Hello, I made a video about my strategy of how I get things done fast in day to day work

https://www.youtube.com/watch?v=YAOq5-w3YXw

https://redd.it/13v2hbx
@r_devops

GitHub Actions equivalent of Azure Devops pipeline resources?

Azure DevOps yaml syntax allow us to define resources for pipeline -> https://learn.microsoft.com/en-us/azure/devops/pipelines/process/resources?view=azure-devops&tabs=schema

For example, I might have separate pipelines for CI and CD. I would like to be able to select specific CI pipeline artifact when dispatching CD pipeline. With Azure DevOps I have achieve this by:

resources:
  pipelines:
  - pipeline: ci
    source: App-CI

trigger:
- none

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    echo $(resources.pipeline.ci.pipelineID)
    echo $(resources.pipeline.ci.runName)
    echo $(resources.pipeline.ci.runID)
    echo $(resources.pipeline.ci.runURI)
    echo $(resources.pipeline.ci.sourceBranch)
    echo $(resources.pipeline.ci.sourceCommit)
    echo $(resources.pipeline.ci.sourceProvider)
    echo $(resources.pipeline.ci.requestedFor)
    echo $(resources.pipeline.ci.requestedForID)

So in my CD pipeline I am specifying it uses CI pipeline as resource. Now, when dispatching CD pipeline through UI, I am able to select specific CI run (which produces artifact).

Is there any equivalent for GitHub Actions? I would like to have separate CI & CD workflows and when dispatching CD workflow I would like to be able to select specific CI run to use for CD workflow.

https://redd.it/13v2fl8
@r_devops