How does your company do secret management? AWS/GCP/Azure/Vault/CyberArk etc. thoughts?

It seems like every company has a different stack and setup and I'm hoping that y'all can shed more light and educate me (and everyone here really).

What does your company use for secrets management (stack, hosting, setup)?
What your thoughts are on AWS/GCP/Azure secret managers vs. Vault. Is Vault overkill or should AWS/GCP/Azure secret managers suffice even for most enterprise cases? Put differently, why would a company self-host Vault if they can use a native secrets manager from any one of the major cloud providers that already offers a ton of benefits like secret versioning, rotation, audit logging, scalability & availability, etc.
Does your company use one or multiple secret managers? If multiple, what for?
Do you set environment variables in platforms like CircleCI, GitLab, Netlify, etc. manually or do you manage to pull them in from secret managers? What's the typical approach here?

https://redd.it/121rorl
@r_devops

What type of programming do you do based on your role?

Let me explain… I’m a full time CS student who has roughly 4 years of operations experience and at my last operations role, I was automating the windows and linux end user workstation stack using PowerShell + bash and used PowerShell intensely with Active Directory and Exchange. I feel in love with automation.

The main three types of programming styles (correct me if I’m wrong) is Procedural; Object Oriented; and Functional programming. And I’ve noticed that the code I’ve written previously in my last job was strictly Procedural, very simple in my opinion and straight to the point.

Now in college OOP(Java to be exact) is kicking my ass and the concepts is extremely new to me so I’m learning/coding as I go but at this point I’m not too comfortable with the language, but I’m willing to learn.

So my final question is since I want into get into a DevOps role post college, is the automation more procedural or is it mixed with OOP and Functional Programming?

Curious to read your thoughts….

Thanks!

https://redd.it/121u50z
@r_devops

Code project help



Hi, so I created a project using azure, netlify, github, replit and notepad. The github readme for it:

"This project contains a simple chatbot interface that can be integrated with Microsoft Teams using the Direct Line API. The interface is built using HTML, CSS, and JavaScript, and it allows users to send and receive messages through the Microsoft Teams platform.

Getting Started

To use this chatbot interface, you will need to have a Microsoft Teams account and a Direct Line secret key. The Direct Line secret key can be obtained by creating a new Direct Line channel in the Microsoft Bot Framework portal.

Once you have your Direct Line secret key, you can update the index.html file in the chatbotsite folder with your secret key, and deploy the folder to a web hosting service like Netlify.

Usage

To use the chatbot interface, simply open the URL where you have deployed the chatbotsite folder, and start sending messages. The interface will send your messages to the Direct Line API, which will then forward them to your Microsoft Teams account. Likewise, any messages sent to your Microsoft Teams account will be displayed in the chatbot interface."

Now I think I've been able to get it created as ive done the azure stuff, deployed to netlify and configurated the bot using developer giving it permissions etc. The only issue is that i cant test preview it before i publish or even when i publish as i only have teams on my work laptop and i dont think my it admin has given me the necessary permissions to add any old app to Teams. Can someone suggest a way round this? Or even test preview the bot for me to see if what im trying to achieve can be done ?

I've spent the last three to four days on this hellish journey (i have no prior coding knowledge or experience at all) do through a combination of googling and chatgtp(literally feeding it errors line by line and correcting it myself on occasion), ive got to where i am now. I have no intention of just giving up-, need to see this through so any help would be appreciated.

https://redd.it/121v2ms
@r_devops

Alternative to alpine

What other small linux distro you guys use for docker image.

https://redd.it/121xq1m
@r_devops

How much Docker knowledge is enough to be hirable?

I'd like to get into DevOps but my current and past jobs mostly deal with Windows and AWS and doesn't touch anything Docker or Kubernetes. With this I'm left with self-studying them in the hopes of getting good enough to qualify for DevOps roles.

I asked ChatGPT what projects should an applicant do on Docker to showcase intermediate knowledge in it and it got me these:

1. Multi-service application: Ask the applicant to create a multi-service application using Docker Compose. They should create a simple web application with a front-end (e.g., React or Angular), a back-end (e.g., Node.js or Python), and a database (e.g., PostgreSQL or MongoDB). The applicant should create a Dockerfile for each service and a docker-compose.yml file to set up the entire application.

2. Dockerizing a legacy application: Provide the applicant with a legacy application or let them choose an existing open-source project. They should create a Dockerfile and any necessary configuration files to containerize the application. This will demonstrate their ability to work with existing codebases and adapt them to Docker.

3. Optimizing Docker images: Give the applicant an existing Dockerfile and ask them to optimize the resulting Docker image. They should focus on reducing the image size, using multi-stage builds, and improving build times. The applicant should also document the changes they made and provide reasons for their decisions.

4. CI/CD pipeline: Ask the applicant to set up a Continuous Integration and Continuous Deployment pipeline using a platform like GitHub Actions, GitLab CI/CD, or Jenkins. The pipeline should include building Docker images, running tests inside containers, and deploying the application to a container orchestration platform like Kubernetes or Docker Swarm.

5 .Container monitoring and logging: The applicant should set up a monitoring and logging solution for containerized applications. They can use tools like Prometheus and Grafana for monitoring, and Elasticsearch, Logstash, and Kibana (ELK Stack) or Fluentd, Elasticsearch, and Kibana (EFK Stack) for logging. The applicant should create a docker-compose.yml file to set up these tools and demonstrate how they can be used to monitor and analyze the performance and logs of the containerized application.

6. Networking and security: Provide a scenario in which the applicant should create a custom Docker network and secure the communication between containers. They should demonstrate the use of network segmentation, encrypted communication using TLS, and proper implementation of user privileges and secrets management.

I understand that I need to learn many other tools to get into DevOps but I'd just like to know if doing these projects could get someone a pass on Docker. If you have any additional suggestions or advice, please share them. Thanks!

P.S. I'd probably post another one for Kubernetes once I'm done with Docker.

https://redd.it/121tur0
@r_devops

Monitoring using Serverless

Has anyone used serverless workers (Cloudflare Workers, AWS Lambda, GCP Cloud Functions) or CI/CD service (GitHub Actions, Jenkins) for monitoring?

I'd like to monitor a service, and normally I'd make an exporter for it and have a server for Prom/Grafana or InfluxDB or whatever.

But I'd like to not have to manage a server, since this is just a hobby project.

One option is paying for some Prom-aaS hosted service, but I think it'd be interesting to run something on GitHub Actions or Cloudflare Workers, since those are both essentially free for paid users.

I don't need Grafana levels of visualization, honestly I could generate hourly/daily/weekly/monthly/yearly graphs with rrdtool and be good with that.

Has anyone tried this before?

What did you use for a your (time-series) database? A file-based db like sqlite in git, or maybe in object storage? How was performance like?

How would you display your graphs? In git? How do you make sure your git fs doesn't become bloated?

https://redd.it/12200dv
@r_devops

How do you improve your speaking skills, especially when you're ruffling some feathers with a new process?

Need your help on something, and I'll create a hypothetical situation.

Let's say you're helping out a team at your organization (you're on team X, you're helping team Y). Team Y works slow, is backlogged, and doesn't understand how to use tech to its advantage. Let's say they do a process that takes them 1 hour, and you propose a solution where you can automate it where it takes 10 seconds.


You think to yourself: "Everyone is going to love this, this meeting is gonna be cake". So you get to the meeting, explain the process, and although half the team loves it, you have those people who hate change in regards to everything. Now I some of you are going to think "Well maybe they have a point", and you're right, they might...I'm ALWAYS open to suggestions, but for the sake of argument let's just assume these people argue against any change just because.

For instance, they might say "Well what if this thing fails and takes down the whole thing! We won't even be able to do it manually", they'll come up with this bizarre hypotheticals that have almost 0.0001% chance of happening, and like, I go blank....because the only responses in my head sound patronizing...like if I said doing it manually is prone to error, they might say "Well yes but we're in control", you know, that sorta thing.

Is this a skill that you developed yourself? Do some people just have it and just people dont?

https://redd.it/12239ji
@r_devops

You're messing up if you're not meditating like this

https://www.youtube.com/watch?v=WsVox90y6ow

https://redd.it/120t8vl
@r_devops

DevOps 101 for a Dev Who Doesn’t Like Ops

The basics of DevOps 

So, what is DevOps? At its core, DevOps is a culture and set of practices that aim to break down the barriers between development and operations teams to improve collaboration and efficiency. It involves automating and streamlining the software development process, from code creation to deployment and beyond.  DevOps is not just a set of tools or processes, but a way of thinking about software development. It’s about creating a culture of collaboration, communication, and continuous improvement. With DevOps, developers and operations teams work together to build, test, and deploy software faster and more reliably. 

Additionally, DevOps promotes collaboration and communication between different teams, which leads to a more efficient and streamlined development process. By breaking down the silos between development and operations teams, everyone is on the same page, working towards the same goal. This results in faster and more reliable releases, as well as overall better quality of the product. In short, DevOps is a time-saving and collaborative approach to software development that ultimately leads to better outcomes for everyone involved. 

Read more

https://redd.it/120kp5c
@r_devops

iniciante

Tenho interesse em entrar na faculdade porém gostaria de saber qual o melhor curso pra quem deseja aprender a programar do zero e sair no fim do curso com uma boa bagagem da área de tecnologia

https://redd.it/120hl1x
@r_devops

Need some advice on my infrastructure and ci/cd

First of all I am not an expert in devops, with that said this is the idea I have in mind for a b2b software I am involved in, the software has the following components

A sql database for each customer, giving a dedicated database (just database not server) makes it a lot easier to scale and the entire system won't slow down when one user abuses it, also gives me time to analyze bottlenecks when it happens since it won't impact everyone at once

About 3-7 web applications per client, some of them are ssr (sveltekit) and will run on either cloud run (gcp) or cloudflare workers, no big server side stuff just loading data and rendering pages

An api server that has an api, websocket server and probably a redis instance for some caching (stuff like logs go into redis before being flushed to sql, I can put it on the same server and not worry about persistence because I don't care about it since it's only for stuff like logs and gets flushed frequently)

All of that is per customer, there are some industry standards so giving everyone their own environment makes sense, also they will charge a couple hundred dollars a month so paying 20-30 for hosting is not a big deal

So when it comes to deploying I need to do the following steps

1. Generate the static web applications and deploy to the correct instances

2. Deploy the ssr web applications

3. Migrate the database

4. Deploy the api server

For testing (only when there are changes pushed) I have the following in mind

1. Deploy in a staging environment and create the database migrations on a populated database

2. Run all unit tests (could be performed in a simple github action)

3. Run all integration tests in the staging environment this includes using the client applications to interact with the api just like in prod, I am also including some tests to mock offline behavior

4. In this industry reliability is important so I am designing a stress test, it will consist of a staging environment simulating thousands of request basically mocking a days worth of activity (or maybe just a few hours) and log the results (speed, percentage of failed requests) and then evaluate based on the percentage, ideally I would automate this step but I think that it would be better if I could create an excel sheet or something and check the results before approving the commit, for example if all failed requests come from something trivial the team may choose to bypass it, in any case this is how I am planning it

One thing I haven't figured out yet is how to store sensitive info, each customer has a couple hundred variables like settings that should be stored in a central application, the problem is with stuff like database passwords I am not sure how to store it, I would just store it in a sql database (maybe encrypt it) and restrict access to that database but that doesn't seem right, I was looking into gcp secret manager but I don't wanna lock myself into an ecosystem

Additionally the only way I know how to trigger automatic builds is via a github webhook but I am sure it's possible to just forward the webhook to my own server that could in turn start the deploy process for each environment keeping in mind some restrictions that users might have enabled (like no changes during business hours) I also need to be able to do that from an internal admin tool

I have no idea if any of this makes sense or not, also I can write all of that in code but I am sure there are existing solutions, that could help me out I am just not aware of it

I know in the long run this project will require a full time devops person or team but we are not public yet and at this point I am just trying to get an mvp going without building uo technical debt

Any guidance will be appreciated, I am also willing to invest time to learn so if I am completely off with this whole approach please nudge me in the right direction

https://redd.it/1229frj
@r_devops

Meetup: ¿Todavía no has oido hablar de DevOps?

Es este MeetUp os ayudaremos a entender la cultura DevOps y su uso dentro de los proyectos.
enlace del meetup

https://redd.it/120hkwc
@r_devops

Is it necessary to maintain a logical layer on top of your codebase?

## Background

Although Git is a widely used platform for version control and collaboration, it does not have the capability to analyze and interpret the logic of code.

Assuming my program now wants to know what is on line 35 of the extract_test.go file. If using GitLab, it is possible to find the corresponding text information, which may look like this:

func TestExtractString(t testing.T) {
fileResult, err := ExtractFromString(javaCodeForExtract, &ExtractConfig{
LangType: core.LangJava,
ExtractType: extractor.TypeExtractFunction,
})
if err != nil {
panic(err)
}
for _, each := range fileResult.Units {
core.Log.Debugf("result: %s", each.GetDesc())
}
}

However, my program does not know what this text represents. This is even more difficult for programs written in other languages (such as Java or Python).

Becides, there are many tools require extracting information from source code for their secondary development. They have to re-implement the extractor once and once again. It's a huge cost.

## What we do

As a member of the infra team, we have decided to add an additional layer of logic to the code repository, so that all third-party tools can access the logic in the code repository using our unified API, reducing their costs in compatibility and language parsing.

​

With our API, they can easily extract metadata of different languages from code files without the need for any additional parsers to be written.

{
"repo_id": "sibyl2",
"rev_hash": "e995ef44372a93394199ea837b1e2eed375a71a0",
"path": "extract_test.go",
"signature": "sibyl2||TestExtractString|testing.T|",
"tags": ,
"func": {
"name": "TestExtractString",
"receiver": "",
"namespace": "sibyl2",
"parameters":
{
"type": "*testing.T",
"name": "t"
}
,
"returns": null,
"span": {
"start": {
"row": {
"$numberLong": "34"
},
"column": {
"$numberLong": "0"
}
},
"end": {
"row": {
"$numberLong": "45"
},
"column": {
"$numberLong": "1"
}
}
},
"extras": {},
"lang": "GOLANG"
}
}

By using MongoDB, we have opened up the data we collect to our data warehouse, which makes it possible to analyze data at the warehouse level. Users can easily analyze the repository situation of a specific version, or even between versions, based on this.

## Discussion

We have already given a speech on QCON 2023 in Beijing, China.

It is indeed able to meet the needs of some Chinese teams and has received some praise. But I'm not sure if this is a suitable and universal solution.

Feel free to leave your comments :)

## Others

This is not strictly speaking a promotion, but if you're interested, this is our GitHub link.

https://github.com/opensibyl/sibyl2

Thanks!

https://redd.it/122f8wu
@r_devops

Is there a tool tracking releases of infrastructures/cncf projects?

Recently, as the product grows, more infrastructures are being added to existed tool stacks, since we are a very small team and right now most of the infrastructures' version are not synchronized, I was quite scared of missing out big updates, then I suggested my team start having "release reviews" each quarter to see what have been updated or deprecated and discuss on whether to upgrade.
I know that to start implementing such routine, we'll need to start tracking releases of tools and compare with current version in used, luckily ArgoCD supports sourcing from different repos right now, so I put all the infrastructure together with application sets and wrote a tiny script that basically runs like this.


1. project repo, name, category, current version in used and latest stable release are manually listed inside a google sheet document
2. script pulls from sheets and getting information either from github or artifact hub
3. if versions are newer or different, it will then be updated back to the sheet and a slack notification will be sent to the channel tracking releases

​

I was wondering if there's any tool that works similarly to this, or is this even a good idea, because I cannot find tools that work like this, so I thought maybe people don't have this type of problem?


As I wish to contribute to the open source community, lots of projects that I'm familiar with are beyond my technical level since I don't have many years of experience in coding, so I guess this is a good place to start.


What do you think? How would you tackle this problem?

https://redd.it/122g5zs
@r_devops

Kubernetes Operations Survey

Hello, y'all!

I’ve started a new Kubernetes operations survey. The goal is to better understand so-called "2nd day operations" with Kubernetes clusters e.g. upgrades, maintenances, disaster recovery, etc.

I've already posted it on r/kubernetes, but I want to collect as many responses as possible.

You can check the results of the previous year’s survey here.

P.S. Last year, some folks suggested that Google forms have flaws, which is true. However, I don’t collect your email - I don’t care about that. This is purely my personal initiative. Also, it looks like Google Forms is the only free tool that is good enough for both form creation and some basic graph building, as well as it’s easy to use. Yet, if you have any suggestions on what can replace it - I would be happy to check those alternatives!

https://redd.it/122jopm
@r_devops

Redirect Ingress GKE trafic to an external domain

Hello,

We do have few endpoints that are too critical to be handled by our monolith app (even if it is replicated in k8s). we thought we could outsource them to cloud run and redirect traffic depending on the path to cloud run or the backed carried by k8s. like this

​

apiVersion: networking.k8s.io/v1
kind: Ingress
spec:
defaultBackend:
service:
name: monolith
port:
number: 80
rules:
- http:
paths:
- backend:
REDIRECTTO: "HOSTNAMECLOUDURUN.a.run.app"
path: /serverless
pathType: ImplementationSpecific

Each ingress rule is expecting a "service" backend.

The issue is a cloud run hostname is not a service, but a domain like https://serverless-xxxxx.a.run.app/

I just need to forward a certain path in my ingress to the hostname of our cloud run.

PS: I can do that via GCP console and update the load balancer, but this doesn't seem a good idea as it will be overwritten by k8s

Best regards

https://redd.it/122kfd4
@r_devops

Is it inevitable every DevOps job is high stress, over-worked?

Admittedly, my current company has a shit-ton of technical debts, no processes and crap management.

But how much better can it get? I'm wondering if a career in a company that doesn't have it's own app (cloud work, modern sysadmin if you like) might be a better bet (lower paid but less stress). I mean, there's a reason why the average tenure in a DevOps position is barely 1.5 years.

https://redd.it/122pq29
@r_devops

Learning about cloud security?

How do you learn about cloud security?

Are there some blogs or YouTube channels that are particularly helpful?

I tried searching but found that there are a lot of channels for cyber security but none for cloud security.

By cloud security, I mean Docker, k8s best practices, how to store passwords in a docker container, common pitfalls, etc.

https://redd.it/122p71b
@r_devops

Is it true that if you don't have proper grip over programming language you can't be successful in devops?

Is it a misconception or is it true that you don't need to know programming language to be successful in devops

https://redd.it/122sf0i
@r_devops

How do you get the list of successful builds in the "CD/Deploy" job from the "CI/Build" job in Jenkins Declarative Pipeline?

I am planning to keep the build multi-branch pipeline separate from the deployment pipelines (one pipeline each for pre-prod and production environments).

I wanted to list all the successful builds of the "main" branch of the build multi-branch pipeline in the deployment jobs so that the "Deployer" can select one of them (latest build during a "normal" deployment and one of the previous builds in case of a rollback).

In particular, the build pipeline pushes a docker image to JFrog artifactory. I want to list the image tags of successful builds that can be deployed. Deployment is done by using `helm upgrade --install` which accepts image tag as one of the options.

I am unable to find any resource that tells how this can be achieved in declarative pipeline.

Any suggestion would be extremely helpful!

https://redd.it/122scro
@r_devops

DevOps Junior

Hello everyone, I recently got a job as a junior devops, I had never worked in IT before and I would like to know as a senior devops what is expected of me in a period of six months.

context about me 2 aws certifications solutions architect and sysops certifications and some projects related to terraform, aws and python.

https://redd.it/122rnrf
@r_devops