Cli tool or library for obfuscating sensitive data

Hi,

Does anyone knows a library for obfuscating and masking the usernames and password inside of text file content?

Like Gitlab Ci does it, masking all sensitive content

I don’t care about the language in which it is written( hope it is not .net,lol)

Basically, want to use it as a post processing step in one of my own devops tools , which gonna save all the commands from bash history

Update:
Found new tool myself, teller. Need to try it though, they say it can remove sensitive data and manage secrets

https://redd.it/n4k3j3
@r_devops

DevOps team : how to grow and move forward

Hi All,

We have around 10 DevOps engineers , mostly working on their own projects inside same company, for different clients

Just to simply, 1-2 engineer per a project

Head count growth steadily with like +3-5 engineers a year

There are some overlaps in technologies between projects, but no more than 40%. It is almost impossible to unify them, due to different domains( finance, healthcare, ai and etc)
Project of different duration, scale and complexity

I do work as someone like a ‘team lead’ in our devops chapter, covering some of the onboarding , mentoring and other responsibilities. 80% hands-on , 20% on org things

As we work all on different projects, we have a bit weak cross project/team knowledge sharing and oftenly implement same stuff with different approaches
Ive heard from many of colleagues of same company size ( about 200peolle) about the same things. Mostly all companies under <1k people have it to certain extend. Learning and sharing requires the whole framework and long term investment

It is easier when u work inside same project, then there is a clear benefit and interest in continuous development & sharing

But when you work separately and shared some piece with someone, there is always a drift when project continues to grow and someone has to maintain it

We done some small internal workshops in past, like open ended discussion on technologies and how they are used in projects & potential opportunities
But it is not always clear of which benefits we get out of it and how it helps us. Clear part is that we know what people are working on and bit diverse their typical working routine


Maybe someone was in our situation.
What to do strategically to make whole thing more standard?
Are tips and trick for short and long term success?
How you managed to solve the cross team communication?
Is there anything we can do now to build a good foundation?
Anything that we should look at in first turn?
Does anyone have a success story they can share?

Best Regards,
Vlad

https://redd.it/n4hfk8
@r_devops

Deployment Config vs Deployment

Hello,

I am new to Openshift, we have tasks of deploying few containerized applications onto the Openshift 4? I like to understand from the experts here, what’s are the differences between the Kubernetes API kind deployment vs Openshift deployment config? Openshift supports both deployment and deployment config? If yes, which is the recommended way of deploying apps?

Thanks for the help

https://redd.it/n4fsod
@r_devops

Looking for light weight kubernetes alternative for local development and learning

Hey. I wanted to check out the world of devops so over the past few weeks I've dockerized a simple application and got it running in microk8s with a nodePort for accessing it. I use a dell t30 and prior to using kubernetes I've never pushed this tower to 3% cpu usage, but kubernetes running idle / just my simple hello world image brought the average up to 15% with frequent peaks over 25%, much of that being just a process named kube-apiserver.

I don't use this home server for much but I don't think I want to let kubernetes consume most of it. I'm new to the space so pardon my asking - is there even an alternative for what I'm asking? Or once you enter this space, do all similar technologies turn that host machine into a container manager without much room to do much else besides manage the cluster? I am reading about nomad now as a simple alternative to kubernetes - simple sounds good for my interest in learning, but will I see the same insane overhead / cpu usage?

My needs are very simple, I'm trying to make my website and other small projects easier to manage, upgrade, redeploy, etc and do so in a modern way. Currently my website is a compiled golang binary living in a tmux pane! I'm also of the old mindset that handing my application over to docker and handing that docker image to kubernetes will continually add overhead and performance degredation. I'm willing to try it and see for myself - although I suspect doing all of this on a single local machine will have drawbacks because it's not what the tech is meant for.

So, tldr, are there any container (or non container) managers solving a similar goal but on a smaller scale or at least with a smaller footprint?

https://redd.it/n4w6dp
@r_devops

Guide on migrating off a self-managed Gitlab instance to gitlab.com

We recently migrated from a self-managed Gitlab instance to gitlab.com. The system administrator of the self-managed Gitlab instance said this would simply entail a git pull && git push and that the migration will be done quickly - depending on your usage of Gitlab, this is either a naïve oversimplification or straight forward dangerous. The truth is more nuanced and entails quite a bit more work. Since the migration took us a couple of full working days and we wrote some reusable checklists and code in the process, we have quickly jotted these down. Maybe somebody else can also make use of it.

https://redd.it/n4sz23
@r_devops

Does Nexus Repository OSS 3.30.0-01 support Capabilities via Rest API

Hi,
I was able to successfully create a repository through Rest API in Nexus Repository OSS 3.30.0-01. But couldn’t find any API references to create capabilities for that particular repository under IQ: Audit and Quarantine wasn’t able to find any Rest API from the documentation.

Any help will be appreciated.
Thank you.

https://redd.it/n4rfcn
@r_devops

Interactive map of Google Cloud Services

Hi guys, I created this interactive map to help get an overview of Google Cloud Platform.

Interactive map of Google Cloud Services

Might come in handy for anyone working towards a certification, or for anyone working with GCP from day to day.

I am now working on a similar map covering Kubernetes, and I was wondering if you had any specific topics that you would like to see covered? 🥳

https://redd.it/n4qbs2
@r_devops

Where to start

Hello. I just graduated with a B.A. in computer science and am considering the DevOps route. Where should I start my focus, also what are some important key pointers for beginners? Thanks in advance!

https://redd.it/n50150
@r_devops

Transition to DevOps without getting burned

Guys how's it going
As the title says i'm looking to make a transition to DevOps role, but i'm burned out haha

I have a solid background as a Sysadmin using Linux, docker, AWS, bash..and also I have my CCNA
The thing is i'm trying to learn a lot of techs at the same time and it's frying my brain

Last night i've stayed up to 1am (after 9hs of work) with Python/K8s/Ansible/Terraform.. and I got completely ruined.. and got nothing out of it

So i guess my question would be, what to learn next? Python? K8s? More Cloud? Terraform? CI/CD?
Tere are so much things that I honestly don't know where to begin and focus on.

I'm 35 now, and I want to keep learning, but i feel completely stuck.

Thanks!

https://redd.it/n4zroe
@r_devops

Web development - Smooth transition or clean cut?

Hi everyone,

I am in a kind of "luxury situation" and would appreciate different opinions on my situation in order to make a sound decision.

I worked as a sneior it project manager at a big e-commerce company and right now I am doing a full-time web developer coding bootcamp, which is ending in around 5 weeks. I am having an offer from my old company to start again as a senior it project manager. My ultimate long-term goal would be to work remotely as ruby/JS backend developer.

Should I take the job and transition slowly (doing code wars and own projects on the side), advancing my skills only in my freetime and search a new job on the side or do a clean cut and search a jr dev. position (probably earning 40% less money for the next years) and having more time to focus on advancing my skills as my day-to-day job?

Thanks for your support girls and guys :)

https://redd.it/n4z4eo
@r_devops

What and How each stream works?

Can you ELI5 what each service/framework does that's bolded?

For data ingestion, you write and build a piece of code in IDE (Gradle? is used and Artifactory? is one of the confgs setup for Gradle, Metorikku and DTSv3 for a version config), you commit this code to Stash, use Bamboo to do CI/CD, create a keytab, conf.json, and generate dts credential on a terminal, copy this json file to s3 bucket, and finally you trigger on Airflow?

https://redd.it/n5cln3
@r_devops

Running Jenkins and Gitea itself as container managed by Kubernetes or locally on a server?

Dear Community,

**Fix assumption:** I have a RHEL 7 or 8 server (physically) to setup some CICD Tools. (I know that there are better operating systems for my use case, at least following opinions in certain blogs.)

**Goal:** Setup an experimental DevOps environment with the goal to gather experience to setup in a distant future a real DevOps environment for a small team. I want to use the following tools:

* Gitea
* Jenkins
* tests, deployments etc. are run in pods using a container service and kubernetes to orchestrate the pods containing the containers

**Question:** There are a lot of guides telling you to run Gitea and Jenkins themself as containerized application inside a kubernetes cluster. I would like to understand why and the pro and cons. So which of them should be run as container inside a pod (Gitea, Jenkins, both?)? Why and why maybe not.

**Thoughts:** The probably major factor to run those applications in pods is that the system becomes more resilient. Disadvantage could be that it is more difficult to deal with persistency and consistency of databases and storage. I also already started a thread towards this topic where I also added as comment to some answers this question: [https://www.reddit.com/r/devops/comments/mw6jp7/setting\_up\_cicd\_git/](https://www.reddit.com/r/devops/comments/mw6jp7/setting_up_cicd_git/)

&#x200B;

I appreciate all your help and thank you very much for you help, time and considerations.

https://redd.it/n5dnqa
@r_devops

Carbon cost of infra-as-code

I've been toying with the idea of showing carbon emission estimates as part of the free/open source Infracost CLI tool for Terraform projects.

I've seen estimates mention that data centers consume around 1% of the global electric supply [1\] and this could increase to between 3-13% by 2030 [2\]. The wider ICT ecosystem accounts for 2% of the world's carbon emissions, putting it on par with the entire aviation industry [3\].

It seems like it might be possible to show "carbon costs" for basic compute (ec2), storage (s3) and data transfer but not easy for services that build on top of these raw primitives, e.g. DynamoDB. However, I'm wondering if people would find that helpful, or if it would change anything about "cloud waste"? That waste is estimated to be around $17bn out of the $50bn that was spent on IaaS in 2020 [4\]. The main causes of the waste are idle resources and over-provisioned resources so maybe if devops/SREs/devs have the carbon costs, they can incentivize people to use those resources more efficiently? Anyone seen infra carbon costs in their organization's carbon accounting reports?

1. https://www.iea.org/reports/data-centres-and-data-transmission-networks
2. https://www.mdpi.com/2078-1547/6/1/117
3. https://www.nature.com/articles/d41586-018-06610-y
4. https://www.gartner.com/en/newsroom/press-releases/2019-11-13-gartner-forecasts-worldwide-public-cloud-revenue-to-grow-17-percent-in-2020

https://redd.it/n5huzs
@r_devops

HIRING Kubernetes Administrator - London

I am currently recruiting for a new Kubernetes Administrator position with a Gartner Magic Quadrant group building massive scale data storage tech.

The team have grown their UK tech team to over one hundred people since opening last year (as part of a large global tech group) and are building out a large and varied Operations and Reliability Engineering group, focused on container growth as part of a large Kubernetes/OpenShift project. They are looking for those who enjoy working with Kubernetes and had proven record building clusters and supporting wider team with use of those technology.

The team are based near St Pauls in more normal circumstances (currently fully remote with flexible post Covid) and we can look at salaries from mid level all the way upto £110,000 plus bonus, pension and private health package.

For more information:

💻 drop me a message on LinkedIn
📩 [email protected]
📞 01727225558

https://redd.it/n5j5x6
@r_devops

NPM+NODEJS

Hello guys, hope all of you are doing well, yesterday my team lead asked me to do a small session about NPM, explain its purpose and how can we use it as a DevOps engineer, knowing that I have a piece of good knowledge in javascript (basic staffs such function, oop, etc) but I have never used npm, so my question is as a DevOps engineer why we need to learn node js & npm, and where we can use them?

Thanks.

https://redd.it/n5mm07
@r_devops

Choosing proper tool for infrastructure/servers state validation

Hi! We are small devops team deploying openshfit/k8s clusters. We need some tool to validate cluster state, e.g. if k8s API is accessible, image registry is routed and so on. Potentially we might have more devops joining so we want everything as a code, so everyone could run tests and see if a system diverges.

&#x200B;

I choose between:

1. chef inspec \- Pros - I like the syntax and many out of the box features. And I don't mind to write Ruby DSL as well. Cons - if chef still a thing? Seems like they've dropped opensource support. Also installation footprint seems a bit overkill for us ( will require ruby or maybe other dependencies ).
2. goss \- Pros - One binary install as it's written on go, so easy to deploy. Cons - I am not a fan of YAML DSL / coding, also seems like goss does not show a command's stdout in it's reports which I consider a significant flaw. According to the latest commits date, the project seems a bit abandoned, at least not actively maintained.
3. write our own solution. Using python/bash/whatever. Pros - maximum flexibility. Cons - It'd take some time and efforts. I don't want reinvent the wheel if a tool I really like exists.

https://redd.it/n5ozee
@r_devops

What is the difference between devops and SRE?

Dear colleagues.

What is the difference between devops and SRE?
Could you please provide an example?

Thanks in advance!

https://redd.it/n5xfix
@r_devops

Question about moving puppet infrastructure to docker

We use Jenkins to setup puppet infrastructure and install product. There are many 3 components involved. Puppetserver 6.x, Jenkins and Nginx acting as package manager. If this setup to be converted, what is best approach? like clubbing Jenkins and puppet server in on image OR seperate? Nginx will be a separate container.

https://redd.it/n5zzw2
@r_devops

Help required to setup vault with RAFT HA and database storage backend.

I am trying to setup Hashicorp Vault with raft as high availability and postgres as storage backend with TLS enabled. The only problem I'm facing at the moment is that, I am unable to join the various vault nodes into the raft HA cluster.

I'm running vault on docker [ the three nodes are a part of the same docker network \] and used openssl to generate a self-signed certificate to test the TLS setup.

This is my vault.hcl

hastorage "raft" {
path = "/vault/file/"
nodeid = "vault3"
}

storage "postgresql" {
connectionurl = "postgres://<username>:<password>@postgres:5432/<dbname>?sslmode=disable"
}

listener "tcp" {
address = "0.0.0.0:8220"
tlscertfile = "/etc/certs/kms.crt"
tlskeyfile = "/etc/certs/kms.key"
}

defaultleasettl = "2208h"
maxleasettl = "4320h"
disablemlock = true
ui = true
clusteraddr = "https://vault3:8221"
apiaddr = "https://vault3:8220"

The first node, upon unseal and initialization, joins itself to a new raft cluster.
The second, which is unsealed using the keys generated upon init of the first node, goes into standby mode. When I try to join the second node into the raft cluster of the first node, I get the following error :

vault operator raft join -leader-client-cert=/etc/certs/kms.crt -leader-client-key=/etc/certs/kms.key
I also used the -client-cert and -client-key options, same error

core: attempting to join possible raft leader node: leaderaddr=https://vault1:8200
vault1 [INFO] http: TLS handshake error from 172.25.0.6:39286: remote error: tls: bad certificate

vault2 WARN core: join attempt failed: error="error during raft bootstrap init call: Put "https://vault1:8200/v1/sys/storage/raft/bootstrap/challenge": x509: certificate is not valid for any names, but wanted to match vault1"

vault2 [ERROR] core: failed to join raft cluster: error="failed to join any raft leader node"

I recreated the certificate with vault\1 as the FQDN, this gives me the following error :


core: attempting to join possible raft leader node: leaderaddr=https://vault1:8200

vault2 [WARN] core: join attempt failed: error="error during raft bootstrap init call: Put "https://vault1:8200/v1/sys/storage/raft/bootstrap/challenge": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0"

vault2 [ERROR] core: failed to join raft cluster: error="failed to join any raft leader node"
vault1 INFO http: TLS handshake error from 172.18.0.5:47794: remote error: tls: bad certificate

I set the environment variable GODEBUG=x509ignoreCN=0, it didn't fix anything.

Any help would be much appreciated!

https://redd.it/n5znsz
@r_devops

Test API of docker container in Azure DevOps CI/CD pipeline

Hi!

I'm working on setting up some ci/cd pipelines for a couple of small containers. The pipeline should be as follow:

1. Build docker image
2. Start container
3. Query the REST api of said container
4. Make sure the response is "reasonable"
5. Push to ACR
6. Deploy to AKS

It's number 3 and 4 that I'm struggling with. It seems kinda basic but I haven't found any good resources online. I'm new to DevOps and I'm guessing I'm just googling the wrong terms, as this sounds like a basic and standard thing one would do in a pipeline. One way, I guess, would be to just docker run the container, curl it with a bash command, regex the response and run exit if the response contains "error". But I'm thinking there's probably a prettier solution out there.

Any suggestions or references to online resources would be highly appreciated!

https://redd.it/n63w2v
@r_devops

I developed a tool to train neural networks on AWS with a single command

Hey everyone,

My friend and I developed Nimbo, a dead-simple CLI that wraps AWS CLI, allowing you to run code on AWS as if you were running it locally. GitHub: https://github.com/nimbo-sh/nimbo. Docs: https://docs.nimbo.sh.

We decided to build this because we were frustrated with how cumbersome using AWS was, and we just wanted to be able to run jobs on AWS as easily as we run them locally. All in all, we didn't like the current AWS DevOps user experience, and we thought we could drastically simplify it for the machine learning/scientific computing niche.

For this reason, we also provide many useful commands to make it faster and easier to work with AWS, such as one-command Jupyter notebooks on EC2, easily checking prices, logging onto an instance, or syncing data to/from S3 (you can see some useful commands here).

Unlike other similar services, we are solely client-side, meaning that the code runs on your EC2 instances and data is stored in your S3 buckets (we don't have a server; all the infrastructure orchestration happens in the Nimbo package).

We have tons of ideas for Nimbo, such as docker support and one-command neural network deployments.
.

We are happy to receive any feedback and suggestions you have.

https://redd.it/n6486v
@r_devops