Greenmask release v0.1.10. Database anonymization tool

A [new release](https://github.com/GreenmaskIO/greenmask/releases/tag/v0.1.10) introduces improvements and bug fixes.

[https://github.com/GreenmaskIO/greenmask](https://github.com/GreenmaskIO/greenmask)

Changes:

* Fixed panic caused in `RandomString` transformer
* Fixed wrong table size calculation. Now the table size includes TOAST table size
* Added custom transformer interaction API defaults if not set
* Changed docker workdir to greenmask home
* Removed bucket name from object path prefix

If you are not familiar yet with Greenmask - it is a Database anonymization tool that brings wide anonymization functionalities and techniques. Check out the [Playground page](https://greenmask.io/playground/) to get started

https://redd.it/1c6e37j
@r_devops

Question about Ansible strategies

Hi all!

We've hired an outsourcing company to help us with the automation of a part of our infrastructure. They've been using Ansible and have made a 2000+ lines long playbook (not counting the vars files and the roles that are used in the playbook), that creates 12 different VMs, hardens them, adds roles, DBs, does some configs, etc.

Problem is that, as I said, it's all one huge playbook that does it all. If you want to just do a part of the job (e.g. create one of the VMs) you have to run the playbook with 10+ parameters that are implemented into the playbook and on the backend, the parameters tell Ansible to ignore blocks of the playbook.

Personally, I think that's dumb and we've discussed it, but the company claims that this is the way it has to be done and that we may be loosing flexibility, but we are gaining speed.

Can you advise me on some best practices on the matter? I know it's all a bit vague without the actual playbook, but I just can't believe that it's better to have a huge lump of an unreadable yaml file, as opposed to decoupling it all and having it easier to maintain.

https://redd.it/1c6eloz
@r_devops

Tiron - A new open source automation tool as an alternative to Ansible/Chef/Puppet/Salt

https://github.com/lapce/tiron

A few highlights are: Tiron uses HCL instead of YAML. It has a built in TUI for displaying outputs.

​

https://redd.it/1c6auk8
@r_devops

Kafka UI - For AWS MSK

Hi, I have setup an MSK cluster. But from development perspective MSK does not have any UI to see producers/consumers/topics etc , test pushing messages etc. I can see these from cli but I am looking for a user friendly UI for dev/QA teams.

Is anyone using any UI with kafka (paid or opensource) . I saw kafka-ui (OSS) but had issues connecting it with Okta (+ AD). Has anyone got this setup working? Conduktor seems to provide UI but the setup includes postgres RDS etc, while the solution I am looking for is only for lower environments to be able to test/verify create topics etc.

Any suggestions/pointers would be helpful.

Thanks

https://redd.it/1c6fsaz
@r_devops

What are the recent projects you discovered that you Think will gain a lot of traction now ?

The landscape of devops is biggus hue mungus. Im asking the same question every few months to at least check whats new in areas like:

- cicd

- observability & monitorinh

- application orchestration

- security / compliance

And many other we have to supervise. What are the new OSS and not only projects that you recently had a plasure to work with that greatly improved your work ?

Mostly interested in whats new in the landscape.

https://redd.it/1c6ltf1
@r_devops

How do I mock a Google or AWS EC2 instance?

I want to test some Github action I wrote for deploying a php app, how do I test it without creating a Google or AWS account on my local machine?

https://redd.it/1c6osz7
@r_devops

Ingress Controller Suggestion

I'm looking for a suggestion for an Ingress controller. Currently experimenting native ingress controller in GKE with NEGs and it is very confusing. There are many configurations that are in parallel to K8s way of doing. "BackendConfigs" along with "K8s Services". It comes with its own health checks, etc.. very confusing. I wanted to go with a more "cloud-native" ingress controller. Would really appreciate some recommendations. Few capabilities that are important:

JWT token validation
Sticky sessions, preferably using URL param
CORS
Ability to create multiple Ingresses (or IngressRoute equivalents) behind a single LB

Of course needs to be free & open-source. Thanks!

https://redd.it/1c6qecw
@r_devops

What would the ideal interview process look like?

I don't even know if there is such a thing based on the variability in DevOps roles, responsibilities, expectations, engineering culture, etc.

Nevertheless, it seems clear to me based off my own experiences (on both sides of the table) and from reading the many posts on this sub that most hiring processes aren't efficient at finding talented engineers.

The emphasis on efficiency is critical here. I've been part of some ridiculously lengthy interview processes. Statistically speaking, an 8-hour interview process should be more effective at evaluating a candidate than a 3-hour one, but is it scalable or ethical?

The goal of any interview process should be to evaluate candidates in the least amount of time so that other priorities don't grind to a halt. If you could have an interview process that took 3 days instead of 3 weeks, you would have a much larger pool of candidates. Happily employed candidates don't want to go through a lengthy interview process, but they might if it were very short. Or other candidates who have offers in hand and can't start a fresh 2-3 week process.

In the following section, I'll outline the common practices I've seen used for evaluating the technical chops of an engineer during an interview process. Please note, the focus here is on evaluating the technical skills/knowledge of a candidate (in the most efficient way), there are other factors (culture fit, leadership, etc.) that would inevitably be involved but those are outside the scope of this discussion.

1. Technical screen - Q&A style

This usually happens early in the process with a Principle Engineer.

Pros:

Answers are honest since there isn't time to Google or ask ChatGPT
Showcases candidate's ability to think on their feet
Typically effective at determining specific knowledge

Cons:

Highly dependent on the interviewer
Favors candidates who are confident speakers or good interviewers
Great tech screens don't always mean they're great engineers "on the job"

2. Take-home/offline assignment

I've had large projects where I was paid 8-hours at market rate, but most take-home assignments are designed to be done within 2 hours. Writing Terraform/Ansible to stand up a 3-tier application, writing AWS Config custom rule in Python, diagramming an architecture to support the given requirements, etc.

Pros:

Showcases candidates "on the job" skills (logic, organization, attention to detail, code comments, etc.)
Candidates should be prepared for deep-dive discussions related to the assignment which can alleviate nerves being a factor

Cons:

Have to assume ChatGPT did 95%, puts the burden on the interviewer to find the fakers
Could favor the unemployed candidate who has more time to invest; hard to know if they spent 2 hours like instructed or 8 hours

3. Live exercise

Paired with another engineer they present a problem and you are required to solve it while talking through your solution. Sometimes this is leetcode style, other times this is in a Google doc.

Pros:

Impossible to fake your way through
Candidates that do well should perform well on the job (coding in the given language)

Cons:

Many will do poorly due to the pressure; good engineers could be missed
Depending on the time spent coding day-to-day for typical DevOps roles, might not be the best metric

4. Work trial

This has become more popular in the last few years, but it's still few and far between. Typically 1-2 weeks on-the-job work trial working with the team on some short project.

Pros:

Most effective at determining competency, soft-skills, culture-fit, etc.
Probably the best way to avoid regrettable hires

Cons:

Big time investment, not scalable
Security/IP concerns; malicious candidate could cause harm
Logistical nightmare
Favors unemployed candidate; difficult for an employed candidate to take off 1-2 weeks short notice, and if they don't get hired they've just used a good chunk of their PTO and pissed off their team

Please feel free to add to this list

and I'll make edits.

So, what does the ideal process look like? Is there such a thing as an ideal process that would be widely applicable? If there were an ideal process, would it eventually be reverse-engineered and made obsolete?

Please share your ideas or positive experiences with a focus on efficiency, ethical expectations, and equality.

For the sake of discussion, assume this is for a senior-level position. Feel free to offer suggestions for other levels but I wanted to avoid this derailing about leveling.

https://redd.it/1c6tsas
@r_devops

How do you guys ensure database in production and uat are identical?

My company is facing this issue.

We have many databases in production, using SQL Server and Stored Procedures.

Our operations team may change the database stored procedure in production during maintenance or an issue.

But usually these dont get implemented in UAT due to human error.

How do u guys handle this?

https://redd.it/1c6vtky
@r_devops

Pointing F5 Big Ip to a Kubernetes Cluster

In my company, we are provisioning a Kubernetes cluster with Kubespray on VMware VMs. The ingress path for this cluster, both for the management API (kubectl and port 6443) and the Ingress for applications, will be handled by F5.

We are struggling since we get:

k get nodes -v=10  
I0404 10:59:25.328555   21011 loader.go:395\] Config loaded from file:  kubeconfig.conf  
I0404 10:59:25.329875   21011 round\_trippers.go:466\] curl -v -XGET  -H "User-Agent: kubectl/v1.29.1 (darwin/arm64) kubernetes/bc401b9" -H "Accept: application/json;g=apidiscovery.k8s.io;v=v2beta1;as=APIGroupDiscoveryList,application/json" 'https://xxxx.xxxxx.es:6443/api?timeout=32s'  
I0404 10:59:25.332888   21011 round\_trippers.go:495\] HTTP Trace: DNS Lookup for xxxx.xxx.es resolved to \[{192.168.xx.xx }\]  
I0404 10:59:55.332089   21011 round\_trippers.go:508\] HTTP Trace: Dial to tcp:192.168.xxx.xxx:6443 failed: dial tcp 192.168.xx.xx:6443: i/o timeout  
I0404 10:59:55.332242   21011 round\_trippers.go:553\] GET https://xxx.xxx.es:6443/api?timeout=32s  in 30002 milliseconds  
I0404 10:59:55.332276   21011 round\_trippers.go:570\] HTTP Statistics: DNSLookup 2 ms Dial 29999 ms TLSHandshake 0 ms Duration 30002 ms  
I0404 10:59:55.332296   21011 round\_trippers.go:577\] Response Headers:  
E0404 10:59:55.333498   21011 memcache.go:265\] couldn't get current server API group list: Get "https://xxx.xxx.es:6443/api?timeout=32s": dial tcp 192.168.xxx.xxx:6443: i/o timeout  
I0404 10:59:55.333532   21011 cached\_discovery.go:120\] skipped caching discovery info due to Get "https://xxx.xxxx.es:6443/api?timeout=32s": dial tcp 192.168.xxx.xxx:6443: i/o timeout

If we try a cURL:

curl -vvvv https://xxx.xx.es:6443  
\*   Trying 192.168.xx.xx:6443...  
\* Connected to xxx.xxx.es (192.168.xxx.xxx) port 6443 (#0)  
\* ALPN, offering h2  
\* ALPN, offering http/1.1  
\*  CAfile: /etc/ssl/certs/ca-certificates.crt  
\*  CApath: /etc/ssl/certs  
\* TLSv1.0 (OUT), TLS header, Certificate Status (22):  
\* TLSv1.3 (OUT), TLS handshake, Client hello (1):  
\* OpenSSL SSL\_connect: Connection reset by peer in connection to xxx.xxx.es:6443   
\* Closing connection 0  
\* TLSv1.0 (OUT), TLS header, Unknown (21):  
\* TLSv1.3 (OUT), TLS alert, decode error (562):  
curl: (35) OpenSSL SSL\_connect: Connection reset by peer in connection to xxx.xxx.es:6443 

We suspect about certificates but appreciate any pointers or documentation you might have.

Note: This F5 is shared among other infrastructure. So installing the f5 k8s controller is not an option for now.

https://redd.it/1c6wt01
@r_devops

Stupid question: k3s identity provider keycloak

Currenlty I have kubectl hooked up to keycloak and works pretty well but now got me thinking I can remove the need for passwords using the cluster api oidc as an identity provider.


my first usecase is to authenticate the standard serviceaccount token on minio, so that minio will validate the token with keycloak and keycloak with the cluster-api? anyone done this?


Then we coudl rolebase, postgres, minio for backups and a few other usecases I have all by serviceaccount token and a role annotation





​

https://redd.it/1c6wrdy
@r_devops

Semantic Release with Maven

I am assigned a task to implement semantic versioning to a maven project at my organization, I am using the semantic-release npm package to achieve the same. So far, I have achieved the desired behavior but one thing is bothering me a lot.

We did not use to follow any commit message standard here and the current pom.xml version is 2.5.

I want semantic release package to detect the current version of pom.xml and prepare the next version accordingly, instead of starting the versioning from scratch. How can I achieve that? I am using Jenkins as a CI agent and .releaserc.js for semantic release configuration. Please let me know the possible solutions, is there any possible way to detect versions from tags either?

https://redd.it/1c6xw61
@r_devops

K8s EKS Learning Path

Hi All

I'm a python dev who has devops experience and don't have k8s experience.

I have experience on whole aws intra including ecs. But no experience on k8s.

I have a deadline to be proficient on k8s in next 20 days for a job offer on devops ( in-company migration)

I know k8s is vast. Is there any way I can learn k8s in this time frame ?

Cloud used here is aws and eks is used. Please share your thoughts on this and please share resources.


I know this is unrealistic timeframe but I have to be proficienct enough to manage clusters if given.

I'm willing to give 6hrs a day for next 25 days to learn.

Kindly give your suggestions.

https://redd.it/1c70vpb
@r_devops

How do I fix these deployment/git workflow issues?

We have a massive and very old .NET project at work. It's codebase is a mess, there's no test suite, it's running on very old tech, and we're startup so don't have the capacity to really change any of this. But, it's our most important service.

Some quick info on our infra, we have `prod`, `uat` and `dev` environments. QA testing is done manually on `uat`.

Our git branches are setup as so:

* `master` branch -> `prod`
* `dev` branch -> `uat`
* PR against `dev` branch -> `dev`

So when we're working on a feature branch, we can deploy to `dev` via a PR, once checks have passed and it's been approved it's merged into the `dev` branch and deployed to `uat`.

**First issue**: this deployment strategy means that if I'm working on feature 1 and deploy to dev, someone else can then work on feature 2 and deploy to `dev`, overriding my feature. And when deployments are 40 minutes, this can be a bit of a blocker (and it means we can't QA test on `dev`).

**Second issue**: if feature 1 and 2 pass code review and goes out to `uat` for QA testing. But feature 2 takes longer to test, or a bug is found, feature 1 is now blocked from going to `prod`. Our temp fix for this is to cherry pick feature 1 onto a "release" branch and merge into `master` (`prod` deployment) but this is causing merge conflicts and is a bit messy

I'm pretty new to DevOps and have been asked to solve these issues. Been reading up on GitFlow and trunk based solutions but can't seem to get around our blockers

https://redd.it/1c71mpm
@r_devops

Best way to store nested json data

What's a better way to store nested jaon data

Ive been using azure tables recently, they fit my needs of storing and recalling data when I need it in powershell/python/js.

I have a system that manages renewals for a suite of products, when a renewal is coming up, I store the renewal data as a Json in a single cell.

Later the rowkey is used as a query parameter and the json is retrieved and rendered.


Over simplification but that's the idea.

I know I could flatten the json and store the data flat, but that is so much manipulation and translation that it doesn't t seem worth it.

Without flattening it, I can't directly query it in the table.

Should I be using a no-sql database to store Json always?

I do not know any no-sql, but I'm willing to start learning if that's really the best path.



https://redd.it/1c72n45
@r_devops

Understanding AWS End of Service Life Is a Key FinOps Responsibility by Marry Henry

Mary Henry from Fairwinds wrote an article about AWS increasing their pricing on EKS extended support, meaning, running out of support Kubernetes versions on EKS.

Never had that on AWS myself, but we ran into it once or twice on Azure, simply because we had no time to upgrade 🫣

AWS wants $0.60 instead of $0.10 per hour in this case. I can't remember we paid anything extra on Azure. That is insane to increase to 6x the price 🤯

https://www.fairwinds.com/blog/understanding-aws-end-of-service-life-is-a-key-finops-responsibility

https://redd.it/1c72l1y
@r_devops

How Jersey Mike's Rebuilt their Infrastructure during COVID

During the COVID pandemic, Jersey Mike’s ran an advertising campaign offering 50% off all sandwiches.

The ad went viral, jumping from an average of 800 orders a day to 80,000+ orders within the first four hours.

Jersey Mike's rebuild their infrastructure to meet high user demand, optimize cloud spend, and reduce the operation workload for their team.

https://www.fullstackexpress.io/p/how-jersey-mikes-rebuilt-infrastructure-covid

https://redd.it/1c723dz
@r_devops

Prometheus instance with data

Hey everyone

Does anybody know a good option for creating a Prometheus instance loaded with sample data for testing?

Thanks!

https://redd.it/1c76hrb
@r_devops

Have we reached a stage of AI/ML models for DevOps related tasks?

Like as simple as providing a solution for certain use cases like provisioning storage capacity as needed or open up n/w ports for a new service that gets spinned up?


I want to see if there are any development out there around data science to deal with day-to-day tasks or even provisioning infrastructure when a use case is provided.


Cheers!

https://redd.it/1c789lc
@r_devops

How is Your Company's Use of AWS Evolving?

Are your companies increasing their use of AWS services, maintaining their current level of usage, or are there instances where projects are being moved back on-premises?

I'm interested in understanding the reasons behind these decisions as well. Whether it's due to cost, security concerns, performance issues, or any other factors....

If you're comfortable sharing, I'd appreciate if you could also mention the industry your company operates in and the scale of your AWS usage. :-)

https://redd.it/1c75bpv
@r_devops