Kubernetes and containers security simulator

Has anyone seen or tried the Kubernetes security simulator created in ControlPlane? It’s an Open Source project (GitHub) providing ready-to-use tooling to deploy a bunch of K8s-based security scenarios in AWS. For now, it has 9 of them created for the CNCF, with different difficulty levels. (And I guess we can expect more coming later.) Really cool to have some fun (at least) or even use for educational purposes.

Recently, we tried it and enjoyed it a lot. Here is an overall impression and full guidance from my teammate through the Seven Seas scenario (it’s an easy one).

Another similar project I recently spotted is the Kubernetes LAN Party by Wiz. Haven’t tried it yet, but any feedback is welcome.

https://redd.it/1bvkasi
@r_devops

I sold my startup because of bugs: I wish I had this serverless repository

I ran a startup for almost 2 years, first with a simple approach of Cron, Queue, Workers.
Then I moved to AWS - EventBridge, SQS, and Lambdas (Vendor Locking)
I wrote an article about the main problem that caused me to close it in the end:
https://nevo.hashnode.dev/i-sold-my-startup-because-of-bugs-i-wish-i-had-this-serverless-repository

https://redd.it/1bvlq2h
@r_devops

breakglass account setup for cloud access? (AWS, GCP or Azure)


Does anyone have an example of a breakglass account setup for cloud access? (AWS, GCP or Azure)

I am looking at what people are implementing with regard to privilege account management (PAM).

The problem is what do you do during an incident or maintenance to temporarily increase permission of a developer in a secure, traceable and auditable way?

https://redd.it/1bvmdto
@r_devops

How do I access a ci value to a golang variable using werf?

To preface I'm not proficient at all in devops and what I need to do is to log the commit hash after a succesful build. Project has a werf.yaml file in which there's this piece of configs:

shell:
beforeInstall:
\- useradd -M -s /bin/bash app
setup:
\- cd /usr/src/app/api
\- go mod tidy -compat=1.17
\- CGO_ENABLED=0 GOOS=linux go install -a -ldflags "-X 'main.buildVersion=${CI_COMMIT_SHA}


the go install -ldflags works fine with a fixed value (e.g. it sets buildVersion to "1.0.0" with main.buildVersion="1.0.0") but I cant figure out how to access the CI_COMMIT_SHA gitlab variable.
How can I achieve that? Or what kind of documentation I should dig into?
Sorry in advance for poor description as I said I'm not the best at devops or technical stuff in general.

https://redd.it/1bvobgm
@r_devops

Preproduction postgres database best practices

Hi,

I am in a small team of development. In order to test new releases before going production we need to link our preproduction server to a preproduction database with fresh data (we fill prod db with frequent cron jobs requesting third party services).

The easiest solution I have found yet is to clone my postgres db when I need to test but I find it quite tedious. Are there better ways of doing this and what are the best practices in general regarding preproduction databases ?

Thanks :)

https://redd.it/1bvonu7
@r_devops

Image process server benchmark

The reproducible showdown between Thumbor, imagor, and imgproxy.

https://redd.it/1bvqpfh
@r_devops

Is anyone using Contour ingress controller?

Hey folks. So I'm reading about Envoy and I came across Contour project. As you might know it's an ingress controller based of Envoy.

My question is, is anyone using it? What are your experiences about this project?

I'm asking because I want to use Envoy as a load balancer (L4 & L7) and also want to migrate from NGINX ingress to Contour.

https://redd.it/1bvsym4
@r_devops

Is it possible to remove a pod from a service, let it finish its process, and then delete it?

Hi

We have ProxySQL instances deployed on the K8S, when we want to make some changes in the ProxySQLs, by deleting the older pods and bringing the new ones, the application takes a while to realize the older connection is not usable so it has to create a new connection, this will cause some errors and spike on CPU usage on the application.

Connections to the ProxySQL have a lifetime, so it will get closed after some time, I wonder if it is possible to do this scenario when we want to update the configs on the ProxySQL:

1. Remove the pod from the service, prevent it from accepting new connections

2. Wait for the ProxySQL pod to handle the existing connection and when all connections are closed, delete the pod

For the second phase, maybe a sidecar can realize the state of the current number of connections, but I have no idea about the rest of the process


Is it possible to do so in Kubernetes?

https://redd.it/1bvujwp
@r_devops

Is this possible in terraform? ("anonymous blocks"?)

I like using the `terraform-aws-modules/security-group/aws module`, but I'm wondering if a definition like this:

module "test_sg" {
source = "terraform-aws-modules/security-group/aws"
name = "test-sg"
vpc_id = "vpc-123"


ingress_with_source_security_group_id = [
{
rule = "postgresql-tcp"
source_security_group_id = "sg-0adf3535bed76a6c6"
},
{
rule = "postgresql-tcp"
source_security_group_id = "sg-0c0bd45fd61a70af2"
}
]
}


Can be somehow expressed with a for-loop or dynamic loop for each `source_security_group_id`, even though the blocks don't have "names". I'm hoping to do something like this, but don't see if that's possible:

locals {
sgs = ["sg-0adf3535bed76a6c6", "sg-0c0bd45fd61a70af2"]
}


# tfsec:ignore:aws-ec2-no-public-egress-sgr
module "test_sg" {
source = "terraform-aws-modules/security-group/aws"
name = "test-sg"
vpc_id = "vpc-123"

ingress_with_source_security_group_id = [
dynamic {
for_each = local.sgs
content {
rule = "postgres-tcp"
source_security_group_id = each.key
}
}
]
}

https://redd.it/1bvufx3
@r_devops

How do you get over guilt from letting people go?

Hi all, different question to usual I expect. Maybe the wrong place to ask but it's my role, cut a long story short recently I worked with someone as a senior to onboard them.

I pretty much did as much as I could but they were hire as a mid when really they were a junior. We went through probation etc tried to get them on the ball but in the end it never worked out and they had to go.

I just feel like I could've done more to help, I feel awful especially with a tough market the person made so much effort just didn't have the skills.

I've never been involved to this level of hire/fire and I feel fucking rotten that they've lost their job.

Is it normal to feel this way or is it just the way it is?

https://redd.it/1bvwhsr
@r_devops

Terraform, Ansible and GitHub Action: best practice?

Hello,

I am having some doubts on where are the boundary between Terraform, Ansible and GitHub Actions...

I am quite new to the all CI-CD pipeline, therefore please be patient with me...

I have a small webapp that uses AWS EC2, route 53, S3.

within the EC2, I have installed two containers (not using the EKS services, but just pulled them form Dockers):

one for the DB (it is a containers with PostgreSQL)
one for the Web App (API developed in JAVA Spring boot)

my understanding is that I shoucl use the following tools:

Terraform to create the infrastructure:
S3
EC2
Certificate
A record in route 53
Ansible to create the basic app environment:
load static files in S3
load PostgreSQL docker
load Java runtime docker
Git Hub Action to upload the code in the two dockers

I struggle to understand if this is ok, or because of the overlapping between Ansible and GitHub Action, I should drop one or the other...

suggestions, please?

​

​

https://redd.it/1bvu4t2
@r_devops

Applying to Senior/Lead roles

Howdy,
I am a senior DevOps engineer and have been in senior / team lead roles for about 5 years. I find myself looking to change jobs, and I guess I'm just not used to this competitive job market.

I'm confident that I do good work and that I'm a good hire, but I cannot seem to get even a single interview, even just a simple HR screening. I am tailoring resumes, writing cover letters, I've lost count of how many applications. It's a bit exhausting, to be honest. I'm not too sure what I am doing wrong.

Am I literally supposed to just include every single buzzword and technology I ethically can from the job posting? Anyone else experiencing this? Any advice?

https://redd.it/1bvzhsf
@r_devops

How to get to Python in an MS shop

Hey all- I’m a looking to get some experience in Python, and I’d like to do so on the job. We do almost everything in MS: .NET, Windows, Azure, Azure DevOps, etc. All our scripting is currently done in PowerShell. We haven’t done anything too sophisticated yet, but I have a hard time coming up with a scenario where Python would be better over PS, especially in Azure. Any clear ways forward here? I’m thinking about just writing some and letting people complain after the fact, but I’d love to have a good reason why besides resumé builder.

https://redd.it/1bw0vd7
@r_devops

Senior in Bay Area Onsite?

I'm coming to an end of my contract job. I can move from small town to family guesthouse in Bay Area / Silicon Valley. Remote jobs are very competitive right now. How is the on-site market for a Senior Devops / SRE in Bay area?

https://redd.it/1bw2nnb
@r_devops

Why does DevOps leadership suck so much?

Granted, there are a few edge cases where management / directorship over a DevOps group knows what they’re doing, or they know enough to know what they don’t know and allow the engineers to influence project planning and priority. If you work for either of these two, count yourself lucky because the rest of us are being directed by idiots who haven’t the slightest clue what CI/CD means or the fact that I can’t produce B without first producing A.

https://redd.it/1bw4cuy
@r_devops

How do you make a Github repo just to test things out on Jenkins?

I don't want to make an API project or a library just to test out Jenkins. What do people do? I tend to create github projects to make small projects to use as a prototype, but I am not sure what I should do to do some "prototyping" with Jenkins if that makes sense.

https://redd.it/1bw679u
@r_devops

Deploying to unknown infrastructure

I have a “web application” that consists of multiple angular projects, multiple .NET APIs, at least one auth server, and at least one database. My experience is mostly on the dev side, although I have done some ops and networking, I'm open to learn anything.

I need to deliver it to multiple customers, in that essentially they will host it on whatever infrastructure they have, ranging from an IIS or Linux machine in the back office all the way to their own private cloud. We can set some baseline requirement if need be..

Currently, the code is building container images, and the auth server has a container image.

What’s the best way to go about building something that will be as universal and still simple for customers to get up and running?

Can I package up all my containers in one “package” that they can easily spin up all with the correct configurations?

How do I handle certificates, and url routes?


https://redd.it/1bw3nar
@r_devops

What do you love about your job and DevOps?

Recently I’ve seen a lot of negative posts regarding redditors and their DevOps job. Let’s flip it around:

r/DevOps, what do you love about your job?

https://redd.it/1bwa2ki
@r_devops

Navigating DevOps in a Startup Jungle

I'm reaching out to you for some detailed advice and guidance regarding a project I've been spearheading at my current startup. As someone relatively new to DevOps, and without direct senior guidance, I've been navigating this journey largely on my own, striving to learn and implement best practices along the way.
When I first joined the company, there was a noticeable gap in knowledge regarding cloud infrastructure, containerization, and deployment practices among the development team. Recognizing the need for improvement, I took it upon myself to understand our objectives and address immediate needs effectively.
We opted for AWS as our cloud provider due to its flexibility and robust features. Initially, I set up separate EC2 instances for the backend and frontend. Over time, I implemented several changes to improve our infrastructure:
CI/CD Pipeline with Jenkins: I introduced Jenkins pipelines for continuous integration and continuous deployment (CI/CD), enabling us to automate the build and deployment processes. While the pipeline is functional, I recognize there's room for improvement to enhance its efficiency and reliability.
Process Management with PM2: To better manage our Node.js applications, I integrated PM2 for process management, ensuring our applications remain stable and responsive even during high loads.
DNS Setup with Route 53 and Elastic IPs: I optimized our DNS setup by leveraging Route 53 for domain management and assigning Elastic IPs to our EC2 instances for static IP addresses. This has helped streamline our access and improve reliability.
Recently, we made the decision to transition to a microservices architecture to improve scalability and maintainability. As part of this transition, I deployed four separate servers for different functionalities, including the frontend, dashboard backend, user app backend, and chat backend.
However, as we continue to grow, I'm faced with the challenge of effectively scaling our infrastructure. Currently, we're using EC2 instances with Elastic IPs routed to simple records for access. I'm uncertain about the best approach to scale this setup using load balancers, or whether transitioning to Docker or Kubernetes would be more appropriate.
While I've been learning Kubernetes on the side, I'm still unclear on how to replicate our current setup in a Kubernetes environment, particularly considering the associated costs and complexities.
In addition to scaling concerns, I've made efforts to automate and streamline our processes, but I'm eager to learn from more experienced individuals like yourself. If you have any insights, advice, or recommendations on how I can improve our infrastructure, scale effectively, and navigate the transition to containerization and Kubernetes, I would greatly appreciate it.
Thank you for considering my request, and I look forward to hearing from you.
TLDR: I am a really curious learner interested in Tech and Cloud, need advice to figure architecture at my company before it goes down :D

https://redd.it/1bwateq
@r_devops

Pixie in production

I have been tinkering around with Pixie and the new relic integration in my clusters.

It seems to work ok, but only tried the community cloud for visuals. Is anyone running Pixie in production clusters? Are you doing this with the community cloud, or some self-hosted solution?

We're also struggling with the insanely expensive Full Platform licenses in NewRelic and my current thoughts is:

* Push K8 data and basic app metrics to NewRelic so alerting and Operations guys have what it takes
* Leave regular devs out of New Relic (only some free dashboards), they can troubleshoot (with traces) in a Pixie UI

https://redd.it/1bwc84p
@r_devops

Anyone here work for capgemini in the UK?

I have 2 offers, one is from capgemini and I would have taken it but the recruiter for my other offer out doubts in my mind because he “used to work with them”. I’m very interested in working with them, but now I’m not sure.

If anyone here works for this company can you tell me how they are with remote working? I don’t mind going to client sites every not and then, or for a week or 2 in a row. But if a client said they wanted somebody on site every day for 3 months or something that would be a hard no for me. Please choose one of your other thousands of engineers to go.

My other offer is fully remote but for slightly less salary (less than my current) and the benefits package is also pretty bad compared to cap (and I do use benefits).

https://redd.it/1bwd6gb
@r_devops