How to manage all different pieces of Infrastructure in a perfect world - Production cluster

Hi Everyone,

What I'm looking out in this thread is to put up the discussion about how to manage all the different pieces of infrastructure that there is in a production ready world!

I have some experience in the devops world, however I've just came into a new job and my role basically is to automate and to lay all the infrastructure that there has been build via "clickOps" :P

I'm pretty excited with this and I think this is the most funny, enthusiastic and growth potencial, experience that someone can have in this sort of position, because you can build things from ground up, right? :)

However I found myself having tooooo much options to go, and now knowing exactly what is the right way to go!

I have some experience with terraform, terragrunt and helm and I want to leverage that. And I will need to manage many different parts of the infrastructure and basically I'm having some struggle in thinking through all the different pieces and that's why I'm here :)

Let me explain a bit the struggle in the hopes that you can help me!

I will need to setup a K8s cluster in GKE. I will do it in Terraform, Terragrunt for multi environment management.

I need to create LB's and Ingresses, which i will use maybe Helm charts for it.

Helm charts will be in a repo in Github for example.

And this is only what i thought in a bit, and i'm pretty sure that i will have way more Infrastructure to handle afterwards, so what's the best way to start in an organized way? What about the automation part of it!?

When i change something in the helm charts i don't want to run kubectl by hand. I would like to automate that process. Should i do it via Terraform integrated with Helm? or should i have Argo for example checking for github changes and deploy it automatically?

When I change something in terraform i need to apply those changes! Should i automate that? How?

Probably this is just to start the discussion as probably there are so many missing parts here, but i would like to hear your opinions about it.

Any Architectural guidance or "mental picture" would be welcome!

https://redd.it/10wsiih
@r_devops

Seeking feedback from Front-End Developers (think Shark Tank)

Hi DevOps Community!

I’m helping a startup that’s building a developer tool that turns design images into front-end code. They’re looking for feedback from Front-End developers on their product vision (think Shark Tank) over a 30-Minute call. We keep you anonymous so you're not being sold to (unless you choose to expose your identity). We just ask that you provide your brutally honest feedback on the clarity & uniqueness of their pitch. You can set your rate for the engagement.

If you're open to it, please review the project details below [persona fit is a requirement to participate\]:

Description of Startup Seeking Feedback: Developer tool that turns UX designs into fully functional ReactJS, using computer vision, the same technology in self-driving cars, to render perfect code with the goal to eliminate manual HTML, CSS, and JS development to allow companies to ship code faster and cheaper.

Desired Persona:
Front-End Developer at a venture-funded UI-based SaaS organization (Series A to C) that has raised at least $1M. They actively use Figma, React JS and Material UI. [Use of Material UI is a requirement to participate in this project\]

Their environment has a minimum of 3 engineers (max 25) and an in-house or contract-based designer for software design.

Requirements to Participate: Your answers to the following questions will help the Startup ensure that it's a fit for both parties before moving forward with the call.

1. Can you please describe your responsibilities as a front-end developer at your current organization? What size is your engineering team? Please elaborate.
2. Does your company currently have a custom component library it uses to build markup? What is your biggest challenge in the front-end development pipeline to ship code? What is the scale of your UI? Please explain.
3. Do you personally write markup in your organization? If so how many hours per month do you spend writing markup? [must be using Figma, ReactJS, and Material UI to qualify for this project\]

If this is of interest please send me a dm with your responses to the above required questions, your LinkedIn profile, and your email address so I can invite you to the project.

Thanks!

https://redd.it/10vn2a2
@r_devops

What do you guys use to manage .env files?

I have worked on a lot of projects over the years. I have yet seen a good solution to managing env, secret and config. The standard is “can you send me your env file”. You might need one for dev one for stage. So many bugs, because someone updated env without telling anyone else. I made a tool for it now that manage my env I can connect it to AWS ssm and azure key vault. I’m really happy with it, but I forgot to check if any thing existed before I made my own cli. I’ll still use my baby but would be nice to see other solutions

https://redd.it/10wujbq
@r_devops

Testing Terraform infra - terratest alternatives?

Hi all,

So we've got our AWS EKS etc infra in Terraform Cloud / Enterprise, and I've setup a pipeline to apply / destroy a separate ephemeral cluster similar to existing ones - just to daily test the TF code.

There are multiple states of it, and after deploy / before destroy I want to run some tests, like see if security groups actually have the ports open / closed, DNS records are correct etc etc (we have a shitton of stuff to test like that tbh with dozens of IAM roles, buckets, CF distributions and so on). Application E2E tests will be added to this pipeline later as well.

Terratest isn't an option since it doesn't work with a remote backend (at least not without some stupid hacks I guess), and it deploys / destroys infra by itself, which my pipeline already does.

I'm considering something like BATS, but maybe there are other specialized tools? Ofc I could just write some bash myself and add to that as the time goes on, but there has to be a better way.

https://redd.it/10wwlyp
@r_devops

GitHub Actions Manager for JetBrains IDEs

I created a JetBrains plugin to manage GitHub Actions from IntelliJ/PyCharm/Rider.

Download it here: https://plugins.jetbrains.com/plugin/19347-github-actions-manager

https://redd.it/10vljba
@r_devops

Looking to get feedback from DevOps/Engineering Professionals!

The startup I'm working with is looking for SaaS leaders like you to provide feedback on a new automated solution they are building for the DevOps/engineering community. The goal is to enable SaaS companies to quickly automate software deployments to the cloud using a visual interface, saving users time by not having to create 1,000+ lines of configurations. This solution is being developed for companies that don’t have a large engineering team.

We want to show you a prototype within the next couple of weeks. This will take less than an hour. Please book here: https://calendly.com/d/zvr-39k-95x/opscanvas-prototype-review 

Thank you for your time in advance.

https://redd.it/10vl52p
@r_devops

Any government GS employees that are DevOps? How's work/life balance?

A friend of mine is a GS13 and the way he explained it is that all he does is oversee architectural meetings with his team and contractors. Basically building out, deploy, and provide support for cloud environments (mainly Azure). Says he gets to telework but comes in sometimes out of boredom. Has a set schedule. The contractors handle all the off hours support. He asked me to apply for a role, but I'm hesitant. I make WAY more in my current role in the private sector. It'll essentially be a pay downgrade, but great benefits and a pension. Was wondering if any federal employees can chime in that worked both public/private sectors? Is a GS position worth it with even with less pay?

https://redd.it/10x1vrq
@r_devops

How attached are you to production?

I often find myself checking in on weekends/after work to make sure everything is fine, even though I am not oncall.
When something happens - I have a real strong sense of ownership, trying to find RCA to remidiate from the cause.
However, I sometimes feel that's not shared for everyone. Seems to me like some people just want to pass on the crap to the next shift for them to solve.

How is this for you? Do you feel attached to prod?

https://redd.it/10x3o3m
@r_devops

Best practices for setting up ansible servers?

So at work we have about 2000 windows and 2000 linux servers, we do not use ansible tower for any of them (nor do we have any plans to do so). However, many departments runs the occasional ansible job. Some use it a lot, some system administrators use it to quickly set passwords and login rights. Others use it for openshift, elasticsearch, logstash and Kibana.

Our linux servers are almost al RHEL based and we use vault for secrets.

Currently the system works like this

Log into a private very restricted jumphost with your administrator user.
Log into ansible-runner from this jumphost (this is done as root!)
Here ansible jobs can be run against hosts, given that the host has been set as an ansible host. (This permission is also very restricted). Mostly single playbooks, very few roles, little to no system.
Some projects are git controlled others are not

My plan is to upgrade this system, making it more traceable. However, I would also like to not add too much friction for our users. Only a handful of projects are using it, and our main CMS is much better protected. The security is quite good as you need both access to a restricted jumphost, and the target host also needs a very specific role to be reachable

My current plan is something like the following:

Setup `ansible-runner-prod01` and `ansible-runner-prod02`, these are used to configure each other
Set a policy among our employees that access to these machines should be done through another jumphost, and login as a admin user, not root.
Each department will have their own ansible user with username and password stored in vault.
Each department will be given their own git controlled folder.
Ansible jobs will be run through a wrapper shell script [deploy.sh](https://deploy.sh) which logs into vault and then logs in as the ansible user.
Finally hosts can have different roles, so for instance only department XXX can run ansible jobs against it.

Thoughts? Are there other ways, better ways, more secure ways?

https://redd.it/10x2tld
@r_devops

Companies stealing/taking credit for DevOps Roadmap

https://roadmap.sh/devops created a fantastic roadmap for DevOps years ago, I've been following it for quite some time and I see they keep adding updates, changes and edits to it.

On LinkedIn I came across a post from some company called "Decipher Zone" with the exact same roadmap- LITERALLY the exact same, but it's just a got their companies logo watermarked on it and taking credit for it without giving all the same resources as roadmap.sh does.

https://www.decipherzone.com/blog-detail/devops-roadmap \- here is the link.

​

I know there's nothing I can do about it, just wanted to vent. The original creator is https://www.linkedin.com/in/kaamranahmed/ \- and he does not work for Decipher Zone.

Anyways, that's all. I'm not looking for any particular response, just felt like putting it out there.

https://redd.it/10x6kdz
@r_devops

devops generalist vs specialist

Are you expected to learn all the devops tools in depth or be Jack of all tools and master of none. As there are so many tools to learn and you need to know good knowledge about networking and a bit of coding to rectify errors in dev team etc.,

https://redd.it/10x65mv
@r_devops

Noob Question About Chef Attributes

So in a chef cookbook I am looking at, there are a bunch of

default[value\][value\] etc etc,

However, in the recipes, there are a bunch of values with the syntax

node[value\][value\] etc.

I don't see anywhere node values are set, where are the recipe's getting the node attributes from?

https://redd.it/10x403j
@r_devops

Is it a good idea for a CMS system to generate a CloudFormation template for the backend of a website?

The idea is about a CMS system that makes it easy for the user to create and manage their website's content. The CMS allows the user to directly edit content files in a GitHub repository through its user-friendly interface. This same repository holds the source code for the website, which is built using NextJS. The website's code includes API functions that fetch dynamic data from a database or authorize users.

The CMS takes things a step further by automatically generating a CloudFormation template, which can be uploaded to the user's AWS account. This template sets up the backend infrastructure for the website, including the database and authorization configuration. By using this template, the user can quickly and easily set up the backend for their website without having to worry about managing the infrastructure themselves.

https://redd.it/10x88eu
@r_devops

If you could define your own responsibilities...

I've had the golden offer, a job where I define my own responsibilities in a senior role within a large tech organisation

​

If you were in this position, what would you do?

https://redd.it/10x66dd
@r_devops

I want to get certified on GCP, Azure or AWS. Which organisation would give me the best future prospects?

Title says it all really. What do you think?

It feels like AWS is more popular.

https://redd.it/10x62ur
@r_devops

Master's Grad in 2022. Cannot land a first Round interview.

Hello y'all, Trying to get into Mid - Sr : SRE/DevOps

Having a tough time since I graduated and can't seem to land even first round interviews. Spent 100+ hours on this resume thinking something I am doing is wrong.Still unable to land even a SINGLE interview round!

I know my DSA's, have 3+ years of legit experience but things are getting bad very steeply. Shooting a shot here to see if someone can critique my resume.

XXX-XX-XX | XXXX, Open to relocation | LinkedIn Profile | GitHub | [email protected]

An AWS Certified Solutions Architect with a master’s degree in Computer Engineering and 4+ years of hands-on experience in developing event driven cloud native applications in public clouds like AWS with services like Lambda, Fargate, ECS,S3 etc... I am highly skilled in platform reliability, microservices design, serverless technologies, test automation and DevOps practices.

SKILLS:Languages (4+ years): Python (boto3, NumPy, Requests), Bash (Linux/CentOS), JavaScript (React/Redux/Node).

AWS services (5+ years): AWS CLI/SDK | Lambda, Fargate, CloudFormation, ECS, CloudWatch, S3, RDS, Kinesis.DevOps (4+ years): Network Administration, NetApp, Docker, Kubernetes, Jenkins, CircleCI, Git, GitHub.Amazon Web Services (5+ years): AWS CLI/SDK, Serverless Framework, AWS Backups, VPC, API Gateways, Lambda, EC2, EBS, EKS, CloudFormation, CloudWatch, S3, DynamoDB.

Web Development / Others: Authentication/Authorization (OAuth, JWT, RBAC, SSO), Microsoft Power BI, Databases (MySQL, PostgreSQL, Redis).

PROFESSIONAL ACHIEVEMENTS

Software Engineer – DevOps 2018 - 2021Organization Durham, NC

· Raised KPI’s expected in an DevOps role, consistently delivered 99.99% SLA, took ownership of all infrastructure for release/change management, performed zero downtime deployments, and completed a fully automated software testing system in under 2 months.

· Spearheaded agile teams in implementing well architected frameworks/best practices in AWS which led to successful ISO 9001:2015 Quality Management certification for XXXX.

· Patched 500+ issues in React/Redux front end systems, Relational Databases and configuration management.

· Created/Managed custom AMI’s, Docker Images and Kubelets to complete deployments. Performed patch management for EC2 instances, shifted security and QA practices to the left of SDLC.

· Authored reusable Infrastructure as Code templates (CloudFormation, Makefiles) for dynamic provisioning of AWS resources like EKS, EC2, Lambda, S3 within private/public VPC’s on AWS cloud.

· Authored workflows that preprocessed and analyzed code (CircleCI, Jenkins, Veracode) to identify many software vulnerabilities in early stages of SDLC. Competitively performed Code Reviews for Python, ReactJS and Infra.

· Managed Public/Private VPC’s. Ensured highly available and resilient architectures for enterprise software platforms.

EDUCATION

Master’s in Computer Science Jan 2021 - Dec 2022

(Some research information here) GPA: 3.69

Bachelor’s in Computer Science and Engineering Aug 2013 – May 2017 GPA: 3.75

​

​

https://redd.it/10x5b4w
@r_devops

How to set up alert monitor for data dog over a function of time?

Trying to setup a data dog monitor to check if a pod is in a certain status for over ten minutes? How can I do this? Datadog sub Reddit is locked and I could not find the relevant information in datadog docs, maybe I’m not looking in the right place. Help would be appreciated!

https://redd.it/10x4rv3
@r_devops

A better way to manage secrets in Kubernetes

Wrote an article on how to better manage secrets in kubernetes by using a custom operator i made. The operator will fetch secrets and put them in a Kubernetes secret and can auto reload deployments that depend on the fetched secrets. You can think of it like a wrapper around native kubernetes secrets

Article: https://maidul.medium.com/kubernetes-secrets-management-on-autopilot-36e0c6373024

https://redd.it/10xjxtf
@r_devops

Looking for DevOps learning partner

Hello everyone, I’ve recently started learning devOps and also looking for someone who is eager to learn and share knowledge together.
I intend to study AWS, Azure DevOps, Docker, Kubernetes, Terraform, an other related technologies.
Hit me up if you’re interested

Discord username: Illusive man#1442

https://redd.it/10xko9j
@r_devops

work sucks

The best vote to end this wins, submit your ideas! maybe your idea will make the news!!! how exciting!

https://redd.it/10xmibq
@r_devops

Do these sentences make sense?

I'm a tech writer interviewing DevOps engineers who are english as a second language. I just wanted to ask, if these sentences would make sense and are properly ordered. I know some of the tools, but not all.

Just want to make sure it's not something redundant like I know adobe creative cloud, photoshop, illustrator, creative suite, figma...

What tools for DevOps have you worked with?

”I’ve worked with many tools with CI/CD like Jenkins, ArgoCD, Github Actions, Ansible, Terraform, CloudFormation, Docker and Chef."

What's your ideal tech stack?

"Docker, Python, Kubernetes, AWS EKS, ArgoCD, Github Actions, Terraform Cloud. I'm already working with most of them."

https://redd.it/10xmwsr
@r_devops