Terraform modules that create the same resource

I have two Terraform modules that create cloudflare_zone resources on the same zone. The first module works fine, but the second one fails because the zone already exists.

Is there a way to let Terraform "merge" the two modules together?

https://redd.it/yc1qtr
@r_devops

Running cypress tests for svelte app with Github Actions

I'm really new to Github Actions. I have a repo with a 'frontend' and a 'backend' folder. Backend is fastapi, frontend is svelte and inside the frontend/app, there are 'src', 'cypress', 'package.json', 'cypress.config.js', 'rollup.config.js'.

In the root folder of the repo, there is the .github folder, and there the main.yml. So I want to build and run my svelte application (and the fastapi backend) and then do cypress tests on them. Running fastapi backend works so far. For the frontend I tried runnning svelte on a separate step than the cypress tests, but then cypress.visit() couldn't find the localhost, where the app is located.

That's why I put everything on the same step:

- name: Install packages
run: |
cd frontend/app
npm install
npm install cypress

- name: Cypress
uses: cypress-io/[email protected]
with:
build: npm --prefix frontend/app run build
start: npm --prefix frontend/app run start
config-file: frontend/app/cypress.config.js
wait-on: 'https://localhost:3400'

The "Install packages" steps works fine, but in the cypress step I get an error

Error: ENOENT: no such file or directory, open '/Users/runner/work/<repo>/<repo>/package-lock.json'

So it never checks the right path (frontend/app) so I probably ignores the --prefix parameter. I also tried: (build: cd frontend/app && npm run build) but it still checks the root dir.

Does anyone has a clue why this happens and how I can let them find the right package file? Has been very frustrating.

Thanks

https://redd.it/ybxzs4
@r_devops

Suggestion on plan to deep dive into devops part of application in production.

Hello folks I'm a ruby on rails web developer(>3 YOE), and so far has very minimal "ops" knowledge.For smaller client works and personal projects I've been using - CI: github-actions/circle- CD: Heroku to Handle all the deployment of PR builds, staging & production apps.Currently working for a larger client, here it is devops heavy(with a large dedicated devops team),& I'm kinda clueless & awestruck by the work the devops have done here, hence this plan for deep dive.Also, since Heroku going through some major overhaul, rather than looking for a similar alternatives I would like to deep dive into the operations part of web apps.After some research I came up with a personal ROADMAP (thanks to roadmap.sh & other sources from this sub).Along with 2 personal projects CI/CD setup as to put my learning into practice.

{
networking: "basics", "Protocols", "nginx", "Web Server", "Network Tools"
linuxscripting: ["ruby/go", "bash", "terminal commands", "Compiling apps"],
cloud: ["aws","IAM", "ec2", "vpc", "aws-cli", "s3" "rds", "load balancing", "Auto scaling groups", ],
containerization: ["docker", "docker-compose", "Docker Swarm", "Swarm cluster", "ECS", "ECR", "Fargate", "Fargate cluster", "ECS load balancing with ALB"],
cicd: "jenkins", "github actions", "gitlab", "travis/circle/semaphore" "heroku", "Pipeline Setup With Jenkins and AWS ECS",
sysprovisioningconfiguration: "Terraform","Ansible",
monitoringandlogging: "Prometheus", "Grafana", "cloud watch", "ELK" ,
orchestration: "kubernetes", "eks",
intermediateproject: {
// No orchestration required
// Should I use Jenkins/other CI tools like circle/gitlab/github actions
appcomponents:
"rails_app monolith(front end using jsbundling-rails or webpacker",
"postgress_db"
,
ci:
"PRs should run test suite, static code analysis(rubocop + eslint)", "check for 90% test coverage", "only merge if green run",
cd: "PR merge to master must auto deploy the containerized app to production",
"deploy a rails_app monolith(including front end) + postgress_db"
advancedproject: {
appcomponents:
"rails backend",
"postgres db",
"sidekiq (requires redis)",
"react js frontend",
"setup metabase for the DB (optional future feature)",
"setup rabbitMQ for messages (optional future feature)"
,
ci:
"PRs should run test suite",
"create PR builds for testing",
"static code analysis(rubocop + eslint)",
"check for 90% test coverage",
"auto create PR apps to testing", // is setting up pr builds too much???
"only merge if green run",
cd:
"PR merge to master must auto deploy the containerized app to production",
"deploy",
,
setupmonitoring: ["Prometheus", "Grafana", "cloud watch", "ELK"],
automate: ["provisioningconfiguration", "terraform", "ansible"]
}
}

Am I overdoing anything here? or am I missing out on anything crucial?.
I know Jenkins kinda has a bad rap around here, I personally would like to learn it.
I'm planning to achieve this in 4-5 months, is it too much?.
Any suggestions regards to above points would be much appreciated.Thanks.

https://redd.it/yc5pnj
@r_devops

GitOps with Advanced Cluster Management for Kubernetes

Manage multiple clusters and deploy apps across several target clusters with Argo CD and Advanced Cluster Management for Kubernetes: https://piotrminkowski.com/2022/10/24/gitops-with-advanced-cluster-management-for-kubernetes/

https://redd.it/yc8bp8
@r_devops

New OSS Project: Helm-Dashboard, the missing UI for Helm

Some time ago, we at Komodor, started working on a new OSS project called Helm-Dashboard. Now, it’s finally ready for users, and I’m excited to share with the world. You can see its GitHub repo here: https://github.com/komodorio/helm-dashboard


So Helm-Dashboard is basically a GUI for Helm, and it's designed to solve some of the more acute painpoints of Helm users by visualizing changes in Helm charts. The goal is to help beginner Helm users to get started with Helm, and for more experienced users to speed up operations.


I invite everyone who is using Helm to give it a try and share your feedback. Feel free to open GitHub issues, send PRs, ask questions in a dedicated #helm-dashboard channel on our Slack Kommunity: https://join.slack.com/t/komodorkommunity/shared\_invite/zt-1dm3cnkue-ov1Yh\~\_95teA35QNx5yuMg

Give it a star if you liked it :)

https://redd.it/yccfb5
@r_devops

Helm Manager - App health / Helm Release TTL / Helm Rest API / Helm Web UI

We are working on a tool that helps better manage Helm applications.

[https://github.com/JovianX/Service-Hub](https://github.com/JovianX/Service-Hub)

Helm Manager includes:

* Show application health (All deployed workload resources are healthy).
* Helm releases Time-To-Live(TTL) schedule for temporary apps.
* Manage Helm releases across multiple clusters.
* Helm Restful API with basic RBAC
* Helm UI with basic RBAC

I would much appreciate your feedback and thoughts!

If you like the project, or find it helpful please STAR ⭐ the repo.

https://redd.it/yceela
@r_devops

Devs deploying to Production and separation of duties?

Ok. So I work in security and I have to care about PCI, SOC2, SOX, Alphabet soup, etc…

How are you creating a compliant separation of duties scenario that auditors are going for?!

I know people are doing it. I want to drive developer enablement and push us to streamline the whole path to production.

https://redd.it/ycdafh
@r_devops

DevOps Interview questions

What are some common interview questions for devops in aws and python? I have an interview coming up and need to prepare for it….anything helpful is much appreciated!

https://redd.it/ycjmhv
@r_devops

DevOps Bulletin - Issue #74: AWS prices in your Google Sheets

Hey folks 👋🏻
Check out a sneak peek of the topics covered on this weekly issue:

* 🔐 "**EKS Security Checklist: 10 best practices for a secure cluste**r" - The best way to tighten your cluster security is to implement practices that have become industry standard and are recommended by the Kubernetes community. Here are 10 most EKS security tactics every team needs to protect its clusters.
* 🗂 "**Searchable Pod logs on Kubernetes in minutes**" - Containers generate huge volumes of log data. In a production environment, SREs, DevOps folks need a central location to query and analyse these logs, to do their job well.
* 📦 "**Quirks, caveats, and gotchas In SQLite**" - This document strives to highlight the principal differences between SQLite and other SQL implementations, as an aid to developers that are porting to or from SQLite or who are trying to build a system that works across multiple database engines.
* 🤖 "**An easier way to install tools for GitHub Actions**" - This blog post covers how to use Arkade in your GitHub Actions to get the typical kinds of tools you'll need to build code and run end-to-end tests.
* ⚡️ "**CloudFront vs Cloudflare, and how to reduce response times for both by \~35%+**" - Turn on Origin Shield if you use CloudFront and care about response times, even for dynamic / uncacheable content.
* 💡 "**Solving common problems with Kubernetes**" - This article is aimed at engineers who need to deploy their code using Kubernetes, but have no idea what Kubernetes is or how it works.
* 📹 Podcast of the week goes to "Leaving the Cloud" by Rework Podcast. **Cloud services have been used by tech companies for many years**, but it’s not the only way. Hear why 37signals is making the decision to go in another direction. David Heinemeier Hansson, co-founder of 37signals, and Eron Nicholson, Director of Operations, discuss why 37signals is making the move away from the cloud.
* 🛠 Project of the week goes to "**The AWS Pricing Google Sheets add-on**" - The add-on allows you to incorporate the latest AWS pricing data in your Google Sheets spreadsheets. This makes it easy to perform cloud cost analysis directly in Sheets without error-prone copy-and-paste from pricing website.
* 📕 Review of "**Infrastructure as Code, Patterns and Practices**" - The book teaches you to automate infrastructure by applying changes in a codified manner. You’ll learn how to create, test, and deploy infrastructure components in a way that’s easy to scale and share across an entire organization.

Complete issue: [https://www.devopsbulletin.com/issues/aws-prices-in-your-google-sheets](https://www.devopsbulletin.com/issues/aws-prices-in-your-google-sheets)

Feedback is welcome :)

https://redd.it/ycdy1i
@r_devops

Am I missing something about SAML?

We're currently undergoing a SAML implementation for third party services to simplify MFA enforcement and user management but SAML seems to be missing some things.

1. When a user leaves, if we delete the user from the IDP then they still exist in the third party service. Some services will still charge for that user.

2. Some services don't support role mapping so we have to go into the service UI anyway.

3. Some service providers don't let you either disable non-saml based login or enforce MFA.

Am I missing something here? Seems like it's not solving many of the problems that we'd hoped it would? Sure it makes creating users simple but often assigning the right roles and removing users doesn't seem much improved?

https://redd.it/ycocer
@r_devops

Next gen deployment strategy for a Hackathon?

Our group is running a Hackathon in a few weeks: brainstorm a project idea, and split into a groups and each build that project.

In preparation for that, I'm thinking about setting up a new environment for the project, maybe something like: Fly.io or Rancher or K3s, where we could try some new methods of building and deploying apps.

Fly.io seems to be doing a lot of interesting things. ISTR I've seen a demo or Rancher where it integrated into VSCode and you could get your code change up and running with a custom URL to test it out, something like that might be slick.

Our current environment is: Windows workstations, test locally, commit and push, then build in Jenkins, which creates Deb packages and pushes those to the Linux staging environment. It'd be nice to try something completely different. Our apps will likely be Java backend, Vue frontend. Probably will need something that can connect back to our existing database via a VPN, or be set up on a VM in on-prem.

Any ideas on something worth looking at getting set up to try?

https://redd.it/ycq8zt
@r_devops

Where do you generate your trace header?

I'm curious about where the distributed trace header (W3C Traceparent, etc) is typically injected into the request in people's application architectures.

I can imagine client devices, LBs or gateways being the origin, but would like to hear from folks who have actually implemented tracing.

https://redd.it/ycmyqc
@r_devops

Terraform -> Ansible -> Kubernetes -> Docker Containers

Am I missing anything? All those technologies seem to synergize with each other. Is there overlap, could this be simplified or improved?

1. Terraform to provision infrastructure
2. Ansible for configuration management of that infrastructure
3. Kubernetes for docker container orchestration

https://redd.it/ycup6d
@r_devops

Platform Engineering: DevOps evolution or a fancy rename?

What is Platform Engineering? How is it different from DevOps, SRE or PaaS? Who should use it, and when should we introduce it?

As a veteran of the PaaS, and a DevOps enthusiast at present, I decided to delve into this topic, peel off the hype, and see what it’s about in practice. This also inspired my recent episode of OpenObservability Talks with George Hantzaras, Director, Cloud Platform Engineering at Citrix Systems.

https://horovits.medium.com/platform-engineering-devops-evolution-or-a-fancy-rename-a0cad2cdc819

https://redd.it/yd2fnx
@r_devops

DevOps Projects

Hi, I'm a beginner in DevOps and Cloud areas and I like to do projects to get some hands-on experience.

I will share a small project I did recently. It's a backend application and I wanted to deploy it to the Kubernetes cluster. So here are the things I did.

Infrastructure:-

1. Provisioned an Azure Kubernetes cluster (AKS) and Azure container registry using the console.

2. Then I wrote the Terraform Infrastructure as Code and tried to deploy those resources using IaC.

3. Configured the Terraform backend.

4. Added the Terraform code into a GitHub repository.

5. Configured GitHub Actions workflow and tried to provision resources using the CI/CD pipeline.


Application Deployment:-

1. I wrote the Dockerfile for the application.

2. Built the application Docker image and pushed it to the Azure container registry.

3. Wrote the Kubernetes manifest files for the application deployment.

4. Deployed application to the Kubernetes cluster manually.

5. Then added the Dockerfile and Kubernetes manifests to my backend application code repository.

6. Configured GitHub Actions workflow and tried to deploy the application using the CI/CD pipeline.

By doing this simple project I got some hands-on experience in Terraform, Docker, Kubernetes and GitHub Actions.

Can you guys share some good project ideas like this? It will helpful for self-learning guys like me.

Thanks for your idea and your time. 🫡

https://redd.it/yd2k77
@r_devops

How does DNS work on a massively scaled system?

Apologies as this isn't necessarily a "devops" related question, but this is quite an active subreddit and I know a lot of people here know networking well, so I thought you could help me understand something I can't wrap my head around. By all means please tell me which subreddit to re-post this too if it's not suited for here.

I'm a SWE, and I understand at a high level the concept of DNS, what it does etc. But I don't understand it in relation to massively scaled systems like Facebook, Amazon etc.

The analogy used is often it's akin to a phonebook. You type in Facebook.com into your browser, and behind the scenes this gets resolved to an IP. But Facebook isn't all hosted on one singular server with a static IP address. I'm pretty sure they serve content based on region and things like that. So if I type in their address on my browser while in London, it gets routed to their London or Europe instances of the site, and the same for other regions (please correct me if this is wrong).

My best guess is it's similar to something to a K8s Service, in that the IP the DNS resolves to is actually some sort of load balancer, that then routes the request to the relevant server based on location etc. But that's a pure guess. I'm really interested in learning more about how this actually works. So if anyone here can shed some light on it, I'd greatly appreciate it!

https://redd.it/yd5orz
@r_devops

Deploy ASP .NET 6 MVC Web App on Google Cloud Run using Cloud Build

Learn how to Deploy ASP .NET 6 MVC Web App on Google Cloud Run using Cloud Build

In this tutorial, we will see a methodical way to implement (CD) Continuous Deployment of an ASP .NET 6 MVC Web App on Google Cloud Run with the help of Google Cloud Build Trigger.

By the end of this tutorial, you will be able to have a full understanding of enabling Continuous Delivery of ASP .NET 6 applications to Cloud Run via Cloud Build.

This tutorial covers in-depth concepts of working with Cloud Build triggers, Cloud Run features such as Logs, Revisions, SLOs etc.

The tutorial also helps you understand how to troubleshoot the Continuous Deployments on Cloud Run.

https://youtu.be/5M9yzZOJXaQ

#cloud #google #aspnetcore #postgresql #cloudstorage #cloudarchitect #devops #cicd #cloudbuild #googlecloudplatform

https://redd.it/yd2zip
@r_devops

How to Map a Custom Domain to Google Cloud Run Service

Learn how to Map a Custom Domain to Google Cloud Run Service.

The Custom domain is with Godaddy registrar, and the DNS management is delegated to Cloud Flare.
We will first verify our domain ownership with GCP.
Then we will add a CName record to point our subdomain or custom domain to ghs .googlehosted .com.

https://youtu.be/lDtvpUYAFzA

https://redd.it/ydctek
@r_devops

Does Traefik by default, act as a firewall ?

When my Traefik proxy container is running on my server, I cant access it through SSH. I just get connection timed out

https://redd.it/yd2jap
@r_devops

I'm giving myself carpal tunnel... suggestions?

I think something about my chair armrest/desk height/mouse/something else configuration is compressing my median nerve in just the right way to start causing issues. Occasional feeling like, not like my hand is asleep, but like its just starting to fall asleep, and problems with tightness and popping in my wrist the next day if, after work, I do anything in the workshop involving hand tools.


I don't know enough about desk ergonomics to know what to do to prevent this. While we're at it, I have lower back issues too, so maybe I need a new chair?


Any insight is appreciated. I'm 34... if I do this until I'm 60 I'm gonna look like Quasimodo.

https://redd.it/ydepwg
@r_devops

What might cause the master instance to not reboot?

I have a proxy and I have three instances. Two slaves and one master, and the master when we shut it down manually won't restart and stay healthy. What might be causing this? I have something set up in Nomad, but I can't figure out what might be the causes for this, and what to look for to troubleshoot.

https://redd.it/ydp0co
@r_devops