Deploying Next site + Node app and database.

Hey all, looking to host a community site I built in NextJS that is using server side rendering which is the least of my worries but I'm trying to host an instance of directus (node app that interacts with a DB and adds an API layer), I'm trying to find a cost effective way of hosting this setup that is reliable, I don't mind doing heavy lifting but also don't want to over engineer and prefer if if I could find a way to rebuild similar setups for the future.

I figured I could go with the droplet route and setup everything myself, which I went ahead today and did 90% of just that but ran into issues with the reverse proxy on nginx, also it was a bit hefty but I could probably build an ansible role to do most of it, but I was thinking maybe docker would be an thought to make this happen? Directus has a docker image and I could use a postgres image as well, I just don't know how well that works in production or if I should just host that on a droplet or a container service (sounds pricier but idk)

Sorry, if I'm not super clear just trying to find way to make this way and keep it cheap, I imagine traffic will be pretty damn minimal so I don't think I need much. I also don't want to over engineer or have things band-aided together.

thanks.

https://redd.it/ylq6md
@r_devops

GCP from AWS

Beyond searching for the equivalent of the services between cloud providers (e.g. EC2 vs Compute Engine), are there any tips and advice one could share for organizations switching from AWS to GCP?

For starters, I’ve found that there are no accounts, but instead groupings based on “projects” in GCP.

https://redd.it/ylpz5h
@r_devops

Scaling Your Team From 5 to 250+ Engineers: FULL Checklist from your feedback!

A few weeks ago I shared a post on here about scaling your engineering organization from 2 to 250 engineers. It was a long blog post that detailed the stages of growth and what to do in terms of Velocity, Quality and Outcomes.

The feedback I got on that post was honestly overwhelming!

I love this community and your comments and suggestions were truly valuable, as I've been putting together something a bit more extensive for engineering leaders... a full checklist to help navigate these stages, step by step. What to focus on in terms of yourself as a leader, your teams and your processes. I included items on culture (something which a lot of you brought up) and each checklist items has extra resources so you can explore more :)

It came out on Product Hunt a couple of hours ago, so you can check it out there, and if you like it, give it an upvote!

This checklist is a living thing, and it really wouldn't be possible without this community, so, if you have more feedback and suggestions, let me know in the comments, as I'll be adding more items and resources as they come!!

Thank you so much for all your support on this!

https://redd.it/yltt9e
@r_devops

4 DevOps KPIs for better software productivity

We made a video a while back about DevOps KPIs! We hope it is useful to some of you.

Let us know if you use any other important KPIs, or maybe none at all, which would also be interesting!

https://redd.it/ylw11c
@r_devops

Using a single Flux instance and single repo to deploy workload on remote cluster with a kubeconfig secret

I am trying to configure multiple K8s clusters via a single Flux instance and a single repo with the following process. Please note cluster provisioning is handled outside of Flux.

The following example installs flux on the management cluster and sync with the repo.
Once Flux is setup, I clone the repo and add sync files for it to deploy workloads on the remote staging cluster using a kubeconfig secret.

When I omit the kubeconfig from the staging-sync.yaml, all get deployed on the management cluster (which is logic but not what I want to achieve), and as soon as i ass the kubeconfig, I get the following error: `Kustomization/flux-system/apps dry-run failed, error: no matches for kind "Kustomization" in version "kustomize.toolkit.fluxcd.io/v1beta2"` and i can't get it work.

But in [this](https://github.com/fluxcd/flux2/discussions/2258) github discussion from January, a member of the Flux project stated that targeting remote clusters using kubeconfig is fully supported.

```sh
# Store Gitlab known_hosts file in a variable
locals {
known_hosts = "${file("${path.module}/known_hosts")}"
}

# Generate an SSH keypair
resource "tls_private_key" "main" {
algorithm = "RSA"
rsa_bits = "4096"
}

# Generate manifests
data "flux_install" "main" {
target_path = var.target_path
}

data "flux_sync" "main" {
target_path = var.target_path
url = "ssh://git@${var.gitlab_base_url}:2222/${var.gitlab_owner}/${var.repository_name}.git"
branch = var.branch
}

# Create Namespace for Flux
resource "kubernetes_namespace" "flux_system" {
metadata {
name = "flux-system"
}

lifecycle {
ignore_changes = [
metadata[0].labels,
]
}
}

# Split multi-doc YAML
data "kubectl_file_documents" "install" {
content = data.flux_install.main.content
}

data "kubectl_file_documents" "sync" {
content = data.flux_sync.main.content
}

# Convert documents list to include parsed yaml data
locals {
install = [for v in data.kubectl_file_documents.install.documents : {
data : yamldecode(v)
content : v
}
]
sync = [for v in data.kubectl_file_documents.sync.documents : {
data : yamldecode(v)
content : v
}
]
}

# Apply manifests on the cluster
resource "kubectl_manifest" "install" {
for_each = { for v in local.install : lower(join("/", compact([v.data.apiVersion, v.data.kind, lookup(v.data.metadata, "namespace", ""), v.data.metadata.name]))) => v.content }
depends_on = [kubernetes_namespace.flux_system]
yaml_body = each.value
}

resource "kubectl_manifest" "sync" {
for_each = { for v in local.sync : lower(join("/", compact([v.data.apiVersion, v.data.kind, lookup(v.data.metadata, "namespace", ""), v.data.metadata.name]))) => v.content }
depends_on = [kubernetes_namespace.flux_system]
yaml_body = each.value
}

# Generate a Kubernetes secret with the GitLab credentials
resource "kubernetes_secret" "main" {
depends_on = [kubectl_manifest.install]

metadata {
name = data.flux_sync.main.secret
namespace = data.flux_sync.main.namespace
}

data = {
identity = tls_private_key.main.private_key_pem
"identity.pub" = tls_private_key.main.public_key_pem
known_hosts = local.known_hosts
}
}

# Create a repository
resource "gitlab_project" "main" {
name = var.repository_name
visibility_level = var.repository_visibility
initialize_with_readme = true
default_branch = var.branch
}

# Deploy generated SSH public key to Gitlab
resource "gitlab_deploy_key" "main" {
title = "flux"
project = gitlab_project.main.id
key = tls_private_key.main.public_key_openssh

depends_on = [gitlab_project.main]
}

# Deploy generated manifests to Gitlab project
resource "gitlab_repository_file" "install" {
project = gitlab_project.main.id
branch = gitlab_project.main.default_branch
file_path = data.flux_install.main.path
content =

base64encode(data.flux_install.main.content)
commit_message = "Add ${data.flux_install.main.path}"

depends_on = [gitlab_project.main]
}

resource "gitlab_repository_file" "sync" {
project = gitlab_project.main.id
branch = gitlab_project.main.default_branch
file_path = data.flux_sync.main.path
content = base64encode(data.flux_sync.main.content)
commit_message = "Add ${data.flux_sync.main.path}"

depends_on = [gitlab_repository_file.install]
}

resource "gitlab_repository_file" "kustomize" {
project = gitlab_project.main.id
branch = gitlab_project.main.default_branch
file_path = data.flux_sync.main.kustomize_path
content = base64encode(data.flux_sync.main.kustomize_content)
commit_message = "Add ${data.flux_sync.main.kustomize_path}"

depends_on = [gitlab_repository_file.sync]
}

# Create the kubeconfig secret for the staging cluster
resource "kubernetes_secret" "kubeconfig_staging" {

metadata {
name = "kubeconfig-staging"
namespace = data.flux_sync.main.namespace
}

data = {
"value.yaml" = base64decode(data.terraform_remote_state.downstream_cluster.outputs.kubeconfig)
}
}
```


## Repository structure after pushing new files

```sh
.
├── apps # Added files after Flux is setup
│   ├── base
│   │   └── podinfo
│   │   ├── kustomization.yaml
│   │   ├── namespace.yaml
│   │   └── release.yaml
│   └── staging
│   ├── kustomization.yaml
│   └── podinfo-values.yaml
├── clusters
│   ├── management
│   │   ├── flux-system # Files generated by Terraform flux provider
│   │   │   ├── gotk-components.yaml
│   │   │   ├── gotk-sync.yaml
│   │   │   └── kustomization.yaml
│   │   ├── kustomization.yaml
│   │   └── staging-sync.yaml
│   └── staging # Added files after Flux is setup
│   ├── apps.yaml
│   └── infrastructure.yaml
├── infrastructure # Added files after Flux is setup
│   ├── kustomization.yaml
│   ├── nginx
│   │   ├── kustomization.yaml
│   │   ├── namespace.yaml
│   │   └── release.yaml
│   ├── redis
│   │   ├── kustomization.yaml
│   │   ├── kustomizeconfig.yaml
│   │   ├── namespace.yaml
│   │   ├── release.yaml
│   │   └── values.yaml
│   └── sources
│   ├── bitnami.yaml
│   ├── kustomization.yaml
│   └── podinfo.yaml
└── README.md
```

## Files content

### clusters/management/flux-system

Note, I omitted the gotk-componenets.yaml intentionally because of it's length.

```sh
# clusters/management/flux-system/gotk-sync.yaml

# This manifest was generated by flux. DO NOT EDIT.
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
name: flux-system
namespace: flux-system
spec:
interval: 1m0s
ref:
branch: master
secretRef:
name: flux-system
url: URL
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: flux-system
namespace: flux-system
spec:
interval: 10m0s
path: ./clusters/management
prune: true
sourceRef:
kind: GitRepository
name: flux-system
```

```sh
# clusters/management/flux-system/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-sync.yaml
- gotk-components.yaml
```

### clusters/management
Those are the files that target the remote staging cluster using the kubeconfig for that cluster.

```sh
# clusters/management/staging-sync.yaml

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
name: flux-system
namespace: flux-system
spec:
interval: 1m0s
ref:
branch: master
secretRef:
name: flux-system
URL: URL
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: flux-system
namespace: flux-system
spec:
interval: 10m0s
kubeConfig:
secretRef:
name: kubeconfig-staging
timeout: 2m10s
path: ./clusters/staging/
prune: true
sourceRef:
kind: GitRepository
name: flux-system
```

```sh
# clusters/management/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
-

staging-sync.yaml
```

### apps, infrastructure and clusters/staging

Those folders are taken from https://github.com/fluxcd/flux2-kustomize-helm-example as a test to deploy workloads.

https://redd.it/ylvo9f
@r_devops

Infrastructure as Code Through Ansible

Latest webcast from the Software Engineering Institute, https://youtu.be/PSUDNYXAONA

It's a basic beginners introduction to the concept and tool. Any thoughts?

Description: Infrastructure as code (IaC) is a concept that enables organizations to automate the provisioning and configuration of their IT infrastructure. This concept also aids organizations in applying the DevOps process (plan, code, build, test, release, deploy, operate, monitor, repeat) to their infrastructure. Ansible is a popular choice within the IaC tool landscape for realizing this goal.

What Attendees Will Learn:

• What is Infrastructure as Code (IaC)?

• How does Ansible fit into the IaC tool landscape?

• How do I get started with Ansible?

Who Should Attend:

• system administrators

• IT operations managers

• automation engineers

• DevOps engineers

https://redd.it/ylzsbp
@r_devops

Switching to DevOps

Hi all,

I'm your typical Linux sysdmin (I have a RHCE cert), also working with SAN and VMware, some basic network knowledge (routing, vlan, LACP, NAT), I have worked with deployments before (gitlab to staging and to prod), and briefly worked on some AWS infrastructure, but I'd consider myself a beginner in that role. I have experience with docker, but not with Kubernetes. I got a job offer that pays significantly more than my current sysadmin job, I was very upfront about my knowledge and what I'm missing, and of course now my imposter syndrome is kicking in.

Is it realistic to learn terraform, more AWS and kubernetes (administration, not setting it up) in like 2 months or so? They of course offered I learn it on the job, they offered me any courses or certifications I'd want, but it's still quite a bit different from what I worked on until now. I am interested in this work otherwise and have been meaning to start learning it, I just haven't gotten around to it yet. I think I learn pretty quickly, and I am definitely willing to learn, but what if I'm biting off more than I can chew?

Thanks for any comments.

https://redd.it/ylx9yp
@r_devops

DevOps & DevSecOps: What Are the Key Differences Between the Two?

The terms DevOps and DevSecOps have been in the air of technology for a long time. But, still, the concepts of these two terms have been misunderstood by many; many are not even aware of the differences these terms have.

Let's dive into the concepts of DevOps and DevSecOps in detail.

https://redd.it/yly6je
@r_devops

Best certification for DevOps?

Yes, I know it's not required, I have 3 years of experience and no one ever bothered me with it besides a ton of free courses, but still, it sure is a nice peace of paper.

Looking at this certificate. I feel like I could get most of these right off the bat, except Cloudformation, which I never touched, as I only ever used Terraform. I actually completed the training for it since we were subscribed to AWS Skill Builder. What's the passing grade on the exam? Is it really worth it?

https://redd.it/yly4iq
@r_devops

Career Change

Hi everyone,

Today is my last day at my current job. I'm taking a huge risk by quitting, pulling my pension to pay off my debt and pay for rent so that I can take some time myself and really figure out the trajectory of the rest of my life. At 33, it might seem a little late but I just need a change in life to work on something that I've become passionate about.

Over the last few years, I've been immersed in the self-hosting, coding, web-development, networking, cyber security, all-things-IT world. I really want to continue to and build a new career from it. But as you can probably tell, I'm having a hard time focusing and deciding on which pathway I should take.

I have the most experience working with Docker and self-hosting although I have yet to deploy anything that doesn't get taken down within a week or so because I either screw it up or change my mind. I also have been working a lot with front-end web development using JavaScript, HTML, & CSS, mostly using ReactJS.

There's a reason why I have spent most of my time in self-hosting using Docker and frontend development with React. It's because I enjoy having ownership over my data and privacy (self-hosting) and love exercising my creative side which is where frontend web development comes in.

With all that being said, I'm just looking for some guidance or possible mentorship in continuing this journey. I find that I do better when there is structure in my life so I've been looking for some type of online course or even just a lesson plan that I follow. I don't want to go back to a traditional classroom structure, I don't think that'll work for me. Online courses and bootcamps are expensive and risky.

Does anyone have any suggestions, tools, resources, YouTube channels, lesson plans, or pathways that I should take?

Thank you!

https://redd.it/ym6o6i
@r_devops

How common areas devops jobs without on call

Throughout my career, I find that I in less interested the business domain, but more interested in technology. I care less about the features my company delivers to customers, but I care more about infrastructure (terraform, kubernetes, AWS, Azure, CI/CD build and deployment, grafana, elastic, security, service mesh, java, javascript). However, I dread being on call, and fire-fighting.

How common are these types of jobs? I find that these jobs that companies are hiring for usually involve being on call, and fire-fighting. I also find that jobs that are strictly backend (i.e., java, .NET), front-end (i.e., react) involve less on-call and fire-fighting. What are my options?

https://redd.it/ymatwn
@r_devops

My job title is “DevOps Engineer” but the work doesn’t line up. Help?

I 32F have been a “DevOps Engineer” for two years & made a switch from being a Big Data Engineer for three years.

I made the switch. I was headhunted, it’s a great company to have on my LinkedIn & the work that I was told we’d be doing sounded exciting but none of it has actually happened.

Since then, our team stack has changed to a point where I know that it’s not really DevOps anymore.

We do use Config Management tools being Puppet & Terraform. We no longer look after CI/CD tools being Jenkins & Spinnaker. It’s now maintained by Release Engineers in another team. We do look after Logging tools being ELK, InfluxDB & Grafana.

We were told that we would be looking at adding Docker as a containerisation tool only for that to be full-steam-ahead by Release Engineers.

I genuinely feel like a fraud. Having a job title with tasks & tools that don’t align. I’ve spent most of the year doing documentation & on-call on random things. I feel like tech support & I hate that.

I’m not growing at all. I’m incredibly bored. I’ve barely done any code all year. I’ve been doing a lot of self-learning to fill my knowledge gaps that it doesn’t look like I’ll get in this team.

I’ve been told we’re changing our names to SREs but that doesn’t make sense either.

I’m not a DevOps Engineer really, am I. Any advice?

https://redd.it/yle5jk
@r_devops

I have an Extra GCP exam voucher

I have bought two vouchers for myself to attempt PCA but fortunately pass the Exam in first attempt, so now I left with an extra voucher(You can use this voucher register for any GCP Exam or associate level exam) , If any of you are interested ping me, Price is very negotiable.

https://redd.it/ylxdui
@r_devops

Security scanning

Have you ever needed to convince your senior security engineer of static code analysis? For some reason ours does not see value in that and just says that we should focus on our biggest attack vector: social engineering and lost devices.

I think you should do it all but static code analysis is such a simple thing to catch a lot of stupid mistakes for such little effort, it is a low hanging fruit when compared to retraining the entire staffs security hygiene or hardening devices.

Thoughts?

https://redd.it/ymodt1
@r_devops

Course: Real world DevOps project from start to finish

Hello everyone,


I've made a DevOps course covering a lot of different technologies and applications,
aimed at startups, small companies and individuals.

To get this out of the way - this course doesn't cover Kubernetes or similar -
I'm of the opinion that for startups and small companies you probably don't need Kubernetes.
For small teams it usually brings more problems than benefits,
and unnecessary infrastructure bills buried a lot of startups before they got anywhere.

As for prerequisites, you can't be a complete beginner in the world of computers.
If you've never even heard of Docker, if you don't have even the slightest idea what DNS is,
or if you don't have any experience with Linux, this course is probably not for you.
That being said, I do explain the basics too, but probably not in enough detail
for a complete beginner.

The course is available at Udemy, and here's a 100% OFF coupon URL (1000 of those available):

https://www.udemy.com/course/real-world-devops-project-from-start-to-finish/?couponCode=FREEDEVOPS2211CJDUA

Be sure to BUY the course for $0, and not sign up for Udemy's subscription plan.
The Subscription plan is selected by default, but you want the BUY checkbox.
If you see a price other than $0, chances are that all coupons have been used already.

I encourage you to watch "free preview" videos, but here's the gist:

The goal of the course is to create an easily deployable and reproducible server
which will have "everything" a start up or a small company will need - VPN, mail, Git,
CI/CD, messaging, hosting websites and services, sharing files, calendar, etc.
It can also be useful to individuals who want to self-host all of those - I ditched Google 99.9%
and other than that being a good feeling, I'm not worried that some AI bug will lock my account
with no one to talk to about resolving the issue.

Considering that it covers a wide variety of topics, it doesn't go in depth in any of those.
Think of it as going down a highway towards the end destination, but on the way there I show you
all the junctions where I think it's useful to do more research on the subject.

We'll deploy services inside Docker and LXC (Linux Containers). Those will include
a mail server (iRedMail), Zulip (Slack and Microsoft Teams alternative), GitLab (with GitLab runner and CI/CD),
Nextcloud (file sharing, calendar, contacts, etc.), checkmk (monitoring solution), Pi-hole (ad blocking on DNS level),
Traefik with Docker and file providers (a single HTTP/S entry point with automatic routing and TLS certificates).

We'll set up WireGuard, a modern and fast VPN solution for secure access to VPS' internal network,
and I'll also show you how to get a wildcard TLS certificate with certbot and DNS provider.

To wrap it all up, we'll write a simple Python application that will compare a list of the desired backups
with the list of finished backups, and send a result to a Zulip stream.
We'll write the application, do a 'git push' to GitLab which will trigger a CI/CD pipeline
that will build a Docker image, push it to a private registry, and then,
with the help of the GitLab runner, run it on the VPS and post a result to a Zulip stream with a webhook.


If you apply the coupon, I'd appreciate if you leave your feedback on Udemy
after you're done and spread the word about the course. It would mean a lot.

If you'd like to support me, you can instead use this 80% OFF coupon:

https://www.udemy.com/course/real-world-devops-project-from-start-to-finish/?couponCode=80OFFDEVOPS2211FJKDA


I hope that you find it useful.


Happy learning,
Predrag

https://redd.it/yla3qp
@r_devops

Metrics, Logging and Application Tracing Solutions

Hi r/devops,

I am a K8s administrator in a relatively small cluster hosted in Digital Ocean. I was using Prometheus+Graphana for Metrics and Logging but there is a need for something more concrete and something that could be also used for Application Tracing. EFK Stack came to mind but ElasticSearch is a bit heavy and the cluster is already at its limits. While researching I found Datadog (that is a bit expensive) and New Relic.

What do you suggest? What are you using for Metrics, Logging and Application Tracing?

https://redd.it/ymsrgu
@r_devops

Getting Started, Automating WITH DOCKER

I have an idea for a project around automation and I'm not sure where to start.

I want a project/repo where when I check in code it spins up some docker containers to run tests. What terminology shoud I research, and what tools should I learn?

https://redd.it/ymrif8
@r_devops

Static Code Analysis Tools with CWE compatibility for bachelor thesis

Hello guys.

I am currently writing my bachelors thesis and I need to analyze Open Source Static Code Analysis Tools which are CWE compatible and have explicit mappings to the weaknesses defined in CWE. The focus of the tools should be on security. Do you have any recommendations for tools I should have a look at?

Thank you in advance

https://redd.it/ymxb9h
@r_devops

What is the difference between ECR and CodeArtifact?

Title.

https://redd.it/ymsug0
@r_devops