Company wide identity provider

In the future we want to use a central identity provider in our company to authenticate and authorize our internal and external users against our applications and APIs.

The project is mainly driven by middle management and product managers. I am in the role of the DevOps engineer and take care of the infrastructure of our applications. Deploying an identity provider kind of falls under my tasks.

I've been working with AWS for years, so a natural choice would be to go with Cognito. While I like AWS products in general, I have doubts about whether Cognito is the right choice. Especially since people who are not fit in AWS should also work with it (operate it). But also because Cognito cannot be customized to the same extent as other identity providers.

In our company, there is currently only (dangerous) half-knowledge on the subject. I'm familiar with the basics of OAuth and OIDC, and I'm just getting more familiar with how it works. However, I am still a bit confused about which criteria to use to select a suitable identity provider.

My questions are: How should I proceed? Which criteria are decisive in your opinion? Should the software developers have a say in the selection? Should we hire an expert to assist us? Is there anyone here who has switched from Cognito to another authentication service? If so, why? Is there another recommendation? Is self-hosting a viable option?

Although this might not be something DevOps engineers usually do, I really enjoy digging into the nuts and bolts of this topic. I feel confident, that given enough time, I can make an informed decision. But management wants us to start as soon as possible with the first prototype.

https://redd.it/117yqln
@r_devops

Company wide identity provider

In the future we want to use a central identity provider in our company to authenticate and authorize our internal and external users against our applications and APIs.

The project is mainly driven by middle management and product managers. I am in the role of the DevOps engineer and take care of the infrastructure of our applications. Deploying an identity provider kind of falls under my tasks.

I've been working with AWS for years, so a natural choice would be to go with Cognito. While I like AWS products in general, I have doubts about whether Cognito is the right choice. Especially since people who are not fit in AWS should also work with it (operate it). But also because Cognito cannot be customized to the same extent as other identity providers.

In our company, there is currently only (dangerous) half-knowledge on the subject. I'm familiar with the basics of OAuth and OIDC, and I'm just getting more familiar with how it works. However, I am still a bit confused about which criteria to use to select a suitable identity provider.

My questions are: How should I proceed? Which criteria are decisive in your opinion? Should the software developers have a say in the selection? Should we hire an expert to assist us? Is there anyone here who has switched from Cognito to another authentication service? If so, why? Is there another recommendation? Is self-hosting a viable option?

Although this might not be something DevOps engineers usually do, I really enjoy digging into the nuts and bolts of this topic. I feel confident, that given enough time, I can make an informed decision. But management wants us to start as soon as possible with the first prototype.

https://redd.it/117yqln
@r_devops

Calculate replicability score on a project/repository

In my current company, we are trying to renovate our work methodology and one of the goals is to create project assets and/or libraries that can be replicated across multiple clients in order to accelerate development and generate marginality.

We have been asked to generate an internally shared replicability score of a project based on language, technology, and complexity.

Based on your experience, if you were to calculate a replicability score for a project what metrics would you use? What analysis strategy would you use?

https://redd.it/1180ijl
@r_devops

Site Reliability Engineer - Automotive AI experience - Available Immediately - USA

Hi all,

Using this platform more as a punt more than anything else.

I've been referred a very talented Site Reliability Engineer who has been laid off recently by one of US's biggest AI organisations. Mid-way through a very difficult personal period, he has reached out to myself and one other recruiter for opportunities on the market. Unfortunately, the opportunities I have for him would require him to be on-site atleast once a week but prefers remote.

If there are any hiring managers in the US who are looking for great SRE talent, this candidate can be vouched for by his recent and previous organisations and has refrained from using Linkedin because of past bad experience with external recruiters.

Happy to share some more details about his profile, please feel free to DM me. He's available for interview early next week.

https://redd.it/1181sdq
@r_devops

In-terminal debugging and CLI assistance https://github.com/HeyCLI/heyCLI_client

Hello all,

HeyCLI is a GPT3 powered command line helper that allows you to use the terminal in natural language. See [https://heycli.com](https://heycli.com)

We have added two new features to HeyCLI:

1. In-terminal debugging of python, nodejs, kubectl, gcloud, aws and many other command errors. When you get an error, just type: hey debug the error above (or something like that, be creative!) 
2. we attached HeyCLI to some commands so you can type "yes" to execute them. 

To try these features, follow the instructions here [https://github.com/HeyCLI/heyCLI\_client](https://github.com/HeyCLI/heyCLI_client)

https://redd.it/118071p
@r_devops

How to get notified on Error Events in NewRelic?

I am currently doing this in the browser:

newrelic.noticeError(err);

I am seeing them arrive in New Relic/Events, but how do I make it to get notifications via Email when error is logged via client side?

https://redd.it/11842bs
@r_devops

Bootstrapping and updating CI/CD permissions

I'm trying to determine the best way to bootstrap and update permissions for CI/CD minimizing manual steps but also keeping it secure, least privilege. I'm sure this is a common need. I'd like to understand good practices around it that I might be missing.

For example, setting up a GitHub repository with GitHub Actions managing infrastructure and apps in AWS. In order for GitHub Actions to have authorization, I need to add an OIDC provider and a role restricted to the GitHub repo with a policy with reasonably least privilege.

I'm OK with this being done by checking out the repo and running the initial IaC with local AWS permissions (I use SSO so that no long-lived Access Keys in \~/.aws). Bootstrapping done, no problems.

But now as the applications grow, more permissions are needed by GitHub Actions to manage new infrastructure. I don't want GitHub Actions in particular to have Administrator Access on the account. So to add the new permissions, I would update the permissions in the bootstrap IaC and apply manually.

Everything is still in git. Recreating or replicating the environment wouldn't have any manual steps apart from running the bootstrap IaC. It seems like the way to go.

The only thing that rubs me the wrong way is that permissions to manage app specific resources would have to be managed outside of the app. They could be pulled in from the app repo/folder and the bootstrap admin would have to verify the diff when applying it.

So my question is: How do you handle the bootstrapping and updating of bootstrapped permissions for your CI/CD?

https://redd.it/1184jz3
@r_devops

CDKTF FAQ

I was recently tasked with a CDKTF project. As I started to dig into CDKTF, I had a lot of questions. I realized that there wasn't really a good FAQ online so I decided to share what I've learned. CDKTF can be kind of confusing as it's new and I suspect many people have questions around it (I saw a question the other day here about it). I plan on updating this FAQ it as I continue to learn more about CDKTF.

https://terrateam.io/docs/cdktf/faq

https://redd.it/11881mp
@r_devops

Packer provisioning on a Ubuntu 20.04 refuses to work

Trying to create a golden image with cloud-init support on Proxmox.
Without any provisioning I can get a iso/template but as soon as I want to configure something with Packers provisioning shell it just hangs forever. I could work around this with further provisioning using Ansible & Co but I refuse to accept defeat in this case.

Packer + Proxmox = Custom Cloud-Init template, then use Terraform to create actual VMs

I know I can use any cloud image from Ubuntu/Debian/etc and let Terraform use it as template however I wanted to include this as a step. For now I can continue to use my CI/CD starting at Terraform but I want to get this working.
Any reference I saw so far was just dishing out shell provisioning commands with sudo without any further tweaks. I let it run for hours just to confirm I wasnt too impatient.

PACKER_LOG=1 shows nothing, output just straight up stops when it should run the specified commands

There is a current problem with Proxmox 7.3 and Packer Plugin 1.1.1, forcing me to use 1.1.0
Also no DHCP, thats why I set the network settings manually

hcl file

packer {
required_plugins {
proxmox = {
version = "1.1.0"
source = "github.com/hashicorp/proxmox"
}
}
}

source "proxmox-iso" "proxmox-ubuntu-20" {
proxmox_url = "url"
vm_name = "packer-ubuntu-20"
iso_url = "https://www.releases.ubuntu.com/focal/ubuntu-20.04.5-live-server-amd64.iso"
iso_checksum = "5035be37a7e9abbdc09f0d257f3e33416c1a0fb322ba860d42d74aa75c3468d4"
username = "user"
password = "pw"
token = "token"
node = "proxmox"
iso_storage_pool = "local"

ssh_username = "packer"
ssh_password = "ubuntu"
ssh_timeout = "20m"
ssh_pty = true
ssh_handshake_attempts = 20

http_directory = "http"
boot_command = [
"<esc><wait><esc><wait>",
"<f6><wait><esc><wait>",
"<bs><bs><bs><bs><bs>",
" ip=${cidrhost("10.0.104.0/24", 80)}::${cidrhost("10.0.104.0/24", 1)}:${cidrnetmask("10.0.104.0/24")}::::${cidrhost("10.0.104.0/24", 1)}",
" autoinstall ds=nocloud-net;s=https://{{ .HTTPIP }}:{{ .HTTPPort }}/ ",
"--- <enter>"
]
boot = "c"
boot_wait = "5s"
insecure_skip_tls_verify = true

template_name = "packer-ubuntu-20"
template_description = "packer generated ubuntu-20.04.3-server-amd64"
unmount_iso = true

memory = 4096
cores = 1
sockets = 1
os = "l26"
qemu_agent = true
cloud_init = true
cloud_init_storage_pool = "local-lvm"
# scsi_controller = "virtio-scsi-pci"
disks {
type = "scsi"
disk_size = "30G"
storage_pool = "local-lvm"
storage_pool_type = "lvm-thin"
format = "raw"
}
network_adapters {
bridge = "vmbr1"
model = "virtio"
firewall = true
vlan_tag = 104
}
}

build {

name = "ubuntu-server-focal-docker"
sources = ["source.proxmox-iso.proxmox-ubuntu-20"]

# Provisioning the VM Template for Cloud-Init Integration in Proxmox #1
provisioner "shell" {
inline = [
"while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done",
"echo 'Sudo now'",
"sudo rm /etc/ssh/ssh_host_*",
"sudo truncate -s 0 /etc/machine-id",
"sudo apt -y autoremove --purge",
"sudo apt -y clean",
"sudo apt -y autoclean",
"sudo cloud-init clean",
"sudo apt update -y",
"sudo apt upgrade -y",
"sudo rm -f /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg"

"sudo sync"
]
}

# Provisioning the VM Template for Cloud-Init Integration in Proxmox #2
provisioner "file" {
source = "files/99-pve.cfg"
destination = "/tmp/99-pve.cfg"
}

# Provisioning the VM Template for Cloud-Init Integration in Proxmox #3
provisioner "shell" {
inline = [ "sudo cp /tmp/99-pve.cfg /etc/cloud/cloud.cfg.d/99-pve.cfg" ]
}
}

user-data

#cloud-config
autoinstall:
version: 1
locale: en_US
keyboard:
layout: de
ssh:
install-server: true
allow-pw: true
disable_root: true
ssh_quiet_keygen: true
allow_public_ssh_keys: true
identity:
hostname: packer-ubuntu-20
password: "$6$qiEN6LwtNwuOZoim$8nvdVicI/.oDb5W4ynnyhToYKegBUGDEWgomK6kymT6xalkuQaoqHhAY4xcurVQ50wDEBhF.OzHUKkm4NvoNe/"
username: packer
realname: packer
packages:
- qemu-guest-agent
- sudo
storage:
layout:
name: direct
swap:
size: 0

https://redd.it/1189igt
@r_devops

Learning So Much Info & It's Hard To Juggle It All

Hey all,

Feel free to remove this post for redundancy reasons - it's not much different than what's been asked many times over on this sub.

I've been following the DevOps roadmap (I know some of you find this controversial - I understand), and just to clarify, I graduated last year with a Bachelor's in Computer Science, so I already have a decent development background (although I could definitely improve, which is something I am working on). I find all of this a bit overwhelming, but I have a strong desire to learn, and I spend time (I try to get to 2 hours of concentrated learning after work, I'm 22 years old and no other obligations mostly) after work learning every single day.

I'm currently in more of a Security-based role at a very large company, which does have me integrated with my company's cloud architecture, so I am learning AWS naturally from that, and would like to get the CCP cert in time. I like the work, and I think some of it will definitely translate to my next job, but DevOps has been an attraction ever since I graduated, and I like the idea of touching so many different tools and technologies, and there's a ton to learn (which again, I enjoy).

I guess the point I'm getting at is, how do you (or did you, depending on your current experience level) juggle all of this new information? For example, I was never exposed to Linux in detail in school. Currently it's something that I'm diving into deeper, but it takes time. Another example is Networking - Networking is huge. I want to learn a ton about Networking, it's interesting! But I find myself frustrated sometimes with the knowledge gaps I have on different topics.

I don't want to just be an average engineer, I want to be good. I'm willing to put in the time. I guess I'd just like some advice on how to juggle all of this information. I know I'm young and have a lot to learn, but I don't want that to be an excuse to take things slow (within reason).

Thank you all!

https://redd.it/118bwzu
@r_devops

Am I being ripped off?

Some context: I job switched internally at my company to be an Associate DevOps Engineer. (I work at a small startup >500 employees). They had an opening for a full DevOps engineer and I told HR I was interested in joining, but didn’t have nearly enough experience. HR ended up telling the team and the hiring manager for the role contacted me and basically said he’d love to bring me on but as an associate, and they’d have to hire a senior to help train me. I didn’t know much about DevOps, just knew a ton of networking, Python, and some bash/linux.

Long story short eventually I got brought on the team and I’m about 3 months in. The only downside is I’m making the same as what I made in my previous role (entry level digital forensics) - $55,000. I love the team and it’s so cool learning all the tools (K8s, Docker, Azure, Debian, Terraform, etc) but I’m definitely doing more work than I was before. There was no pay increase as it was seen as a lateral movement.

The upside I see is that I don’t have to spend 10 yrs as a sys admin to break into DevOps. And I get to learn pretty much everything or anything I want.

Am I being ripped off? Should I be making more? Or is this worth it for the experience?

https://redd.it/118d7ax
@r_devops

is practical DevSecOps worth it ?

Hey guys
I'm considering taking this: https://www.practical-devsecops.com/certified-cloud-native-security-expert/

As I want to establish myself in k8s and cloud native security but the price tag is insane (1k USD) considering that I'm in a country with a relatively weak currency compared to the dollar. I don't care about the certs, I'm after the skills I'll gain in the process

If you think it isn't worth it and there are other resources to learn what this cert offers let me know

https://redd.it/118h416
@r_devops

Learning Azure vs AWS

I can't for the life of me understand Azure as much as I do with AWS. I feel as though Azure is more granular than AWS which makes it a tad bit more difficult, at least for me, to learn. Is this normal or just me?

https://redd.it/118hsdv
@r_devops

Has your org started to evaluate the pros/cons of purchasing ChatGPT Premium?

Just curious what all is going on out there related to adoption of AI technologies like this for professional purposes. Are you starting to evaluate it in any serious way? Is the mere suggestion enough to get laughed out of the stand up?

https://redd.it/118e0yz
@r_devops

Updating stuff inside a pod

Our app's SSL certificates will expire in a few weeks. The app is deployed in a K8's cluster and to updates the certificates we're supposed to exex into the pod and upload the certs into a folder, then run s script to "activate' them. I'm still a Kubernetes noob, so I'm confused: would these changes persist when the pod is deleted and a new one created?

https://redd.it/1188khv
@r_devops

How do i setup ci cd pipeline from gitlab to ECS 2 for next js?

Hi I’ve been looking for tutorials on this and I can’t seem to find one that incorporate gitlab even if I find one they are either not working or outdated. Can someone tell me how to set one up for next js ? Or if I can may I message any of you that are capable I mostly work on front end and backend but never on devops or ci cd now I am confused. Thanks

https://redd.it/118ls12
@r_devops

what do you use shell scripts in your day to day work as a devops practitioner?

I've learnt the syntax and understood shell scripting, I'm finding it hard to understand where exactly do one use in their every day practices as a devops engineer.

https://redd.it/118n051
@r_devops

TF401025 'repoName' is not a valid name for a Git repository - solved

I'm just dumping this here, hoping to help someone googling this issue in the future.

When trying to create a repository named 'Server' you get the error TF401025
'repoName' is not a valid name for a Git repository or
'newRepositoryName' is not a valid name for a Git repository

You just can't name a repository 'Server' in DevOps, as explained here

https://redd.it/117zykd
@r_devops

Should we use OpenTelemetry traces for running tests?

Writing tests that check that side effects across the system are actually happening was always difficult (for me at least).

Then came OpenTelemetry, which basically gives you amazing visibility on what's happening across your system. So if you want to check that a user gets an e-mail everytime they register - theoretically you just need to search for the right spans.

We thought it might be an interesting concept to explore, so we started writing an open source testing framework that does exactly that - with Jest. We've actually managed to get into YCombinator with that so we can work on it full time.

Does it make sense? Would you be using something like that?

(Adding here our repo - https://github.com/traceloop/jest-opentelemetry, although this honestly doesn't mean to be a self-promoting post. I'm actually looking for honest feedback).

https://redd.it/118q0xn
@r_devops

Is anyone great at drawing (have a strong artistic side)?

So I think networking/devops is all very interesting, but I have this strong skill in drawing very realistic art. I'm not trying to brag, but I can really draw some great stuff - professional portrait artist level.

Sometimes I feel like this skill is going to waste and people say "oh you should just do frontend/UI developer" type of work.

Are there any devops or network/systems engineers that have a strong drawing/artistic side? Maybe there is some link here in terms of visual talent and working with computer systems?

Btw, this post is not a joke. I seriously am at odds sometimes in terms of choosing/settling on a career path due to having a strong artistic/drawing side. If someone could chime in or offer advice it might really help settle a lot of internal conflict that I have.

https://redd.it/118qixb
@r_devops