Terraform, pull docker image from ECR

Hello everyone, I'm a software engineer transitioning into DevOps and recently began working with Terraform. I must say, I'm loving it! Terraform is an amazing tool. Currently, I'm working on small projects involving Lambdas, S3, and more.
My latest task was to deploy a Node.js container on an EC2 instance. I've managed to set up almost everything successfully. Here's a snippet of my EC2 instance configuration:

resource "awsinstance" "ec2instance" {
dependson = aws_iam_role.ec2_role, aws_ecr_repository.ecr_repo
ami = var.instance
instancetype = var.instancetype
subnetid = awssubnet.main.id
keyname = awskeypair.ec2keypair.keyname
vpcsecuritygroupids = [awssecuritygroup.instancesg.id]
associatepublicipaddress = var.allowpublicip

iaminstanceprofile = awsiaminstanceprofile.ec2instanceprofile.name

userdata = <<-EOF
#!/bin/bash
sudo apt update -y
sudo apt install -y docker.io awscli
sudo service docker start
aws ecr get-login-password --region ${var.region} | sudo docker login --username AWS --password-stdin ${awsecrrepository.ecrrepo.repositoryurl}
sudo docker pull ${awsecrrepository.ecrrepo.repositoryurl}
sudo docker run -d -p 80:3000 ${awsecrrepository.ecrrepo.repositoryurl}
EOF
tags = {
Name = var.instancename,
"source" = "terraform",
"environment" = terraform.workspace
}
}

The challenge I'm facing is with running the "user_data" section. The steps seem correct, and when I print the variables and execute the steps individually over SSH, everything works fine. I can even access my application. So, it appears that the steps are correct.
I've confirmed that Docker and the AWS CLI are installed, and I can successfully log in. When I SSH into the instance, I can pull the Docker image without any issues and run it.
What could I be missing? Any insights would be greatly appreciated!

https://redd.it/1cfv9ia
@r_devops

Should I use AWS for my (temporarily) local startup or manage my own server?

So I've started planning to develop an app, deploy it in my home city first, with a young audience. Keep in my mind, I'm also in college. So I'm thinking of keeping costs as low as possible, and given that only one region needs to be served I thought that would be simple enough. Let's say (if we're lucky) the platform reaches a total of 10 thousand daily active users, which is roughly 7% of the young population of the city. The idea is that we first launch in this metropol we live in, and If we reach a decent amount of users and MRR we expand.


Anyways:

I was looking into how I would deploy the app once I actually develop it, so I took a look at AWS. I was surprised to see that a EC2 service with 4 vCPU and 8GiB memory (with 2 Baseline and 4 peak instances) costs around 200 dollars a month. RDS for Postgres was also unexpectedly expensive.I didn't even bother to check the rest (S3, Monitoring, CI/CD Pipeline,...). I then remembered that the little VPS I had started renting from a more local provider (for silly personal projects and some websites) costs me only 6 euros a month, and that is advertised as 4 vCore (x86), 4 GB RAM, 80 GB SSD (RAID10), 80 TB Traffic. Increase the budget to 100 euros a month, and you've got yourself a dedicated server with 24 cores and 128 GB of RAM. Sure managing my own VPS was a bit of a headache, and I understand the amazon perhaps performs at a way higher standard, but this price difference is just mind boggling to me. If I'm misunderstanding something about these pricings, please point it out.


My question is, given these price differences, what (other than the hundreds of additional services, that are totally unnecessary for my business, and the easy scaling) would be reasons to still choose AWS?

https://redd.it/1cfzzwn
@r_devops

KusionStack step inside Platform Tooling Landscape

It's true, KusionStack has recently been included in the Platform Tooling Landscape Platform Orchestrator category. It is also the first-ever open-source project which included in both the CNCF Landscape and the Platform Tooling Landscape that focuses on the field of Platform Engineering.

https://redd.it/1cg06j3
@r_devops

Should each component of a project live on its own server/instance?

A basic question regarding architectural best practices!

I'm setting up an open access data portal where people can visit a website and find visualised datasets related to my industry.

I've identified a few open-source components to help do this well (it's a non-profit idea so that has swayed a lot of the decision-making so far):

\-> Metabase for the frontend / data visualisation

\-> Airbyte for data pipeline management

\-> PostgresSQL (offsite) as a standalone cluster for the data itself

\-> Wordress/Ghost/Drupal for a blog

Approaches I've considered:

\-> 1 large VPS running all the moving pieces with DNS routing onto the various components (say, pipeline.myproject.com pipes onto Airbyte, etc, etc).

\-> Each component lives on its own VPS with a VNC in the background for efficient communication between the "nodes"

And my question:

Is there a right option and a wrong option here? Is instance-per-component more ideal?

TIA!

&#x200B;

&#x200B;

https://redd.it/1cg30r2
@r_devops

url redirect to internal app system design

is there any opensource app with url redirect similar to team1.slack.com team2.slack.com

was looking for any app,db with inbuilt url redirect to internal microservices on github opensource apps public

tried to search

https://slack.engineering/?s=system%20design

https://redd.it/1cg1kzo
@r_devops

Terminal Interface for Hashicorp Vault

Hi everyone!
Just wanted to share a tui app, I've been working on for a while.
It's inspired by other popular ones like k9s/wander etc, to give you a nice interface while never leaving your terminal for your work with Vault.
Repo: https://github.com/dkyanakiev/vaul7y


Hopefully this brings more people to it so they can give it a spin and share any feedback and/or bug to report!


&#x200B;

https://redd.it/1cg5tgc
@r_devops

What would you say if your Boss suddenly says you are responsible for some legacy python2 app that is the core of the platform and have to move it to python3 together with multiple Other improvements to the code base (as a devops)

Just curious if something like this happened to you and how did you react / proceeded.

Also curious how many of devopses suddenly become responsible for parts of the core products (maintenance/development)?

Edit. This is not a small app, its hundreds of thousends of lines in frameworks that are not maintained in over 5 years, written over few years.

https://redd.it/1cg9jrb
@r_devops

SIEM OVERAGES

👉I’m looking for IT and #DevOps pros who are struggling with their Splunk spend and interested in reducing those overages!(with or without replacing their SIEM) 💭Let’s connect for 30 minutes! You’ll learn about the latest in observability tooling and use cases!!! In exchange for 30 minutes of your time you’ll also be fed🔥🔥🔥 Let’s do this!!! Your sharing of this post will be equally appreciated 😇

https://redd.it/1cggiaa
@r_devops

User monitoring

What tools or software do y’all use to monitor user behavior. Things like what buttons a user clicks on, how long they stay on pages, and other data used to analyze user experience.

https://redd.it/1cghqfm
@r_devops

An "all-in-one" cli tool vs separate tools

We currently have a repository with a hundred different scripts and binaries all written in different languages that people took on as side projects to automate parts of our job.

There is an incentive to move all of these tools into a single Golang CLI application, but there are some drawbacks. We currently don't have any quality control or ownership of these tools whatsoever, so we need to figure out some sort of process for this. Besides, none of the team members have a development background, so it's been pretty wild what you see in the code of the current tools.

Would love to hear some thoughts on whether this would be a good/bad idea and what to look out for.

https://redd.it/1cghg2e
@r_devops

Will anyone be at RSA in MAY ?

Would love to meet other individuals in the same space

https://redd.it/1cgghmd
@r_devops

An article citing important S3 bucket pricing "vulnerability": How an empty S3 bucket can make your AWS bill explode

Thought it was important to disseminate the lessons in this blog post:

S3 charges you for unauthorized incoming requests
Anyone who knows the name of any of your S3 buckets can ramp up your AWS bill as they like.
Adding a random suffix to your bucket names can enhance security.
When executing a lot of requests to S3, make sure to explicitly specify the AWS region.

Read more here: https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-can-make-your-aws-bill-explode-934a383cb8b1

https://redd.it/1cgklx9
@r_devops

Learning AWS

What is the best way to learn how to use AWS without paying to much money?
Note: I have previous experience in using MS Azure products.

https://redd.it/1cgld0o
@r_devops

Automate investigations for common issues

Hi everyone, I've developed an open source framework, Playbooks, to standardize the investigation process for on-call issues. Drawing from my experience as a service owner of critical services, I created Playbooks to consolidate and automate the expertise of senior engineers. This framework allows you to define steps in a playbook that automatically executes upon receiving a related alert, and then delivers a summary of the findings. It's designed to expedite issue resolution and reduce on-call escalations.
Check it out and contribute here: Playbooks on GitHub. https://github.com/DrDroidLab/playbooks
If you frequently handle on-call duties, I'd appreciate your thoughts on how this could help streamline your workflows and make incident management more efficient.

https://redd.it/1cglqro
@r_devops

Python or Golang for Long term?

I have 3 years of overall experience in Software and IT Engineering.

For automation, I have used Shell Scripting so far. But Now I want to learn a new language for Automation tasks and Software Development too.

The project I'm working on uses Golang extensively but the leads have given me free hand to explore any language I want.

My Initial thoughts were to learn Python, keeping in mind future career aspects, but Golang is used in my Project. Since I have planned to stay here for next 2 years, I'm confused which language to choose at this point of time.

Your opinions/suggestions/advices are most welcome!

https://redd.it/1cgm5fl
@r_devops

A list of Free DevOps Courses from Coursera.

Learn Docker, Kubernetes and Azure DevOps from Free Courses on Coursera. Here are 7 free courses from Coursera which you can take for free without any Certificate. **Main Article**

https://redd.it/1cgs7u1
@r_devops

Moving away from docker, old school setup ideas?...

We have this server (VPS) running docker and deploying via docker and it's been a game of babysitting docker since the origin of time; mysterious crashes that only occur inside of docker, segmentation faults inside JavaScript code only on docker, it's slow, network configuration is a nightmare, and the list adds up.

Finally I am fed up with it and I am thinking of moving away from it, the system works via clusters (each cluster is a VPS), and three main databases.

The databases were taken away from docker after they had terrible performance and mysterious crashes; so they are now dedicated and have been working fine ever since.

But now the cluster is to be moved away, the cluster is just composed of a nginx balancer/SSL, a redis cache, and a bunch of self spawning NodeJS services; the nginx server load balances; the NodeJS services can crash, for example due to bad code or some sort of serious error or oversight, they log what happened, sending a warning over the databases, before committing seppuku.

And a new instance replaces it, on the same port.

Multiple clusters can work at the same time, independently of each other, they are technically copies other than the master cluster.

The cluster is the part that needs consistent redeployment, over different machines in different geographical regions; one may be in a dataserver, other may be on a potato for a machine somewhere in Africa; you ssh in the machine, stop the cluster, rsync the new system, start the cluster; but clearly something must watch all the processes, something must determine the scale, something must install and setup the dependencies (nginx, node and redis).

I could write a bash script by hand to do all this, make style (like a makefile), or maybe there's something better I am ignoring.

https://redd.it/1cgtw97
@r_devops

Open Source Datadog Guide

We published an open source guide to help customers make sense of datadog. It's meant as a reference for engineers and organizations that want to make the most of their datadog usage and avoid serious gotchas that could result in significant higher costs or effort. It's still early days and the guide is far from comprehensive - please help by contributing 🙏

https://github.com/nimbushq/og-datadog

https://redd.it/1cgv55f
@r_devops

OpenTofu 1.7.0 is out with State Encryption, Dynamic Provider-defined Functions

Hey there, technical lead of the OpenTofu project here!
We’re proud to announce that OpenTofu 1.7.0 is now officially out!
It includes State Encryption, Provider-defined Functions, Declarative removed blocks, loopable import blocks, and much more!
You can find the launch post here, as well as the release itself here.
Looking forward to hearing what you think!

https://redd.it/1cgwgec
@r_devops

How do you get your mojo back after burnout?

A few years ago I burnt myself out, like really burt out. I spiraled for a bit, had a leave of absence but eventually got my feet back under me.

I'm now trying to get back in my groove but I can't find it for the life of me. I've tried a few different positions at different companies but no matter what the work is I just can't do more then the bare minimum. I admit part of my spiral was due to the lack of recognition for the work that put me into it. I think this plays a big role in this but I just can't get my mojo back.

https://redd.it/1cgw05j
@r_devops

Developer experience session with Confluent's platform team

We are hosting a Zoom session with Confluent to talk about their internal pluggable service runtimes - language-specific, plugin-based component frameworks with the most common components that teams need included out-of-the-box.


About the guest:
Cody is an OG at Confluent currently managing the Platform Engineering Team (Service Foundations team). https://www.linkedin.com/in/codyaray/


When: Friday May 3rd, 2024 at 11am PDT | 2pm EDT
RSVP: https://forms.gle/Td1xzX8iFXTbdmAq9

https://redd.it/1cgyq8k
@r_devops