AWS Permission Bouncers: Letting Loose in Dev, Keeping it Tight in Prod

Hi there, I wrote a blog post that y'all may be interested in. It discusses how to manage cross-account IAM permissions for different teams. Would love feedback!

https://www.noq.dev/blog/aws-permission-bouncers-letting-loose-in-dev-keeping-it-tight-in-prod

https://redd.it/136tnkd
@r_devops

remote connection to GCE Vm machines

Not sure if this is appropriate sub, but how do you guys connect to your vms within GCE with no external interfaces? For now we just have openvpn server through which most of the devs connect. I know that there is Cloud Interconnect and Cloud IPSec VPN native solutions, but they all assume org on-premises network which we don't really have since all of us work remotely. What are best practices here? Thank you in advance and if this isn't appropriate sub feel free to delete.

https://redd.it/136pmjo
@r_devops

Sysadmin/DevOps Engineer without Terraform experience looking for relocation feeling hopeless advice needed.

Hi everyone.

I am Sysadmin or DevOps Engineer without Terraform experience. I have 4 years of experience. I spend 2 years in on-premise environments doing mostly dealing with CI/CD pipelines, deployments and general infrastructure and apps support, after that 2 years I joined a startup from its first moments as a only guy for infrastructure and build infrastructure based on AWS, EKS, RDS. However I did not have an environment to use Terraform collaboratively, and mostly used console and eksctl when building.

In summary, I have knowledge about cloud, kubernetes, databases, ci/cd, helm, monitoring, little bit of gitops and even understand how terraform and terragrunt works and wrote the same infrastructure i created in terraform but i have no real experience.

I need to relocate and even though i pass first 2-3 interviews, I get rejected due to having no real experience with terraform. Unfortunately i have reasons to relocate, if it is not going to happen, I will have to seek asylum after my visa expires.

I am thinking to getting certifications of Terraform and AWS however I don't know it will help or not.

What will be the next steps that can help?


Thanks.

https://redd.it/136yccz
@r_devops

Deploying Similar Applications on Kubernetes

We started our K8s journey in 2022 Q3. We have a dev cluster. The stack is pretty well fleshed out with 3rd party software. We have CI and CD working with a test application that is similar to our 1st party software. I could manually create the manifests for our applications like I have for this test application, but I'm thinking about this from an automation perspective.

Our applications are similar in that they made up of the same K8s resources: Namespace, Deployment, Service, ServiceMonitor, NetworkPolicy, PodDisrutionBudget, and so on. I wonder if we can use a tool to template the resources and deploy these applications all using the same template.

Is this a use case for Helm? Could we use a single Helm chart to create the manifests for all similar applications?

https://redd.it/136zmd0
@r_devops

OWASP ZAP and Jenkins

Has anyone made OWASP ZAP work with Jenkins to trigger DAST security scans automatically? Or is there another way to make sure I am scanning everything?

https://redd.it/136zhlx
@r_devops

How do you scale Longhorn in k8s?

Hi all,

My current research is how to implement Longhorn in DigitalOcean managed kubernetes environment and below I need to use DigitalOcean Volumes. All is good until I wonder about the scaling. I have more than 10TB of data that is being generated/used by 10-15 pods and the tendency is the data and pods to grow.

So how do Longhorn scales? How he can automatically increase it's volume? How does it work under?


I really appreciate any help you can provide.

https://redd.it/136pguj
@r_devops

tool for pushing AWS costs in analytics tool like Metabase, Looker etc..

Hi devops community,

Need some advice/suggestions on a new tool I've written, the scope for this tool if it's made open source for the community etc..

Context:

Our CFO, and finance team has no access to AWS console but there came a requirement for them to visualise the costs data in Metabase so they can see the monthly and daily costs for each AWS service and can see and prepare internal cost reports for AWS INFRA, as we use Metabase for our analytics purpose they asked me if this can be pushed to Metabase.

Currently, I couldn't find any tailor-made open-source solution for this, although Vmware has something called CloudHealth tool that does something similar but it's super expensive.

So using the AWS SDK for Go I was able to write a tool that gets the cost data daily, push to Mysql DB and from there Metabase can read it and the analysts team can perform the required action on this data via Metabase.

A snippet of the output is give below: https://ibb.co/2y3DQqC

So my ask is how helpful is this tool per your opinion?
If I’m able to helmify everything and put this in a Github repo as open source so anyone can install it, will it help anyone with a similar-use case ? What's the scope for this project?

Can this be converted into an OSS product idea?

Need your valuable feedback folks :)

https://redd.it/13779d7
@r_devops

Tips on keeping track of complex code...?

I dont think i had this problem during college, but since ive joined a company and started working with very very big and complex code.. i have a very hard time remembering "where things were" especially when reading code.

For example there are functions that create objects and this points to there and that function actually calls this and that and blah blah blah.... u need to connect all this..

I just get really frustrated keeping track of these intermediate info.. and when it gets complicated my brain kinda shuts down. I think you need to be able to some sort of remember where things were or at least save some in your memory to keep on going and read code.

Has anyone struggled with this before? Are there any tips that made you better at this? Such as practicing some thinking patterns, tools or writing it down.

Also does this get better by time? I think some people are just good at this compared to other people.

https://redd.it/1376knz
@r_devops

DevOps Learning Series Infrastructure as Code (IaC): The Foundation for Automated Infrastructure Management

Series index

It's been a couple of weeks since the last post, and I'm excited to dive deeper into the world of DevOps. Today, we'll be discussing Infrastructure as Code (IaC), a critical component of automated infrastructure management in DevOps practices. We'll explore the benefits, key concepts, popular IaC tools, and walk through some specific examples to give you a good beginning to start understanding this essential aspect of DevOps.

What is Infrastructure as Code?

Infrastructure as Code refers to the practice of managing and provisioning infrastructure resources (such as networks, servers, storage, etc) using code and version control systems, rather than relying on manual configurations or ad hoc scripts. IaC enables organizations to treat their infrastructure in a similar manner to their application code, thus ensuring consistent, repeatable, and automated deployment of environments.

Key Benefits of Infrastructure as Code

Adopting IaC brings several significant benefits to organizations, including:

1. Consistency and predictability: IaC enables organizations to maintain a standard, version-controlled infrastructure configuration, which minimizes configuration drift and ensures consistent deployment across environments.
2. Faster provisioning: IaC allows for rapid and automated provisioning of infrastructure resources, reducing the time and effort required to set up new environments.
3. Increased collaboration: By managing infrastructure code in a version control system, development and operations teams can collaborate more effectively, share knowledge, and establish a single source of truth for infrastructure configurations.
4. Enhanced security and compliance: IaC allows organizations to enforce security best practices and regulatory requirements throughout the infrastructure lifecycle, improving security and auditability.

Popular Infrastructure as Code Tools

There are several IaC tools available, each with its own strengths and trade-offs. Some of the most popular IaC tools include:

1. Terraform: An open-source IaC tool by HashiCorp that enables provisioning and management of infrastructure resources across multiple cloud platforms through declarative configuration files.
2. AWS CloudFormation: An AWS-specific IaC service that allows users to define, manage, and provision AWS resources using JSON or YAML templates.
3. Azure Resource Manager (ARM) Templates: A native IaC solution for Microsoft Azure that enables users to define, deploy, and manage resources through JSON templates.
4. Google Cloud Deployment Manager: An IaC service for Google Cloud Platform that automates the creation, deployment, and management of resources through YAML configuration files.

Example: Creating and Managing Infrastructure with Terraform

To better understand IaC in action, let's take a look at a practical example using Terraform. Imagine you're working on a web application project that requires a load balancer, two web servers, and a database server. With Terraform, you can create a configuration file (using HashiCorp Configuration Language, or HCL) to define these resources and their dependencies.

Here's a simple, and untested, example of what your Terraform configuration file might look like:

resource "awsvpc" "example" {
cidrblock = "10.0.0.0/16"
}

resource "awssubnet" "example" {
vpcid = awsvpc.example.id
cidrblock = "10.0.1.0/24"
}

resource "awssecuritygroup" "example" {
vpcid = awsvpc.example.id

ingress {
fromport = 80
toport = 80
protocol = "tcp"
cidrblocks = ["0.0.0.0/0"]
}
}

resource "awslb" "example" {
name = "example-lb"

internal = false
loadbalancertype = "application"
securitygroups = [awssecuritygroup.example.id]
subnets = [awssubnet.example.id]
}

resource "awslbtargetgroup" "example" {
name = "example-target-group"
port = 80
protocol = "HTTP"
vpcid = awsvpc.example.id
}

resource "awslblistener" "example" {
loadbalancerarn = awslb.example.arn
port = 80
protocol = "HTTP"

defaultaction {
type = "forward"
targetgrouparn = awslbtargetgroup.example.arn
}
}

module "web" {
source = "./modules/webserver"
instancecount = 2
securitygroupid = awssecuritygroup.example.id
subnetid = awssubnet.example.id
}

resource "awsdbinstance" "example" {
allocatedstorage = 20
engine = "mysql"
engineversion = "5.7"
instanceclass = "db.t2.micro"
name = "exampledb"
username = "admin"
password = "password"
vpcsecuritygroupids = [awssecuritygroup.example.id]
subnetgroupname = awsdbsubnetgroup.example.name
}

resource "awsdbsubnetgroup" "example" {
name = "example"
subnetids = aws_subnet.example.id

tags = {
Name = "example-db-subnet-group"
}
}

In this example, we define an AWS VPC, subnet, security group, load balancer, target group, listener, two web servers (using a module), and a database instance. By running `terraform init` and `terraform apply`, Terraform will create and manage these resources for you, handling dependencies and updates as needed. If you need to make changes to your infrastructure, simply update the configuration file and run `terraform apply` again. Terraform should calculate the necessary changes and apply them accordingly.

Conclusion

Infrastructure as Code is a foundational concept in the DevOps landscape, allowing organizations to automate the provisioning and management of infrastructure resources while maintaining consistency, predictability, and security. By embracing IaC and incorporating it into their DevOps workflows, organizations can streamline their operations and foster greater collaboration between development and operations teams.

​

Further Reading:

1. ***Infrastructure as Code: Managing Servers in the Cloud***. O'Reilly Media, Morris, K., & Kordyban, K.
2. ***Terraform: Up & Running***. O'Reilly Media, Brikman, Y.
3. Terraform - **What is Infrastructure as Code (IaC)?**
4. Amazon Web Services - **AWS CloudFormation.**
5. Microsoft Azure - **Azure Resource Manager (ARM) Templates.**
6. Google Cloud - **Google Cloud Deployment Manager.**
7. **Terraform Commands (CLI).**

​

I hope this more in-depth look at some Infrastructure as Code tools has been informative and helpful. As always, feel free to share your thoughts, experiences, and questions in the comments below!

https://redd.it/137bwv9
@r_devops

I am signing my jar files and problem is that it is treating my jar as unsigned.

I am signing my jar files and problem is that it is treating my jar as unsigned.


For example:

- Signed by "CN="Signer""
     Digest algorithm: SHA-1 (disabled)
     Signature algorithm: SHA1withRSA (disabled), 2048-bit key

WARNING: The jar will be treated as unsigned, because it is signed with a weak algorithm that is now disabled by the security property:

  jdk.jar.disabledAlgorithms=MD2, MD5, RSA keySize < 1024, DSA keySize < 1024, SHA1 denyAfter 2019-01-01

This is the response that I am getting when I verify my build. I am not getting solution for this one.

I run this command to sign my jar.

jarsigner -storetype pkcs12 -keystore mycertificate.pfx -tsa https://timestamp.comodoca.com/rfc3161 -digestalg SHA-256 file.jar 'aliasname' 

please help me if any of you have been through or knows the solution. It is urgent for me.

https://redd.it/137bjx2
@r_devops

Security of EC2 RI cost optimizing services (e.g. Zesty)

^(I am considering using Zesty or some other cost optimization service provider, basically a RI marketplace trader on my behalf. What are the potential security issues that might arise from such activity in my account? I imagine a buggy software could reserve a lot of instances on my behalf, is there anything I can do about this?)

https://redd.it/137doml
@r_devops

From Chaos to Clarity: How to Secure Your Supply Chain with Attestations

Attestations are signed pieces of evidence gathered at various points along the SDLC. How can you use Attestations and cryptographic sign/verify techniques to help secure your development process and your software supply chain? Check out the model described in this article.

https://redd.it/137f7ak
@r_devops

User quotas on AWS EFS

Hello,

We currently have a volume made up of EBS shares that host user directories. The volume is an XFS partition and we use quotas to limit how much each user can store. We want to migrate this setup to EFS to take advantage of the hot/cold tiering and pay as much as you use features of EFS. However EFS itself does not support any OS level quotas. I looked into this and NFSv4 itself does support quotas, however that feature is not supported in EFS. I also tried mounting the EFS share to an EC2 instance and exporting it as NFS. However, that seems unsupported as well.
Any ideas?
We want to be able to have user home directories and storage limits for the users on ten EFS share. Has anyone encountered a use case like this?
Thank you!!

https://redd.it/137hui3
@r_devops

Intelligent next-gen editor for infra-as-code

Hi everyone, we’ve been working on an adaptive architecture engine as part of the open-source Klotho (github), and are now adding an intelligence layer on top of it we call InfraCopilot that features:

- High level editor - Start with a high level design and let InfraCopilot fill in the details
- Rearchitecting with confidence - Change architecture components and designs while InfraCopilot adjusts the rest to work with your changes
- Text + Visual editing - Use the UI or text based commands for maximum productivity
- Exporting Infrastructure-as-Code - Generate a deployable version of any of your architectures.
- Syncing with Github - Save and resume your work using your GitHub repo

We’re sharing the early access with the community - What are the last 2 Infra-as-Code tasks you had to do that took you more time than you were hoping?

https://redd.it/137k01e
@r_devops

Tutorial: Deploying a Django & django-q application with Kubernetes at Digital Ocean with CI/CD Github actions and SSL certificate using HTTPS communication.

Let me know if you have any thoughs about this, I'll try to keep the article updated. I've had issues deploying to Digital Ocean when having a django-q application, so I opted for using Kubernetes, but didn't found a good guide for this, so here is one! Hope you'll enjoy it.

https://medium.com/@alexanderlindgren\_17992/deploying-a-django-django-q-application-with-kubernetes-at-digital-ocean-with-ci-cd-github-e1580cc96ea5

If you enjoy this article and want to keep track of my future articles regarding Django, web-development and entrepreneurship, click here: https://medium.com/@alexanderlindgren\_guttae/subscribe

https://redd.it/137ouon
@r_devops

Questions to ask new team

What questions do you guys ask new team members?

https://redd.it/137qxub
@r_devops

Why does it seem like GitHub goes down every other week these days?

It is currently having a minor outage: https://twitter.com/isdownapp/status/1654152908399681537
But it is starting to seem really frequent: https://isdown.app/integrations/github?c=1683215760


https://twitter.com/githubstatus


Any ideas why?

https://redd.it/137s18r
@r_devops

Is GitHub less stable this year or just me?

Seems like the service has been having issues almost every month now, which is causing a slowdown/downtime in our org. https://www.githubstatus.com/history

Is it just me? I'm really hoping this isn't an indication of a pattern here...

https://redd.it/137ryrk
@r_devops

I write an article about how to automate a ecs task creation and execution using event rules

Terraform ECS article

https://redd.it/137tcx9
@r_devops

Is it possible to install Rancher to manage an already functioning K8S?

Hey folks...

At my workplace, we're implementing K8S on premise and everything was done manually with Kubeadm, but it's starting to get complex to manage directly in K8S as the environment is growing.

We use RHEL 9 and I'm thinking about using Rancher to help with the administration. Questions:

1 - Can I install Rancher to manage an already functioning K8S?

2 - We use Podman instead of Docker, is that okay with Rancher?

3 - If you have any other suggestions besides Rancher, they're also welcome!

I forgot to mention that we only have a single node cluster... so, rebuilding/altering the cluster is a possibility.

https://redd.it/137wcs2
@r_devops