terracove - open-source to instantly test the health of your terraform/terragrunt repository

https://github.com/jatalocks/terracove

Hey r/devops.

After much trial and error, I made a prototype of a tool that my team pretty much needed but that didn't yet exist to my knowledge.

What my problem was originally is that we had a lot of technical debt and drift on our terraform/terragrunt repositories, and tons of leftover modules that haven't been planned in years of missing CI. Instead of going one by one, planning it and checking if it's valid, I built terracove.

What it does in parallel is basically init/plan/show using terratest on every subdirectory on your repository tree or provided paths. The output is either a JSON summary or a custom made Junit XML test file you can ingest into your tests reader. It took it around 8 minutes to map the entirety of our bloated repository.

All you have to do to set it up is either:

brew tap jatalocks/jatalocks
brew install terracove
terracove --junit --json .

Or (with the correct ENV variables and secrets if necessary):

docker run --rm -v $(pwd):/data ghcr.io/jatalocks/terracove /data --json --junit

I hope anyone finds this useful, Any feedback is welcome!

https://redd.it/12hcafq
@r_devops

Quick question about release management

I'm interested in understanding how many hours a week are spent (in total) company-wide to ensure that software version status information is accurate.

For example, if every week a release manager needs to meet with 3 team leaders for 10 minutes each, then that constitutes 1 hour a week (2 participants * 3 meetings * 10 minutes).

Consider how many hours it takes to:

1. ensure that all tickets are up-to-date and that their status is accurate

1. Compile a release's overall status - to understand what has been deployed to integration/staging, what has been tested, etc.

2. attend meetings (even one-on-one) to sync on status

​

Thanks!

https://redd.it/12hemca
@r_devops

I'm planning to deploy all tools in the same cluster as where my microservice application is deployed. Is this generally a good design?

For additional context, I'll be deploying this on both EKS cluster and Minikube (w/ stuff like karpenter not deployed on Minikube). The tools would be coming from their official helm charts.

For EC2 Managed Group, I wonder what instance should I deploy this on.

For Minikube I have 16gb of RAM but I'm planning to upgrade it soon, would 32gb suffice?

https://imgur.com/a/wtFd2ox

https://redd.it/12hdle9
@r_devops

SonarQube Testlink Integration

Hello guys, recently my manager asked me to find that is there any possible way that we can download a “Testlink” plugin into SonarQube (or vice-versa).

I’ve been searching online for a lead but didn’t find anything.
Please help me out!

Also let me know if there’s any other way of integrating both the tools!

P.S - Currently we are using the SonarQube’s community edition.

https://redd.it/12hg8uk
@r_devops

How do I win over my new team and get them to use the stuff I was hired to implement?

This might just be a vent, but here goes: I was senior devops in a fairly nimble startup and got lured over to a more established company with a lot more dev needs in January.


The CTO was particularly impressed with the suite of tools and processes I had introduced (and my team was using), and was largely hired to bring that same mindset and arsenal of ops tools over.


Two months in, I’m struggling to get the new dev team to just make use of a CI/CD tool that should help with code reviews.


This is just the first of about 5 things I thought would be implemented and used by now… but I’m getting a lot of trouble getting traction/buy-in.


Are there technical solutions to this or am I going to have to use my personality?

https://redd.it/12hlcfm
@r_devops

Effortlessly Manage Kubernetes with this Intuitive New GUI – See It in Action!

Demo on youtube: https://www.youtube.com/watch?v=DtQnuSDmodg
The open source project can be found at https://github.com/sharadregoti/devops-cli


​

https://redd.it/12hmupz
@r_devops

What's that one time you realised a client/employer just wasn't ready for DevOps?

My story in the replies.

https://redd.it/12hoxjz
@r_devops

Ship CloudWatch Log to Grafana using lambda

I'm setting up a lambda function to push log from cloudwatch to grafana I then put a concurrency limit on my lambda so it won't throttle itself but I noticed that some of the logs are missing in grafana compared to cloudwatch. I was thinking about sending log in batch with sqs but cloudwatch subscription filter doesn't support it. How do you guys ship your cloudwatch log to lambda in batch?

https://redd.it/12ho81s
@r_devops

DevOps Learning Series Principles and a look at their impact.

Series index

Hey r/devops, continuing the Devops Learning series, this post focuses on the core principles of DevOps and the benefits they offer. Understanding these principles will help you embrace the DevOps mindset and successfully implement its practices in your organization.

## Devops Principles:

1. Collaboration: Encourage close cooperation and communication between development and operations teams, breaking down traditional silos, and fostering a shared understanding and ownership of the software delivery process.
2. Automation: Streamline and automate manual, repetitive tasks throughout the software development lifecycle, including testing, deployment, and infrastructure management, to reduce human error and increase efficiency.
3. Continuous Integration (CI): Regularly integrate code changes into a shared repository, triggering automated builds and tests to catch integration errors early and maintain a consistently releasable product.
4. Continuous Delivery (CD): Automate the release of code changes to production environments, ensuring rapid, reliable, and consistent deployments with minimal manual intervention.
5. Continuous Improvement: Embrace a culture of constant learning and adaptation by regularly assessing processes, tools, and practices, and making data-driven decisions to optimize performance and address issues.
6. Rapid Feedback: Implement robust monitoring and feedback mechanisms to quickly identify issues, learn from user behavior, and inform future development efforts, thereby ensuring continuous alignment with customer needs.
7. Infrastructure as Code (IaC): Manage and provision infrastructure resources using code and version control systems, enabling consistent, repeatable, and automated deployment of environments.
8. Security and Compliance: Integrate security best practices and regulatory requirements throughout the development lifecycle, promoting a proactive approach to risk mitigation and fostering a shared responsibility for security among team members (DevSecOps).
9. Flexibility and Adaptability: Embrace an iterative, experimental mindset that allows for rapid response to changing requirements, technology advancements, and market demands, fostering a resilient and agile organization.
10. Customer-Centricity: Prioritize the needs and expectations of end users, using their feedback to inform development decisions, and ensuring the continuous delivery of value-added features and improvements.

## Benefits

Let's take a closer look at the data supporting the effectiveness of DevOps principles. By exploring various studies and reports that shed light on the impact of DevOps, we'll further our understanding of the value these principles bring to organizations.

Deployment Frequency and Speed

One of the key indicators of a successful DevOps implementation is the frequency and speed of deployments. According to the 2021 State of DevOps Report by Puppet, high-performing DevOps organizations deploy software 200 times more frequently than low performers (1). These rapid deployment rates are enabled by effective automation, streamlined processes, and close collaboration between development and operations teams.

Additionally, the same report found that high-performing DevOps organizations have a lead time for changes (i.e., the time it takes for code changes to move from commit to production) that is 2,604 times faster than low-performing organizations (1). This acceleration highlights the efficiency gains that DevOps practices can bring to the software delivery process.

Reduced Change Failure Rates and Improved Recovery Times

DevOps principles not only accelerate the software delivery process but also improve the quality and reliability of releases. The 2021 State of DevOps Report revealed that high-performing organizations experience a 3 times lower

change failure rate compared to low performers (1). This improvement in reliability can be attributed to rigorous testing, automated deployment, and better collaboration between development and operations teams.

When failures do occur, DevOps practices contribute to faster recovery times. The same report found that high-performing organizations have a mean time to recovery (MTTR) that is 2,604 times faster than their low-performing counterparts (1). By implementing robust monitoring, swift incident response, and effective collaboration, DevOps teams can quickly identify and resolve issues, minimizing the impact on customers.

Enhanced Customer Satisfaction and Business Outcomes

Embracing DevOps principles not only improves technical performance but also leads to better business outcomes. A 2019 Accelerate State of DevOps Report by DORA and Google Cloud demonstrated a strong correlation between high-performing DevOps teams and superior business performance (2). High performers reported 1.7 times greater customer satisfaction, 1.7 times higher profitability, and 1.8 times higher market share compared to their peers (2).

In this post I communicated examples highlighting the value of adopting DevOps principles for both the technical and business aspects of organizations. By embracing collaboration, automation, continuous improvement, and customer-centricity, teams can deliver high-quality software rapidly and efficiently, leading to improved customer satisfaction and business performance.

As always, I encourage you to share your thoughts and experiences with DevOps principles and their impact on your organization in the comments below.

Sources and more reading:

(1) Puppet Labs. (2021). State of DevOps Report 2021. https://www.puppet.com/resources/report/state-of-devops-report/

(2) Forsgren, N., Humble, J., Kim, G., Brown, K., Kersten, M., Rivas, R., & Shipp, S. (2019). Accelerate State of DevOps Report 2019. DORA and Google Cloud. https://cloud.google.com/files/DevOps/state-of-devops-2019.pdf

https://redd.it/12hrjt4
@r_devops

Calling OpenAI API Service using Python

https://medium.com/techbeatly/calling-openai-api-service-using-python-9e1d51416e7a?sk=20568c9675cea21735d6b90ee0ee2d13

https://redd.it/12hnvfw
@r_devops

Need some help with terraformer

Hey All,

I am very new to this terraform world. I'm working on importing previously built hand-built AWS infrastructure into TerraForm.

I was using the terraform import function, however, it was time-consuming and difficult to use. I stumbled upon Terraformer today which is supposed to automate the task for you. I keep getting the following error message when performing the import. I was hoping someone with more experience than me could help me out.

​

Thanks!

​

​

Andrews-iMac:Import acaplice$ terraformer import aws -r vpc --regions=us-east-1 -O hcl --profile=ProfileName

2023/04/10 16:44:45 aws importing region us-east-1

2023/04/10 16:44:45 aws importing... vpc

2023/04/10 16:44:50 aws error initializing resources in service vpc, err: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, exceeded maximum number of attempts, 3, request send failed, Get "https://169.254.169.254/latest/meta-data/iam/security-credentials/": dial tcp 169.254.169.254:80: connect: host is down

https://redd.it/12hvsjl
@r_devops

Tools for checking your code?

What code tools do you use for verifying code you wrote,? Sonarqube ?

https://redd.it/12hmmjt
@r_devops

Building in public: Cloud pricing calculators are super annoying - so here is one based on natural language

For now I've only implemented AWS services, hence it's in alpha. But I'd be curious to hear what the most common pricing prompts are for you - and whether you mostly need estimates (only set the most important parameters on calculator.aws) or exact prices.

​

Also, how often do you look up cloud prices? Which ones do you look up? Let me know in the comments.

​

https://go.kubiya.ai/cloud-pricing (a video demo)

https://redd.it/12hz7wy
@r_devops

Platform Engineering for Hashicorp Vault?

We're setting up HCP Vault (the SaaS) to be a platform for other dev groups to use for their secrets. My team will be managing it, so I'm looking to templatize/standardize as much as possible.

I've read over this guide Onboarding Applications to Vault Using Terraform: A Practical Guide - https://www.hashicorp.com/blog/onboarding-applications-to-vault-using-terraform-a-practical-guide \- which is a very baby-steps way of structuring the terraform. Does anyone know of any guides with more complex terraform? I'm looking for other ideas for how different teams manage this, and not seeing a lot in public repos, maybe I'm missing it, or maybe anyone doing this work is keeping it private.

https://redd.it/12i17s4
@r_devops

Is it reasonable for a company to give you academic/behavior assessments before you’ve even gotten a phone screener?

I’ve been looking for jobs lately and have come across a few instances where a company wants me to take a Wonderlic type logic test or a 200 question personality assessment as part of the standard application. Sometimes multiple. And they’ll quote an expected 1-2 hours of time needed to complete them. Is this now normal? I personally think it’s ridiculous to ask for 1-2 hours of someone’s time just to submit a resume, especially because there’s no guarantee after spending that time I’ll even get the 10 minute phone screener. If I’m already in the running for the position and we’ve at least talked before and decided to continue, I’d be much more willing, but having to do them just to get a resume on a desk seems like an unreasonable ask.

Im asking because I’m not sure if I’m out of touch for how applications go these days or if this is a bit much for just an application

https://redd.it/12hmhuf
@r_devops

GitHub Secrets: Is it safe to store an environment secret to a file from within a GitHub Action?

In other words, is it safe to have something like this in a .yml file:

# Add API key
- name: Add API key secret to file
env:
API_KEY_SECRET: ${{ secrets.API_KEY_SECRET }}
run: echo "$API_KEY_SECRET" > ~/work/MyProject/MyProject/secrets/api_key.txt

To clarify, the secret is not pushed to the repository, initially it only exists as an environment secret. I'm not sure if the .txt file created above ends up somewhere unsecure. The reason I ask is because if this is secure, then I can have a convenient location for programs to access secrets stored in files both on my machine (either .gitignore or outside the local repo) and when running on the server, without storing secrets in the repository.

https://redd.it/12i56ye
@r_devops

DevOps/DevSecOps hero to zero

Hi guys,can you share your thoughts and a roadmap for being devops/devsecops, i am currently working as SOC Analyst and learning pentesting,i want to add DevOps to my skill and become DevSecOps.
Maybe in future i want to change my job to DevOps
Last question,what are the chances find remote work as a DevOps

https://redd.it/12i9csf
@r_devops

How much building a big following on the dev community can help someone get better job opportunities?

Im not sure if this is the best place for asking it but it’s something that makes me really curious! Especially concerning frontend professionals, since i see so many people apparently doing a big effort towards getting a bigger following on the community while in other hand a lot of other devs apparently aren’t active on social medias at all

https://redd.it/12ic2et
@r_devops

Open Policy Agent

I recently came across OPA. Currently exploring courses from Styra. I wanted to know if learning it is worth for a DevOps engineer? Is it in demand? Does this have any future In DevOps market?

https://redd.it/12icbhu
@r_devops

How much would you earn as a Senior Consultant in DevOps in AWS Poland?

Can anyone tell me the salary range of a Sr DevOps Consultant in Poland?

https://redd.it/12idrgi
@r_devops