Multipurpose pure bash script that helps automatize common tasks for web developers

Hi everyone!

Few years ago I started a project to learn BASH. At the begining the script helps me to automatize backups for my agency ([BROOBE](https://www.broobe.com/)), but now it has more features:

* LEMP automated installer (Nginx, MySQL/MariaDB, PHP).
* Files and database backups (MySQL or MariaDB).
* Upload backups to Dropbox (with Dropbox-Uploader).
* Restore backups from Dropbox or URL.
* WordPress automated installer.
* WP-CLI actions helper.
* Let's Encrypt actions helper.
* Monit installer and configuration helper.
* Netdata installer and configuration helper.
* Certbot installer and configuration helper.
* Cloudflare support (via API).
* PHP-FPM optimization tool (beta).
* Image optimization tools.
* Security Tools with malware scanners.
* IP/Domain blacklist checker.

I would love to receive your comments and feedback!

Link to the repo: [/lemp-utils-scripts: LEMP Bash Utils for Ubuntu 18.04+ (github.com)](https://github.com/lpadula/lemp-utils-scripts)

https://redd.it/lutqkq
@r_devops

demonstrate X technology skills without direct prod experience

I'm currently looking for Mid to Senior DevOps roles and have been unable to get past the phone screen for jobs paying above my current compensation. Most jobs will ask me about my prod experience with k8s, IaC, go/python, which I don't have, and the process ends. I have built out proof of concepts using k8s and terraform and demo'd it to my current company, but this carries little weight when speaking to the recruiter. Unfortunately, my company is very reluctant to embrace any new technologies much less put them into prod. So, I find myself trying to learn Go so I can build a product on the side and then using that as my showcase for all the infra stuff. Is this a good way to demonstrate skill with a technology absent direct prod experience?

https://redd.it/lutbfw
@r_devops

Getting "ssh: handshake failed" error

Hi all,

I am trying to run an Ansible to create an EC2 instance using this example.

It is creating the EC2 instance along with the security groups and VPC, however, at some point, I am getting the following error:

>Error: timeout - last error: SSH authentication failed ([email protected]:12): ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey\], no supported methods remain

​

Also, I am not sure if it has something to do with the error, but when running the deploy.sh file, I am getting:

❯ sh deploy.sh
+ ssh-add -A
No identity found in the keychain.
+ cd terraform
+ terraform init

How can I fix this issue and let Ansible SSH into my instance?

Thanks!

https://redd.it/luouj5
@r_devops

Deep Dive into Docker Internals - Union Filesystem

Hi /r/devops

Today i published article detailing inner workings of union mount filesystem (OverlayFS) which is used by Docker. I think this might be an interesting read for some of you here.

So, here's the link: https://itnext.io/deep-dive-into-docker-internals-union-filesystem-5a1fbcd426b5

https://redd.it/lv3g0v
@r_devops

Monitoring across frameworks

Gone are the times when developers' jobs ended with the release of the application. Nowadays, developers care more and more about the operational side of IT: perhaps they operate applications themselves, but more probably, their organization fosters increased collaboration between Dev and Ops.

I started to become interested in the Ops side of software when I was still a consultant. When Spring Boot released the Actuator, I became excited. Via its convention-other-configuration nature, it was possible to add monitoring endpoints with just an additional dependency.

Since then, other frameworks have popped up. They also provide monitoring capabilities. In this post, I’d like to compare those frameworks concerning those capabilities.

Read more

https://redd.it/lul7xs
@r_devops

How to migrate a complex infrastructure to terraform?

Hello everyone, I would like to know your opinions and experiences on how to migrate a complex infrastructure on AWS (mostly EC2s and VPCs) to Terraform.
Years ago we started building it with ansible, since we liked the declarative approach, but the more we go on the more it's difficult to keep up with the shortcomings of the tool (and Ansible's community, while fantastic for everything, does not really shines on the AWS side), so we are finding ourselves scripting more and more using the aws cli.
Also, it's a nightmare testing infrastructure provisioning with a procedural tool.
So a lot of people suggested me to start switching to TF, but no one that I've talked to has worked on a migration from a tool to another...or, to put in another way, to migrate an established AWS infrastructure to an IaC tool.

So, I would like to know your opinions, what approaches you can suggest. At the moment I'm working to migrate at least the global components (s3, iam, etc.) that are quite easier compared to the rest of the envs, but I'm quite lost when it will come to migrate the dynamic pieces of the infrastructure.

https://redd.it/lujk1a
@r_devops

Thin clients connecting to a cloud-vm as the main PC for personal use

Guys, I have 3 laptops in 2 locations. They house my personal stuff and also some (non-heavy) engineering programs to do some calculations.

I don't like how they all have different programs installed, different files and have different configurations. I want to standardise them all to have the same state. I'm not talking about containers and swapping these containers around in order to standardise things. I would like to be able log off a machine at night and go somewhere else, log into another machine and start working again as if I never left the previous machine. It will be ultra convenient and accessible from anywhere.

1. Can I use a cloud-vm for this? The idea is to start a cheap VM with auto-turn-off, call some web-function and it turns on and then I start work again. Is this feasible?
2. I also won't ever be paranoid about people stealing my laptops anymore. Which providers do you recommend to keep the costs low but still have enough redundancy that I'll never lose that VM for the rest of my life.
3. How would you backup this system?

Thankkkk yoouuu

https://redd.it/luh8yt
@r_devops

Convert Gitlab Raw Log Into HTML Using PHP

This tutorial help to convert gitlab raw log into html color view. The gitlab ci/cd are generating logs for each jobs. You can view color view logs using gitlab view.

https://www.phpflow.com/laravel-7/convert-gitlab-raw-log-into-html-using-php/

https://redd.it/lug2gc
@r_devops

What are the key skills of DevOps?

Hi, Developer trying to learn a bit of Devopsy stuff to make my job less painful. I'm not trying to become a DevOps professional, Or be a great or even good DevOps engineer. I just want to learn basics for personal use. Not being confused by server things, build tools ect and basic computer skills.

My question is, If you could only keep like 20% of your knowledge, What would it be? What should I focus on?

https://redd.it/lueijd
@r_devops

Prometheus/Alertmanager/Kube-state-metrics

Hello guys, I am kind of new with Prometheus monitoring and I really need some help with slack alerts, I am a complete beginner when it comes to golang and metrics.

I have Prometheus deployed in my kubernetes cluster and alertmanager that is supposed to send slack notifications, everything runs great but i am having trouble getting my node name in that slack alerts.

Kube-state-metrics are colleting info that I need, like kubernetes_node_info and that metrics contain node names that I would like to use in my slack alerts.

I am able to get node names in kube-state-metrics alerts, like this one

- alert: KubernetesNodeReady
expr: kubenodestatuscondition{condition="Ready",status="true"} == 0
for: 5m
labels:
severity: critical
annotations:
description: 'Node {{ $labels.node }} has been unready for a long time\n VALUE = {{ $value }}\n LABELS: {{ $labels }}'
summary: 'Kubernetes Node ready (instance {{ $labels.instance }})

But what I would like to accomplish is to get that node name for every alert that I have configured, is that possible? I would be more than grateful with any help or at least if you could point me in the right direction?

Also, this is my alertmanager.yml


alertmanager.yml:
route:
groupby: alertname, job, group, app
receiver: slack

receivers:
- name: slack
slackconfigs:
- apiurl: 'https://hooks.slack.com/services/xxxxxxx'
channel: '#channel'
sendresolved: true
title: |-
[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }} for {{ .CommonLabels.job }}
{{- if gt (len .CommonLabels) (len .GroupLabels) -}}
{{" "}}(
{{- with .CommonLabels.Remove .GroupLabels.Names }}
{{- range $index, $label := .SortedPairs -}}
{{ if $index }}, {{ end }}
{{- $label.Name }}="{{ $label.Value -}}"
{{- end }}
{{- end -}}
)
{{- end }}
text: >-
{{ range .Alerts }}
*Alert:* {{ .Annotations.summary }} - `{{ .Labels.severity }}`
*Description:* {{ .Annotations.description }}
*Graph:* <{{ .GeneratorURL }}|:chartwithupwardstrend:> Runbook: <{{ .Annotations.runbook }}|:spiralnotepad:>
Details:
{{ range .Labels.SortedPairs }} • {{ .Name }}: {{ .Value }}
{{ end }}
{{ end }}


Thank you!

https://redd.it/lvccoe
@r_devops

Monthly 'Getting into DevOps' thread - 2021/03

What is DevOps?

[AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

Books to Read

The Phoenix Project - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
[The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
Google's Site Reliability Engineering - Google engineers explain how they build, deploy, monitor, and maintain their systems.
[The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book
The Unicorn Project - the "sequel" to The Phoenix Project.
[DevOps for Dummies](https://www.amazon.com/DevOps-Dummies-Computer-Tech-ebook/dp/B07VXMLK3J/) - don't let the name fool you.

What Should I Learn?

Emily Wood's essay - why infrastructure as code is so important into today's world.
[2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
This comment by /u/mdaffin - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
[This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.
Roadmap.sh - Step by step guide for DevOps or any other Operations Role

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

Previous Threads
https://www.reddit.com/r/devops/comments/la7j8w/monthlygettingintodevopsthread202102/

https://www.reddit.com/r/devops/comments/koijyu/monthlygettingintodevopsthread202101/

https://www.reddit.com/r/devops/comments/k4v7s0/monthlygettingintodevopsthread202012/

https://www.reddit.com/r/devops/comments/jmdce9/monthlygettingintodevopsthread202011/

https://www.reddit.com/r/devops/comments/j3i2p5/monthlygettingintodevopsthread202010/

https://www.reddit.com/r/devops/comments/ikf91l/monthlygettingintodevopsthread202009/

https://www.reddit.com/r/devops/comments/i1n8rz/monthlygettingintodevopsthread202008/

https://www.reddit.com/r/devops/comments/hjehb7/monthlygettingintodevopsthread202007/

https://www.reddit.com/r/devops/comments/gulrm9/monthlygettingintodevopsthread202006/

https://www.reddit.com/r/devops/comments/gbkqz9/monthlygettingintodevopsthread202005/

https://www.reddit.com/r/devops/comments/ft2fqb/monthlygettingintodevopsthread202004/

https://www.reddit.com/r/devops/comments/fc6ezw/monthlygettingintodevopsthread202003/


https://www.reddit.com/r/devops/comments/axcebk/monthlygettingintodevopsthread/

Please keep this on topic (as a reference for those new to devops).

https://redd.it/lvet1r
@r_devops

Streamline setting up local dev env with docker?

We plan on growing our development team by 2-3x over the next year.

In order to minimize onboarding time, we have been considering if we can someone containerize a local development environment so all new devs can easily work in the same environment, and minimize time needed for set up.

For context, we’re primarily a Java shop.

Any thoughts, suggestions, best practices would be appreciated. Have done some research online but haven’t found anything them feels definitive yet.

Thanks!

https://redd.it/lvg7q3
@r_devops

Best resources/tutorials for setting up basic CI for personal full stack web app

Hi, devops.

I'm a student trying to learn about continuous integration by setting up a basic CI pipeline for a web app. The app uses Python's Flask framework for the back end and React for the front end. I've been trying to use Travis CI.

I've had a lot of questions while trying to write the .travis.yml file to install the front end and back end and then run tests for each. For example, I'm not sure if I should create separate git branches (each with their own yml file) for the Flask app and React app, or if I should use a single yml file that somehow directs the Travis VM to call the installation and testing scripts in the front end and back end subdirectories.

I've been surprised by how few tutorials I've found for setting up a basic CI workflow for a project like this with some unit tests and maybe some tests that test the front and back ends together. I'm definitely a noob with devops though, so I might be asking the wrong questions.

Are there any tutorials or resources you would suggest for learning how to set up a basic CI pipeline for a simple personal project like this?

https://redd.it/lvmtol
@r_devops

MTV Cribs for DevOps

I work at a small company and I'm a junior engineer. I'm getting into DevOps but neither of my coworkers are really equipped to teach me about it. I've been reading lots of articles, but most are either about how to use specific technologies or about very general concepts like CI/CD. I want something between those two: specific enough to be practically useful, but general enough that the whole DevOps infrastructure is described. I'd love to learn how those that know more than me are doing things by having them show us round their systems and explain their rationales, a bit like how a pen tester will walk you through their exploit. Does this exist anywhere?

https://redd.it/lvmjeu
@r_devops

Jobs hiring trend in 2021

Working from remote would be a new normal?


what do you think or see the trend going to be or happening right now?

https://redd.it/lvl7v6
@r_devops

I'm conducting a market research and I have a conjecture, WDYT?

My conjecture is that due to the scarcity of DevOps engineers (from various reasons), or the fact that many times infrastructure and deployment mechanisms are designed and created by software developers without DevOps expertise, many companies suffer from rickety cloud infrastructures and deploy mechanisms which are not robust, not secure and not made by best practices.

How wrong do you think I am?

https://redd.it/lvjzpu
@r_devops

Devops training help needed

Hello all.

I have taken the initiative to deliver sort of an internal training/seminar on the devops culture. There aren't going to be any tools specifics (maybe mentions here and there), but the intention is to help the company understand what devops is and isn't, and set the foundations for next projects to come.

At least 80% of the company are junior to mid developers, so I see this opportunity as particularly important.

What are your thoughts on some topics I could go through, and how would you approach them?

If you can think of any games/excercise that could help prove a point they would be very helpful.

Thanks, really appreciate your help, and keep delivering faster!!

Edit: typos

https://redd.it/lvg1ym
@r_devops

HashiCorp Certified: Terraform Associate discount code?

Any discount codes for the exam?

https://redd.it/lvfx4d
@r_devops

Creating Lambda and ECR in the same terraform

Hello devops masters.
We would like to create a lambda function that runs from a container, we would also like to create the ECR at the same time. The problem is that the lambda cannot be created with a non existing image.. (ECR is created first but it is empty) and the lambda cannot be created..

Since it's kind of a new feature I couldn't find any documentations 😔.

We want to manage the build of the code and the pushing to the ECR seperatly from the terraform which is infra only.

Does anyone have a way around that?

resource "awsecrrepository" "arepo" {
name = var.ecrname
imagetagmutability = "MUTABLE"
}

resource "awslambdafunction" "alambda" {
functionname = "a-lambda"
role = awsiamrole.a-lambda-role.arn
imageuri = "${awsecrrepository.arepo.repositoryurl}:latest"
packagetype = "Image"
tags = var.tags

}

Any comment is much appreciated

https://redd.it/lva06o
@r_devops

Best courses to learn Kubernetes, Dockers and Containers from scratch

Hello

I am looking for some suggestions from experts here to help me with the courses to learn the important concepts of DevOps.
I have Linux Academy membership and can also buy courses from Udemy.

Please let me know where to start with.

Thank you

https://redd.it/lvdqij
@r_devops

Infrastructure State Visualisation For Terraform

Hi everybody!

This one is for the Terraformers among you. I’ve always felt the plan/apply output is pretty messy and could be improved upon, so I decided to do something about it. I’m currently working on a little project called Pluralith. We’re essentially talking about an application that visualises your Terraform state and integrates directly with the Terraform CLI.

The application is currently in an early state with a private alpha starting soon. I demoed the application at HashiTalks 2021 some time ago and the reception was overwhelmingly positive. I already posted in r/Terraform a while ago, but since Terraform is a pretty popular devops tool these days I thought I’d share the project in here and collect some more feedback from people all over the devops landscape. The project could take many directions in the future so if you’re not a Terraform user, but have any ideas or questions that involve any other tools in your workflow I’d also very much like to hear about that :)

If you want to get a better feel for the project check out the website **HERE**. If you are interested in being an early tester or simply want to know when the thing is available for download for your OS leave me your email and I’ll make sure to notify you when you can get your hands on it.

I’ve also created a little subreddit for user feedback, discussions, feature requests etc. We are already over 90 people strong so head over there and join r/Pluralith if you are interested in the project!

I’ve also created a little Linkedin page here so if you’re a Linkedin type of person you can find Pluralith **HERE**

As said feedback, questions, comments etc. are very welcome!

Cheers,

Dan

https://redd.it/lvd0bi
@r_devops