IT infrastructure

Hi, folks. IS there any recommendations on IT infrastructure books which contains everything from Load Balancers to Server Clusters, Databases, Network, Storages etc

https://redd.it/ltib21
@r_devops

Apache Maven Features

Simple project setup that follows best practices.

Consistent usage across all projects.

Dependency management including automatic updating.

A large and growing repository of libraries.

Extensible, with the ability to easily write plugins in Java or scripting languages.

https://redd.it/lu32ku
@r_devops

Secrets Managers for Kubernetes (Vault (Hashi), Conjur (CyberArk), Platform Specific, etc)

I’ve recently been researching this topic, have a few notes, and would love anyone’s feedback or experience using these tools. Overall, my takeaways are these:

Vault:

* Allows authentication via K8s Service Accounts (given natively to every pod)
* Allows pods to make requests as needed via their SA assuming they have access
* Provides an agent and admission control which will (upon request by specific configurations) mutate and mount secrets as a volume for pods.

Conjur:

* Allows roles to be defined via K8s attributes and provides access based on these roles.
* Provides a secrets synchronizer agent to make secrets available to pods as K8s secrets.

If anyone has any corrections or sees errors/omissions please let me know, these are just my main notes after exploring both docs.

Overall, it seems that vault and Conjur provide the same services but using different solutions. They both seem equally complex, but not always in the same areas.

Does anyone have any experience using either of these tools? Is there a tool I’m ignoring? I’ve been spending a lot of time researching this and have had a hard time finding direct comparisons or recommendations for secrets management in k8s.

https://redd.it/ltziy0
@r_devops

How do you get better as a devops developer in an platform team

At my company, my team (as a platform team) has build the platform for kubernetes cluster. We let our developers build their applications on the cluster. We have been been maintaining it, and adding better monitoring and fixing security vulnerabilities for the past year. Unfortunately, I am struggling with becoming better at kubernetes. Even though, the technical knowledge of kubernetes is very vast and broad, my team has been involved in only the operations side of it for the past year. I feel that there is so much more to kubernetes than the operations side. I know that I can learn the non-operations side of kubernetes during my own time, but for my day to day job, is it common for kubernetes cluster operators to only have only worked with the operations side, but not so much the development side?

https://redd.it/lu07z5
@r_devops

Is it worth learning Powershell?

Is it worth learning Powershell or is everything done on bash?

View Poll

https://redd.it/ltz9a8
@r_devops

Building a New Web App - Rate My Setup

The high-level overview is that the web app is a dashboard for e-commerce business owners that can connect to multiple platforms.

1. API built with Spring boot(java11) and deployed via war file
2. API Application server tomcat 9
3. front end built in VUE
4. Front end server nginx(latest)
5. Both API and front end running on Centminmod (centOS flavor) Digital Ocean instance
6. Data stored in MySQL 8 fully managed Digital Ocean Database
7. Images, Documents, other media stored on object storage Digital ocean space

Before I get to production, I want to get kubernetes setup. I already have environmental configs for both the front end and back end projects.

In production, I'll have a load balancer and auto scaler setup to handle our load. I'm expecting heavy I/O across the cluster.

Does anyone see any places of improvement for my setup based on the above information?

Are there any potential bottlenecks that are blatantly obvious?

I'm planning to load test before final release to really dial in the instance types and auto scaling settings.

https://redd.it/luwuq5
@r_devops

Multipurpose pure bash script that helps automatize common tasks for web developers

Hi everyone!

Few years ago I started a project to learn BASH. At the begining the script helps me to automatize backups for my agency ([BROOBE](https://www.broobe.com/)), but now it has more features:

* LEMP automated installer (Nginx, MySQL/MariaDB, PHP).
* Files and database backups (MySQL or MariaDB).
* Upload backups to Dropbox (with Dropbox-Uploader).
* Restore backups from Dropbox or URL.
* WordPress automated installer.
* WP-CLI actions helper.
* Let's Encrypt actions helper.
* Monit installer and configuration helper.
* Netdata installer and configuration helper.
* Certbot installer and configuration helper.
* Cloudflare support (via API).
* PHP-FPM optimization tool (beta).
* Image optimization tools.
* Security Tools with malware scanners.
* IP/Domain blacklist checker.

I would love to receive your comments and feedback!

Link to the repo: [/lemp-utils-scripts: LEMP Bash Utils for Ubuntu 18.04+ (github.com)](https://github.com/lpadula/lemp-utils-scripts)

https://redd.it/lutqkq
@r_devops

demonstrate X technology skills without direct prod experience

I'm currently looking for Mid to Senior DevOps roles and have been unable to get past the phone screen for jobs paying above my current compensation. Most jobs will ask me about my prod experience with k8s, IaC, go/python, which I don't have, and the process ends. I have built out proof of concepts using k8s and terraform and demo'd it to my current company, but this carries little weight when speaking to the recruiter. Unfortunately, my company is very reluctant to embrace any new technologies much less put them into prod. So, I find myself trying to learn Go so I can build a product on the side and then using that as my showcase for all the infra stuff. Is this a good way to demonstrate skill with a technology absent direct prod experience?

https://redd.it/lutbfw
@r_devops

Getting "ssh: handshake failed" error

Hi all,

I am trying to run an Ansible to create an EC2 instance using this example.

It is creating the EC2 instance along with the security groups and VPC, however, at some point, I am getting the following error:

>Error: timeout - last error: SSH authentication failed ([email protected]:12): ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey\], no supported methods remain

​

Also, I am not sure if it has something to do with the error, but when running the deploy.sh file, I am getting:

❯ sh deploy.sh
+ ssh-add -A
No identity found in the keychain.
+ cd terraform
+ terraform init

How can I fix this issue and let Ansible SSH into my instance?

Thanks!

https://redd.it/luouj5
@r_devops

Deep Dive into Docker Internals - Union Filesystem

Hi /r/devops

Today i published article detailing inner workings of union mount filesystem (OverlayFS) which is used by Docker. I think this might be an interesting read for some of you here.

So, here's the link: https://itnext.io/deep-dive-into-docker-internals-union-filesystem-5a1fbcd426b5

https://redd.it/lv3g0v
@r_devops

Monitoring across frameworks

Gone are the times when developers' jobs ended with the release of the application. Nowadays, developers care more and more about the operational side of IT: perhaps they operate applications themselves, but more probably, their organization fosters increased collaboration between Dev and Ops.

I started to become interested in the Ops side of software when I was still a consultant. When Spring Boot released the Actuator, I became excited. Via its convention-other-configuration nature, it was possible to add monitoring endpoints with just an additional dependency.

Since then, other frameworks have popped up. They also provide monitoring capabilities. In this post, I’d like to compare those frameworks concerning those capabilities.

Read more

https://redd.it/lul7xs
@r_devops

How to migrate a complex infrastructure to terraform?

Hello everyone, I would like to know your opinions and experiences on how to migrate a complex infrastructure on AWS (mostly EC2s and VPCs) to Terraform.
Years ago we started building it with ansible, since we liked the declarative approach, but the more we go on the more it's difficult to keep up with the shortcomings of the tool (and Ansible's community, while fantastic for everything, does not really shines on the AWS side), so we are finding ourselves scripting more and more using the aws cli.
Also, it's a nightmare testing infrastructure provisioning with a procedural tool.
So a lot of people suggested me to start switching to TF, but no one that I've talked to has worked on a migration from a tool to another...or, to put in another way, to migrate an established AWS infrastructure to an IaC tool.

So, I would like to know your opinions, what approaches you can suggest. At the moment I'm working to migrate at least the global components (s3, iam, etc.) that are quite easier compared to the rest of the envs, but I'm quite lost when it will come to migrate the dynamic pieces of the infrastructure.

https://redd.it/lujk1a
@r_devops

Thin clients connecting to a cloud-vm as the main PC for personal use

Guys, I have 3 laptops in 2 locations. They house my personal stuff and also some (non-heavy) engineering programs to do some calculations.

I don't like how they all have different programs installed, different files and have different configurations. I want to standardise them all to have the same state. I'm not talking about containers and swapping these containers around in order to standardise things. I would like to be able log off a machine at night and go somewhere else, log into another machine and start working again as if I never left the previous machine. It will be ultra convenient and accessible from anywhere.

1. Can I use a cloud-vm for this? The idea is to start a cheap VM with auto-turn-off, call some web-function and it turns on and then I start work again. Is this feasible?
2. I also won't ever be paranoid about people stealing my laptops anymore. Which providers do you recommend to keep the costs low but still have enough redundancy that I'll never lose that VM for the rest of my life.
3. How would you backup this system?

Thankkkk yoouuu

https://redd.it/luh8yt
@r_devops

Convert Gitlab Raw Log Into HTML Using PHP

This tutorial help to convert gitlab raw log into html color view. The gitlab ci/cd are generating logs for each jobs. You can view color view logs using gitlab view.

https://www.phpflow.com/laravel-7/convert-gitlab-raw-log-into-html-using-php/

https://redd.it/lug2gc
@r_devops

What are the key skills of DevOps?

Hi, Developer trying to learn a bit of Devopsy stuff to make my job less painful. I'm not trying to become a DevOps professional, Or be a great or even good DevOps engineer. I just want to learn basics for personal use. Not being confused by server things, build tools ect and basic computer skills.

My question is, If you could only keep like 20% of your knowledge, What would it be? What should I focus on?

https://redd.it/lueijd
@r_devops

Prometheus/Alertmanager/Kube-state-metrics

Hello guys, I am kind of new with Prometheus monitoring and I really need some help with slack alerts, I am a complete beginner when it comes to golang and metrics.

I have Prometheus deployed in my kubernetes cluster and alertmanager that is supposed to send slack notifications, everything runs great but i am having trouble getting my node name in that slack alerts.

Kube-state-metrics are colleting info that I need, like kubernetes_node_info and that metrics contain node names that I would like to use in my slack alerts.

I am able to get node names in kube-state-metrics alerts, like this one

- alert: KubernetesNodeReady
expr: kubenodestatuscondition{condition="Ready",status="true"} == 0
for: 5m
labels:
severity: critical
annotations:
description: 'Node {{ $labels.node }} has been unready for a long time\n VALUE = {{ $value }}\n LABELS: {{ $labels }}'
summary: 'Kubernetes Node ready (instance {{ $labels.instance }})

But what I would like to accomplish is to get that node name for every alert that I have configured, is that possible? I would be more than grateful with any help or at least if you could point me in the right direction?

Also, this is my alertmanager.yml


alertmanager.yml:
route:
groupby: alertname, job, group, app
receiver: slack

receivers:
- name: slack
slackconfigs:
- apiurl: 'https://hooks.slack.com/services/xxxxxxx'
channel: '#channel'
sendresolved: true
title: |-
[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }} for {{ .CommonLabels.job }}
{{- if gt (len .CommonLabels) (len .GroupLabels) -}}
{{" "}}(
{{- with .CommonLabels.Remove .GroupLabels.Names }}
{{- range $index, $label := .SortedPairs -}}
{{ if $index }}, {{ end }}
{{- $label.Name }}="{{ $label.Value -}}"
{{- end }}
{{- end -}}
)
{{- end }}
text: >-
{{ range .Alerts }}
*Alert:* {{ .Annotations.summary }} - `{{ .Labels.severity }}`
*Description:* {{ .Annotations.description }}
*Graph:* <{{ .GeneratorURL }}|:chartwithupwardstrend:> Runbook: <{{ .Annotations.runbook }}|:spiralnotepad:>
Details:
{{ range .Labels.SortedPairs }} • {{ .Name }}: {{ .Value }}
{{ end }}
{{ end }}


Thank you!

https://redd.it/lvccoe
@r_devops

Monthly 'Getting into DevOps' thread - 2021/03

What is DevOps?

[AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

Books to Read

The Phoenix Project - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
[The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
Google's Site Reliability Engineering - Google engineers explain how they build, deploy, monitor, and maintain their systems.
[The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book
The Unicorn Project - the "sequel" to The Phoenix Project.
[DevOps for Dummies](https://www.amazon.com/DevOps-Dummies-Computer-Tech-ebook/dp/B07VXMLK3J/) - don't let the name fool you.

What Should I Learn?

Emily Wood's essay - why infrastructure as code is so important into today's world.
[2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
This comment by /u/mdaffin - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
[This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.
Roadmap.sh - Step by step guide for DevOps or any other Operations Role

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

Previous Threads
https://www.reddit.com/r/devops/comments/la7j8w/monthlygettingintodevopsthread202102/

https://www.reddit.com/r/devops/comments/koijyu/monthlygettingintodevopsthread202101/

https://www.reddit.com/r/devops/comments/k4v7s0/monthlygettingintodevopsthread202012/

https://www.reddit.com/r/devops/comments/jmdce9/monthlygettingintodevopsthread202011/

https://www.reddit.com/r/devops/comments/j3i2p5/monthlygettingintodevopsthread202010/

https://www.reddit.com/r/devops/comments/ikf91l/monthlygettingintodevopsthread202009/

https://www.reddit.com/r/devops/comments/i1n8rz/monthlygettingintodevopsthread202008/

https://www.reddit.com/r/devops/comments/hjehb7/monthlygettingintodevopsthread202007/

https://www.reddit.com/r/devops/comments/gulrm9/monthlygettingintodevopsthread202006/

https://www.reddit.com/r/devops/comments/gbkqz9/monthlygettingintodevopsthread202005/

https://www.reddit.com/r/devops/comments/ft2fqb/monthlygettingintodevopsthread202004/

https://www.reddit.com/r/devops/comments/fc6ezw/monthlygettingintodevopsthread202003/


https://www.reddit.com/r/devops/comments/axcebk/monthlygettingintodevopsthread/

Please keep this on topic (as a reference for those new to devops).

https://redd.it/lvet1r
@r_devops

Streamline setting up local dev env with docker?

We plan on growing our development team by 2-3x over the next year.

In order to minimize onboarding time, we have been considering if we can someone containerize a local development environment so all new devs can easily work in the same environment, and minimize time needed for set up.

For context, we’re primarily a Java shop.

Any thoughts, suggestions, best practices would be appreciated. Have done some research online but haven’t found anything them feels definitive yet.

Thanks!

https://redd.it/lvg7q3
@r_devops

Best resources/tutorials for setting up basic CI for personal full stack web app

Hi, devops.

I'm a student trying to learn about continuous integration by setting up a basic CI pipeline for a web app. The app uses Python's Flask framework for the back end and React for the front end. I've been trying to use Travis CI.

I've had a lot of questions while trying to write the .travis.yml file to install the front end and back end and then run tests for each. For example, I'm not sure if I should create separate git branches (each with their own yml file) for the Flask app and React app, or if I should use a single yml file that somehow directs the Travis VM to call the installation and testing scripts in the front end and back end subdirectories.

I've been surprised by how few tutorials I've found for setting up a basic CI workflow for a project like this with some unit tests and maybe some tests that test the front and back ends together. I'm definitely a noob with devops though, so I might be asking the wrong questions.

Are there any tutorials or resources you would suggest for learning how to set up a basic CI pipeline for a simple personal project like this?

https://redd.it/lvmtol
@r_devops

MTV Cribs for DevOps

I work at a small company and I'm a junior engineer. I'm getting into DevOps but neither of my coworkers are really equipped to teach me about it. I've been reading lots of articles, but most are either about how to use specific technologies or about very general concepts like CI/CD. I want something between those two: specific enough to be practically useful, but general enough that the whole DevOps infrastructure is described. I'd love to learn how those that know more than me are doing things by having them show us round their systems and explain their rationales, a bit like how a pen tester will walk you through their exploit. Does this exist anywhere?

https://redd.it/lvmjeu
@r_devops

Jobs hiring trend in 2021

Working from remote would be a new normal?


what do you think or see the trend going to be or happening right now?

https://redd.it/lvl7v6
@r_devops