Cloud hosted log management tool for Spring boot application?

I have two Java applications written using Spring Boot and running on AWS. Unfortunately, browsing logs in an application hosted by AWS beanstalk is not very convenient. Therefore I am thinking about a cloud solution for collecting and viewing logs. I want to avoid self-management of ELK stack. What solutions do you recommend / use?


I thinking about :
\- logz.io
\- logdna.com
\- datadog
\- cloud hosted graylog?

https://redd.it/kn827j
@r_devops

What have you done to OS config lately with automation

Just wondering what you guys have recently done with IAC and why?

Last thing i did was to setup LVM on linux openSuse
And enabled windows features with packer to help cover all our bases when releasing software

https://redd.it/kn5mey
@r_devops

Getting Terraform to apply only when a change exists using Make

This is a post by me on how I cut down the time required required for a Terraform apply if no changes are to be done by making use of Terraform's detailed exit codes

It's a big long-ish, but hope it helps. https://sathyasays.com/2020/12/31/terraform-apply-only-on-change/

https://redd.it/knokx5
@r_devops

DevOps Trends for 2021

I'm thinking that the following would be the DevOps trends to watch for 2021:

1. Bundles and Software Bills of Materials.
2. Complexity vs Simplicity paradox.
3. Vertical integration in DevOps vs “Best Tool for the Job”.
4. Widening gap between high performers and low performers.
5. Kanban.
6. CI is the new Version Control – CD will come next.

Expanded version with details - https://worklifenotes.com/2020/12/30/devops-trends-for-2021/

Happy New Year everyone!

https://redd.it/knt49l
@r_devops

Deploying Python FastAPI on a Ubuntu VM with Caddy 2 Web Server

Hello DevOps Community,

As FastAPI is gaining popularity in contrast to Flask for building REST APIs. Most of the new prodcut development initiatives for REST APIs considering FastAPI in the Python space. It is also important to understand from the deployment stand point that FastAPI runs on any operating system and cloud.

As the runtime stack for FastAPI is Python, Ubuntu VM comes with Python already installed. So I thought trying to deploy the FastAPI on Ubuntu 18 VM. FastAPI runs on WSGI Web Server Gunicorn with the help of ASGI worker uvicorn worker class. The tutorial covers real time deployment by exposing the FastAPI running on Gunicorn as a reverse proxy via an opensource Web Server Caddy 2.

Hope you find it helpful.

Thanks!

https://redd.it/knobi6
@r_devops

A Six Article Series for Learning Docker

Hey folks!

I just completed my article series on Docker. I've tried to keep it as beginner friendly as possible. I hope it helps people out who are just getting started with Docker. If you have some experience with it I'll be glad if you could give it a read and let me know if you have any suggestions for me. Thanks a lot and a Happy New Year!

https://redd.it/knnk5b
@r_devops

what should be the ipv4_address when deploying docker compose cluster on azure vm?

I have the following docker compose file that I want to deploy to a Azure Viurtual Machine. I am new to deployment and cloud. I am really confused as to what to right in place of ipv4_address? In the following connection if I try to connect from within the VM it connects as the ipv4_address is 192.168.98.10.

But, it fails to connect from outside the vm over the global ip that azure provides.
```
---
# Source: tendisplus/templates/tendisplus.yaml

version: '3'
services:
tendisplus-master-0:
image: tencentdbforkv/tendisplus:latest
container_name: tendisplus-master-0
ports:
- "30000:51002"
networks:
tendisplus-net:
ipv4_address: 192.168.98.10
environment:
- REDIS_PASSWORD=password
- CLUSTER=yes
volumes:
- "/data/tmp/tendisplus/30000:/data1/tendis/51002"
restart: on-failure:3
tendisplus-slave-0:
image: tencentdbforkv/tendisplus:latest
container_name: tendisplus-slave-0
ports:
- "50000:51002"
networks:
tendisplus-net:
ipv4_address: 192.168.98.20
depends_on:
- tendisplus-master-0
environment:
- REDIS_PASSWORD=password
- CLUSTER=yes
volumes:
- "/data/tmp/tendisplus/50000:/data1/tendis/51002"
restart: on-failure:3
networks:
tendisplus-net:
driver: bridge
ipam:
config:
- subnet: 192.168.98.0/24
```
I also tried adding my global ip in place of the 192.168.98.10 but it sends this error
```
ERROR: for tendisplus-master-2 Cannot start service tendisplus-master-2: Invalid address 13.92.1.119: It does not belong to any of this network's subnets

ERROR: for tendisplus-master-2 Cannot start service tendisplus-master-2: Invalid address 13.92.1.119: It does not belong to any of this network's subnets

ERROR: for tendisplus-master-0 Cannot start service tendisplus-master-0: Invalid address 13.92.1.119: It does not belong to any of this network's subnets

ERROR: for tendisplus-master-1 Cannot start service tendisplus-master-1: Invalid address 13.92.1.119: It does not belong to any of this network's subnet
```

https://redd.it/knkbmi
@r_devops

Why Packer?

My boss wants our team to learn and implement Packer starting in 2021. That's cool, and I'm always down to learn, but I was wondering what the big benefit of Packer was. Specifically, what's so great about "config -> deploy" vs "deploy -> config" if you're using the same provisioner either way?

https://redd.it/ko3cwq
@r_devops

Kubevious v0.8 Release

As a gift for all Kubernetes users, we released Kubevious v0.8.

https://youtu.be/jrYlrdRjxDE

https://redd.it/ko4adl
@r_devops

How Do You Know You're Ready For A DevOps / SRE Position?

Hey All,

sorry for such a nebulous question I dont really have a network to reach out to about things like this. I'm currently a jr / sys admin at a large corporation and I'm stagnating. I'm not learning anything new Professionally or technologically and I'm not being exposed to new technology. so How do I know I'm ready to step up into a DevOps / SRE position or if I should wait a while longer and look for another kind of position and if I DO look for another position, what should it be?

I have my AZ900 and CCP, 1 year ex as a sys admin and work with powershell and python regularly. I feel comfortable digging in to things and coming up with a hypothesis. previously I've done everything from technical support to POS implementations. Insight or guidance would be MUCH appreciated, trying to make '21 my year.

https://redd.it/ko479t
@r_devops

Building OVAs with Packer

I have a requirement to generate Ubuntu and Windows OVAs for consumption by VMware Workstation or Player, and I've been experimenting with a few ways to do this in GCP using vmware-iso. My initial efforts have relied on Cloud Build, both with community remote builder and packer containers. I also tried doing a custom Ubuntu remote builder based on an image that had nested virtualization enabled. The pipeline looks something like this:

Launch builder from Google Cloud Build
Install Linux package dependencies on builder with apt
Install VMware Workstation on builder
Run Packer with vmware-iso builder, configured to perform an OVA export

The above fails because vmware-iso (and the other VMware builders from Packer) rely on a VMware installation on the builder, which won't launch without the right virtualization extensions. The GCP nested virtualization is limited to KVM. The closest thing I've found to a solution is using VMware Engine, which seems complex/pricey based on the documentation I've reviewed. If I could spin up / spin down a builder in VMware Engine, that might be possible, but it doesn't seem like this will easily integrate with Cloud Build as a remote builder. It also doesn't seem to cover smaller requirements like this one.

I'm interested in knowing if anyone has tried connecting VMware Engine with Cloud Build, or has another perspective on delivering this requirement in any cloud provider.

https://redd.it/knyntg
@r_devops

Help with an API deployed in Azure that doesn't work and i dont know why.

Hello, i deployed my API to azure via bitbucket piplene using FTP, but when i open the link of the API it displays "Error 500.30"

I want to kudos and did dotnet projectName.dll and it showed that it cant find newtonsoft.json version = 12.0.0.0 . I cant realy understand why this message shows as i have installed version 12.0.0.2 and localy, even when i do dotnet publish i works fine.

Any ideas?

https://redd.it/kohp3u
@r_devops

Monthly 'Getting into DevOps' thread - 2021/01

What is DevOps?

[AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

Books to Read

The Phoenix Project - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
[The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
Google's Site Reliability Engineering - Google engineers explain how they build, deploy, monitor, and maintain their systems.
[The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book
The Unicorn Project - the "sequel" to The Phoenix Project.
[DevOps for Dummies](https://www.amazon.com/DevOps-Dummies-Computer-Tech-ebook/dp/B07VXMLK3J/) - don't let the name fool you.

What Should I Learn?

Emily Wood's essay - why infrastructure as code is so important into today's world.
[2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
This comment by /u/mdaffin - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
[This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.
Roadmap.sh - Step by step guide for DevOps or any other Operations Role

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

Previous Threads
https://www.reddit.com/r/devops/comments/k4v7s0/monthlygettingintodevopsthread202012/

https://www.reddit.com/r/devops/comments/jmdce9/monthlygettingintodevopsthread202011/

https://www.reddit.com/r/devops/comments/j3i2p5/monthlygettingintodevopsthread202010/

https://www.reddit.com/r/devops/comments/ikf91l/monthlygettingintodevopsthread202009/

https://www.reddit.com/r/devops/comments/i1n8rz/monthlygettingintodevopsthread202008/

https://www.reddit.com/r/devops/comments/hjehb7/monthlygettingintodevopsthread202007/

https://www.reddit.com/r/devops/comments/gulrm9/monthlygettingintodevopsthread202006/

https://www.reddit.com/r/devops/comments/gbkqz9/monthlygettingintodevopsthread202005/

https://www.reddit.com/r/devops/comments/ft2fqb/monthlygettingintodevopsthread202004/

https://www.reddit.com/r/devops/comments/fc6ezw/monthlygettingintodevopsthread202003/

https://www.reddit.com/r/devops/comments/exfyhk/monthlygettingintodevopsthread2020012/

https://www.reddit.com/r/devops/comments/ei8x06/monthlygettingintodevopsthread202001/

https://www.reddit.com/r/devops/comments/axcebk/monthlygettingintodevopsthread/

Please keep this on topic (as a reference for those new to devops).

https://redd.it/koijyu
@r_devops

Best way to learn DevOps

Hey r/devops!

I am a self-taught Python developer, and so far I have experienced building various projects. From developing a desktop GUI app, writing web automation, and building web apps and APIs using Flask.

I am at a point where I want to start looking for a job in a field that involves scripting (Bash/Python) and "making things work" so I started to learn about DevOps and found it very interesting.

I do not have any degree/diploma and everything I know I learned by myself.

Although my full-stack skills are decent, I am not attracted to this field. And as I don't have a CS degree, finding a job as a Python developer can be pretty hard (as they always prefer someone with a degree).

This is why I want to get a grasp on DevOps as I truly believe that this is something I am capable of doing, only I am not sure how hard it will be to find a job once I am ready to start looking for one.

I would really appreciate it if someone could shed some light and what exactly I need to cover and learn to be at a point where I can start applying for a DevOps position.

Thanks!

https://redd.it/kofxzi
@r_devops

What is Vagrant ? Why Vagrant is important in DevOps World ? | Part 1 | Beginner | Vagrant Commands

Hello All, 
In this tutorial, we all will learn complete vagrant in three different parts.
[Part 1 includes](https://www.randomskool.com/2021/01/what-is-vagrant-why-vagrant-is.html) **-** 

* *Introduction of vagrant*
* *Why vagrant is used?*
* *Different commands used in vagrant*
* *Example - Provisioning using a simple bash script*

### [Introduction of Vagrant](https://www.randomskool.com/2021/01/what-is-vagrant-why-vagrant-is.html) -

A vagrant is a tool for building and managing virtual machine environments in a single workflow.
Vagrant is used to setting up one or more virtual machines by:


* *Importing pre-made images (called "boxes")*
* *Setting VM-specific settings (IP address, hostnames, port forwarding, memory, etc.)*
* *Running provisioning software like Ansible and SaltStack.*

### [Why vagrant is used?](https://www.randomskool.com/2021/01/what-is-vagrant-why-vagrant-is.html)

Vagrant allows us very easily to share setups between team members allowing a very easy spin-up of a work environment. Suppose, one reason to use Vagrant is to test how your deployment works, i.e. provisioning, locally before pushing those changes to other environments.

For Complete Tutorial - [https://www.randomskool.com/2021/01/what-is-vagrant-why-vagrant-is.html](https://www.randomskool.com/2021/01/what-is-vagrant-why-vagrant-is.html)

https://redd.it/koeu14
@r_devops

Question: How to make your own server

I have been making websites and purchasing vps to host them, I know how to configure things and start the server to listen to any elastic ip address, I figure having a raspberrypi running my server would be far more economical,

I figured that I could just set up the nginx with proper configurations there, my question is, how do I make it accept requests over the internet, do I need a custom dns? Also how would that ip be static for that to work? Is there any tutorial I can refer to?

https://redd.it/koapek
@r_devops

How worth it is a degree at this point in my career? What's the max earning potential with vs without one for an IC technical tract?

So I've reached a point where I'm making six figures in a medium COL area in my late 20s without a degree and am doing alright for myself, but I've been thinking about going for WGU BS Cloud Computing degree to see what additional options would open up. Is it worth the time, effort, and cost though? Should I just grind leetcode instead? I can get interviews easily enough though passing them is another story altogether.

https://redd.it/kop54x
@r_devops

How to move to Devops

Hello,
I need your advice on transitioning my career to DevOps from a developer role. Currently, I work at a small-medium company doing their operations like deploying manually to servers and sometimes doing developments with C#, etc. But my role is designated as a junior architect as I take care of mainly Website Deployments, database deployments, etc to different environments. By the way we are not cloud yet. I also take care of the TFS. Code reviews and check in before the deployments. Please advise me some steps to begin with. Thanks.

https://redd.it/kp27ie
@r_devops

What is the best way to solve a problem

So devops and the cloud are all the rage theses days. A lot of the problems that we are facing these days aren't necessarily new to the industry but they may be new to you. I have years of experience as a developer but not a lot of experience in operations and in IT. As a senior devops member on my team, I don't think it is expected that I have answers for everything but should know how to get to a reasonable solution. As all companies are different, there are many ways to solve a problem and it depends on the context of the problem and how it affects the company. How do you develop insight of all the different factors that relate to a problem. One of the ways is to get first hand experience or to learn from somebody in your company. But what happens when you or anybody your company doesn't have insight? If they don't have any idea what they are doing, they should be honest and admit that they don't know

Here is an example. I don't have any experience with protecting systems against DDoS nor does anyone else in my company. But suppose I am now the person responsible for resolving the issue. I can search on Google, AWS docs or ask reddit, but is the industry standard way of finding solutions for problems that are new to you. Another resource would be to refer to books on devops, security or Google SRE on possible solutions.

How do you approach problems that you are unfamiliar with? Is an acceptable answer to tell your team that you don't know and will need to spend a day reading a book or AWS articles on DDoS, in order to get enough background information

https://redd.it/kpaojh
@r_devops

Windows IAC?

Is it possible to setup a windows environment in a way where everything is defined as code, or is this a futile thought? I'm vaugly familiar with some devops practices but I work in a smaller business where Windows is a large share of our environment. Would be nice to have everything in code so if something needs to be rebuilt it could be done quickly.

https://redd.it/kpbr6z
@r_devops

How to practice devops learnings?

Hi I am an release engineer and planning to move to devops role. I've been learning different tools like ansible, docker, kubernetes. But I do not have scope to implement them at my work. Can you please help me finding a best way to get hands-on experience and practice with the devops tools I learn.

https://redd.it/kp5udn
@r_devops