Considering Dev-Ops or AWS. OSCP holding Brother suggests AWS to get started.

I have a brother who's doing particularly well in Dev-Ops, he's rather young, and I want a change. I am a decade older (almost 30), have a 4 year business degree, and some basic experience on a few various things--but nothing that's great.

Since he's young & a family member I don't like pestering him since it involves business.
I do not wish to look for a "leg up" with his employer but want to find a pathway that works for me and talk to him if I gain competence down the road.

He told me AWS (always found it rather interesting when I play with LightSail) doesn't pay as well but is in decent demand in the US (we live in the US) and if I can learn enough to do some basic projects I should have a chance of scoring a job--that could set me up for a position down the road.

I'm unable to study for the OSCP or burn 1000+ hours right now (as he did)--but do have 1-2 hours a day, learn quickly, and am self motivated with everything I do. I have a family to take care of and am highly motivated to take it.

If you were in my situation of limited time, carried some technical aptitude, learned at a decent pace, and wanted to get into the first step what would you consider doing?

(Thus far I've read the entire AWS whitepaper, done some basic tutorials on Amazon, and will start learning Python--to get a bearing of what's actually out there)

\---------------------------------

Currently I make a living with CNC routers fabricating complex 3d models that I cut out and market direct online in the states. I manage all of the marketing, customer service, design work, CAD/CAM, and all the stuff involving the machines....I've dabbled with LinuxCNC (CLI for setup and installs), Arduino (C++ but just basic things of using premade libraries), Wordpress (HTLM, CSS, DNS, SSL, random things involving Cloudflare, Lightsail, etc), and a lot of graphics/CAM/CAD programs over the years.

https://redd.it/neuirk
@r_devops

AWS Lambda support for Node.js 10 is becoming to an end in August 2021. It’s time to switch!

It’s the end of AWS Lambda support for Node.js v10. AWS Lambda support for Node.js 10 is due to end in August 2021. It’s time to switch! In this article, we’re discussing and comparing the differences of working with Node.js 10 and Node.js 14 + AWS Lambda, the impacts, and benefits of this change:

https://dashbird.io/blog/aws-lambda-nodejs-10-vs-14/

https://redd.it/nej1jl
@r_devops

Super interesting links for DevOps Interview

Heya Techies!

Would you mind listing down some perfect links which would help me in cracking the Devops interview.

Really needed....!!!!

https://redd.it/ne7dnx
@r_devops

Log (Observability) Overview

Observability and logging are always complex and bespoke solutions. Over the years, I understood fragments, and recently I assembled the knowledge into a mental mindmap that I thought others would like.

The core roles of components:

Log (data) shipper \- send logs somewhere (such as router or aggregator). These are also called log forwarders.
Log (event) router \- collect logs, and send them somewhere based on rules you define. These are also called log collectors, log processors, and log forwarders.
Log aggregator \- centralized datastore + indexing; essentially where to store and retrieve logs
Visualization \- create graphical dashboards for data sources (like centralized logging)
Ingestion \- buffer logs so that it doesn't overwhelm the system (HA), these vary, some platforms integrate this, have direct support, or can be hacked into any solution.
Alerting \- an alert system based on metrics (metrics extracted from logs, such as a single event, or past a threshold)

There's a lot of overlap with log shipping, log routing, and log collecting, as tools have varying levels of this. For example, Filebeat and Promtail can fetch from files or use discovery with docker and Kubernetes, while Fluentd, Fluentbit, and Logstash have plugins to collect logs and other data from a variety of sources.

These components may have different features, common ones:

Transformation Pipeline \- filter and transform logs, e.g. apply regex to break up a log line into JSON, create metrics from logs, or transform to another text format.
Discovery \- find systems to get logs, such as Docker container labels or container names, or Kubernetes labels.

Different tools may fill one or more of these roles, and you may not use all of the components, as logging and in general observability is always a bespoke solution.

Examples:

Stacks (combination of roles): PLG (promtail-loki-grafana), TICK (Telegraf-InfluxDB-Chronograf-Kapacitor), ELK (ElasticSearch-Logstash-Kibana), EFK (ElasticSearch-Fluentd|Fluentbit-Kibana), ElasticStack (ELK + Beats)
Log (Event) Routers: Logstash, Fluentd
Log (Data) Shippers†: syslog, syslog-ng, rsyslog2, Filebeat, LogStash, Fluentbit, Fluentd, Promtail, Telegraf
Log aggregators: ElasticsSarch, Graylog, InfluxDB, Loki
Visualization: kibana, grafana, chronograf
Transformation: logstash, Fluentd, rsyslog2, syslog-ng, ElasticSearch ingestion pipeline
Discovery: Logstash, Fluentd, Fluentbit, FileBeat, Promtail, Telegraf†††
Alerting: Kapacitor, Grafana, Kibana, Prometheus AlertManager††

† Logging libraries like Log4j may have built-in capability to ship logs, such as supporting syslog protocol. Docker for example through Docker driver resource supports a variety of sources.

†† In PLG, alerts can be triggered by Grafana or Prometheus Alertmanger. Prometheus would be something akin to a metrics aggregator or centralized data source for metrics, and it scrapes logs from a variety of sources but also supports pushing metrics to it. Loki uses the same Prometheus library, but logs are pushed (shipped) to it from promtail (or another shipper like Fluentbit) and stored in an object-store. Promtail can create metrics that are based on log data that can be used to trigger alerts.

††† Telegraf can extract logs as well as metrics (so it like Prometheus in this aspect) as well as traces (so combined with InfluxDB, like Jaeger w/ ElasticSearch or Cassandra, or Tempo with an object store).

Many of these tools, such as Fluentbit, FluentD, and Logstash have plugins to gather data beyond just logs, including metrics or SNMP, and the Beats family of tools have specialized beats for different data sources, such as Filebeat for logs and MetricBeat for metrics.

https://redd.it/ne740z
@r_devops

Question about tech stack/servers setup using kubernetes. Dealing with uploading images

Hey all,

Current project is broke down into 3 servers right now.

* Auth server
* Main API server
* Chat server

I'm currently setting up the uploading images for users and I'm wondering if it would be smart to separate it from the Main API server.

It's my first time handling image uploads and I'm looking at using [multer](https://github.com/expressjs/multer#readme).

I'm wondering if anyone who has experience with uploading files could point me in the right direction with the tech stack they used and how their servers were setup.

Appreciate any insight,

Cheers

https://redd.it/ne44ds
@r_devops

Looking for advice - I have to automate all the company applications and don't know where to start

Hello, let me tell you the whole story:

I work in a small startup, we have around 3 to 4 SaaS products all hosted at AWS.

I'm currently acting as a Jr Devops (moved from embedded dev) in a 5 dev team. The Company used to give all the devops responsibility to one senior dev in our company but he just takes too long (he hates it) and well, now it's my responsibility to make it all as quick and seamless as possible but the thing is, I don't know where to start.

I've already written and deployed the CI/CD in bitbucket pipelines, but I still have to manually get into every instance and set every new env.

Every time we need to scale up or down, I have to reinstall everything through the terminal command by command.

From my point of view I have to:

1) Change all the different deploys from manual installation to containers. (That way I don't have to install everything by hand, nor the devs)

2) Use Terraform (Ansible?) to start using infrastructure as code instead of logging every time to the AWS console and manually set up instances, buckets, R53 and IAMs

3) Kubernetes (Docker Swarm?) To control and orchestrate said containers. (Is it really necessary?)

4) Monitor everything with Prometheus and forget about having to ssh to each instance so see if the API collapsed

5) Plug and pray

I want to know if I'm headed in the right direction given this is my first devops experience.

Thanks!

https://redd.it/ne3vrk
@r_devops

Help identifying tool icons - in comments

I just joined a new devops team and am trying to get up to speed on their tools. Could anyone identify the tools by icon below (I tried reverse google image search - no luck)

https://redd.it/nfduev
@r_devops

Deploying a transformer-based text classification NLP model with FastAPI

Hello all,

I recently wrote an article about deploying spaCy NLP models with FastAPI for entity extraction. As many people told me it was helpful, I did a new article about deploying transformer-based NLP models with FastAPI for text classification (using Facebook's Bart Large MNLI model).

FastAPI is a great is great framework for API development in Python in my opinion. It helped me save a lot of troubles when developing the NLPCloud.io API.

Here's the article:

https://nlpcloud.io/nlp-machine-learning-classification-api-production-fastapi-transformers-nlpcloud.html

I'd love to have your feedback on this! Have you ever deployed transformer-based models to production? If so, which tools did you use?

Thanks!

https://redd.it/nfao41
@r_devops

What are your killer tips for kubernetes cost optimization?

Title says it all

https://redd.it/nfdtct
@r_devops

Major Pagerduty Downtime

Pagerduty experiencing a major downtime

https://status.pagerduty.com/incidents/wt6p4xl7htft

https://redd.it/nfk60i
@r_devops

What’s the Difference Between Django and Flask

Flask and Django are mature, extensible web frameworks that fundamentally provide similar functionality in handling requests and maintaining documents but differ in scope.

Let's check out the difference between Flask vs Django performance.

Most of the differences between the two frameworks stem from different approaches, the rest from excellent basic design decisions. Here are a few key differences that might influence your decision:

Request object – Flask uses local streams, and Django passes the request where it needs to be.

Forms – Django is available with built-in forms that integrate with the ORM and site admin area. Flask doesn’t support forms by default, but you can use WTForms to fill that gap.

Databases – Django is available with a built-in ORM and migration system that can manage databases. Flask doesn’t do that, but tools like SQLAlchemy provide similar functionality (or even more).

Authentication and User Privileges – Django provides an authentication application that provides a default implementation for user control and privileges. Flask provides secure cookies as a tool for your own implementation.

Admin Panel – Django includes a fully integrated admin interface for managing application data. Flask doesn’t have these features, but Flask-Admin is a very popular extension that can be used to create a similar admin tool.

https://redd.it/nf9ngo
@r_devops

DevOps Query

I’m pretty much new to the IT world and I would like some insight on where to start from scratch to land a future career in DevOps please help.

Does anyone here know how I would I be able to pursue a dev ops career from scratch? Someone I know recommended this IT path for me and I really don’t know where to start so any help or suggestions would be greatly appreciated. Seeing as though it’s a very lucrative field that would provide me with some sort of financial stability and I could grow to really like it.

Furthermore what certs should I attain first to work my way up to this role. I posted this question in the IT sub Reddit and they directed me here so I would really appreciate it if someone could provide some insight on where I should begin. Also how long does it take to work your way up the ladder to reach this role because I’m being told it’s not for beginners.

https://redd.it/nfna31
@r_devops

Is learning DevOps tools from Kode kloud any good?

I am interested in learning DevOps...and I am completely new to this technology so y'all suggestions would be really helpful !!
If not Kode kloud any other resources please :)

https://redd.it/nflc40
@r_devops

GitHub Enterprise vs. GitLab (Explanation for a Non Tech Person)

Hi folks,

I'm prepping for a non-tech interview with GitHub this week. During the interview, it's likely I'll need to articulate the competitive landscape for GitHub - particularly as it relates to GitHub's Enterprise product. The obvious comparison seems to boil down to GitHub vs. GitLab (or maybe I'm missing something?).

Ask: Any bullet point differences between the two to help my non tech brain contextualize how they stack up to each other? Perhaps points as to why you chose (or didn't choose) to use GitHub over others?

Huge thanks in advance for any tips!

https://redd.it/nf9mu5
@r_devops

Large Icon libraries for CI/CD diagramming?

I’ve been using a mish-mash of various found and edited icons over the years such as the AWS architecture SVGs for internal diagramming of flows, but I now need something publishable.

Free use or commercially licensed, i’ve been trying to find a library of cohesively styled SVGs to cover the wide range of concepts in DevOps like git operations, merge requests, different tests, reports, issue creation, deploying, etc. I’ve gotten pretty far editing together some basic graphical primitives but many concepts need a designer’s hand and I just don’t have the time or budget to do it all from scratch. Surely there are some good solutions already available?

https://redd.it/nf4nnz
@r_devops

OPEN SOURCE APM TOOLS

Heya. I'm doing a study/graduate thesis for a relatively large company that's looking to revamp its network infrastructure, and as a part of the process, they're looking into getting an APM to monitor their systems with. I've acquainted myself with the tools and elicited some requirements regarding what needs people actually have regarding it, and I've gotten somewhat familiar with Dynatrace, Appdynamics, New Relic and Datadog APM already as a part of it all.

I'd like to look into open source solutions too, though, but they aren't discussed a lot. Gartner for example basically ignores their existence, from what I can tell. I'm aware of things like Pinpoint and SkyWalking, but I'm a bit unsure how they compare to the paid solutions.

Does anyone have any experience with them? Are they worth looking into? And for that matter, are there any non-open source APMs that I should consider, beyond just those four?

https://redd.it/nf4ikw
@r_devops

How long do your pipelines take? How long SHOULD they take?

Like, your test, deploy, linting, whatever pipelines. All of them. Do you have a lot of small short ones? Do you have a few long, enormous ones?

I was just wondering, since I have a pretty large one that I was thinking of turning into to a batch job and I was curious how others approached this situation.

https://redd.it/nfr3aq
@r_devops

A Devops Dissertation Idea

Hi Reddit,

I'm approaching the end of my DevOps course and am required to write a dissertation covering a relevant / interesting research question.

My idea is to compare multi-cloud IaC tools (terraform, pulumi, ansible) in terms of performance / ease of development etc.

I'm trying to think of a good use case to implement with each of these tools on AWS, Azure and GCP. Wondering if any person with experience in the field has a novel idea for this? I have no background with cloud platforms so any suggestions would be appreciated 🙂

https://redd.it/nggjz6
@r_devops

CI/CD ideas for DevOps student

Hi!

I have been studying DevOps technologies for last three month. And now I need to choose what final project should I deploy. It's some kind of final task that I need to do to getting to the next course. But I don't have any ideas what to deploy.

I know only Spring/PetClinic, but I don't know java and I have not much time to learn it. Maybe somebody knows some open source project which I will be able to deploy with CI/CD instruments like Jenkins, Ansible and Terraform.

Thanks for advices!

https://redd.it/ngd2ss
@r_devops

Is running a Database like MySQL in a docker container any good?

Hello DevOps Enthusiasts

I am looking for some guidance here, what would be the best practice to run a database - directly on the host or in a container?

Host seems to be the answer to me, but I would like to know if anyone running in containers for production environments and are there any pitfalls/benefits doing this.

Thanks

https://redd.it/ngce5d
@r_devops

The configuration system that has to exist

I feel like I'm taking crazy pills. At work, we've got configuration scattered in various systems. Some application configuration is set in puppet (hiera) which we use for the bulk of our system configuration management. Secrets are mixed in there, so access for view or edit by devs is off limits. Some configuration is set in a git repo that holds a tsv file, which is actually more useful than you might think. Other configuration is scattered among various code repos like you might expect.

I am looking for a configuration to bind them all together, and this is what I'm looking for:

management of secrets (vault would be ideal)
version control of all config (need to know who changed what when, rollback, etc.)
usable by both puppet (hiera) and various other tools (cli / REST api / etc.)
flexible acls/permissions to various parts of the tree

I feel like I'm missing the forest for the trees, because I know I'm not unique, but I can't find this system. Consul or etcd is nice for making configuration available, but doesn't have the sort of version control I want (I'm also not really in need of service discovery). Git repos with structured config might work with hiera, but doesn't have the kind of api or secret management I'd like. The closest I came to ideal was Jerakia (https://jerakia.io/) but this project looks to be pretty much abandoned.

Can someone point me towards a system that can do what I need? Any clues to put me in the right direction would be great - I just need a push.

https://redd.it/ngm8rc
@r_devops