Question about tech stack/servers setup using kubernetes. Dealing with uploading images

Hey all,

Current project is broke down into 3 servers right now.

* Auth server
* Main API server
* Chat server

I'm currently setting up the uploading images for users and I'm wondering if it would be smart to separate it from the Main API server.

It's my first time handling image uploads and I'm looking at using [multer](https://github.com/expressjs/multer#readme).

I'm wondering if anyone who has experience with uploading files could point me in the right direction with the tech stack they used and how their servers were setup.

Appreciate any insight,

Cheers

https://redd.it/ne44ds
@r_devops

Looking for advice - I have to automate all the company applications and don't know where to start

Hello, let me tell you the whole story:

I work in a small startup, we have around 3 to 4 SaaS products all hosted at AWS.

I'm currently acting as a Jr Devops (moved from embedded dev) in a 5 dev team. The Company used to give all the devops responsibility to one senior dev in our company but he just takes too long (he hates it) and well, now it's my responsibility to make it all as quick and seamless as possible but the thing is, I don't know where to start.

I've already written and deployed the CI/CD in bitbucket pipelines, but I still have to manually get into every instance and set every new env.

Every time we need to scale up or down, I have to reinstall everything through the terminal command by command.

From my point of view I have to:

1) Change all the different deploys from manual installation to containers. (That way I don't have to install everything by hand, nor the devs)

2) Use Terraform (Ansible?) to start using infrastructure as code instead of logging every time to the AWS console and manually set up instances, buckets, R53 and IAMs

3) Kubernetes (Docker Swarm?) To control and orchestrate said containers. (Is it really necessary?)

4) Monitor everything with Prometheus and forget about having to ssh to each instance so see if the API collapsed

5) Plug and pray

I want to know if I'm headed in the right direction given this is my first devops experience.

Thanks!

https://redd.it/ne3vrk
@r_devops

Help identifying tool icons - in comments

I just joined a new devops team and am trying to get up to speed on their tools. Could anyone identify the tools by icon below (I tried reverse google image search - no luck)

https://redd.it/nfduev
@r_devops

Deploying a transformer-based text classification NLP model with FastAPI

Hello all,

I recently wrote an article about deploying spaCy NLP models with FastAPI for entity extraction. As many people told me it was helpful, I did a new article about deploying transformer-based NLP models with FastAPI for text classification (using Facebook's Bart Large MNLI model).

FastAPI is a great is great framework for API development in Python in my opinion. It helped me save a lot of troubles when developing the NLPCloud.io API.

Here's the article:

https://nlpcloud.io/nlp-machine-learning-classification-api-production-fastapi-transformers-nlpcloud.html

I'd love to have your feedback on this! Have you ever deployed transformer-based models to production? If so, which tools did you use?

Thanks!

https://redd.it/nfao41
@r_devops

What are your killer tips for kubernetes cost optimization?

Title says it all

https://redd.it/nfdtct
@r_devops

Major Pagerduty Downtime

Pagerduty experiencing a major downtime

https://status.pagerduty.com/incidents/wt6p4xl7htft

https://redd.it/nfk60i
@r_devops

What’s the Difference Between Django and Flask

Flask and Django are mature, extensible web frameworks that fundamentally provide similar functionality in handling requests and maintaining documents but differ in scope.

Let's check out the difference between Flask vs Django performance.

Most of the differences between the two frameworks stem from different approaches, the rest from excellent basic design decisions. Here are a few key differences that might influence your decision:

Request object – Flask uses local streams, and Django passes the request where it needs to be.

Forms – Django is available with built-in forms that integrate with the ORM and site admin area. Flask doesn’t support forms by default, but you can use WTForms to fill that gap.

Databases – Django is available with a built-in ORM and migration system that can manage databases. Flask doesn’t do that, but tools like SQLAlchemy provide similar functionality (or even more).

Authentication and User Privileges – Django provides an authentication application that provides a default implementation for user control and privileges. Flask provides secure cookies as a tool for your own implementation.

Admin Panel – Django includes a fully integrated admin interface for managing application data. Flask doesn’t have these features, but Flask-Admin is a very popular extension that can be used to create a similar admin tool.

https://redd.it/nf9ngo
@r_devops

DevOps Query

I’m pretty much new to the IT world and I would like some insight on where to start from scratch to land a future career in DevOps please help.

Does anyone here know how I would I be able to pursue a dev ops career from scratch? Someone I know recommended this IT path for me and I really don’t know where to start so any help or suggestions would be greatly appreciated. Seeing as though it’s a very lucrative field that would provide me with some sort of financial stability and I could grow to really like it.

Furthermore what certs should I attain first to work my way up to this role. I posted this question in the IT sub Reddit and they directed me here so I would really appreciate it if someone could provide some insight on where I should begin. Also how long does it take to work your way up the ladder to reach this role because I’m being told it’s not for beginners.

https://redd.it/nfna31
@r_devops

Is learning DevOps tools from Kode kloud any good?

I am interested in learning DevOps...and I am completely new to this technology so y'all suggestions would be really helpful !!
If not Kode kloud any other resources please :)

https://redd.it/nflc40
@r_devops

GitHub Enterprise vs. GitLab (Explanation for a Non Tech Person)

Hi folks,

I'm prepping for a non-tech interview with GitHub this week. During the interview, it's likely I'll need to articulate the competitive landscape for GitHub - particularly as it relates to GitHub's Enterprise product. The obvious comparison seems to boil down to GitHub vs. GitLab (or maybe I'm missing something?).

Ask: Any bullet point differences between the two to help my non tech brain contextualize how they stack up to each other? Perhaps points as to why you chose (or didn't choose) to use GitHub over others?

Huge thanks in advance for any tips!

https://redd.it/nf9mu5
@r_devops

Large Icon libraries for CI/CD diagramming?

I’ve been using a mish-mash of various found and edited icons over the years such as the AWS architecture SVGs for internal diagramming of flows, but I now need something publishable.

Free use or commercially licensed, i’ve been trying to find a library of cohesively styled SVGs to cover the wide range of concepts in DevOps like git operations, merge requests, different tests, reports, issue creation, deploying, etc. I’ve gotten pretty far editing together some basic graphical primitives but many concepts need a designer’s hand and I just don’t have the time or budget to do it all from scratch. Surely there are some good solutions already available?

https://redd.it/nf4nnz
@r_devops

OPEN SOURCE APM TOOLS

Heya. I'm doing a study/graduate thesis for a relatively large company that's looking to revamp its network infrastructure, and as a part of the process, they're looking into getting an APM to monitor their systems with. I've acquainted myself with the tools and elicited some requirements regarding what needs people actually have regarding it, and I've gotten somewhat familiar with Dynatrace, Appdynamics, New Relic and Datadog APM already as a part of it all.

I'd like to look into open source solutions too, though, but they aren't discussed a lot. Gartner for example basically ignores their existence, from what I can tell. I'm aware of things like Pinpoint and SkyWalking, but I'm a bit unsure how they compare to the paid solutions.

Does anyone have any experience with them? Are they worth looking into? And for that matter, are there any non-open source APMs that I should consider, beyond just those four?

https://redd.it/nf4ikw
@r_devops

How long do your pipelines take? How long SHOULD they take?

Like, your test, deploy, linting, whatever pipelines. All of them. Do you have a lot of small short ones? Do you have a few long, enormous ones?

I was just wondering, since I have a pretty large one that I was thinking of turning into to a batch job and I was curious how others approached this situation.

https://redd.it/nfr3aq
@r_devops

A Devops Dissertation Idea

Hi Reddit,

I'm approaching the end of my DevOps course and am required to write a dissertation covering a relevant / interesting research question.

My idea is to compare multi-cloud IaC tools (terraform, pulumi, ansible) in terms of performance / ease of development etc.

I'm trying to think of a good use case to implement with each of these tools on AWS, Azure and GCP. Wondering if any person with experience in the field has a novel idea for this? I have no background with cloud platforms so any suggestions would be appreciated 🙂

https://redd.it/nggjz6
@r_devops

CI/CD ideas for DevOps student

Hi!

I have been studying DevOps technologies for last three month. And now I need to choose what final project should I deploy. It's some kind of final task that I need to do to getting to the next course. But I don't have any ideas what to deploy.

I know only Spring/PetClinic, but I don't know java and I have not much time to learn it. Maybe somebody knows some open source project which I will be able to deploy with CI/CD instruments like Jenkins, Ansible and Terraform.

Thanks for advices!

https://redd.it/ngd2ss
@r_devops

Is running a Database like MySQL in a docker container any good?

Hello DevOps Enthusiasts

I am looking for some guidance here, what would be the best practice to run a database - directly on the host or in a container?

Host seems to be the answer to me, but I would like to know if anyone running in containers for production environments and are there any pitfalls/benefits doing this.

Thanks

https://redd.it/ngce5d
@r_devops

The configuration system that has to exist

I feel like I'm taking crazy pills. At work, we've got configuration scattered in various systems. Some application configuration is set in puppet (hiera) which we use for the bulk of our system configuration management. Secrets are mixed in there, so access for view or edit by devs is off limits. Some configuration is set in a git repo that holds a tsv file, which is actually more useful than you might think. Other configuration is scattered among various code repos like you might expect.

I am looking for a configuration to bind them all together, and this is what I'm looking for:

management of secrets (vault would be ideal)
version control of all config (need to know who changed what when, rollback, etc.)
usable by both puppet (hiera) and various other tools (cli / REST api / etc.)
flexible acls/permissions to various parts of the tree

I feel like I'm missing the forest for the trees, because I know I'm not unique, but I can't find this system. Consul or etcd is nice for making configuration available, but doesn't have the sort of version control I want (I'm also not really in need of service discovery). Git repos with structured config might work with hiera, but doesn't have the kind of api or secret management I'd like. The closest I came to ideal was Jerakia (https://jerakia.io/) but this project looks to be pretty much abandoned.

Can someone point me towards a system that can do what I need? Any clues to put me in the right direction would be great - I just need a push.

https://redd.it/ngm8rc
@r_devops

Logging the 12 Factor App Way

So the 12 Factor App methodology recommends logging everything to standard out and using log routers to send logs to their final destination. I definitely like the idea, it abstracts the dependency of the logging technology out of the application, removes the need for the developer to be familiar with logging technology, makes logging more portable, etc.

Opinions of the 12fa aside, I'm looking for some good examples of the "log routers" they are referring to. I see them reference LogPlex (which is depricated) and fluentd on the 12fa website but I'm curious to seewhat else is out there and what experiences others have had with this approach.

Thanks in advanced!

https://redd.it/ng8nac
@r_devops

I am in a DEVOPS role, and can't code.

Good Morning,

First time poster. I recently as of December got a new role within my company. I went from being Tier 2 support for network escalations and a team technical lead to a Devops role.

I got this role because I know networking (10+ years experience, CCNA, Security+) So I am now 6 months in. I am a good fit I was actually loaned to this team for half a year and they wanted to keep me so I can do the job. But I want to grow and go above and beyond.

Python will be a huge asset and I want to learn it but I have no idea where to start. I have learned to bash script as part of my role (testing devices and automating tests) So I am not a complete noob.

Where can I start learning Python? I have tried to use guides online and libraries like paramiko / netmiko but they don't work well in my environment due to SSH encryption differences. So instead of going down that route I want to learn it as it pertains to Networking Automation etc and not rely on unsupported libraries and copy and pasting code.

https://redd.it/ng9451
@r_devops

smee.io to forward webhooks to Jenkins behind firewall?

Trying to setup webhooks between Jenkins and Bitbucket cloud, there's several proxy services out there but smee is the only one that appears to support selfhosting. Anyone have experience with it specifically or recommendations for another way around this issue?

https://redd.it/ng8wxo
@r_devops

Workflow for CircleCI and Terraform

I have recently begun to use the Terraform API to be able to trigger workflows from our CICD CircleCI. One of the obstacles we are trying to overcome is preventing the entire Terraform workflow from Circle from triggering with every single commit. We are utilizing filters, and the apply will only run on the main branch, but we want to be sure that Circle is not being abused/run with every commit on the backend.

We are currently using filters to prevent the workflow from running a Terraform Apply unless it is a merge into a master/main branch; however, we want our users to be able to test their Terraform. Right now in their development branches, the workflow is triggered with every commit. We want the workflow to be run only when intentionally "specified" (Ideally we would want this to be done from git command line, not logging into the circle CI interface and running the workflow). This way a user can make 10 commits to a branch before they test without triggering 10 workflows behind the scenes.

I thought about using branches with prefixes which trigger the terraform plan workflow in Circle, but that wont prevent a user from making half a dozen commits to that branch and triggering half a dozen workflows. It doesn't look like tagging is best method of triggering the workflow either. Is there a best practice or a similar use case that you all are aware of for this?

https://redd.it/ng6z77
@r_devops