Recomendations for a backend deployment

Hi.

im developing a backend with python and Django, the reason for choose this options, well simple i just have more expereince using Django and DRF(Django REST Framework). Im not a seasoned developer by any means, my only experiences deploying a backedn is in the free tier that Render . com offers.

so here is a little context about the project.

I need to call an external API to get some data, but i need do it using a long polling aproach,so i'll need to set up a task that will repeat around maybe every 5 seconds that makes a request to the API, then i will filter some stuff form the body of the response and store this data(probably in a cache, because of the fast access) so that way when the client makes a resuqest to my DRF endpoint i will return this data(the one in the cache) that i have saved from the previus request made to the external API. As right now i only need one task for a single espesific endpoint of the external API, but this could increase to around 4 or 5 task that will do something similar but for diferent enpoints of the same external API.

so the solution that i have come to for this is the following one:

the retpeted task that i mentioned before will be execute using celery and celery beat.

I will have 3 machines: one for the Django server, one for redis and one that will run the celery workers. this because in this way the repeted tasks of making the requests to the external API wont take any CPU or memory from the server that will by runing Django. And i will also have a DB, postgres in this case.

So the thing will work like this, the celery worker will make the requests to the external API, then after filtering the response the resulting data will be store in the redis cache; Then when i client makes a request to my django server(the endpoint i mentioned before) the sever will have to validate an API key that the client has, then the django server will just look for the the value that the celery worker has previusly store in the cache and then take this value and return it to the client.

Right now im thinking about of deploying this in must of the stuff in aws, like this:

AWS ElastiCache for the redis machine
An EC2 instance for the machine that will run the celery workers
I was thinking on using AWS app runnner instead of and EC2 for the django server, this because from i have read is similar to what deploying on render . com is
for the DB i was thinking initially to use Render . com, since is a bit cheaper at the beginig and also includes the backups and then maybe if the DB grows mirgrating into AWS RDS

So what do you guys think of this, is this a good way to sstructure my backend for this problem?
shouuld i structure the cloud set up in a different way?
should i use a different cloud provider?
Any tips or tricks that you now that my be helpfull ?

thanks a lot for your help guys.

https://redd.it/1caxdku
@r_devops

Is CodiumAI or GitHub copilot worth it?

The vscode extension requires a subscription after 30 days. To those who have a subscription, does copilot worth the 10$/month subscription?

Every service needs a 10$/month subscription, and it quickly adds up.

https://redd.it/1cayerp
@r_devops

What are the biggest challenges you /team faced while adopting a CI/CD tool ?

We are in discussion of finalizing circle ci but wanted to understand how difficult it is to adopt a new platform/tool and what are the initial or late challenges ?

https://redd.it/1cb26j5
@r_devops

Suggestion for topics for tech talk session. Something fundamental but missed by most people

I was suppose to give a tech talk on some devops topic. I am looking a fundamental topic/process but somehow not being followed or missed by many people. I want it to be useful session please help

https://redd.it/1cb28y8
@r_devops

How to enable HTTPS on a python app.

Hey guys,

i have a python app running using python `app.py` in a azure VM .
the app is accessible at https://<public-ip>:3000

I want to run it on https://<public-ip>:3000 or https://<azure-dns>:3000
can someone help and suggest how can I achieve it.

https://redd.it/1cb4bp1
@r_devops

An Opinionated Guide to Managing Observability Pipelines

Observability data, when left on its own, behaves like entropy - it increases without bounds (until you run out of money). Observability pipelines help organizations take control of their data and limit volume before sending it upstream to vendors.

https://bit.kevinslin.com/p/an-opinionated-guide-to-managing

https://redd.it/1cb60ez
@r_devops

Imposter syndrome kickin my ass

I was made redundant this year and finally got an offer. Its near 2x my previous salary which is making me feel like a FRAUD. I have never used K8s before at work but part of their interview process was troubleshooting k8 clusters and fixing it. I guess they liked how I approached it considering it was the SRE team watching me do this the whole time but idk.

Its probably been asked so many times but how the hell can i get this feeling out. I genuinely don’t feel like im THAT good but how can i basically not embarrass myself. I have like 2 and 1/2 years DevOps experience so not that much


https://redd.it/1cb6aob
@r_devops

How do you handle feature flags in production ?

Feature flags are very interesting when you want to enable/disable a feature for everyone or a specific set of users in deployed applications.

How do you handle them in your applications/infra ?

* Environment variables ?
* SaaS (PostHog, Flagsmith, etc.) ?
* Self hosted open source solution (Flagsmith, etc.) ?
* Harcoded ?
* App database (Like a settings/parameters table) ?

Also, when was the last time you had to disable a feature flag in emergency because a feature deployed to production had flaws ?

https://redd.it/1cb3g46
@r_devops

Coroot 1.0 – simplified open source observability for modern environments

Exciting news – Coroot 1.0 has arrived! As DevOps practitioners, reliable monitoring tools are essential.

Coroot 1.0 offers a simple, comprehensive view of environments, supporting Kubernetes, VMs, and cloud services. It's easily deployable with technologies like eBPF and Netlink, gathering crucial information without complex setups. Additionally, it provides actionable insights without overwhelming users.

We're fostering a community and value feedback from users of all experience levels. Join us in shaping Coroot's development.


Check out our features overview and our repo on GitHub.

https://redd.it/1cbb8f7
@r_devops

What is the interesting AI GPT project for DevOps/Platform Engineering team?

Do you have any project working for AI just for internal purposes within your DevOps/Platform team? Please share with us.

https://redd.it/1cbceut
@r_devops

The more HashiCorp shares pictures in front of NASDAQ the worse their products will become

just saying.

after some posts of their CTO and their team in front of the stock exchange this only goes one way. down the shitter.


I work for a Fortune 500 corp and the law of average products ramps up really high once you go public

https://redd.it/1cbelva
@r_devops

IBM nearing deal for cloud software provider HashiCorp, source says

April 23 (Reuters) - International Business Machines (IBM.N), opens new tab is nearing a deal to buy cloud software provider HashiCorp (HCP.O), opens new tab, according to a person familiar with the matter.
Source https://www.reuters.com/markets/deals/ibm-nearing-buyout-deal-hashicorp-wsj-reports-2024-04-23/

https://redd.it/1cbebi3
@r_devops

Elastic search DR

Just trying to set up DR for elastic cluster. 300gb+ data. DR site in another continent.

Does it make sense to do uni or bi directional CCR? So we can fail back and forth easily.

Anything you guys recommend?

The site to site vpn has good bandwidth.

https://redd.it/1cbdl55
@r_devops

How is the job market these days?

Sorry if these kind of posts aren't allowed, but I am genuinely curious about the job market for devops/sre. I used to work as a front-end web developer, and that field is very saturated imo among juniors and mid-level devs. Can the same be said for devops/sre? You will commonly see 1000+ applicants for SWE roles within a few days (granted most of them are probably unqualified but still daunting).

With software developers, obviously there was a lot of over-hiring and hoarding during the pandemic which can be said the same for other fields. But I think software developers have it especially tougher because offshoring is making a resurgence and AI is at least giving managers the impression that you don't need as many developers. I don't know if devops/sre is experiencing those kind of blowbacks.

Anyways I personally did not enjoy my career as a web developer. It got very stale solving similar CRUD problems and I don't have the energy to grind out LEETCODE problems. I want to go get a CS degree and transiton to devops but if it's very difficult to get into it, then I must re-assess. I guess I am looking for advice.

https://redd.it/1cbifqz
@r_devops

Segmenting VPC/vNet for a SaaS

Hey,

Title says it all, how do you guys segment your virtual networks for a SaaS? This includes static web apps for the frontend, databases, Redis caches etc.

How complicated is this, have any of you guys done it for a SaaS?

https://redd.it/1cblgcu
@r_devops

Backing up RDS instance to S3 bucket via Lambda

I have an EventBridge event triggering my Lambda function to backup my RDS (MySQL) database to an S3 function with an SNS topic emailing me the results as the destination.

I connected my RDS database to my Lambda function directly under the configuration tab and assumed this meant I wouldn’t have to account for the connection strings in my lambda function code body?

I Frankensteined some Python code from the interwebs but keep getting timeouts 😩

Can anyone point me in the right direction here?

https://redd.it/1cbnvd9
@r_devops

We switched our enterprise product to cater to young startups. Here's why (and we'd love to have you as beta testers)

[Disclaimer: I'm the founder of Facets.cloud, a DevOps solution built from my experiences as a CTO.\]

Hey everyone,

Rohit here. I want to share my experience from a few years ago and how that led us to pivot our strategy.

Here goes something...

I used to work at a late-stage startup where our cloud infrastructure had become a complex beast.

We faced many challenges with our infra, from launching in different regions to managing deployments across environments.

It was a constant struggle, and our tech debt kept growing.

To address these issues, we built an in-house "architecture-first" DevOps solution.

The idea was simple: make architectural documentation the single source of truth.

Any changes, whether in software or infrastructure, would be made at the architecture level and then cascade down to the environments.

But we didn't stop there.

We took it a step further by including alerts, observability, monitoring, CI/CD pipelines, and database schemas in the architectural model. This allowed us to manage critical operational concerns uniformly across the board.

The experiment was a success, so we turned it into a product called Facets.cloud.

We raised funding and built a comprehensive feature set for the DevOps space.

However, after a while, we identified two key problems that we'd overlooked:

1. Facets had become a complex enterprise product where we missed out on early user feedback.
2. Developers, especially those at early-stage startups, needed a more self-service and simple solution.

That's why we're releasing Facets 2.0 - focused on quick, clean cloud deployments optimized for early-stage startup developers.

We're still committed to our "architecture-first" approach, but we're simplifying the platform to make it accessible to any developer or DevOps engineer.

I don't have a trial version ready just yet, but I'd love to get your early feedback on the idea.

We've opened our Beta program, and I'm eager for you all to join us as beta testers: **https://www.facets.cloud/facets-for-startups-2**

Thanks for reading, and I look forward to your thoughts and suggestions!

https://redd.it/1cbs5sa
@r_devops

What do you think about support roles?

So far Ive had only devops / sre / cloud engineer roles but in my current unemployed job hunt I am often asked if I would be ok for a "support" role, like Cloud support engineer, DevOps support engineer and when I look into the description of these roles doesn't sound so different to what I used to do in the previous roles just that instead of the focus being the internal team now the one serving tickets is the client?


What I wanted to know is , taking one of those roles is considered a downgrade? a side change on career? or is just one more step in the line of these career.

https://redd.it/1cbsaub
@r_devops

We pivoted our startup from enterprise-only to SaaS. Here's why...

Hey everyone,

Rohit here, one of the founders of Facets.cloud.

I wanted to share my experience from a few years ago and how that experience led us to pivot Facets to a SaaS product from an enterprise-only product.

Here goes something...

I used to work at a late-stage startup where our cloud infrastructure had become a complex beast.

We faced many challenges with our infra, from launching in different regions to managing deployments across environments.

It was a constant struggle, and our tech debt kept growing.

To address these issues, we built an in-house "architecture-first" DevOps solution.

The idea was simple: make architectural documentation the single source of truth.

Any changes, whether in software or infrastructure, would be made at the architecture level and then cascade down to the environments.

But we didn't stop there.

We took it a step further by including alerts, observability, monitoring, CI/CD pipelines, and database schemas in the architectural model. This allowed us to manage critical operational concerns uniformly across the board.

The experiment was a success, so we turned it into a product called Facets.cloud.

We raised funding and built a comprehensive feature set for the DevOps space.

However, after a while, we identified two key problems that we'd overlooked:

Facets had become a complex enterprise product where we missed out on early user feedback.
Developers, especially those at early-stage startups, needed a more self-service and simple solution.

That's why we're releasing Facets 2.0 - focused on quick, clean cloud deployments optimized for early-stage startup developers.

We're still committed to our "architecture-first" approach, but we're simplifying the platform to make it accessible to any developer or DevOps engineer.

I don't have a trial version ready just yet, but I'd love to get your early feedback on the idea.

We've opened our Beta program, and I'm eager for you all to join us as beta testers:
https://www.facets.cloud/quick-cloud-deployments

Thanks for reading, and I look forward to your thoughts and suggestions!

>[Disclaimer: I'm the founder of Facets.cloud, a DevOps solution built from my work experience.\]

PS. I had to delete and repost since it didn't let me edit text. What am I doing wrong here.

https://redd.it/1cbtdoj
@r_devops

Do I have the chance as a junior devops engineer?

Do you think with this knowledge I can start at a Junior devops role?

So I have knowledge in Python, Git, docker, bash, SQL, Linux and a little powershell scripting.

what other skills do i need for a junior devops role?

https://redd.it/1cbub8j
@r_devops

Does anyone else suffer from "get across the line" syndrome?

I'm not sure what this behaviour is called, but I'm refering to the push by management to deliver tasks in such a way that they just about deliver what's needed in that instance, and then move onto the next deliverable ASAP.

No time for documentation, no time to plan out a body of work into the next few weeks, no time to consider futureproofing, keeping things DRY, reviewing old configuration to confirm it's valid, and so on.

I often hear that "if it's not a (paid for) feature, it's not getting done" is a mindset by product managers and the like, so "unsexy" things like security posture planning, technical debt and even documentation never gets addressed. How common would you find this style of work; this sort of relentless firefighting moving from task focusing on whats 3" ahead rather than 3ft or 3 miles?

https://redd.it/1cbu7sb
@r_devops