Migrating from Node Redis to Ioredis: a slightly bumpy but faster road
Redis is a very important tool in the Ably toolbox, and since most of our backend code is written in Node.js, using the recommended Node Redis client was natural... but we started hitting some performance issues, and Ioredis came into the picture.
In this new blog post of the Ably Engineering series, we tell the story of how and why we migrated from Node Redis client library to Ioredis to suit our use case better, and a couple of bumps we faced along the way:
Migrating from Node Redis to Ioredis: a slightly bumpy but faster road
Hope you enjoy it, feedback is always welcomed 🤗
https://redd.it/p6ritj
@r_devops
Redis is a very important tool in the Ably toolbox, and since most of our backend code is written in Node.js, using the recommended Node Redis client was natural... but we started hitting some performance issues, and Ioredis came into the picture.
In this new blog post of the Ably Engineering series, we tell the story of how and why we migrated from Node Redis client library to Ioredis to suit our use case better, and a couple of bumps we faced along the way:
Migrating from Node Redis to Ioredis: a slightly bumpy but faster road
Hope you enjoy it, feedback is always welcomed 🤗
https://redd.it/p6ritj
@r_devops
Ably Realtime
Migrating from Node Redis to Ioredis: a slightly bumpy but faster road
Here's how and why we migrated from Node Redis client library to Ioredis to suit our use case better, and the couple of bumps we faced along the way.
meta Why do people keep posting YouTube videos instead of gitHub Repos? I want to read and replicate your infrastructure code, not watch a video.
Mini meta rant: how is it that YouTube videos keep getting posted here? Do other people really prefer YouTube content over a solid README with code examples? You know, like what you do on the actual job. Every other day it's another YouTube video and I'm getting a little tired of it. I'm not going to go through your 10 video playlist to learn concepts when there is no way to replicate it in my own homelab without manually typing in what you're saying.
https://redd.it/p6ti1u
@r_devops
Mini meta rant: how is it that YouTube videos keep getting posted here? Do other people really prefer YouTube content over a solid README with code examples? You know, like what you do on the actual job. Every other day it's another YouTube video and I'm getting a little tired of it. I'm not going to go through your 10 video playlist to learn concepts when there is no way to replicate it in my own homelab without manually typing in what you're saying.
https://redd.it/p6ti1u
@r_devops
reddit
[meta] Why do people keep posting YouTube videos instead of gitHub...
Mini meta rant: how is it that YouTube videos keep getting posted here? Do other people really prefer YouTube content over a solid README with...
AWS Cognito vs Auth0
I have an application that I've built and am in the process of rebuilding and am thinking of redoing the auth to use Auth0 instead of AWS Cognito since I have heard Auth0 is better and this seems like a good time to switch before I have to deal with migrations. I've done a bit of research on these two but am curious to see what this sub thinks of them in regards to management as well as just functionality.
https://redd.it/p6uezf
@r_devops
I have an application that I've built and am in the process of rebuilding and am thinking of redoing the auth to use Auth0 instead of AWS Cognito since I have heard Auth0 is better and this seems like a good time to switch before I have to deal with migrations. I've done a bit of research on these two but am curious to see what this sub thinks of them in regards to management as well as just functionality.
https://redd.it/p6uezf
@r_devops
reddit
AWS Cognito vs Auth0
I have an application that I've built and am in the process of rebuilding and am thinking of redoing the auth to use Auth0 instead of AWS Cognito...
What are best practices integration testing as a platform grows?
When multiple changes are in flight across the various services/teams in a platform, how do you best handle this queue?
One option I've seen executed both poorly and well is to only allow one change at a time until it either hits prod or fails and is rolled back. This requires a strong automated testing suite to allow for a fast feedback loop but even then it doesn't scale well once you approach a large number services.
An alternative would be automating a full stack creation/termination for each change and test against current prod but this could create friction (sure, there's workarounds though) if co-dependent services make it to prod which haven't been tested against each other.
Anyone care to comment a workflow that's successful in their org?
https://redd.it/p6ubm1
@r_devops
When multiple changes are in flight across the various services/teams in a platform, how do you best handle this queue?
One option I've seen executed both poorly and well is to only allow one change at a time until it either hits prod or fails and is rolled back. This requires a strong automated testing suite to allow for a fast feedback loop but even then it doesn't scale well once you approach a large number services.
An alternative would be automating a full stack creation/termination for each change and test against current prod but this could create friction (sure, there's workarounds though) if co-dependent services make it to prod which haven't been tested against each other.
Anyone care to comment a workflow that's successful in their org?
https://redd.it/p6ubm1
@r_devops
reddit
What are best practices integration testing as a platform grows?
When multiple changes are in flight across the various services/teams in a platform, how do you best handle this queue? One option I've seen...
Asking advise for cloud based startup
Me and a couple of my friends doing a startup from an idea we got together, I'm a .NET developer on Windows.
Our project's main logic is done inside Windows 10 Pro VM machines, so to do actual work we need Windows 10 VM machines, for that, I can get dedicated servers from a provider and run a bunch of Windows VMs to do the work, the problem is that I have no idea how to implement rest of the things, I even don't know what we need. So I'm asking you for help.
We need one more VM or database were we going to put already processed data, also we need a web page where users will request work from.
So my question is what do we need to accomplish this?
I just found out that API gateway can be used to connect services, etc.
I would appreciate some architecture guide as well, maybe someone already did similarly and shared it on Internet?
​
Imagine we doing "photo processing" and the main work is done in Windows 10 VM machines, we need users to upload photos and later get results, what do we need to set up such an environment?
All kinds of feedbacks are greatly appreciated.
https://redd.it/p6y6e4
@r_devops
Me and a couple of my friends doing a startup from an idea we got together, I'm a .NET developer on Windows.
Our project's main logic is done inside Windows 10 Pro VM machines, so to do actual work we need Windows 10 VM machines, for that, I can get dedicated servers from a provider and run a bunch of Windows VMs to do the work, the problem is that I have no idea how to implement rest of the things, I even don't know what we need. So I'm asking you for help.
We need one more VM or database were we going to put already processed data, also we need a web page where users will request work from.
So my question is what do we need to accomplish this?
I just found out that API gateway can be used to connect services, etc.
I would appreciate some architecture guide as well, maybe someone already did similarly and shared it on Internet?
​
Imagine we doing "photo processing" and the main work is done in Windows 10 VM machines, we need users to upload photos and later get results, what do we need to set up such an environment?
All kinds of feedbacks are greatly appreciated.
https://redd.it/p6y6e4
@r_devops
reddit
Asking advise for cloud based startup
Me and a couple of my friends doing a startup from an idea we got together, I'm a .NET developer on Windows. Our project's main logic is done...
AWS service catalog terraform
hey guys I am new to terraform. Is it possible to launch AWS service catalog product using terraform?
I came across this old thread https://stackoverflow.com/questions/56185045/how-to-launch-a-product-from-terraform but not sure about it.
https://redd.it/p6y5qy
@r_devops
hey guys I am new to terraform. Is it possible to launch AWS service catalog product using terraform?
I came across this old thread https://stackoverflow.com/questions/56185045/how-to-launch-a-product-from-terraform but not sure about it.
https://redd.it/p6y5qy
@r_devops
Stack Overflow
How to launch a product from terraform
I have a Product in my AWS Service Catalog that I would like to create a terraform script that will launch the product. I am probably missing something obvious but I can't see an obvious way of doing
Is there a way to export a ConfigMap file outside of deployment?
I have an application config that needs to be dynamically generated based upon Values. At the same time I would like to be able to still deploy the application based upon the same configuration that would be dynamically generated. Is there a way other than --dry-run --debug to print the contents of the configuration that would be built during install? Ideally I would like to just get the contents of the config file and not the entire contents of the charts. I am looking to save the contents to a local file for convenient local deployment for devs.
https://redd.it/p6sdom
@r_devops
I have an application config that needs to be dynamically generated based upon Values. At the same time I would like to be able to still deploy the application based upon the same configuration that would be dynamically generated. Is there a way other than --dry-run --debug to print the contents of the configuration that would be built during install? Ideally I would like to just get the contents of the config file and not the entire contents of the charts. I am looking to save the contents to a local file for convenient local deployment for devs.
https://redd.it/p6sdom
@r_devops
reddit
Is there a way to export a ConfigMap file outside of deployment?
I have an application config that needs to be dynamically generated based upon Values. At the same time I would like to be able to still deploy...
Is there any point of learning .net core?
Hey, on my current project I am in a devops role but I have opportunity to help developing backend written in .net core. I am aware that .net core is a huge thing and people master this skill many years so here is my question. Is it worth to go for it? Will it help extend my devops knowledge? My main goal is to become devops expert . Will it help me develop better azure functions or will i find configuring msg brokers knowledge useful? (btw. I have some C# background acquired during my studies)
https://redd.it/p71o3e
@r_devops
Hey, on my current project I am in a devops role but I have opportunity to help developing backend written in .net core. I am aware that .net core is a huge thing and people master this skill many years so here is my question. Is it worth to go for it? Will it help extend my devops knowledge? My main goal is to become devops expert . Will it help me develop better azure functions or will i find configuring msg brokers knowledge useful? (btw. I have some C# background acquired during my studies)
https://redd.it/p71o3e
@r_devops
reddit
Is there any point of learning .net core?
Hey, on my current project I am in a devops role but I have opportunity to help developing backend written in .net core. I am aware that .net core...
Self-hosted directory of self-hosted apps
I started self-hosting web apps last year. As I have been discovering more apps, I made https://www.selfhost.is/ to collect and organize them, and gather useful deployment tutorials, for example Umami is one of the apps I am self-hosting. Hopefully it's helpful to people. Let me know what you think. You are welcome to share an app or a tutorial!
https://redd.it/p716ii
@r_devops
I started self-hosting web apps last year. As I have been discovering more apps, I made https://www.selfhost.is/ to collect and organize them, and gather useful deployment tutorials, for example Umami is one of the apps I am self-hosting. Hopefully it's helpful to people. Let me know what you think. You are welcome to share an app or a tutorial!
https://redd.it/p716ii
@r_devops
Terraform/Gruntwork or CloudFormation?
The client I am working for is deciding between sticking with CloudFormation or switching to Terraform. I have been looking in to Gruntwork and was wondering if anyone had any experience, talking points.
https://redd.it/p73won
@r_devops
The client I am working for is deciding between sticking with CloudFormation or switching to Terraform. I have been looking in to Gruntwork and was wondering if anyone had any experience, talking points.
https://redd.it/p73won
@r_devops
reddit
Terraform/Gruntwork or CloudFormation?
The client I am working for is deciding between sticking with CloudFormation or switching to Terraform. I have been looking in to Gruntwork and...
Azure Service Bus
What is Azure Service Bus?
Message buses and queues are a powerful tool communicating between applications. They are used extensively in microservices to decouple applications and services from each other providing benefits like load-balancing, safe routing and control across service and application. Microsoft Azure Service Bus is a reliable cloud messaging as a service (MaaS) with message queues and publish-subscribe topics.
Concepts and Terminologies
Message: Transfer of data between different applications and services is called a message. A message contains business data such as sales and purchase order or any inventory status in different forms like JSON, XML or plain text.
Decouple applications: Service Bus improves reliability and scalability of applications and services. No consumer or producer are required to be online at the same time.
Load Balancing: Allow multiple consumers to read from a queue at an equivalent time, each safely obtaining exclusive ownership to specific messages.
Queues: Sender sends messages to queue and receiver receives the message from queue Messages stay in the queue until the receiver is available to receive and process them.
Topics: On the other hand, topics are same as queues the only difference is that same queue can be used or accessed by multiple receivers. Topics are useful in publish/subscribe scenarios.
When to use Azure Service Bus?
Azure Service bus is the “Swiss Army Knife” service of all other generic messaging services. It is a solid workflow and transaction handling and has robust facilities for handling many applications fault conditions.
Service Bus functions as a bridge between hybrid cloud solutions and systems that include branch office or work site systems. Systems that sit “behind the firewall”, are roaming across networks, or are occasionally offline can’t be reached directly via “push” messaging, but require messages to be sent to a verified pickup location from where the designated receiver can obtain them.
When the business application lies in the cloud and on-site datacenter, then Service Bus queues and topic subscriptions are the ideal choice. This model is especially fashionable SaaS providers in health care, tax and legal consulting, restaurant services, and retail.
Alternative solutions to Azure Service Bus
Azure Messaging provides a fleet of services that permits developers to select a fully-managed service that most closely fits their requirements The services follow common principles and provide flexibility to the developers. The other alternatives to Service Bus are Event Hubs and Event Grid which are complemented by Azure services like Logic Apps, IoT Hubs and Notification Hubs
Read More : https://www.taliun.com/azure-service-bus
https://redd.it/p6nm14
@r_devops
What is Azure Service Bus?
Message buses and queues are a powerful tool communicating between applications. They are used extensively in microservices to decouple applications and services from each other providing benefits like load-balancing, safe routing and control across service and application. Microsoft Azure Service Bus is a reliable cloud messaging as a service (MaaS) with message queues and publish-subscribe topics.
Concepts and Terminologies
Message: Transfer of data between different applications and services is called a message. A message contains business data such as sales and purchase order or any inventory status in different forms like JSON, XML or plain text.
Decouple applications: Service Bus improves reliability and scalability of applications and services. No consumer or producer are required to be online at the same time.
Load Balancing: Allow multiple consumers to read from a queue at an equivalent time, each safely obtaining exclusive ownership to specific messages.
Queues: Sender sends messages to queue and receiver receives the message from queue Messages stay in the queue until the receiver is available to receive and process them.
Topics: On the other hand, topics are same as queues the only difference is that same queue can be used or accessed by multiple receivers. Topics are useful in publish/subscribe scenarios.
When to use Azure Service Bus?
Azure Service bus is the “Swiss Army Knife” service of all other generic messaging services. It is a solid workflow and transaction handling and has robust facilities for handling many applications fault conditions.
Service Bus functions as a bridge between hybrid cloud solutions and systems that include branch office or work site systems. Systems that sit “behind the firewall”, are roaming across networks, or are occasionally offline can’t be reached directly via “push” messaging, but require messages to be sent to a verified pickup location from where the designated receiver can obtain them.
When the business application lies in the cloud and on-site datacenter, then Service Bus queues and topic subscriptions are the ideal choice. This model is especially fashionable SaaS providers in health care, tax and legal consulting, restaurant services, and retail.
Alternative solutions to Azure Service Bus
Azure Messaging provides a fleet of services that permits developers to select a fully-managed service that most closely fits their requirements The services follow common principles and provide flexibility to the developers. The other alternatives to Service Bus are Event Hubs and Event Grid which are complemented by Azure services like Logic Apps, IoT Hubs and Notification Hubs
Read More : https://www.taliun.com/azure-service-bus
https://redd.it/p6nm14
@r_devops
Taliun
Azure Service Bus
Message buses and queues are a powerful tool communicating between applications. They are used extensively in microservices to decouple applications and services from each other providing benefits like load-balancing, safe routing and control across service…
write technical blog in devops will help with future getting hired?
I started to write technical blog, in scribble style, about what I have done in my job and my free time, do you think it will help with getting hired?
Because I tend to forget a lot of stuff during interviews, my ideas, my thoughts, what I used to do and what to improve in details
link: https://yhc-bluemoon.blogspot.com/
but it really looks like scribble style, it's not meant to teach people how to do certain stuff nor tutorials
by the way, I want to move to Vienna, Austria. I am from Southeast Asia country, a self-taught person, with a degree in non-related field, it looks like it could be difficult to me to apply for visa in that sense, since RWR uses point system and EU blue card requires higher salary, but I gonna try. If anyone can help, please let me know.
https://redd.it/p756n4
@r_devops
I started to write technical blog, in scribble style, about what I have done in my job and my free time, do you think it will help with getting hired?
Because I tend to forget a lot of stuff during interviews, my ideas, my thoughts, what I used to do and what to improve in details
link: https://yhc-bluemoon.blogspot.com/
but it really looks like scribble style, it's not meant to teach people how to do certain stuff nor tutorials
by the way, I want to move to Vienna, Austria. I am from Southeast Asia country, a self-taught person, with a degree in non-related field, it looks like it could be difficult to me to apply for visa in that sense, since RWR uses point system and EU blue card requires higher salary, but I gonna try. If anyone can help, please let me know.
https://redd.it/p756n4
@r_devops
Amber, Encrypted secrets management
Amber is an encrypted secret management solution tailored towards CI system. It allows the encrypted secrets to be stored as part of the git repository itself so that changes can be tracked over time. This is the announcement post with more details: https://www.fpcomplete.com/blog/announcing-amber-ci-secret-tool/
This is the github repository with the code: https://github.com/fpco/amber
https://redd.it/p77lvz
@r_devops
Amber is an encrypted secret management solution tailored towards CI system. It allows the encrypted secrets to be stored as part of the git repository itself so that changes can be tracked over time. This is the announcement post with more details: https://www.fpcomplete.com/blog/announcing-amber-ci-secret-tool/
This is the github repository with the code: https://github.com/fpco/amber
https://redd.it/p77lvz
@r_devops
FP Complete
Announcing Amber, encrypted secrets management
We've released a new tool, Amber, to help better manage secrets in Git repositories for CI purposes. Read more about the motivation and how to get started.
Grafana-Azure AD Integration
Hi there,
We have integrated Grafana with Azure AD to enable SSO login. We have created two groups Grafana-admin and Grafana-users on the Azure AD with different members. I need to assign the level of permission’s (admin, editor, viewer) so that the group “Grafana-users” only have the viewer access and “Grafana-admin” has the admin level access? I need help to understand where should I need to define this configuration?
Thanks, and I really appreciate your help on this.
https://redd.it/p798hc
@r_devops
Hi there,
We have integrated Grafana with Azure AD to enable SSO login. We have created two groups Grafana-admin and Grafana-users on the Azure AD with different members. I need to assign the level of permission’s (admin, editor, viewer) so that the group “Grafana-users” only have the viewer access and “Grafana-admin” has the admin level access? I need help to understand where should I need to define this configuration?
Thanks, and I really appreciate your help on this.
https://redd.it/p798hc
@r_devops
reddit
Grafana-Azure AD Integration
Hi there, We have integrated Grafana with Azure AD to enable SSO login. We have created two groups Grafana-admin and Grafana-users on the Azure...
EDI Analyst
Hello all,
I recently apply to a position with EDI (Electronic Data Interchange) Analyst title and my job tags said it belongs to data warehouse. My question is whether the job is devops related and can I gain some knowledge of devops by doing the role. Thank you.
https://redd.it/p78o4e
@r_devops
Hello all,
I recently apply to a position with EDI (Electronic Data Interchange) Analyst title and my job tags said it belongs to data warehouse. My question is whether the job is devops related and can I gain some knowledge of devops by doing the role. Thank you.
https://redd.it/p78o4e
@r_devops
reddit
EDI Analyst
Hello all, I recently apply to a position with EDI (Electronic Data Interchange) Analyst title and my job tags said it belongs to data warehouse....
DevOps Guest Writer
Lucidity is hiring! We’re on the lookout for DevOps guest writers:
If you’re a DevOps/SRE engineer with experience handling AWS infrastructure at your org, and you also happen to write articles on DevOps-related subjects, then you’re the one for us.
We’re actively building out our guest contributors network and would love to team up with DevOps engineers to write insightful articles on managing AWS storage.
This is a fully compensated role and would be perfect for someone who is motivated to share their acumen with the DevOps community.
If you or someone you know fits this role, do drop a comment or you could also email me with some of your previous articles at [email protected].
https://redd.it/p7bjdr
@r_devops
Lucidity is hiring! We’re on the lookout for DevOps guest writers:
If you’re a DevOps/SRE engineer with experience handling AWS infrastructure at your org, and you also happen to write articles on DevOps-related subjects, then you’re the one for us.
We’re actively building out our guest contributors network and would love to team up with DevOps engineers to write insightful articles on managing AWS storage.
This is a fully compensated role and would be perfect for someone who is motivated to share their acumen with the DevOps community.
If you or someone you know fits this role, do drop a comment or you could also email me with some of your previous articles at [email protected].
https://redd.it/p7bjdr
@r_devops
Automating Terraform updates using Dependabot
Hi there, Dependabot received private registry and Terraform 1.0 support last month so I've decided to see it in action. If you're using GitHub as your VCS Dependabot might be something to look at :)
https://sysdogs.com/articles/automating-terraform-updates-with-dependabot
https://redd.it/p7d7l7
@r_devops
Hi there, Dependabot received private registry and Terraform 1.0 support last month so I've decided to see it in action. If you're using GitHub as your VCS Dependabot might be something to look at :)
https://sysdogs.com/articles/automating-terraform-updates-with-dependabot
https://redd.it/p7d7l7
@r_devops
Sysdogs
Automating Terraform updates with Dependabot
2021-08-19 | Kamil Szczygieł | Code
GitLab or GitHub/Azure DevOps
What should I make sure we ask about and look into? I have GitLab experience, I was surprised that the GitHub/Azure DevOps sales pitch actually sounded so good. Enterprise Architects are now doing a test run of each before making a decision. What would I regret not having brought up?
Context: large traditional enterprise that struggles a lot. I’m the new DevOps lead on a struggling/failing project looking to turn stuff around. It appears both would work for what I need, IoC (including pipelines!), GitOps, and automatic (not auto-manual!) deploys and rollbacks. If there’s anything I should look into now is the time to raise objections and point stuff out.
I know the “traditional” adjective sounds bad, and it is tbh. If I succeed in this role I’ll have to excel as a diplomat in addition to technical skill.
https://redd.it/p7e99u
@r_devops
What should I make sure we ask about and look into? I have GitLab experience, I was surprised that the GitHub/Azure DevOps sales pitch actually sounded so good. Enterprise Architects are now doing a test run of each before making a decision. What would I regret not having brought up?
Context: large traditional enterprise that struggles a lot. I’m the new DevOps lead on a struggling/failing project looking to turn stuff around. It appears both would work for what I need, IoC (including pipelines!), GitOps, and automatic (not auto-manual!) deploys and rollbacks. If there’s anything I should look into now is the time to raise objections and point stuff out.
I know the “traditional” adjective sounds bad, and it is tbh. If I succeed in this role I’ll have to excel as a diplomat in addition to technical skill.
https://redd.it/p7e99u
@r_devops
reddit
GitLab or GitHub/Azure DevOps
What should I make sure we ask about and look into? I have GitLab experience, I was surprised that the GitHub/Azure DevOps sales pitch actually...
Harbormaster: The anti-Kubernetes for your personal server
I thought the /r/devops subreddit might be interested in this project I just found!
https://gitlab.com/stavros/harbormaster
https://redd.it/p7frxs
@r_devops
I thought the /r/devops subreddit might be interested in this project I just found!
https://gitlab.com/stavros/harbormaster
https://redd.it/p7frxs
@r_devops
GitLab
Stavros Korokithakis / Harbormaster · GitLab
Migrated to https://github.com/skorokithakis/harbormaster
Downloading from Artifactory
Hello, I am trying to download a package from our artifactory deployment using powershell.
Using Invoke-WebRequest I authenticate correctly however it doesn't download the file, only the loading page. Not sure if I'm missing a setting somewhere or not.
Invoke-WebRequest -Uri "https://artifactory:8082/artifactory/test/testfile.exe" -Headers $header -Method "Get" -ContentType "application/executable" -UseBasicParsing -OutFile "C:\\temp\\testfile.exe"
Any help or pointers in the right would be greatly appreciated.
https://redd.it/p7eyip
@r_devops
Hello, I am trying to download a package from our artifactory deployment using powershell.
Using Invoke-WebRequest I authenticate correctly however it doesn't download the file, only the loading page. Not sure if I'm missing a setting somewhere or not.
Invoke-WebRequest -Uri "https://artifactory:8082/artifactory/test/testfile.exe" -Headers $header -Method "Get" -ContentType "application/executable" -UseBasicParsing -OutFile "C:\\temp\\testfile.exe"
Any help or pointers in the right would be greatly appreciated.
https://redd.it/p7eyip
@r_devops
reddit
Downloading from Artifactory
Hello, I am trying to download a package from our artifactory deployment using powershell. Using Invoke-WebRequest I authenticate correctly...
Devops Engineer interviewing process
Just got off a call with a company that said their final interview section is a 4 hour technical and personal interview. Is that common? Took my by surprise. I'll also add this isn't for a junior position.
https://redd.it/p7hs7p
@r_devops
Just got off a call with a company that said their final interview section is a 4 hour technical and personal interview. Is that common? Took my by surprise. I'll also add this isn't for a junior position.
https://redd.it/p7hs7p
@r_devops
reddit
Devops Engineer interviewing process
Just got off a call with a company that said their final interview section is a 4 hour technical and personal interview. Is that common? Took my...