AWS question: is the reserved subscription retroactive?
Whenever you buy a reserved instance type with specified tenancy and AZ, will this be valid for the already existing instances? E.g. I buy an r5.large, eu-west-1a, default tenancy reserved instance subscription, will the price of the already existing r5.large instances be dropped as well? Or only of the new instances, started after the purchase?
https://redd.it/dmeqcd
@r_devops
Whenever you buy a reserved instance type with specified tenancy and AZ, will this be valid for the already existing instances? E.g. I buy an r5.large, eu-west-1a, default tenancy reserved instance subscription, will the price of the already existing r5.large instances be dropped as well? Or only of the new instances, started after the purchase?
https://redd.it/dmeqcd
@r_devops
reddit
AWS question: is the reserved subscription retroactive?
Whenever you buy a reserved instance type with specified tenancy and AZ, will this be valid for the already existing instances? E.g. I buy an...
The anatomy of DevOps process flow – close the loop to succeed
Business is a holistic system. Its core components are tightly coupled, and their interactions help us create the desired outcome and meet a specific market need. This value creation chain is a process that has to be well-defined, properly executed, and neatly measured to achieve repeatable value delivery. When your business has well-managed processes in place, you can start implementing continual improvements in service quality, delivery time, or production costs. It is clear as day that mature processes are mandatory for steady business growth. It is what ensures clarity
DevOps business processes are implemented in a way to deliver new features, bug fixes, and system enhancements to production as quick as possible. Similar to Lean and Agile practices, the DevOps process flow seeks to eliminate wasteful practices and increase value-producing activities. The promise of building quality software quicker is so appealing that it is being adopted rapidly both in startups and in old-school corporations. According to [the state of DevOps report](https://puppet.com/resources/whitepaper/state-of-devops-report/thank-you) by Puppet, companies that have adopted DevOps methodology benefit from 46x more frequent code deployment, 440x shorter commit-release cycle and 96x faster time to recovery. Such performance is achieved by employing a set of DevOps best practices that reduce re-work and remove overhead from the software development pipeline.
While a healthy [DevOps culture](https://blog.cherryservers.com/how-not-to-implement-devops-culture) lays out the foundation for DevOps adoption, well-defined business processes ensure its success. As I‘ve put it earlier, [DevOps is an intersection of culture, processes and tools](https://blog.cherryservers.com/devops-full-picture-an-intersection-of-culture-processes-and-tools). So let‘s now examine what the stages of the DevOps process flow are and how do they help businesses develop breathtaking software.
**Continuous planning - the smaller, the better**
Forget long-term business strategies that take months to be created and are often severely aimed. Continuous business planning employs the best practices of Lean and Agile methodologies to make your DevOps process flow as smooth as ever. The main idea is to plan software development in short iterations to reduce waste and create a product that your customers would crave.
As Eric Ries stated in his book [The Lean Startup](https://www.amazon.com/Lean-Startup-Entrepreneurs-Continuous-Innovation/dp/0307887898), every new business venture must start with having its core value statement defined – a promise that should drive your customers crazy, make them fall in love with you and ensure everyone a happy ending. Of course, at first, you don‘t know for sure what your target market wants, so you roll up your sleeves, build a minimum viable product and preach it to the early adopters. The goal is to test your idea in production as quick as possible and get raw user feedback, which helps you make data-driven decisions about further development. You move in short iterations releasing new features, bug fixes, and system enhancements. Each release is closely monitored, the data gets analyzed, and your plan is modified accordingly. This way, you are less likely to go astray and can reach a [product-market fit](https://pmarchive.com/guide_to_startups_part4.html) more quickly.
**Continuous integration - set the stage**
Back in the old days, code integration was a long and tedious process. The longer your team worked on a code build, the more painful it had become. Imagine the day when after months of coding, a dozen developers are trying to integrate their code into a single piece of software. Different code branches collide, bug fixing becomes obscure, and your project timeline extends obscenely. By now, you have probably missed your project deadline, and your team is demoralized. That’s integration hell.
Continuous integration helps you get to the integration heaven, making software
Business is a holistic system. Its core components are tightly coupled, and their interactions help us create the desired outcome and meet a specific market need. This value creation chain is a process that has to be well-defined, properly executed, and neatly measured to achieve repeatable value delivery. When your business has well-managed processes in place, you can start implementing continual improvements in service quality, delivery time, or production costs. It is clear as day that mature processes are mandatory for steady business growth. It is what ensures clarity
DevOps business processes are implemented in a way to deliver new features, bug fixes, and system enhancements to production as quick as possible. Similar to Lean and Agile practices, the DevOps process flow seeks to eliminate wasteful practices and increase value-producing activities. The promise of building quality software quicker is so appealing that it is being adopted rapidly both in startups and in old-school corporations. According to [the state of DevOps report](https://puppet.com/resources/whitepaper/state-of-devops-report/thank-you) by Puppet, companies that have adopted DevOps methodology benefit from 46x more frequent code deployment, 440x shorter commit-release cycle and 96x faster time to recovery. Such performance is achieved by employing a set of DevOps best practices that reduce re-work and remove overhead from the software development pipeline.
While a healthy [DevOps culture](https://blog.cherryservers.com/how-not-to-implement-devops-culture) lays out the foundation for DevOps adoption, well-defined business processes ensure its success. As I‘ve put it earlier, [DevOps is an intersection of culture, processes and tools](https://blog.cherryservers.com/devops-full-picture-an-intersection-of-culture-processes-and-tools). So let‘s now examine what the stages of the DevOps process flow are and how do they help businesses develop breathtaking software.
**Continuous planning - the smaller, the better**
Forget long-term business strategies that take months to be created and are often severely aimed. Continuous business planning employs the best practices of Lean and Agile methodologies to make your DevOps process flow as smooth as ever. The main idea is to plan software development in short iterations to reduce waste and create a product that your customers would crave.
As Eric Ries stated in his book [The Lean Startup](https://www.amazon.com/Lean-Startup-Entrepreneurs-Continuous-Innovation/dp/0307887898), every new business venture must start with having its core value statement defined – a promise that should drive your customers crazy, make them fall in love with you and ensure everyone a happy ending. Of course, at first, you don‘t know for sure what your target market wants, so you roll up your sleeves, build a minimum viable product and preach it to the early adopters. The goal is to test your idea in production as quick as possible and get raw user feedback, which helps you make data-driven decisions about further development. You move in short iterations releasing new features, bug fixes, and system enhancements. Each release is closely monitored, the data gets analyzed, and your plan is modified accordingly. This way, you are less likely to go astray and can reach a [product-market fit](https://pmarchive.com/guide_to_startups_part4.html) more quickly.
**Continuous integration - set the stage**
Back in the old days, code integration was a long and tedious process. The longer your team worked on a code build, the more painful it had become. Imagine the day when after months of coding, a dozen developers are trying to integrate their code into a single piece of software. Different code branches collide, bug fixing becomes obscure, and your project timeline extends obscenely. By now, you have probably missed your project deadline, and your team is demoralized. That’s integration hell.
Continuous integration helps you get to the integration heaven, making software
Puppet
2019 State of DevOps Report
Check out the data on security integration (from nearly 3,000 survey respondents) and learn how to make security work in your DevOps practice.
integration a trivial task – you no longer think much of it. How does it work? The mainline of your code lies in the version control system. Before you start working, you make a local code copy from the repository. You then make changes to the production code and automated tests – continuous integration assumes that the lion‘s share of your code is covered by tests. After you finish your work, you create a local code build that gets tested automatically. If the local tests pass, you are allowed to commit your code to the mainline code flow. Then a new code build is created on the continuous integration server where automated tests are rerun to detect potential artifacts that may be caused by the developer‘s local environment. If these tests pass, then and only then your work is done. If things go wrong, fixing the broken build becomes the highest priority task, which, if you have implemented DevOps processes correctly, shouldn‘t take more than 10 minutes. This way, your team is confident that everybody is working with the latest code build.
Continuous integration augments continuous business planning and extends your DevOps process flow. You start with planning small, code in tiny batches – typically no bigger than 20 lines of code – and build on every code commit. This way you know if your code compiles and the initial tests pass. And when they don‘t, you spend much less time debugging the problem, since you have made just so many changes to the system. The team never gets too far from the stable code base and becomes capable of employing the next step of the DevOps process flow.
**Continuous delivery - spread chunks of value daily**
Ever since the [Agile manifesto](https://agilemanifesto.org/), there was a notion of developing software in small batches, but the deployment pipeline still lacked efficiency. Continuous delivery is the next stage of the DevOps process flow, whose primary purpose is to optimize the throughput of the [deployment pipeline](https://martinfowler.com/bliki/DeploymentPipeline.html). While the deployment pipeline starts with continuous integration, there is more to it.
The main idea of continuous delivery is to get fast, automated feedback on the production readiness of your software, every time you make a code commit. Continuous delivery ensures that new features, configuration changes, bug fixes, and experiments flow through your deployment pipeline safely and quickly in a sustainable way. Comprehensive tests take time to complete, yet you still want to make low-risk releases. So how do you balance this?
Continuous delivery goes further than merely compiling and unit testing your code. Typically, you would put more protracted and more expensive tests further away in your deployment pipeline. As a rule of thumb, these tests are also less likely to fail. The final testing suite would depend on the complexity and maturity of the system, which may require additional tests. Later stages of the deployment pipeline may include integration, load, UI, and penetration tests to prevent any performance, usability, and security issues. If the system requires thorough testing, additional tests can be executed on separate machines in parallel. This way, you get continuous feedback about the quality of your code as fast as possible, without lowering the pace of software delivery.
Depending on the maturity of your DevOps processes, continuous delivery may extend into continuous deployment. The only difference here is that with the former you keep your mainline code ready to be released at any time, while with the latter you deploy to production automatically with the condition that all the tests have passed. This way, your DevOps process flow becomes even more agile. Continuous deployment may not be suitable for financial or mission-critical applications that require extensive testing and manual intervention. If that’s not the case for you, start your DevOps journey with continuous delivery and move to continuous deployment when your DevOps processes have matured.
**Continuous operations - scale out to th
Continuous integration augments continuous business planning and extends your DevOps process flow. You start with planning small, code in tiny batches – typically no bigger than 20 lines of code – and build on every code commit. This way you know if your code compiles and the initial tests pass. And when they don‘t, you spend much less time debugging the problem, since you have made just so many changes to the system. The team never gets too far from the stable code base and becomes capable of employing the next step of the DevOps process flow.
**Continuous delivery - spread chunks of value daily**
Ever since the [Agile manifesto](https://agilemanifesto.org/), there was a notion of developing software in small batches, but the deployment pipeline still lacked efficiency. Continuous delivery is the next stage of the DevOps process flow, whose primary purpose is to optimize the throughput of the [deployment pipeline](https://martinfowler.com/bliki/DeploymentPipeline.html). While the deployment pipeline starts with continuous integration, there is more to it.
The main idea of continuous delivery is to get fast, automated feedback on the production readiness of your software, every time you make a code commit. Continuous delivery ensures that new features, configuration changes, bug fixes, and experiments flow through your deployment pipeline safely and quickly in a sustainable way. Comprehensive tests take time to complete, yet you still want to make low-risk releases. So how do you balance this?
Continuous delivery goes further than merely compiling and unit testing your code. Typically, you would put more protracted and more expensive tests further away in your deployment pipeline. As a rule of thumb, these tests are also less likely to fail. The final testing suite would depend on the complexity and maturity of the system, which may require additional tests. Later stages of the deployment pipeline may include integration, load, UI, and penetration tests to prevent any performance, usability, and security issues. If the system requires thorough testing, additional tests can be executed on separate machines in parallel. This way, you get continuous feedback about the quality of your code as fast as possible, without lowering the pace of software delivery.
Depending on the maturity of your DevOps processes, continuous delivery may extend into continuous deployment. The only difference here is that with the former you keep your mainline code ready to be released at any time, while with the latter you deploy to production automatically with the condition that all the tests have passed. This way, your DevOps process flow becomes even more agile. Continuous deployment may not be suitable for financial or mission-critical applications that require extensive testing and manual intervention. If that’s not the case for you, start your DevOps journey with continuous delivery and move to continuous deployment when your DevOps processes have matured.
**Continuous operations - scale out to th
agilemanifesto.org
Manifesto for Agile Software Development
We are uncovering better ways of developing software
by doing it and helping others do it. These are our
values and principles.
by doing it and helping others do it. These are our
values and principles.
e world**
Now we step into the world of IT operations. Here, downtime is not an option. The performance of your system cannot suffer from ongoing releases, updates, and patches which are so frequent when you adopt the DevOps process flow. Continuous operations work hand in hand with continuous monitoring to bring peace of mind for agile businesses that are building software at scale.
Continuous in operations, just like everything else DevOps, start with a centralized version control system. Everything lives in it, including your code, your database schema, your server configuration files, and anything beyond that. The main idea here is to have [reproducible builds](https://martinfowler.com/bliki/ReproducibleBuild.html) of your system: if you took a virgin machine, you would be able to recreate your system out of the box. Having all your infrastructure changes logged also helps with compliance and auditing purposes. Using version control for IT operations is also the first step to abstracting away from infrastructure and treating your hardware as code.
[Infrastructure as code](https://www.hashicorp.com/resources/what-is-infrastructure-as-code) is an IT resource management approach that defines compute, storage, and network infrastructure through source code, more precisely – configuration definition files. In conjunction with cloud computing, infrastructure as code augments the DevOps process flow by order of magnitude. Infrastructure costs shrink due to a pay-per-use pricing model, deployment speed increases because of the on-demand provisioning, and infrastructure misconfiguration risks diminish thanks to automated configuration management. Even more, infrastructure as code changes the way you think about IT operations. Instead of architecting to last, you build to fail.
DevOps fits neatly with modern cloud-native applications, making [microservices architecture](https://martinfowler.com/microservices/) the primary way to run your software at scale. As soon as you outgrow the minimal viable product phase, you shatter that monolith application into loosely coupled services and run them in isolation. By doing so, you create a highly available system that is prepared to fail. If one of your services breaks down, you kill it and spin up a new one. If a load of your system increases, you merely deploy new nodes for the specific service. And if you are deploying to production daily – which you should – microservices architecture and the right DevOps tools can facilitate your deployment strategy. It becomes quicker and easier to introduce system changes gradually through canary releases or blue-green deployments.
**Continuous monitoring - "with a thousand eyes, and one"**
Let’s assume you have already implemented DevOps processes that we discussed so far. Software flows swiftly through your deployment pipeline with code integration, testing, and deployment fully automated. You are swiftly releasing new software and deploying it to production, but now you need to make sure your new releases do not cause any performance degradation. Most bugs should have been caught by now, but some artifacts are hard or impractical to test in a staging environment. Thus, testing shifts to production and extends through continuous monitoring. So what exactly is being monitored? The data collected via continuous monitoring can be divided into primary and secondary metrics.
**Primary metrics** are set to evaluate application performance that is experienced by the end-users. First, you may want to monitor *end-user experience* directly – through network port mirroring (passively) or synthetic probes (actively) – to capture latency issues and system inconsistencies as the user interacts with your application. Next, you want to monitor *business transactions* across infrastructure tiers to make sure you are meeting your SLA. Finally, you need to have *system reports* that would consist of a standard set of metrics for each application. These reports allow you to evaluate the performance of the whole system, despite cross-application difference
Now we step into the world of IT operations. Here, downtime is not an option. The performance of your system cannot suffer from ongoing releases, updates, and patches which are so frequent when you adopt the DevOps process flow. Continuous operations work hand in hand with continuous monitoring to bring peace of mind for agile businesses that are building software at scale.
Continuous in operations, just like everything else DevOps, start with a centralized version control system. Everything lives in it, including your code, your database schema, your server configuration files, and anything beyond that. The main idea here is to have [reproducible builds](https://martinfowler.com/bliki/ReproducibleBuild.html) of your system: if you took a virgin machine, you would be able to recreate your system out of the box. Having all your infrastructure changes logged also helps with compliance and auditing purposes. Using version control for IT operations is also the first step to abstracting away from infrastructure and treating your hardware as code.
[Infrastructure as code](https://www.hashicorp.com/resources/what-is-infrastructure-as-code) is an IT resource management approach that defines compute, storage, and network infrastructure through source code, more precisely – configuration definition files. In conjunction with cloud computing, infrastructure as code augments the DevOps process flow by order of magnitude. Infrastructure costs shrink due to a pay-per-use pricing model, deployment speed increases because of the on-demand provisioning, and infrastructure misconfiguration risks diminish thanks to automated configuration management. Even more, infrastructure as code changes the way you think about IT operations. Instead of architecting to last, you build to fail.
DevOps fits neatly with modern cloud-native applications, making [microservices architecture](https://martinfowler.com/microservices/) the primary way to run your software at scale. As soon as you outgrow the minimal viable product phase, you shatter that monolith application into loosely coupled services and run them in isolation. By doing so, you create a highly available system that is prepared to fail. If one of your services breaks down, you kill it and spin up a new one. If a load of your system increases, you merely deploy new nodes for the specific service. And if you are deploying to production daily – which you should – microservices architecture and the right DevOps tools can facilitate your deployment strategy. It becomes quicker and easier to introduce system changes gradually through canary releases or blue-green deployments.
**Continuous monitoring - "with a thousand eyes, and one"**
Let’s assume you have already implemented DevOps processes that we discussed so far. Software flows swiftly through your deployment pipeline with code integration, testing, and deployment fully automated. You are swiftly releasing new software and deploying it to production, but now you need to make sure your new releases do not cause any performance degradation. Most bugs should have been caught by now, but some artifacts are hard or impractical to test in a staging environment. Thus, testing shifts to production and extends through continuous monitoring. So what exactly is being monitored? The data collected via continuous monitoring can be divided into primary and secondary metrics.
**Primary metrics** are set to evaluate application performance that is experienced by the end-users. First, you may want to monitor *end-user experience* directly – through network port mirroring (passively) or synthetic probes (actively) – to capture latency issues and system inconsistencies as the user interacts with your application. Next, you want to monitor *business transactions* across infrastructure tiers to make sure you are meeting your SLA. Finally, you need to have *system reports* that would consist of a standard set of metrics for each application. These reports allow you to evaluate the performance of the whole system, despite cross-application difference
martinfowler.com
bliki: ReproducibleBuild
Builds of software products should be reproducible, so that we can use the differences from past builds to diagnose new problems.
s. Primary metrics matter the most since they help you understand your system as a whole and how your customers experience it.
**Secondary metrics** assess computational resources of the system to ensure there is enough resource capacity to handle the load, and to identify any bottlenecks of the system. *Runtime application architecture monitoring* – that is based on application discovery and dependency mapping – helps you better understand your system topology, service dependencies, and impacts of your changes. In addition to this, you may also want to feel the pulse of your middleware through *deep-dive component monitoring*. Secondary metrics are essential to manage your system and improve its topology.
These two sets of performance metrics are closely monitored to collect the data, understand it, identify trends, and eventually take data-driven actions. By continuously monitoring your users, systems, and network, you detect and contain any incidents. Then you respond by remediating the issues, making a retrospective analysis and applying necessary policy changes. It then allows you to predict future threats and take necessary preventative actions to harden your system. Ultimately, automation strengthens these continuous monitoring stages and enables complex if-then rules to make your system self-aware. This way, you can manage your system more efficiently, self-scale the underlying infrastructure, and make informed business decisions.
Continuous monitoring closes the loop of the DevOps process flow, giving your team feedback about its development efforts. Just like everything else in DevOps, this feedback has to be taken in small sips to help you maintain pace and allow continuous planning of your development tasks.
**On the final note**
It may take time and effort to adopt DevOps culture and implement DevOps processes in your organization, but the benefits are worth the effort. You accelerate innovation, increase efficiency, reduce failures, and enhance the job satisfaction of your IT team. [There is no magic recipe to adopting DevOps](https://devops.com/there-is-no-magic-recipe-to-adopting-devops/) – it is a journey, and, like every other journey, it starts with a small step forward.
​
Originally posted at [https://blog.cherryservers.com/the-anatomy-of-devops-process-flow-close-the-loop-to-succeed](https://blog.cherryservers.com/the-anatomy-of-devops-process-flow-close-the-loop-to-succeed)
https://redd.it/dmgg35
@r_devops
**Secondary metrics** assess computational resources of the system to ensure there is enough resource capacity to handle the load, and to identify any bottlenecks of the system. *Runtime application architecture monitoring* – that is based on application discovery and dependency mapping – helps you better understand your system topology, service dependencies, and impacts of your changes. In addition to this, you may also want to feel the pulse of your middleware through *deep-dive component monitoring*. Secondary metrics are essential to manage your system and improve its topology.
These two sets of performance metrics are closely monitored to collect the data, understand it, identify trends, and eventually take data-driven actions. By continuously monitoring your users, systems, and network, you detect and contain any incidents. Then you respond by remediating the issues, making a retrospective analysis and applying necessary policy changes. It then allows you to predict future threats and take necessary preventative actions to harden your system. Ultimately, automation strengthens these continuous monitoring stages and enables complex if-then rules to make your system self-aware. This way, you can manage your system more efficiently, self-scale the underlying infrastructure, and make informed business decisions.
Continuous monitoring closes the loop of the DevOps process flow, giving your team feedback about its development efforts. Just like everything else in DevOps, this feedback has to be taken in small sips to help you maintain pace and allow continuous planning of your development tasks.
**On the final note**
It may take time and effort to adopt DevOps culture and implement DevOps processes in your organization, but the benefits are worth the effort. You accelerate innovation, increase efficiency, reduce failures, and enhance the job satisfaction of your IT team. [There is no magic recipe to adopting DevOps](https://devops.com/there-is-no-magic-recipe-to-adopting-devops/) – it is a journey, and, like every other journey, it starts with a small step forward.
​
Originally posted at [https://blog.cherryservers.com/the-anatomy-of-devops-process-flow-close-the-loop-to-succeed](https://blog.cherryservers.com/the-anatomy-of-devops-process-flow-close-the-loop-to-succeed)
https://redd.it/dmgg35
@r_devops
DevOps.com
There is No Magic Recipe to Adopting DevOps - DevOps.com
Over the past five years, we have seen more and more organizations of every size trying to adopt DevOps practices. However, we’re still expecting adoption Over the past five years, we have seen more and more organizations of every size trying to adopt DevOps…
(azuredevops) initiate release stages based on package feed view
I was thinking about making a single release pipeline with all enivornments (stages).
I have my builds published in a universal package feed (or can be a nuget feed) and the latest version from any view triggers the pipeline.
Is there a (builtin) way to put in a condition for a stage where it checks from which feed view the package came from?
The use case would be, deploy a prerelease package to staging but not acceptance, or hotfix patches directly to acceptance and skip staging.
https://redd.it/dmsttl
@r_devops
I was thinking about making a single release pipeline with all enivornments (stages).
I have my builds published in a universal package feed (or can be a nuget feed) and the latest version from any view triggers the pipeline.
Is there a (builtin) way to put in a condition for a stage where it checks from which feed view the package came from?
The use case would be, deploy a prerelease package to staging but not acceptance, or hotfix patches directly to acceptance and skip staging.
https://redd.it/dmsttl
@r_devops
reddit
(azuredevops) initiate release stages based on package feed view
I was thinking about making a single release pipeline with all enivornments (stages). I have my builds published in a universal package feed (or...
Alerts on Azure?
Newbie here trying to create alerts for quotas (cores, disks, and public IPs) in Azure.
Would appreciate any pointers on how to get this configured. End goal is to have it post to Slack for alerts.
I was thinking about using the Azure Metadata Instance Service but I’m not sure if their APIs can pull this kind of information. Other idea would be to create a simple bash script on a VM that uses the az-cli along with a web hook to Slack.
https://redd.it/dmrhxy
@r_devops
Newbie here trying to create alerts for quotas (cores, disks, and public IPs) in Azure.
Would appreciate any pointers on how to get this configured. End goal is to have it post to Slack for alerts.
I was thinking about using the Azure Metadata Instance Service but I’m not sure if their APIs can pull this kind of information. Other idea would be to create a simple bash script on a VM that uses the az-cli along with a web hook to Slack.
https://redd.it/dmrhxy
@r_devops
reddit
Alerts on Azure?
Newbie here trying to create alerts for quotas (cores, disks, and public IPs) in Azure. Would appreciate any pointers on how to get this...
Show events on a grafana graph ?
I have a PromQL query showing metrics in a time-series graph. I also have an elasticsearch server that I have with logs. I want to create a dashboard that shows these metrics from prometheus but that can show some events (log lines according to my query) on the graph timeline.
eventually I'd like to see certain metrics relativly to some events.
I tried to draw an example, the vertical lines represent the events and the graph shows a metric.
example:
[https://imgur.com/18lxQdb](https://imgur.com/18lxQdb)
https://redd.it/dlxxgr
@r_devops
I have a PromQL query showing metrics in a time-series graph. I also have an elasticsearch server that I have with logs. I want to create a dashboard that shows these metrics from prometheus but that can show some events (log lines according to my query) on the graph timeline.
eventually I'd like to see certain metrics relativly to some events.
I tried to draw an example, the vertical lines represent the events and the graph shows a metric.
example:
[https://imgur.com/18lxQdb](https://imgur.com/18lxQdb)
https://redd.it/dlxxgr
@r_devops
Imgur
Post with 0 votes and 18 views.
Windows Licensing for Automated Test VMs
I'm relatively new to DevOps - we are trying to get certain things automated in our development shop, one of which is the implementation of automated testing.
Most of our developers use either macOs or some flavor of Linux and our deployments are all Linux-based (mostly CentOS).
Our main product is a web application that our customers access via your standard browsers (IE, FF, Chrome), and our customers are almost exclusively using Windows.
We want to set up a GitLab pipeline that runs a suite of automated tests each night, connecting to Windows VMs.
Is there an "industry standard" method for Windows licensing related to tests like this? I know that they provide ISO's for browser testing where the license is good for 90 days, but we don't have the time to set up new VMs every 3 months or so.
I know other companies out there have figured this out, and it feels like a stupid question...I'm just not sure what the best path forward is. We don't have any kind of MSDN license since no one here in the office uses Windows.
Any advice or tips would be greatly appreciated!
https://redd.it/dmy0bd
@r_devops
I'm relatively new to DevOps - we are trying to get certain things automated in our development shop, one of which is the implementation of automated testing.
Most of our developers use either macOs or some flavor of Linux and our deployments are all Linux-based (mostly CentOS).
Our main product is a web application that our customers access via your standard browsers (IE, FF, Chrome), and our customers are almost exclusively using Windows.
We want to set up a GitLab pipeline that runs a suite of automated tests each night, connecting to Windows VMs.
Is there an "industry standard" method for Windows licensing related to tests like this? I know that they provide ISO's for browser testing where the license is good for 90 days, but we don't have the time to set up new VMs every 3 months or so.
I know other companies out there have figured this out, and it feels like a stupid question...I'm just not sure what the best path forward is. We don't have any kind of MSDN license since no one here in the office uses Windows.
Any advice or tips would be greatly appreciated!
https://redd.it/dmy0bd
@r_devops
reddit
Windows Licensing for Automated Test VMs
I'm relatively new to DevOps - we are trying to get certain things automated in our development shop, one of which is the implementation of...
I made this video about chaos engineering startup Gremlin
Not sure whether this is too trivial for this subreddit, but I thought I'd share it with you nonetheless:
[https://youtu.be/6e5qNWFRXnw](https://youtu.be/6e5qNWFRXnw)
FYI, this is not a sponsored episode, I'm just a small indie content creator interested in this.
https://redd.it/dm0e67
@r_devops
Not sure whether this is too trivial for this subreddit, but I thought I'd share it with you nonetheless:
[https://youtu.be/6e5qNWFRXnw](https://youtu.be/6e5qNWFRXnw)
FYI, this is not a sponsored episode, I'm just a small indie content creator interested in this.
https://redd.it/dm0e67
@r_devops
YouTube
Exploring chaos engineering with Gremlin co-founder and CTO Matthew Fornaciari
Listen to the podcast version 🎙👇🏼 https://apple.co/2W2g0nV Gremlin is a fast growing company in the chaos engineering space founded in 2016. The company has ...
Experience attracting and hiring DevOps engineers?
I'm in a leadership role at a company that is still figuring out DevOps. We've built a mini-DevOps process run by software engineers and data scientists, but it's clear to me that we need a team that is full-time focused on speeding up/automating our deployment process through better processes and tailored CI/CD software solutions. I recently posted a position for a Sr. DevOps Engineer on our company website (and on LinkedIn, Glassdoor, etc.), and haven't gotten a single bite. For reference, my company is a major employer of technology professionals in the Salt Lake City area, and it's rare for a posting like this not to garner any interest.
Has anyone here had success getting the first few DevOps professionals into a company that's just getting off the ground in this regard? What did your postings look like? Did you use agencies? Go to conferences? Any advice appreciated.
EDIT: Since you asked, here's the posting. Hopefully this doesn't count as violating Rule 3, since I'm here to ask for feedback rather than to attract candidates directly. [https://www.linkedin.com/jobs/view/1510886802/?alternateChannel=search](https://www.linkedin.com/jobs/view/1510886802/?alternateChannel=search)
https://redd.it/dmzieu
@r_devops
I'm in a leadership role at a company that is still figuring out DevOps. We've built a mini-DevOps process run by software engineers and data scientists, but it's clear to me that we need a team that is full-time focused on speeding up/automating our deployment process through better processes and tailored CI/CD software solutions. I recently posted a position for a Sr. DevOps Engineer on our company website (and on LinkedIn, Glassdoor, etc.), and haven't gotten a single bite. For reference, my company is a major employer of technology professionals in the Salt Lake City area, and it's rare for a posting like this not to garner any interest.
Has anyone here had success getting the first few DevOps professionals into a company that's just getting off the ground in this regard? What did your postings look like? Did you use agencies? Go to conferences? Any advice appreciated.
EDIT: Since you asked, here's the posting. Hopefully this doesn't count as violating Rule 3, since I'm here to ask for feedback rather than to attract candidates directly. [https://www.linkedin.com/jobs/view/1510886802/?alternateChannel=search](https://www.linkedin.com/jobs/view/1510886802/?alternateChannel=search)
https://redd.it/dmzieu
@r_devops
Linkedin
Progressive Leasing hiring Senior DevOps Engineer in Draper, UT, US | LinkedIn
Posted 2 weeks ago. As a Sr. DevOps Engineer, you will build and manage a continuous integration/continuous deployment…See this and similar jobs on LinkedIn.
Have you any experience with rsync.net storage?
Hi guys!
​
Have anyone any experience with [rsync.net](https://rsync.net)? I need a backup location where can i store database backups, script and other sys tools.
https://redd.it/dn26f7
@r_devops
Hi guys!
​
Have anyone any experience with [rsync.net](https://rsync.net)? I need a backup location where can i store database backups, script and other sys tools.
https://redd.it/dn26f7
@r_devops
www.rsync.net
Cloud Storage for Offsite Backup
rsync.net offers secure cloud storage on an open standards ZFS and SSH platform for offsite backup and disaster recovery
How to estimate amount of code globally in a particular language?
There are some stats in github for popular languages like Java, js, c++. I’m more interested in amount of code for tools like Terraform (HCL), ansible, cloud formation, puluni, etc. Ok if public repos only and even in particular segment/location.
https://redd.it/dn3huc
@r_devops
There are some stats in github for popular languages like Java, js, c++. I’m more interested in amount of code for tools like Terraform (HCL), ansible, cloud formation, puluni, etc. Ok if public repos only and even in particular segment/location.
https://redd.it/dn3huc
@r_devops
reddit
How to estimate amount of code globally in a particular language?
There are some stats in github for popular languages like Java, js, c++. I’m more interested in amount of code for tools like Terraform (HCL),...
Newbie of IaaC: Configuration Orchestration and Management
Background: I am new in IaaC, and learning Terraform. We are a small Microsoft workshop.
Recently I just started my adventure in IaaC.
My goal is to build a clone of our production environment on Azure. Now the goal seems to be a two-stage work: Terraform builds the infrastructure, and a configuration management tool configures the VMs.
After I spent few weeks, I found that Terraform is good at configuration orchestration (create and keep immutable infrastructures), but not configuration management (install software, configure accounts, and deploy our software).
Here are my questions:
1. Did I define the goal correctly? Shall it have more or less stages with different tools?
2. If I am correct, do you have a suggestion of configuration management tool?
Thank you for your reply!
https://redd.it/dn0xac
@r_devops
Background: I am new in IaaC, and learning Terraform. We are a small Microsoft workshop.
Recently I just started my adventure in IaaC.
My goal is to build a clone of our production environment on Azure. Now the goal seems to be a two-stage work: Terraform builds the infrastructure, and a configuration management tool configures the VMs.
After I spent few weeks, I found that Terraform is good at configuration orchestration (create and keep immutable infrastructures), but not configuration management (install software, configure accounts, and deploy our software).
Here are my questions:
1. Did I define the goal correctly? Shall it have more or less stages with different tools?
2. If I am correct, do you have a suggestion of configuration management tool?
Thank you for your reply!
https://redd.it/dn0xac
@r_devops
reddit
Newbie of IaaC: Configuration Orchestration and Management
Background: I am new in IaaC, and learning Terraform. We are a small Microsoft workshop. Recently I just started my adventure in IaaC. My goal...
Flyway migrations kubernetes init containers and helm
I wrote up this github example how to smoothly manage flyway migrations using helm and kubernetes. Hope this helps someone i've seen alot of implementations this was the best i could come up with.
​
Alot of times these patterns for flyway migrations result in bloated release images with flyway tooling or hacky Dockerfiles managing things with [start.sh](https://start.sh) scripts. I like the below pattern because its frictionless once implemented.
​
[https://github.com/up-time/flyway-k8-helm-migrations](https://github.com/up-time/flyway-k8-helm-migrations)
https://redd.it/dn14h2
@r_devops
I wrote up this github example how to smoothly manage flyway migrations using helm and kubernetes. Hope this helps someone i've seen alot of implementations this was the best i could come up with.
​
Alot of times these patterns for flyway migrations result in bloated release images with flyway tooling or hacky Dockerfiles managing things with [start.sh](https://start.sh) scripts. I like the below pattern because its frictionless once implemented.
​
[https://github.com/up-time/flyway-k8-helm-migrations](https://github.com/up-time/flyway-k8-helm-migrations)
https://redd.it/dn14h2
@r_devops
GitHub
GitHub - up-time/flyway-k8-helm-migrations: Helm generating configmaps for k8 init containers running flyway migrations
Helm generating configmaps for k8 init containers running flyway migrations - GitHub - up-time/flyway-k8-helm-migrations: Helm generating configmaps for k8 init containers running flyway migrations
The Complete Overview of DevOps Cloud Native Tools Landscape
Cloud native technologies are revolutionizing the way applications are delivered. They serve as an excellent complement to [DevOps](https://blog.cherryservers.com/devops-full-picture-an-intersection-of-culture-processes-and-tools) by providing the tools and platforms to enable automation and scalability. However, even for those in the industry, understanding what cloud native is (and isn’t) and navigating the entirety of the cloud native landscape can be a challenge.
Fortunately, the [Cloud Native Computing Foundation (CNCF)](https://www.cncf.io/) is helping to standardize the cloud native space and making cloud native more accessible. Their [Cloud Native Landscape interactive map](https://landscape.cncf.io/) provides us with a list of tools and services that enable cloud native computing. Here, we’ll take that a step further and break down each section of the current cloud native landscape. By the end of this piece, you should have a firm grasp of the different aspects of cloud native computing, as well as an understanding of many of the most popular tools to implement them.
## What is cloud native and how does its
## landscape look like?
One of the more admirable things the CNCF has done is [give the term “cloud native” an authoritative definition](https://github.com/cncf/toc/blob/master/DEFINITION.md). It should help us avoid much of the ambiguity surrounding other terms commonly used in the industry, such as the confusion around [the differences between DevOps and Agile](https://blog.cherryservers.com/how-is-devops-different-from-agile-development-methodology). To paraphrase the CNCF, cloud native tech enables organizations to build and run apps in a scalable and dynamic way. Technologies like containers, microservices, immutable infrastructure, and declarative APIs are important parts of cloud native.
The cloud native landscape is the list of tools, services, and platforms that make up the current cloud native ecosystem. Many popular [DevOps tools](https://blog.cherryservers.com/5-devops-tools-you-should-know-in-2019) can be found in the current list. They are broken up into categories based on the functionality they provide and the layer of the cloud native stack they reside in. As opposed to enumerating every tool in the list, in the sections that follow, we’ll explain each category and review some of the most popular tools available.
## App definition and development
The app definition and development category of the cloud native landscape focuses on components directly involved in enabling application functionality, communication between microservices, application data storage, and image creation.
### Databases
Databases have long been a fundamental aspect of application development. In a traditional LAMP/LEMP stack, the database needs to reliably, securely, and quickly enable [CRUD](https://www.techopedia.com/definition/25949/create-retrieve-update-and-delete-crud) operations for the app. Typically, some sort of DBMS (Database Management System) like MySQL, Postgres, or MariaDB was used.
With traditional databases, all the files and resources had to reside on the same host for proper functionality. It’s this requirement that created a challenge for cloud native computing. Databases for cloud native apps cannot have a single point of failure and needs data to be spread across multiple servers. It is something distributed databases do very well.
[Distributed databases](https://en.wikipedia.org/wiki/Distributed_database) enable more resilient and modular application development in the cloud. Data is stored by replicating and duplicating it across multiple discrete servers. By doing so, cloud native developers can increase performance and durability of their systems.
Some of the most popular tools in this category are:
* [**Apache Hadoop**](https://hadoop.apache.org/)\- Technically, Hadoop is the framework used for distributed processing while [Cassandra](https://cassandra.apache.org/) is Apache’s distributed NoSQL database management system
Cloud native technologies are revolutionizing the way applications are delivered. They serve as an excellent complement to [DevOps](https://blog.cherryservers.com/devops-full-picture-an-intersection-of-culture-processes-and-tools) by providing the tools and platforms to enable automation and scalability. However, even for those in the industry, understanding what cloud native is (and isn’t) and navigating the entirety of the cloud native landscape can be a challenge.
Fortunately, the [Cloud Native Computing Foundation (CNCF)](https://www.cncf.io/) is helping to standardize the cloud native space and making cloud native more accessible. Their [Cloud Native Landscape interactive map](https://landscape.cncf.io/) provides us with a list of tools and services that enable cloud native computing. Here, we’ll take that a step further and break down each section of the current cloud native landscape. By the end of this piece, you should have a firm grasp of the different aspects of cloud native computing, as well as an understanding of many of the most popular tools to implement them.
## What is cloud native and how does its
## landscape look like?
One of the more admirable things the CNCF has done is [give the term “cloud native” an authoritative definition](https://github.com/cncf/toc/blob/master/DEFINITION.md). It should help us avoid much of the ambiguity surrounding other terms commonly used in the industry, such as the confusion around [the differences between DevOps and Agile](https://blog.cherryservers.com/how-is-devops-different-from-agile-development-methodology). To paraphrase the CNCF, cloud native tech enables organizations to build and run apps in a scalable and dynamic way. Technologies like containers, microservices, immutable infrastructure, and declarative APIs are important parts of cloud native.
The cloud native landscape is the list of tools, services, and platforms that make up the current cloud native ecosystem. Many popular [DevOps tools](https://blog.cherryservers.com/5-devops-tools-you-should-know-in-2019) can be found in the current list. They are broken up into categories based on the functionality they provide and the layer of the cloud native stack they reside in. As opposed to enumerating every tool in the list, in the sections that follow, we’ll explain each category and review some of the most popular tools available.
## App definition and development
The app definition and development category of the cloud native landscape focuses on components directly involved in enabling application functionality, communication between microservices, application data storage, and image creation.
### Databases
Databases have long been a fundamental aspect of application development. In a traditional LAMP/LEMP stack, the database needs to reliably, securely, and quickly enable [CRUD](https://www.techopedia.com/definition/25949/create-retrieve-update-and-delete-crud) operations for the app. Typically, some sort of DBMS (Database Management System) like MySQL, Postgres, or MariaDB was used.
With traditional databases, all the files and resources had to reside on the same host for proper functionality. It’s this requirement that created a challenge for cloud native computing. Databases for cloud native apps cannot have a single point of failure and needs data to be spread across multiple servers. It is something distributed databases do very well.
[Distributed databases](https://en.wikipedia.org/wiki/Distributed_database) enable more resilient and modular application development in the cloud. Data is stored by replicating and duplicating it across multiple discrete servers. By doing so, cloud native developers can increase performance and durability of their systems.
Some of the most popular tools in this category are:
* [**Apache Hadoop**](https://hadoop.apache.org/)\- Technically, Hadoop is the framework used for distributed processing while [Cassandra](https://cassandra.apache.org/) is Apache’s distributed NoSQL database management system
Cherryservers
What is DevOps? An Intersection of Culture, Processes and Tools
DevOps is a combination of culture, processes and tools that allow engineers to deliver better quality software quicker.
.
* [**MySQL**](https://www.mysql.com/)\- You may be familiar with MySQL if you come from a background of spinning up LAMP/LEMP stacks for web applications. [MySQL Cluster CGE](https://www.mysql.com/products/cluster/) is the organization's distributed database offering focused on enabling scalability and high availability (HA).
* [**TiKV**](https://github.com/tikv/tikv)**-** This open-source distributed NoSQL database provides two key benefits: transactional APIs and [ACID](https://en.wikipedia.org/wiki/ACID) TiKV is incubated by the CNCF as well.
### Streaming & Messaging
Microservices architecture is the cornerstone of both DevOps and cloud native computing. It allows different services to communicate with each other, maintaining data consistently and avoiding data corruption.
Streaming services and message brokers are the middleware of cloud native. They formalize the way distinct nodes communicate with each other in the microservices-enabled system. [AMQP](https://www.amqp.org/) is one of the most commonly used protocols here, but there are others such as [MQTT](https://mqtt.org/) and [STOMP](https://stomp.github.io/).
Each service and protocol works slightly differently, but the underlying principle for messaging is the same: message or event “producers” send information to an intermediary (the broker) that makes delivers it to “consumers” or event receivers to act on the information. There’s a bit of debate on what does and does not constitute streaming, but in a nutshell streaming is simply the ability to do messaging at scale. For a deep dive into the topic of streaming and messaging, check out Roger Rea’s [Streaming Analytics: To Stream or Not to Stream? article](https://www.rtinsights.com/streaming-analytics-to-stream-or-not-to-stream/).
Popular cloud native streaming and messaging applications include:
* [**RabbitMQ**](https://www.rabbitmq.com/)\- A popular open-source message broker that supports a variety of messaging protocols.
* [**Strimzi**](https://strimzi.io/)\- Allows Kafka (Apache’s distributed message streaming platform) to run on a Kubernetes cluster.
* [**Beam**](https://beam.apache.org/)\- A portable and extensible programming model that allows processing batches or streams of data and running them on a variety of execution engines. It supports popular programming languages like Java and Python.
### Application Definition & Image Build
Creating application definitions ([great explanation by Puppet here](https://puppet.com/docs/pe/2018.1/creating_application_definitions.html)) and building images are a vital part of automating the development pipeline (which is a key component of DevOps). Reliable images ensure deployment processes can be scaled and reproduced, enabling automation and HA.
Services in this category make it easier to manage containers and clusters at scale. If you think about how containers and microservices work, the use case for these services begins to become clear. Microservices and containerization are all about breaking down large monolithic apps into smaller chunks. It is great for resilience and scalability, but it leaves you with many images that need to be configured, maintained, and ran. Application definition and image building tools make it simple to define multi-container applications. Using Docker Compose, as a specific example, you can spin up multiple Docker containers as a single service.
Here are some of the most popular cloud native tools in the application definition & image build subcategory:
* [**Docker Compose**](https://docs.docker.com/compose/)\- Containerization is one of the most important aspects of cloud native and Docker is the leader in the containerization market. Docker Compose is used to define multi-container Docker apps. Compose uses YAML files for configuration and allows instantiation with one command.
* [**Helm**](https://helm.sh/)****\- Kubernetes can get complex at scale. Helm allows you to install, upgrade, and define (using YAML files) Kubernetes apps. While Helm may not be a household name yet, it is maintained by the CNCF and has
* [**MySQL**](https://www.mysql.com/)\- You may be familiar with MySQL if you come from a background of spinning up LAMP/LEMP stacks for web applications. [MySQL Cluster CGE](https://www.mysql.com/products/cluster/) is the organization's distributed database offering focused on enabling scalability and high availability (HA).
* [**TiKV**](https://github.com/tikv/tikv)**-** This open-source distributed NoSQL database provides two key benefits: transactional APIs and [ACID](https://en.wikipedia.org/wiki/ACID) TiKV is incubated by the CNCF as well.
### Streaming & Messaging
Microservices architecture is the cornerstone of both DevOps and cloud native computing. It allows different services to communicate with each other, maintaining data consistently and avoiding data corruption.
Streaming services and message brokers are the middleware of cloud native. They formalize the way distinct nodes communicate with each other in the microservices-enabled system. [AMQP](https://www.amqp.org/) is one of the most commonly used protocols here, but there are others such as [MQTT](https://mqtt.org/) and [STOMP](https://stomp.github.io/).
Each service and protocol works slightly differently, but the underlying principle for messaging is the same: message or event “producers” send information to an intermediary (the broker) that makes delivers it to “consumers” or event receivers to act on the information. There’s a bit of debate on what does and does not constitute streaming, but in a nutshell streaming is simply the ability to do messaging at scale. For a deep dive into the topic of streaming and messaging, check out Roger Rea’s [Streaming Analytics: To Stream or Not to Stream? article](https://www.rtinsights.com/streaming-analytics-to-stream-or-not-to-stream/).
Popular cloud native streaming and messaging applications include:
* [**RabbitMQ**](https://www.rabbitmq.com/)\- A popular open-source message broker that supports a variety of messaging protocols.
* [**Strimzi**](https://strimzi.io/)\- Allows Kafka (Apache’s distributed message streaming platform) to run on a Kubernetes cluster.
* [**Beam**](https://beam.apache.org/)\- A portable and extensible programming model that allows processing batches or streams of data and running them on a variety of execution engines. It supports popular programming languages like Java and Python.
### Application Definition & Image Build
Creating application definitions ([great explanation by Puppet here](https://puppet.com/docs/pe/2018.1/creating_application_definitions.html)) and building images are a vital part of automating the development pipeline (which is a key component of DevOps). Reliable images ensure deployment processes can be scaled and reproduced, enabling automation and HA.
Services in this category make it easier to manage containers and clusters at scale. If you think about how containers and microservices work, the use case for these services begins to become clear. Microservices and containerization are all about breaking down large monolithic apps into smaller chunks. It is great for resilience and scalability, but it leaves you with many images that need to be configured, maintained, and ran. Application definition and image building tools make it simple to define multi-container applications. Using Docker Compose, as a specific example, you can spin up multiple Docker containers as a single service.
Here are some of the most popular cloud native tools in the application definition & image build subcategory:
* [**Docker Compose**](https://docs.docker.com/compose/)\- Containerization is one of the most important aspects of cloud native and Docker is the leader in the containerization market. Docker Compose is used to define multi-container Docker apps. Compose uses YAML files for configuration and allows instantiation with one command.
* [**Helm**](https://helm.sh/)****\- Kubernetes can get complex at scale. Helm allows you to install, upgrade, and define (using YAML files) Kubernetes apps. While Helm may not be a household name yet, it is maintained by the CNCF and has
the support of industry giants like Google and Microsoft. It is also quite popular on GitHub with 13,000+ stars as of this writing.
* [**Packer**](https://www.packer.io/)\- Packer makes it easy to automate the creation of machine images. It also comes with out of the box support for images compatible with a variety of platforms and is extensible. New platforms can be added to Packer by way of plugins.
### Continuous Integration & Delivery
CI/CD is at the heart of DevOps processes. Building quality software quicker is what most DevOps teams that leverage cloud native tools strive for. We’ve covered CI/CD in-depth in our [What is DevOps](https://blog.cherryservers.com/devops-full-picture-an-intersection-of-culture-processes-and-tools) post, so I’ll only give a crash course here.
Continuous integration (CI) is all about making sure developers are working with the latest code build. The code in development cannot drift too far from the main branch, otherwise the hell will break loose. CI helps to ensure bugs, are addressed quickly and keeps the main branch stable, ready to be deployed at any time.
Continuous delivery (CD) is the next logical step after implementing continuous integration. CD automates code delivery pipeline by running different suites of automated tests and deploying your code to QA or staging environments. Some companies go further, by implementing continuous deployment. The only difference is that it automatically deploys your code to production after all the tests have passed.
Here are the most popular CI\\CD tools that are listed in the cloud native landscape:
* [**GitLab**](https://about.gitlab.com/)**-**GitLab is a GitHub alternative that is growing in popularity. The idea here is that you can minimize complexity by baking source code management, CI/CD, application monitoring and security into a single holistic solution.
* [**Jenkins**](https://jenkins.io/)\- The Jenkins automation server application is probably one of the first tools that come to mind when you think about CI\\CD. Trusted by many DevOps teams, as well as being highly extensible and scalable, Jenkins is a popular choice for CI\\CD in the world of cloud native.
* [**Drone**](https://try.drone.io/)**-** Drone is a popular open-source continuous delivery system built on top of Docker. It uses YAML configuration files and is compatible with any environment that can run within a Docker container.
## Orchestration & Management
Cloud native application built in a DevOps-driven organization is inherently scalable, automated and resilient. It means the tools used to orchestrate and manage cloud native apps must be able to work at scale and trigger automated workflows.
In a nutshell, this category of tools is all about making microservices work together in harmony. That’s what the tools in these categories help you do.
### Scheduling & Orchestration
Deploying and managing a cluster of containers is one of the major operational tasks required to create a resilient, loosely coupled, and easily scalable cloud native application. Operating containers in a cluster also helps you enhance operational agility and mitigate the single point of failure problem: you no longer try to avoid failure, but rather prepare your application to handle it with minimal consequences.
If your application consists of just a few containers, tools in this category may be unnecessary. Scheduling and orchestration tools become valuable as cloud native applications scale. The benefit of these tools is that they abstract away the complexities of managing a large number of containers. As opposed to managing your application at the container-level, they provide you with a framework to deploy, rollback, load balance, and configure self-healing functionality at scale. For a more in-depth intro to the topic, check our [Why you need Kubernetes and what can it do](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/#why-you-need-kubernetes-and-what-can-it-do). While that piece is specific to Kubernetes, many of the principles can be generally applied.
There ar
* [**Packer**](https://www.packer.io/)\- Packer makes it easy to automate the creation of machine images. It also comes with out of the box support for images compatible with a variety of platforms and is extensible. New platforms can be added to Packer by way of plugins.
### Continuous Integration & Delivery
CI/CD is at the heart of DevOps processes. Building quality software quicker is what most DevOps teams that leverage cloud native tools strive for. We’ve covered CI/CD in-depth in our [What is DevOps](https://blog.cherryservers.com/devops-full-picture-an-intersection-of-culture-processes-and-tools) post, so I’ll only give a crash course here.
Continuous integration (CI) is all about making sure developers are working with the latest code build. The code in development cannot drift too far from the main branch, otherwise the hell will break loose. CI helps to ensure bugs, are addressed quickly and keeps the main branch stable, ready to be deployed at any time.
Continuous delivery (CD) is the next logical step after implementing continuous integration. CD automates code delivery pipeline by running different suites of automated tests and deploying your code to QA or staging environments. Some companies go further, by implementing continuous deployment. The only difference is that it automatically deploys your code to production after all the tests have passed.
Here are the most popular CI\\CD tools that are listed in the cloud native landscape:
* [**GitLab**](https://about.gitlab.com/)**-**GitLab is a GitHub alternative that is growing in popularity. The idea here is that you can minimize complexity by baking source code management, CI/CD, application monitoring and security into a single holistic solution.
* [**Jenkins**](https://jenkins.io/)\- The Jenkins automation server application is probably one of the first tools that come to mind when you think about CI\\CD. Trusted by many DevOps teams, as well as being highly extensible and scalable, Jenkins is a popular choice for CI\\CD in the world of cloud native.
* [**Drone**](https://try.drone.io/)**-** Drone is a popular open-source continuous delivery system built on top of Docker. It uses YAML configuration files and is compatible with any environment that can run within a Docker container.
## Orchestration & Management
Cloud native application built in a DevOps-driven organization is inherently scalable, automated and resilient. It means the tools used to orchestrate and manage cloud native apps must be able to work at scale and trigger automated workflows.
In a nutshell, this category of tools is all about making microservices work together in harmony. That’s what the tools in these categories help you do.
### Scheduling & Orchestration
Deploying and managing a cluster of containers is one of the major operational tasks required to create a resilient, loosely coupled, and easily scalable cloud native application. Operating containers in a cluster also helps you enhance operational agility and mitigate the single point of failure problem: you no longer try to avoid failure, but rather prepare your application to handle it with minimal consequences.
If your application consists of just a few containers, tools in this category may be unnecessary. Scheduling and orchestration tools become valuable as cloud native applications scale. The benefit of these tools is that they abstract away the complexities of managing a large number of containers. As opposed to managing your application at the container-level, they provide you with a framework to deploy, rollback, load balance, and configure self-healing functionality at scale. For a more in-depth intro to the topic, check our [Why you need Kubernetes and what can it do](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/#why-you-need-kubernetes-and-what-can-it-do). While that piece is specific to Kubernetes, many of the principles can be generally applied.
There ar
Packer | HashiCorp Developer
Explore Packer product documentation, tutorials, and examples.
e a variety of options for cloud native container scheduling and orchestration, but the three major players are:
* [**Kubernetes**](https://kubernetes.io/)\- Kubernetes or K8s is the most popular container orchestration solution. K8s started at Google and was released as an [open-source tool in 2014](https://github.com/kubernetes/kubernetes/commit/2c4b3a562ce34cddc3f8218a2c4d11c7310e6d56)
* [**Docker Swarm**](https://docs.docker.com/engine/swarm/)\- Docker is the leading container runtime, so it makes sense that their orchestration tool would be a popular and trusted option. Swarms consist of multiple Docker hosts that operate in a cluster as managers or workers.
* [**Mesos**](https://mesos.apache.org/)**-** Mesos is Apache’s orchestration tool that abstracts away infrastructure resources and enables the creation of highly scalable distributed systems. Architecturally, Mesos is an abstraction layer that orchestrates CPU, memory and storage resources of physical or virtual machines.
### Coordination & Service Discovery
There are thousands of stateless microservices in a cloud native application. Their lifecycle is unpredictable since microservices applications are built with auto-scaling and self-healing in mind. Coordination and service discovery tools enable the dynamic configuration and discovery of such microservices that are otherwise hard to handle. Using these tools can reduce redeployments and help with the implementation of stateless services.
The reason these services are useful for cloud native apps is that each discrete microservice may have a different impact on application performance and resource consumption. This requires services that are “aware” of one another. With coordination in place, failover, load balancing, and performance issues can also be addressed. For a practical example of the need for coordination and service discovery, check out [the work Andy Redko did with Apache Zookeeper](https://www.javacodegeeks.com/2013/11/coordination-and-service-discovery-with-apache-zookeeper.html).
Some of the most popular coordination & service discovery tools are:
* [**ZooKeeper**](https://zookeeper.apache.org/)**-** ZooKeeper is a tool from Apache. It is meant to maintain centralized configuration, naming, and grouping of your services. As a distributed system management tool, it is easily scalable and provides redundancy to your system.
* [**Eureka**](https://github.com/Netflix/eureka)\- Netflix is a big contributor to the world of DevOps tools and cloud native computing. Eureka is an open-source service registry they developed to handle large-scale load balancing and failover.
* [**etcd**](https://github.com/etcd-io/etcd)**-** etcd is an open-source key-value store that aims to provide cloud native services with configuration and services information in a simple, secure, fast, and reliable fashion.
### Remote Procedure Call
Remote Procedure Calls (RPCs) allow applications to call a procedure that is not in the same address space. It is a core part of cloud native computing because it allows discrete microservices to communicate without residing on the same system.
RPCs are by no means new. Sun RPC a.k.a Open Network Computing Remote Procedure Call (ONC RPC) was originally developed in the 1980s. Similarly, Windows Systems having been using RPC for years. The reason RPCs are so attractive in cloud native world is that they enable language independent and low latency client/server communication. For more on the benefits of RPC, and gRPC in particular, check out The New Stack’s [Google’s gRPC: A Lean and Mean Communication Protocol for Microservices](https://thenewstack.io/grpc-lean-mean-communication-protocol-microservices/).
Popular RPC tools include:
* [**gRPC**](https://grpc.io/)**-** gPRC is an RPC framework designed to run in almost any environment. It is one of the most popular development tools for communication between microservices. gPRC is used by major organizations like Netflix, Cisco, and Square.
* [**Dubbo**](https://dubbo.apache.org/en-us/)\- This Apache offering is a high-performanc
* [**Kubernetes**](https://kubernetes.io/)\- Kubernetes or K8s is the most popular container orchestration solution. K8s started at Google and was released as an [open-source tool in 2014](https://github.com/kubernetes/kubernetes/commit/2c4b3a562ce34cddc3f8218a2c4d11c7310e6d56)
* [**Docker Swarm**](https://docs.docker.com/engine/swarm/)\- Docker is the leading container runtime, so it makes sense that their orchestration tool would be a popular and trusted option. Swarms consist of multiple Docker hosts that operate in a cluster as managers or workers.
* [**Mesos**](https://mesos.apache.org/)**-** Mesos is Apache’s orchestration tool that abstracts away infrastructure resources and enables the creation of highly scalable distributed systems. Architecturally, Mesos is an abstraction layer that orchestrates CPU, memory and storage resources of physical or virtual machines.
### Coordination & Service Discovery
There are thousands of stateless microservices in a cloud native application. Their lifecycle is unpredictable since microservices applications are built with auto-scaling and self-healing in mind. Coordination and service discovery tools enable the dynamic configuration and discovery of such microservices that are otherwise hard to handle. Using these tools can reduce redeployments and help with the implementation of stateless services.
The reason these services are useful for cloud native apps is that each discrete microservice may have a different impact on application performance and resource consumption. This requires services that are “aware” of one another. With coordination in place, failover, load balancing, and performance issues can also be addressed. For a practical example of the need for coordination and service discovery, check out [the work Andy Redko did with Apache Zookeeper](https://www.javacodegeeks.com/2013/11/coordination-and-service-discovery-with-apache-zookeeper.html).
Some of the most popular coordination & service discovery tools are:
* [**ZooKeeper**](https://zookeeper.apache.org/)**-** ZooKeeper is a tool from Apache. It is meant to maintain centralized configuration, naming, and grouping of your services. As a distributed system management tool, it is easily scalable and provides redundancy to your system.
* [**Eureka**](https://github.com/Netflix/eureka)\- Netflix is a big contributor to the world of DevOps tools and cloud native computing. Eureka is an open-source service registry they developed to handle large-scale load balancing and failover.
* [**etcd**](https://github.com/etcd-io/etcd)**-** etcd is an open-source key-value store that aims to provide cloud native services with configuration and services information in a simple, secure, fast, and reliable fashion.
### Remote Procedure Call
Remote Procedure Calls (RPCs) allow applications to call a procedure that is not in the same address space. It is a core part of cloud native computing because it allows discrete microservices to communicate without residing on the same system.
RPCs are by no means new. Sun RPC a.k.a Open Network Computing Remote Procedure Call (ONC RPC) was originally developed in the 1980s. Similarly, Windows Systems having been using RPC for years. The reason RPCs are so attractive in cloud native world is that they enable language independent and low latency client/server communication. For more on the benefits of RPC, and gRPC in particular, check out The New Stack’s [Google’s gRPC: A Lean and Mean Communication Protocol for Microservices](https://thenewstack.io/grpc-lean-mean-communication-protocol-microservices/).
Popular RPC tools include:
* [**gRPC**](https://grpc.io/)**-** gPRC is an RPC framework designed to run in almost any environment. It is one of the most popular development tools for communication between microservices. gPRC is used by major organizations like Netflix, Cisco, and Square.
* [**Dubbo**](https://dubbo.apache.org/en-us/)\- This Apache offering is a high-performanc
Kubernetes
Production-Grade Container Orchestration
Kubernetes, also known as K8s, is an open source system for automating deployment, scaling, and management of containerized applications.
It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds…
It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds…
e Java RPC framework. In addition to RPC functionality, it offers intelligent load balancing, an extensible plugin design, automatic service registration, and runtime traffic routing.
* [**Tars**](https://github.com/tarsCloud)\- This RPC framework is based on the name service and the Tars protocol. Today, it supports the Go, C++, NodeJs, PHP, and Java programming languages.
### Service proxy
If you’ve ever had to troubleshoot an application, you’re likely familiar with the difficulty of drilling down to the core of the issue. Is it the app? Is it the network? Such approach may even lead to fingerpointing and make solving problems take longer. One of the core benefits of using a service proxy is being able to decouple network and application problems easily.
Drilling down a bit further, service proxies handle communication between instances in service mesh topologies. Not only do service proxy facilitates network communication, but it also improves observability and facilitates performance tweaks.
Some of the most popular cloud native service proxies are:
* [**Envoy**](https://www.envoyproxy.io/)\- Envoy was created by the Lyft team to serve as a distributed proxy, communication bus, and data plane. The project took inspiration from popular open-source projects like NGINX and HAproxy. It is now used by many respected names in the world of cloud native including SalesForce, Pinterest, AirBnb, and Uber. (were you a little surprised to see Uber using Lyft technology? Score one for the benefits of open source!).
* [**Skipper**](https://opensource.zalando.com/skipper/)**-** Open-source HTTP router and reverse proxy Skipper is designed to handle large amounts of HTTP routes dynamically. It can also serve as a Kubernetes Ingress controller.
* [**NGINX**](https://www.nginx.com/)**-** Much more than just a web server, NGINX provides robust proxy functionality and load balancing. It can also serve as an API gateway…
### API gateway
APIs are an important part of enabling the scalability of cloud native apps. API gateways make API implementation easier by providing a single point of entry for API calls. It makes load balancing, security, and monitoring more effective and simple to manage.
Tyk does a good job of enumerating the benefits of API gateways to microservices in [this piece](https://tyk.io/blog/microservices-api-gateway/). The short version is that API gateways streamline interservice communication, minimize attack surface, and enable service mocking to make testing easier.
There is a bit of an overlap between this and service proxy categories. Apart from NGINX and Envoy, two of the popular API gateways for cloud native apps are:
* [**Kong**](https://konghq.com/)**-** Bosting latencies of less than a millisecond in a world where speed is of the utmost importance, Kong is a very popular API gateway for connecting microservices.
* [**Tyk**](https://tyk.io/)\- Trusted by giants like CapitalOne, Cisco, and Starbucks, Tyk is a popular API Gateway that boasts impressive performance stats. According to their site, on a simple virtual machine with 2 CPU cores and 2 GB of RAM, their API gateway can handle approximately 2,000 requests per second while keeping latency under 85 milliseconds.
### Service Mesh
As cloud native services scale and the architecture becomes complex, a service mesh can help you handle network-based communication of your system more easily. This [Red Hat article](https://www.redhat.com/en/topics/microservices/what-is-a-service-mesh) provides a good crash course on the topic, but in a nutshell: a service mesh is a network of proxies that takes the burden of communication off of microservices. These proxies are often referred to as “sidecars” because they are effectively attached to, but still discrete from, the microservices themselves.
Some of the most popular service mesh tools are:
* [**Zuul**](https://github.com/Netflix/zuul)\- Another Netflix contribution to the cloud native ecosystem, Zuul is a gateway that provides functionality including monitoring, dynamic routing, and security.
* [**Istio*
* [**Tars**](https://github.com/tarsCloud)\- This RPC framework is based on the name service and the Tars protocol. Today, it supports the Go, C++, NodeJs, PHP, and Java programming languages.
### Service proxy
If you’ve ever had to troubleshoot an application, you’re likely familiar with the difficulty of drilling down to the core of the issue. Is it the app? Is it the network? Such approach may even lead to fingerpointing and make solving problems take longer. One of the core benefits of using a service proxy is being able to decouple network and application problems easily.
Drilling down a bit further, service proxies handle communication between instances in service mesh topologies. Not only do service proxy facilitates network communication, but it also improves observability and facilitates performance tweaks.
Some of the most popular cloud native service proxies are:
* [**Envoy**](https://www.envoyproxy.io/)\- Envoy was created by the Lyft team to serve as a distributed proxy, communication bus, and data plane. The project took inspiration from popular open-source projects like NGINX and HAproxy. It is now used by many respected names in the world of cloud native including SalesForce, Pinterest, AirBnb, and Uber. (were you a little surprised to see Uber using Lyft technology? Score one for the benefits of open source!).
* [**Skipper**](https://opensource.zalando.com/skipper/)**-** Open-source HTTP router and reverse proxy Skipper is designed to handle large amounts of HTTP routes dynamically. It can also serve as a Kubernetes Ingress controller.
* [**NGINX**](https://www.nginx.com/)**-** Much more than just a web server, NGINX provides robust proxy functionality and load balancing. It can also serve as an API gateway…
### API gateway
APIs are an important part of enabling the scalability of cloud native apps. API gateways make API implementation easier by providing a single point of entry for API calls. It makes load balancing, security, and monitoring more effective and simple to manage.
Tyk does a good job of enumerating the benefits of API gateways to microservices in [this piece](https://tyk.io/blog/microservices-api-gateway/). The short version is that API gateways streamline interservice communication, minimize attack surface, and enable service mocking to make testing easier.
There is a bit of an overlap between this and service proxy categories. Apart from NGINX and Envoy, two of the popular API gateways for cloud native apps are:
* [**Kong**](https://konghq.com/)**-** Bosting latencies of less than a millisecond in a world where speed is of the utmost importance, Kong is a very popular API gateway for connecting microservices.
* [**Tyk**](https://tyk.io/)\- Trusted by giants like CapitalOne, Cisco, and Starbucks, Tyk is a popular API Gateway that boasts impressive performance stats. According to their site, on a simple virtual machine with 2 CPU cores and 2 GB of RAM, their API gateway can handle approximately 2,000 requests per second while keeping latency under 85 milliseconds.
### Service Mesh
As cloud native services scale and the architecture becomes complex, a service mesh can help you handle network-based communication of your system more easily. This [Red Hat article](https://www.redhat.com/en/topics/microservices/what-is-a-service-mesh) provides a good crash course on the topic, but in a nutshell: a service mesh is a network of proxies that takes the burden of communication off of microservices. These proxies are often referred to as “sidecars” because they are effectively attached to, but still discrete from, the microservices themselves.
Some of the most popular service mesh tools are:
* [**Zuul**](https://github.com/Netflix/zuul)\- Another Netflix contribution to the cloud native ecosystem, Zuul is a gateway that provides functionality including monitoring, dynamic routing, and security.
* [**Istio*
GitHub
TarsCloud
TarsCloud has 12 repositories available. Follow their code on GitHub.