Secure Kubernetes on Hetzner Cloud with a node driver for Rancher
Hi! I wrote a post on how to deploy Kubernetes to Hetzner Cloud (a cheap and very good cloud provider from Germany) using a node driver for Rancher. The node driver is very convenient, but requires that a firewall is configured separately, which is what I show in this post. Any feedback is much appreciated!
https://vitobotta.com/2020/01/17/secure-kubernetes-hetzner-cloud-rancher-node-driver/
https://redd.it/eq1bbw
@r_devops
Hi! I wrote a post on how to deploy Kubernetes to Hetzner Cloud (a cheap and very good cloud provider from Germany) using a node driver for Rancher. The node driver is very convenient, but requires that a firewall is configured separately, which is what I show in this post. Any feedback is much appreciated!
https://vitobotta.com/2020/01/17/secure-kubernetes-hetzner-cloud-rancher-node-driver/
https://redd.it/eq1bbw
@r_devops
has_many :codes
Secure Kubernetes on Hetzner Cloud with a node driver for Rancher
In this post, we’ll see how to deploy Kubernetes to Hetzner Cloud using a node driver for Rancher. By default, when using this node driver Rancher doesn’t set up any firewall, leaving etcd exposed. Here’s how to fix that while still taking advanta...
Hi r/devops, we recently open sourced a command-line cloud automation tool and would love to get feedback on it. We've been using a version of this tool internally for many years and decided to clean it up and open source it.
Disclaimer: I'm co-CTO at Perfect Sense, the company that built Gyro, and the original author of the tool.
With that out of the way, we'd love to get feedback (good or bad) on Gyro. If you've ever used Terraform or Cloudformation you'll feel right at home with Gyro. It allows you to automate any cloud provider with an API. Currently we have good (but not yet 100% of resources) support for AWS and Azure. We're working to add Google Cloud support.
Gyro evolved from an internal tool we built starting way back in 2014. The initial goal of that tool was to automate creating and updating our infrastructure. As time went on we integrated various parts of our infrastructure into this tool. For example, we used it to execute Chef Solo on our hosts, ssh into hosts, interact with our service discovery, and deploy our code using with blue/green deployment model. We found this "one tool for most of your day-to-day ops activities" to be extremely valuable. This is why we decided to refactor the code, make it more flexible, and open source it so others could do the same.
Why the name Gyro? It's short for Gyroscope which is an essential device that allows airplanes to navigate in the clouds. I love aviation. Also, if you read "gyro" and thought of a greek sandwich, you're not the first, definitely won't be the last, haha. That's ok though, gyro sandwiches are super yummy. :)
There are a few things that make Gyro different from similar tools. I'll try to highlight those here but encourage you to read either the developer documentation site ([https://gyro.dev](https://gyro.dev/)) or if you want a higher level quick hits page read [https://getgyro.io/introducing-gyro](https://getgyro.io/introducing-gyro).
**1) Simple: Configuration Syntax (aka the Gyro Configuration Language)**
I know, I know, another Yet Another DSL. Originally we wrote this using YAML but we wanted clean (and limited) logic in our configuration and YAML didn't really fit the bill. We tried a few different language based internal DSLs such as Kotlin, Groovy, and even TCL (don't hate) but the language always bled through and didn't feel right.
Ultimately we decided to design our own simplified, but powerful, language. This allowed us to have greater control over scoping rules, control structures, and runtime execution. Building a tool that generates an internal graph of resources is extremely complex and not having complete control over what is happening during execution makes it much more complex.
[https://gyro.dev/guides/language/](https://gyro.dev/guides/language/)
**2) Logic: Control Structures**
We're well aware of the ongoing debate of whether allowing logic (control structures) in a configuration is a good thing or not. We believe it is as long as you provide reasonable limits. We tried to strike a balance between no logic and a full programming language. To start with we've implemented two control structures we think are most important for configuration logic, "if" and "for".
Control structures are actually an extension of Gyro rather than baked into the language parser.
[https://gyro.dev/guides/language/control-structures.html](https://gyro.dev/guides/language/control-structures.html)
**3) Workflows**
We think this is huge. What we mean when we say workflows is the ability to define transition stages for complex cloud infrastructure updates. Blue/green deployments are a good example of this. With Gyro you can define a stage to create a new load balancer and new virtual machines with your updated code. After this stage executes you can either prompt the user to continue allowing them to validate the new deployment or you can automate it with simple logic. The next stage you define can either drop those new machines into the load balancer taking traffic or you can flip DNS depending on how you like to do blue/green. If at any poi
Disclaimer: I'm co-CTO at Perfect Sense, the company that built Gyro, and the original author of the tool.
With that out of the way, we'd love to get feedback (good or bad) on Gyro. If you've ever used Terraform or Cloudformation you'll feel right at home with Gyro. It allows you to automate any cloud provider with an API. Currently we have good (but not yet 100% of resources) support for AWS and Azure. We're working to add Google Cloud support.
Gyro evolved from an internal tool we built starting way back in 2014. The initial goal of that tool was to automate creating and updating our infrastructure. As time went on we integrated various parts of our infrastructure into this tool. For example, we used it to execute Chef Solo on our hosts, ssh into hosts, interact with our service discovery, and deploy our code using with blue/green deployment model. We found this "one tool for most of your day-to-day ops activities" to be extremely valuable. This is why we decided to refactor the code, make it more flexible, and open source it so others could do the same.
Why the name Gyro? It's short for Gyroscope which is an essential device that allows airplanes to navigate in the clouds. I love aviation. Also, if you read "gyro" and thought of a greek sandwich, you're not the first, definitely won't be the last, haha. That's ok though, gyro sandwiches are super yummy. :)
There are a few things that make Gyro different from similar tools. I'll try to highlight those here but encourage you to read either the developer documentation site ([https://gyro.dev](https://gyro.dev/)) or if you want a higher level quick hits page read [https://getgyro.io/introducing-gyro](https://getgyro.io/introducing-gyro).
**1) Simple: Configuration Syntax (aka the Gyro Configuration Language)**
I know, I know, another Yet Another DSL. Originally we wrote this using YAML but we wanted clean (and limited) logic in our configuration and YAML didn't really fit the bill. We tried a few different language based internal DSLs such as Kotlin, Groovy, and even TCL (don't hate) but the language always bled through and didn't feel right.
Ultimately we decided to design our own simplified, but powerful, language. This allowed us to have greater control over scoping rules, control structures, and runtime execution. Building a tool that generates an internal graph of resources is extremely complex and not having complete control over what is happening during execution makes it much more complex.
[https://gyro.dev/guides/language/](https://gyro.dev/guides/language/)
**2) Logic: Control Structures**
We're well aware of the ongoing debate of whether allowing logic (control structures) in a configuration is a good thing or not. We believe it is as long as you provide reasonable limits. We tried to strike a balance between no logic and a full programming language. To start with we've implemented two control structures we think are most important for configuration logic, "if" and "for".
Control structures are actually an extension of Gyro rather than baked into the language parser.
[https://gyro.dev/guides/language/control-structures.html](https://gyro.dev/guides/language/control-structures.html)
**3) Workflows**
We think this is huge. What we mean when we say workflows is the ability to define transition stages for complex cloud infrastructure updates. Blue/green deployments are a good example of this. With Gyro you can define a stage to create a new load balancer and new virtual machines with your updated code. After this stage executes you can either prompt the user to continue allowing them to validate the new deployment or you can automate it with simple logic. The next stage you define can either drop those new machines into the load balancer taking traffic or you can flip DNS depending on how you like to do blue/green. If at any poi
nt things don't look right Gyro can roll back to a previous stage.
This functionality has been extremely important for us to be able to allow people to do deployments and to quickly roll back should anything go wrong.
[https://gyro.dev/guides/workflows/](https://gyro.dev/guides/workflows/)
**4) Extensibility**
Aside from workflows, extensibility is what I'm most excited about in Gyro. We've included a number of ways you can extend Gyro with plugins.
* Add new commands ([https://gyro.dev/extending/commands/](https://gyro.dev/extending/commands/), [https://github.com/perfectsense/gyro-ssh-plugin](https://github.com/perfectsense/gyro-ssh-plugin))
* Add new language functionality, aka Directives ([https://gyro.dev/extending/directive/](https://gyro.dev/extending/directive/))
* Add custom variable resolvers ([https://gyro.dev/extending/resolver/](https://gyro.dev/extending/resolver/))
* Add custom event hooks (undocumented) to trigger custom logic when various things happen such as a resource is created or updated
TL;DR Check out [https://gyro.dev](https://gyro.dev/) or [https://github.com/perfectsense/gyro](https://github.com/perfectsense/gyro), this the developer/user documentation for Gyro and the main repo. It should give you a good idea of what Gyro is and how it works!
https://redd.it/eq023h
@r_devops
This functionality has been extremely important for us to be able to allow people to do deployments and to quickly roll back should anything go wrong.
[https://gyro.dev/guides/workflows/](https://gyro.dev/guides/workflows/)
**4) Extensibility**
Aside from workflows, extensibility is what I'm most excited about in Gyro. We've included a number of ways you can extend Gyro with plugins.
* Add new commands ([https://gyro.dev/extending/commands/](https://gyro.dev/extending/commands/), [https://github.com/perfectsense/gyro-ssh-plugin](https://github.com/perfectsense/gyro-ssh-plugin))
* Add new language functionality, aka Directives ([https://gyro.dev/extending/directive/](https://gyro.dev/extending/directive/))
* Add custom variable resolvers ([https://gyro.dev/extending/resolver/](https://gyro.dev/extending/resolver/))
* Add custom event hooks (undocumented) to trigger custom logic when various things happen such as a resource is created or updated
TL;DR Check out [https://gyro.dev](https://gyro.dev/) or [https://github.com/perfectsense/gyro](https://github.com/perfectsense/gyro), this the developer/user documentation for Gyro and the main repo. It should give you a good idea of what Gyro is and how it works!
https://redd.it/eq023h
@r_devops
Run pre-deployment policy checks on Terraform using Open Policy Agent with Regula
Hi everyone. We just open sourced Regula, a tool that evaluates Terraform infrastructure-as-code for policy prior to deployment. Regula includes a library of rules written in Rego, the policy language used by the Open Policy Agent project. Where relevant, we’ve mapped Regula policies to the CIS AWS Foundations Benchmark, and we'll be adding more rules as we go.
There are two parts to Regula. The first is a shell script that generates a Terraform plan in JSON format, ready for consumption by OPA.
The second part is a Rego framework that:
* Merges resource info from planned\_values
and configuration in the terraform plan into a more conveniently accessible format.
* Looks for rules and executes them.
* Creates a report with the results of all rules and a control mapping in the output.
We hope you give it a try and would love to hear what you think. [https://github.com/fugue/regula](https://github.com/fugue/regula)
https://redd.it/eq2s6m
@r_devops
Hi everyone. We just open sourced Regula, a tool that evaluates Terraform infrastructure-as-code for policy prior to deployment. Regula includes a library of rules written in Rego, the policy language used by the Open Policy Agent project. Where relevant, we’ve mapped Regula policies to the CIS AWS Foundations Benchmark, and we'll be adding more rules as we go.
There are two parts to Regula. The first is a shell script that generates a Terraform plan in JSON format, ready for consumption by OPA.
The second part is a Rego framework that:
* Merges resource info from planned\_values
and configuration in the terraform plan into a more conveniently accessible format.
* Looks for rules and executes them.
* Creates a report with the results of all rules and a control mapping in the output.
We hope you give it a try and would love to hear what you think. [https://github.com/fugue/regula](https://github.com/fugue/regula)
https://redd.it/eq2s6m
@r_devops
GitHub
GitHub - fugue/regula: Regula checks infrastructure as code templates (Terraform, CloudFormation, k8s manifests) for AWS, Azure…
Regula checks infrastructure as code templates (Terraform, CloudFormation, k8s manifests) for AWS, Azure, Google Cloud, and Kubernetes security and compliance using Open Policy Agent/Rego - fugue/r...
20 years since Y2K
Thought this article might be of interest--it looks back over a couple of big tech incidents and discusses what we've learned: [https://www.transposit.com/blog/2020.01.15-left-pad-heartbleed-and-devops/](https://www.transposit.com/blog/2020.01.15-left-pad-heartbleed-and-devops/)
(full disclosure, a co-worker wrote it.)
https://redd.it/eq3ot9
@r_devops
Thought this article might be of interest--it looks back over a couple of big tech incidents and discusses what we've learned: [https://www.transposit.com/blog/2020.01.15-left-pad-heartbleed-and-devops/](https://www.transposit.com/blog/2020.01.15-left-pad-heartbleed-and-devops/)
(full disclosure, a co-worker wrote it.)
https://redd.it/eq3ot9
@r_devops
Transposit
20 Years Since Y2K: left-pad, Heartbleed, and DevOps
Our software is full of ticking time bombs. This post is an exploration of incidents that changed how we operate since Y2K and how modern IT operations helped stop the bleeding.
Deployments sharing helm charts best practices
What are the best practices as far as deployments sharing helm charts?
The company I started working for is using a single huge, monolithic chart for all their deployments (100's) and that strikes me as very wrong.
Equally, having a helm chart for every deployment/repo has the downside of creating difficulties when you want to change the deployment for a subset of microservices that all share a common pattern, which makes using a shared chart attractive.
So in general, what should be done?
https://redd.it/eq0ecx
@r_devops
What are the best practices as far as deployments sharing helm charts?
The company I started working for is using a single huge, monolithic chart for all their deployments (100's) and that strikes me as very wrong.
Equally, having a helm chart for every deployment/repo has the downside of creating difficulties when you want to change the deployment for a subset of microservices that all share a common pattern, which makes using a shared chart attractive.
So in general, what should be done?
https://redd.it/eq0ecx
@r_devops
reddit
Deployments sharing helm charts best practices
What are the best practices as far as deployments sharing helm charts? The company I started working for is using a single huge, monolithic...
Spin up temporary site for each pull request
I have been looking for a way to create a temporary environment (azure app service) for my application (aspnetcore + react) for each and every pull request that is made against master, so developers can review the code and testers can test the features before they are actually merged into master and Azure Devops sends the changes towards production.
The reasoning behind this is to make sure we are always able to deploy to production (preferrably automatically) without risking untested features are blocking tested features. I would also like us to be able to use only one pull request, and not first a PR for each feature against dev, then against test and then against master when it is ready for production.
Am I insane?
I can't seem to find anyone that spins up new app services on azure from their ci/cd system. We are using Azure DevOps for build and release pipelines now, everything is hosted on Azure. Any suggestions are appreciated.
https://redd.it/epz3bm
@r_devops
I have been looking for a way to create a temporary environment (azure app service) for my application (aspnetcore + react) for each and every pull request that is made against master, so developers can review the code and testers can test the features before they are actually merged into master and Azure Devops sends the changes towards production.
The reasoning behind this is to make sure we are always able to deploy to production (preferrably automatically) without risking untested features are blocking tested features. I would also like us to be able to use only one pull request, and not first a PR for each feature against dev, then against test and then against master when it is ready for production.
Am I insane?
I can't seem to find anyone that spins up new app services on azure from their ci/cd system. We are using Azure DevOps for build and release pipelines now, everything is hosted on Azure. Any suggestions are appreciated.
https://redd.it/epz3bm
@r_devops
reddit
Spin up temporary site for each pull request
I have been looking for a way to create a temporary environment (azure app service) for my application (aspnetcore + react) for each and every...
sonar-cxx: flags to prevent a rule to be triggered
Hi.
I was searching for a possibility to deactivate some rules for a particular piece of code when using [sonar-cxx](https://github.com/SonarOpenCommunity/sonar-cxx).
Here is an example. The `cxx:TooLongLine` is a controversial rule that triggers when a line's length is > 160 characters, but I try to abide by it whenever possible. But in some rare cases, it can be troublesome.
In my latest project, I have to compose web pages using pieces of HTML, CSS and SVG pictures in particular, all of this using C++. You guessed it: SVG pictures produce very long `const char *` strings, and there are a few of them.
So yes, one of my header files has plenty of `cxx:TooLongLine` violations, and I'm searching for a way to deactivate the rule, *one line at a time*.
I remember, back when I was coding in Java, that there was a inline flag that allowed to deactivate a rule for a specific piece of code, for example (might be wrong):
```
//@deactivate(java:TheRuleToDeactivate)
```
So my question is: is there a way to do something similar with sonar-cxx?
Thanks a lot for your help.
https://redd.it/eq1a0b
@r_devops
Hi.
I was searching for a possibility to deactivate some rules for a particular piece of code when using [sonar-cxx](https://github.com/SonarOpenCommunity/sonar-cxx).
Here is an example. The `cxx:TooLongLine` is a controversial rule that triggers when a line's length is > 160 characters, but I try to abide by it whenever possible. But in some rare cases, it can be troublesome.
In my latest project, I have to compose web pages using pieces of HTML, CSS and SVG pictures in particular, all of this using C++. You guessed it: SVG pictures produce very long `const char *` strings, and there are a few of them.
So yes, one of my header files has plenty of `cxx:TooLongLine` violations, and I'm searching for a way to deactivate the rule, *one line at a time*.
I remember, back when I was coding in Java, that there was a inline flag that allowed to deactivate a rule for a specific piece of code, for example (might be wrong):
```
//@deactivate(java:TheRuleToDeactivate)
```
So my question is: is there a way to do something similar with sonar-cxx?
Thanks a lot for your help.
https://redd.it/eq1a0b
@r_devops
GitHub
GitHub - SonarOpenCommunity/sonar-cxx: SonarQube C++ Community plugin (cxx plugin): This plugin adds C++ support to SonarQube with…
SonarQube C++ Community plugin (cxx plugin): This plugin adds C++ support to SonarQube with the focus on integration of existing C++ tools. - SonarOpenCommunity/sonar-cxx
So, how are you guys releasing your artifacts?
Similar to another enquiry I've made in the past about what Artifactory software you are using, but a more general information gathering one this time - how are you releasing artifacts and getting them to your customers?
Our shop isn't a full SaaS (yet), so when we do CI/CD we push changes to our own live/customer environments where we can develop against. Our artifacts strategy at the moment is to release the files on our Jira Board in the announcements page to our respective customers. I want to try and improve this as it's been there since god knows when and it feels like a better process should exist.
Our customers take the artifacts we release and give them to there own QA team who do there own CD process after they've quality checked and raised any tickets we might have missed, followed by pushing them to there own live environment we don't control.
We have a container registry where I've been trying to push changes into our docker image and have our environments refresh that everytime there's a change to our image, but it's just experimental and I want to know how everyone else is doing it.
https://redd.it/epxmjt
@r_devops
Similar to another enquiry I've made in the past about what Artifactory software you are using, but a more general information gathering one this time - how are you releasing artifacts and getting them to your customers?
Our shop isn't a full SaaS (yet), so when we do CI/CD we push changes to our own live/customer environments where we can develop against. Our artifacts strategy at the moment is to release the files on our Jira Board in the announcements page to our respective customers. I want to try and improve this as it's been there since god knows when and it feels like a better process should exist.
Our customers take the artifacts we release and give them to there own QA team who do there own CD process after they've quality checked and raised any tickets we might have missed, followed by pushing them to there own live environment we don't control.
We have a container registry where I've been trying to push changes into our docker image and have our environments refresh that everytime there's a change to our image, but it's just experimental and I want to know how everyone else is doing it.
https://redd.it/epxmjt
@r_devops
reddit
So, how are you guys releasing your artifacts?
Similar to another enquiry I've made in the past about what Artifactory software you are using, but a more general information gathering one this...
Scaling docker container horizontally benefits if on same host
What are the benefits of scaling a service horizontally when they run on the same host? Say two instances of a microservice instead of one.
Wouldn't the resources that could have been used for the single microservice just get split up and used for the second, giving no real benefit. Like instead of having one car go 100km/h now there are two cars each going 50km/h ?
https://redd.it/eq8n66
@r_devops
What are the benefits of scaling a service horizontally when they run on the same host? Say two instances of a microservice instead of one.
Wouldn't the resources that could have been used for the single microservice just get split up and used for the second, giving no real benefit. Like instead of having one car go 100km/h now there are two cars each going 50km/h ?
https://redd.it/eq8n66
@r_devops
reddit
Scaling docker container horizontally benefits if on same host
What are the benefits of scaling a service horizontally when they run on the same host? Say two instances of a microservice instead of...
How to automatically ensure pipeline compliance
So, let's say that we want to ensure that all software pipelines put the code through SAST testing (or any other important step), how do we as DevOps engineers ensure that the pipelines for 100 or 1000+ all confirm to business requirements? What's to stop one developer removing a crucial step in the pipeline and another approving this and it therefore not going noticed for a very long time? or, possibly more likely, what's to stop a developer taking a template pipeline and simply removing a step on the initial pipeline commit?
For context we are currently using gitlab CI but are looking at potentially moving to cloud build, I hope the answer to this is platform agnostic (but assume it won't be)
https://redd.it/epyi34
@r_devops
So, let's say that we want to ensure that all software pipelines put the code through SAST testing (or any other important step), how do we as DevOps engineers ensure that the pipelines for 100 or 1000+ all confirm to business requirements? What's to stop one developer removing a crucial step in the pipeline and another approving this and it therefore not going noticed for a very long time? or, possibly more likely, what's to stop a developer taking a template pipeline and simply removing a step on the initial pipeline commit?
For context we are currently using gitlab CI but are looking at potentially moving to cloud build, I hope the answer to this is platform agnostic (but assume it won't be)
https://redd.it/epyi34
@r_devops
reddit
How to automatically ensure pipeline compliance
So, let's say that we want to ensure that all software pipelines put the code through SAST testing (or any other important step), how do we as...
GuardDuty: The request is rejected because the caller is not authorized to call this API
I have an AWS GuardDuty stack. Inside I have one resource:
IPSet:
Type: "AWS::GuardDuty::IPSet"
Properties:
Activate: True
DetectorId: !Ref Detector
Format: TXT
Location: s3://mybucket/ipset.txt
Name: MyIPSet
I create the bucket:
▶ aws s3 mb s3://mybucket/
make_bucket: mybucket
List stuff in the bucket etc, all works fine.
On deploying the stack:
{
"StackId": "arn:aws:cloudformation:us-east-1:111111111111:stack/test-stack/28801e90-390b-11ea-9d33-0ab1b9b87aaa",
"EventId": "ThreatIntelSet-CREATE_FAILED-2020-01-17T09:25:10.136Z",
"ResourceStatus": "CREATE_FAILED",
"ResourceType": "AWS::GuardDuty::ThreatIntelSet",
"Timestamp": "2020-01-17T09:25:10.136Z",
"ResourceStatusReason": "The request is rejected because the caller is not authorized to call this API. (Service: AmazonGuardDuty; Status Code: 400; Error Code: InternalServerErrorException; Request ID: b70503ba-3598-4f3e-97be-bd6ebaf58b94)",
"StackName": "test-stack",
"ResourceProperties": "{\"Format\":\"TXT\",\"Activate\":\"true\",\"DetectorId\":\"4eb7d968df4f4f7db0be0711b0c027d3\",\"Location\":\"s3://mybucket/threatset.txt\",\"Name\":\"fooorg-audit-ThreatIntelSet\"}",
"PhysicalResourceId": "",
"LogicalResourceId": "ThreatIntelSet"
}
I've seen this answer here in Stack Overflow
[https://stackoverflow.com/questions/48404164/accessdeniedexception-caller-is-not-authorized-to-call-api-using-aws-guardduty/57725109#57725109](https://stackoverflow.com/questions/48404164/accessdeniedexception-caller-is-not-authorized-to-call-api-using-aws-guardduty/57725109#57725109)
But no use. Very frustrated! Any thoughts on what else it could be?
https://redd.it/epy9mp
@r_devops
I have an AWS GuardDuty stack. Inside I have one resource:
IPSet:
Type: "AWS::GuardDuty::IPSet"
Properties:
Activate: True
DetectorId: !Ref Detector
Format: TXT
Location: s3://mybucket/ipset.txt
Name: MyIPSet
I create the bucket:
▶ aws s3 mb s3://mybucket/
make_bucket: mybucket
List stuff in the bucket etc, all works fine.
On deploying the stack:
{
"StackId": "arn:aws:cloudformation:us-east-1:111111111111:stack/test-stack/28801e90-390b-11ea-9d33-0ab1b9b87aaa",
"EventId": "ThreatIntelSet-CREATE_FAILED-2020-01-17T09:25:10.136Z",
"ResourceStatus": "CREATE_FAILED",
"ResourceType": "AWS::GuardDuty::ThreatIntelSet",
"Timestamp": "2020-01-17T09:25:10.136Z",
"ResourceStatusReason": "The request is rejected because the caller is not authorized to call this API. (Service: AmazonGuardDuty; Status Code: 400; Error Code: InternalServerErrorException; Request ID: b70503ba-3598-4f3e-97be-bd6ebaf58b94)",
"StackName": "test-stack",
"ResourceProperties": "{\"Format\":\"TXT\",\"Activate\":\"true\",\"DetectorId\":\"4eb7d968df4f4f7db0be0711b0c027d3\",\"Location\":\"s3://mybucket/threatset.txt\",\"Name\":\"fooorg-audit-ThreatIntelSet\"}",
"PhysicalResourceId": "",
"LogicalResourceId": "ThreatIntelSet"
}
I've seen this answer here in Stack Overflow
[https://stackoverflow.com/questions/48404164/accessdeniedexception-caller-is-not-authorized-to-call-api-using-aws-guardduty/57725109#57725109](https://stackoverflow.com/questions/48404164/accessdeniedexception-caller-is-not-authorized-to-call-api-using-aws-guardduty/57725109#57725109)
But no use. Very frustrated! Any thoughts on what else it could be?
https://redd.it/epy9mp
@r_devops
Stack Overflow
AccessDeniedException (caller is not authorized to call API) using AWS GuardDuty
Aws lambda and CLI both returned "InternalServerErrorException: An error occurred (InternalServerErrorException) when calling the UpdateThreatIntelSet operation: The request is rejected because the
Does Microsoft have their own implementation of microservices?
[https://en.wikipedia.org/wiki/Microservices#A\_comparison\_of\_platforms](https://en.wikipedia.org/wiki/Microservices#A_comparison_of_platforms) shows two implementations of microservices: Spring Cloud and Kubernetes ecosystem.
Does Microsoft have their own implementation of microservices too? [https://dotnet.microsoft.com/learn/aspnet/microservices-architecture](https://dotnet.microsoft.com/learn/aspnet/microservices-architecture) seems to mean that they can create a microservice by deploying a service inside a docker container. Maybe use Docker Swarm to coordinate multiple containers?services? Is this Microsoft's platform/implementation of microservice architecture?
https://redd.it/eqb22x
@r_devops
[https://en.wikipedia.org/wiki/Microservices#A\_comparison\_of\_platforms](https://en.wikipedia.org/wiki/Microservices#A_comparison_of_platforms) shows two implementations of microservices: Spring Cloud and Kubernetes ecosystem.
Does Microsoft have their own implementation of microservices too? [https://dotnet.microsoft.com/learn/aspnet/microservices-architecture](https://dotnet.microsoft.com/learn/aspnet/microservices-architecture) seems to mean that they can create a microservice by deploying a service inside a docker container. Maybe use Docker Swarm to coordinate multiple containers?services? Is this Microsoft's platform/implementation of microservice architecture?
https://redd.it/eqb22x
@r_devops
Wikipedia
Microservices
collection of loosely coupled services used to build computer applications
Kafka is no longer the best choice thanks to Apache Pulsar
Recently on HackerNews someone asked why I was moving off of Kafka and I wrote a somewhat lengthy comment.
This inspired me to do a presentation at a meetup about how Apache Pulsar has come to replace Kafka and how today there's not really a reason to be on Kafka unless you've invested into it heavily already.
Here is the talk:
https://youtu.be/jLruEmh3ve0?t=130
https://redd.it/eppwrp
@r_devops
Recently on HackerNews someone asked why I was moving off of Kafka and I wrote a somewhat lengthy comment.
This inspired me to do a presentation at a meetup about how Apache Pulsar has come to replace Kafka and how today there's not really a reason to be on Kafka unless you've invested into it heavily already.
Here is the talk:
https://youtu.be/jLruEmh3ve0?t=130
https://redd.it/eppwrp
@r_devops
YouTube
TAST00: Kafka is no longer king for big data
Many years of experience with Kafka and a lot of pain points. Time to start to seriously consider Pulsar. Speaker: Cory Darby
Application/Service Inventory Query
Between Dev, Test, and Prod we have a large number of On-Premises servers (Physical and VM). Running various applications like databases, web servers etc.
The trouble is working out what is what. For example, I need to make a change to Customer A's web server, I first need to discover the hostname of the web server.
Now I know in AWS we just query the tags of the instance, easy peasy. But how would we go about this for a physical environment?
Would appreciate some guidance on how this could be acheived.
End result: I want to quickly ask some appliction/service where Customer A's servers are, and filter from there.
https://redd.it/epddoj
@r_devops
Between Dev, Test, and Prod we have a large number of On-Premises servers (Physical and VM). Running various applications like databases, web servers etc.
The trouble is working out what is what. For example, I need to make a change to Customer A's web server, I first need to discover the hostname of the web server.
Now I know in AWS we just query the tags of the instance, easy peasy. But how would we go about this for a physical environment?
Would appreciate some guidance on how this could be acheived.
End result: I want to quickly ask some appliction/service where Customer A's servers are, and filter from there.
https://redd.it/epddoj
@r_devops
reddit
Application/Service Inventory Query
Between Dev, Test, and Prod we have a large number of On-Premises servers (Physical and VM). Running various applications like databases, web...
What software engineering methods are monitoring tools like DynaTrace using to get such detailed metrics?
Sorry if this is the wrong sub, was thinking about posting it in /r/programming or something similar, but it strikes me as a devops-y question. How do DynaTrace and other monitoring tools get metrics and info such as:
* Request routes with response times and tracing
* JVM metrics
* Memory, CPU, etc metrics of hosts and containers
* DB queries and other database information
* All kinds of other information we use for analysis, alerting, etc.
I've implemented tools like DynaTrace and Prometheus at work, and I appreciate how useful they are, but I'd like to understand more about the software engineering that goes into building tools like these. How do they get these metrics? What methods are they using to format and store them? I know I can use top or free to check mem/cpu, or run jstat to look at JVM metrics if I'm on a Linux host, but how are these monitoring tools doing it and making the data easy to understand?
​
I realize this might seem kind of broad, or might be a "duh" question to some people, but I'd really like to learn more about this. A lot of the monitoring tools I use are almost like a black box/magic to me, they just work and I want to know what's happening at the code level for them to accomplish what they're doing.
https://redd.it/epcbz5
@r_devops
Sorry if this is the wrong sub, was thinking about posting it in /r/programming or something similar, but it strikes me as a devops-y question. How do DynaTrace and other monitoring tools get metrics and info such as:
* Request routes with response times and tracing
* JVM metrics
* Memory, CPU, etc metrics of hosts and containers
* DB queries and other database information
* All kinds of other information we use for analysis, alerting, etc.
I've implemented tools like DynaTrace and Prometheus at work, and I appreciate how useful they are, but I'd like to understand more about the software engineering that goes into building tools like these. How do they get these metrics? What methods are they using to format and store them? I know I can use top or free to check mem/cpu, or run jstat to look at JVM metrics if I'm on a Linux host, but how are these monitoring tools doing it and making the data easy to understand?
​
I realize this might seem kind of broad, or might be a "duh" question to some people, but I'd really like to learn more about this. A lot of the monitoring tools I use are almost like a black box/magic to me, they just work and I want to know what's happening at the code level for them to accomplish what they're doing.
https://redd.it/epcbz5
@r_devops
reddit
What software engineering methods are monitoring tools like...
Sorry if this is the wrong sub, was thinking about posting it in /r/programming or something similar, but it strikes me as a devops-y question. ...
Learning to DevOps with RPi?
Hi all,
Been following this thread for a few months now and it's been great, especially the monthly Getting into DevOps thread. I have just finished my first week as a DevOps engineer. I've come from being a developer so a lot of the Linux and networking stuff is completely new to me. Been an intense first week!
Was just wondering... I have a raspberry pi 4gb I bought a while back that I've been meaning to do something with, but was wondering if I could get some suggestions as to what kind of things I could use it for? Specifically to help me learn the fundamentals of DevOps and improve my overall skills.
To be more specific, my day to day includes server monitoring, testing, deployment of new servers and infrastructure, etc. General DevOps-y stuff!
Any suggestions most welcome 😁
https://redd.it/eqchuo
@r_devops
Hi all,
Been following this thread for a few months now and it's been great, especially the monthly Getting into DevOps thread. I have just finished my first week as a DevOps engineer. I've come from being a developer so a lot of the Linux and networking stuff is completely new to me. Been an intense first week!
Was just wondering... I have a raspberry pi 4gb I bought a while back that I've been meaning to do something with, but was wondering if I could get some suggestions as to what kind of things I could use it for? Specifically to help me learn the fundamentals of DevOps and improve my overall skills.
To be more specific, my day to day includes server monitoring, testing, deployment of new servers and infrastructure, etc. General DevOps-y stuff!
Any suggestions most welcome 😁
https://redd.it/eqchuo
@r_devops
reddit
Learning to DevOps with RPi?
Hi all, Been following this thread for a few months now and it's been great, especially the monthly Getting into DevOps thread. I have just...
Is this an opposite ways to the microservice architecture?
Martin Fowler said in his Patterns for Enterprise application Architecture 2004 that
> while many things can be made transparent in distributed objects, performance isn't usually one of them.
Although our prototypical architect was distributing bjects the way he was for performance reasons, in act his design will usually cripple performance, make the system much harder to build and deploy, or, usually, do both.
> we get to my First Law of Distributed Object Design: Don't
distribute your objects!
> How, then, do you effectively use multiple processors? In most cases the way to go is clustering (see Figure 7.2). Put all the classes into a single process and then run multiple copies of that process on the various nodes. That way each process uses local calls to get the job done and
thus does things faster. You can also use fine-grained interfaces for all the classes within the process and thus get better maintainability with a simpler programming model.
Is it to prefer a big program and multiple replica processes running the program?
In microservice architecture, is it to partition a big program into multiple small programs, and run them in different processes?
Are the above two ways of architecture opposite to each other?
Thanks.
https://redd.it/eqb6vz
@r_devops
Martin Fowler said in his Patterns for Enterprise application Architecture 2004 that
> while many things can be made transparent in distributed objects, performance isn't usually one of them.
Although our prototypical architect was distributing bjects the way he was for performance reasons, in act his design will usually cripple performance, make the system much harder to build and deploy, or, usually, do both.
> we get to my First Law of Distributed Object Design: Don't
distribute your objects!
> How, then, do you effectively use multiple processors? In most cases the way to go is clustering (see Figure 7.2). Put all the classes into a single process and then run multiple copies of that process on the various nodes. That way each process uses local calls to get the job done and
thus does things faster. You can also use fine-grained interfaces for all the classes within the process and thus get better maintainability with a simpler programming model.
Is it to prefer a big program and multiple replica processes running the program?
In microservice architecture, is it to partition a big program into multiple small programs, and run them in different processes?
Are the above two ways of architecture opposite to each other?
Thanks.
https://redd.it/eqb6vz
@r_devops
reddit
Is this an opposite ways to the microservice architecture?
Martin Fowler said in his Patterns for Enterprise application Architecture 2004 that > while many things can be made...
Help needed with Canonical MAAS CLI
I've been trying to create a boot-source-selection for CentOS in MAAS, but MAAS spits out not found as stdout.
I only want to use MAAS CLI to select CentOS7 image as well as Ubuntu 18.04 ( which already is selected and imported ).
https://redd.it/eq9k5v
@r_devops
I've been trying to create a boot-source-selection for CentOS in MAAS, but MAAS spits out not found as stdout.
I only want to use MAAS CLI to select CentOS7 image as well as Ubuntu 18.04 ( which already is selected and imported ).
https://redd.it/eq9k5v
@r_devops
reddit
Help needed with Canonical MAAS CLI
I've been trying to create a boot-source-selection for CentOS in MAAS, but MAAS spits out not found as stdout. I only want to use MAAS CLI to...
Implementing DevOps through GitHub Actions for Xamarin iOS & Android
This sub reddit has some awesome posts from the past:
[AWS Solution architect exam and notes](https://www.reddit.com/r/devops/comments/ckpbhl/just_passed_the_aws_solutions_architect_exam/), [Getting Started with DevOps](https://www.reddit.com/r/devops/comments/aqc8cj/interested_in_getting_into_devops_start_here/), [DevOps Interview Questions](https://www.reddit.com/r/devops/comments/dj9ffy/a_collection_of_devops_interview_questions/)
For years DevOps has intrigued me. I always wanted to apply DevOps in mobile apps, and GitHub Actions made it really accessible for me. Once GitHub actions started supporting MacOS runners and "Xamarin", I decided to dive in and figure it out. But I wasn't able to figure it out, getting my most downvoted StackOverflow question, and also no proper response on the GitHub community forums.
After months of trying, I was able to finally get the GitHub Actions working with Xamarin Android & iOS solutions! It helped me learn a lot more about DevOps in general.
I wrote an article on what I learnt, with a video walkthrough, and it became really popular! If you guys are interested
[https://levelup.gitconnected.com/using-github-actions-with-ios-and-android-xamarin-apps-693a93b48a61](https://levelup.gitconnected.com/using-github-actions-with-ios-and-android-xamarin-apps-693a93b48a61) (Unmetered link: [bit.ly/30crWpc](https://bit.ly/30crWpc))
The article was curated by Medium in their "Programming", "iOS Dev" and "Android Dev" categories. In all this, I became more optimistic of the future realized that there's still a divide in Software Development and DevOps, that's continuing to shrink, and there's also a huge demand for developers to learn and improve!
https://redd.it/eq5vg5
@r_devops
This sub reddit has some awesome posts from the past:
[AWS Solution architect exam and notes](https://www.reddit.com/r/devops/comments/ckpbhl/just_passed_the_aws_solutions_architect_exam/), [Getting Started with DevOps](https://www.reddit.com/r/devops/comments/aqc8cj/interested_in_getting_into_devops_start_here/), [DevOps Interview Questions](https://www.reddit.com/r/devops/comments/dj9ffy/a_collection_of_devops_interview_questions/)
For years DevOps has intrigued me. I always wanted to apply DevOps in mobile apps, and GitHub Actions made it really accessible for me. Once GitHub actions started supporting MacOS runners and "Xamarin", I decided to dive in and figure it out. But I wasn't able to figure it out, getting my most downvoted StackOverflow question, and also no proper response on the GitHub community forums.
After months of trying, I was able to finally get the GitHub Actions working with Xamarin Android & iOS solutions! It helped me learn a lot more about DevOps in general.
I wrote an article on what I learnt, with a video walkthrough, and it became really popular! If you guys are interested
[https://levelup.gitconnected.com/using-github-actions-with-ios-and-android-xamarin-apps-693a93b48a61](https://levelup.gitconnected.com/using-github-actions-with-ios-and-android-xamarin-apps-693a93b48a61) (Unmetered link: [bit.ly/30crWpc](https://bit.ly/30crWpc))
The article was curated by Medium in their "Programming", "iOS Dev" and "Android Dev" categories. In all this, I became more optimistic of the future realized that there's still a divide in Software Development and DevOps, that's continuing to shrink, and there's also a huge demand for developers to learn and improve!
https://redd.it/eq5vg5
@r_devops
reddit
Just passed the AWS solutions architect exam, wanted to share my...
[Here](https://github.com/SkullTech/aws-solutions-architect-associate-notes) is a Github repo I created which contains all of my notes, divided...
What is Serverless ?
Welcome everyone,
In this video, we will talk about serverless which is one of the most widely used paradigms in the world of cloud computing.
Don't hesitate to share your thoughts.
PS: it's an educative video
Thanks !!
https://youtu.be/9mZV2ZJBbHM
https://redd.it/eq77az
@r_devops
Welcome everyone,
In this video, we will talk about serverless which is one of the most widely used paradigms in the world of cloud computing.
Don't hesitate to share your thoughts.
PS: it's an educative video
Thanks !!
https://youtu.be/9mZV2ZJBbHM
https://redd.it/eq77az
@r_devops
YouTube
What is Serverless ?
Hi everyone, In this video we will discuss about serverless architecture. Enjoy !! This presentation is inspired by Wes Gruver's course. Track : Syn Cole - F...