Can you save battery by running Docker remotely?
My hypothesis is that if we could use a remote server for Docker builds, we could save battery. Let's look at a few ways we could do that, then try one of them out and finally review the experience.
[https://blog.alexellis.io/save-your-battery-with-docker-over-ssh/](https://blog.alexellis.io/save-your-battery-with-docker-over-ssh/)
https://redd.it/e02r1e
@r_devops
My hypothesis is that if we could use a remote server for Docker builds, we could save battery. Let's look at a few ways we could do that, then try one of them out and finally review the experience.
[https://blog.alexellis.io/save-your-battery-with-docker-over-ssh/](https://blog.alexellis.io/save-your-battery-with-docker-over-ssh/)
https://redd.it/e02r1e
@r_devops
Alex Ellis' Blog
Can Docker over ssh save your battery from running out?
Many developers run a virtual machine on their MacBook, just so that they can build Docker images, but what if there was another way? Find out how SSH may help
Setting up logging and monitoring infrastructure
Hi,
I wish to set up infrastructure mentioned in title, but at this point I am a little bit confused by variety of tools and how they can be connected in different ways.
My constraints:
1.low cost and low maintenance effort (I can spend some time to set it up right but I might not have enough time to take care of it when it suddenly crashes too often).
2. Scalable
For logging I was thinking about:
Client -> Apache Kafka -> Graylog +Grafana
For monitoring:
Client -> Telegraf -> Apache kafka (the same as before) -> Telegraf (again?) -> Prometheus (+ Grafana?)
I would like to be able to create some alerts) notification in future as well as add some custom metrics as needed.
Do I get this right or some connections are not possible/duplicate each other? Any advice, alternatives, as well as info about any of mentioned tools is very welcome.
https://redd.it/e0ilwe
@r_devops
Hi,
I wish to set up infrastructure mentioned in title, but at this point I am a little bit confused by variety of tools and how they can be connected in different ways.
My constraints:
1.low cost and low maintenance effort (I can spend some time to set it up right but I might not have enough time to take care of it when it suddenly crashes too often).
2. Scalable
For logging I was thinking about:
Client -> Apache Kafka -> Graylog +Grafana
For monitoring:
Client -> Telegraf -> Apache kafka (the same as before) -> Telegraf (again?) -> Prometheus (+ Grafana?)
I would like to be able to create some alerts) notification in future as well as add some custom metrics as needed.
Do I get this right or some connections are not possible/duplicate each other? Any advice, alternatives, as well as info about any of mentioned tools is very welcome.
https://redd.it/e0ilwe
@r_devops
reddit
Setting up logging and monitoring infrastructure
Hi, I wish to set up infrastructure mentioned in title, but at this point I am a little bit confused by variety of tools and how they can be...
AWS ElasticSearch Kibana
How do I access Kibana PUBLICLY ( with login ) in AWS Elasticsearch ?
https://redd.it/e0kgxa
@r_devops
How do I access Kibana PUBLICLY ( with login ) in AWS Elasticsearch ?
https://redd.it/e0kgxa
@r_devops
reddit
AWS ElasticSearch Kibana
How do I access Kibana PUBLICLY ( with login ) in AWS Elasticsearch ?
[Packer] - Automatically configure network settings for VirtualBox images (*.ova) ?
I am setting up an \*.ova" VirtualBox image with Packer and later provide it with the necessary confiugrations over ansible. However, configuring the network adapters seem to be an absolute pain. By default when nothing is explicitly specified in my template I can not reach the guest from the host. So, I tried adding a bridged adapter with vboxmanage.
However, now Ansible is not able to connect over SSH anymore to do the configurations.
How can I ensure that the network adapters are configured in a way that I can always reach the guest from the host (all via the packer template, that is) ?
Hope to get some hints, as I am relatively new to the subject.
https://redd.it/e0k5ji
@r_devops
I am setting up an \*.ova" VirtualBox image with Packer and later provide it with the necessary confiugrations over ansible. However, configuring the network adapters seem to be an absolute pain. By default when nothing is explicitly specified in my template I can not reach the guest from the host. So, I tried adding a bridged adapter with vboxmanage.
However, now Ansible is not able to connect over SSH anymore to do the configurations.
How can I ensure that the network adapters are configured in a way that I can always reach the guest from the host (all via the packer template, that is) ?
Hope to get some hints, as I am relatively new to the subject.
https://redd.it/e0k5ji
@r_devops
reddit
[Packer] - Automatically configure network settings for VirtualBox...
I am setting up an \*.ova" VirtualBox image with Packer and later provide it with the necessary confiugrations over ansible. However, configuring...
Vulnerability Management Tools
What are people using for their Cloud hosted VM's in term of vulnerability management and scanning?
I've recently been look at Qualys ,RAPID7 and Tenable
https://redd.it/dyli9u
@r_devops
What are people using for their Cloud hosted VM's in term of vulnerability management and scanning?
I've recently been look at Qualys ,RAPID7 and Tenable
https://redd.it/dyli9u
@r_devops
reddit
Vulnerability Management Tools
What are people using for their Cloud hosted VM's in term of vulnerability management and scanning? I've recently been look at Qualys ,RAPID7 and...
Sending message to all service instances.
Hello, I have a situation where a message has to be sent to all the instances of the service running in the cluster(messages are for increasing or decreasing the amount of logs for a particular customer). For example, if there are 200 instances of the service are running, now i need to send info to all the instances. Have you ever faced a situation like this? What is the best approach you would provide for this problem?
One approach is to use RabbitMq and send a message to all the service instances. But the rabbitMq is not built for this use case, and I'm not sure if this approach will scale. Might need to maintain a queue for each instance.
https://redd.it/dyjz62
@r_devops
Hello, I have a situation where a message has to be sent to all the instances of the service running in the cluster(messages are for increasing or decreasing the amount of logs for a particular customer). For example, if there are 200 instances of the service are running, now i need to send info to all the instances. Have you ever faced a situation like this? What is the best approach you would provide for this problem?
One approach is to use RabbitMq and send a message to all the service instances. But the rabbitMq is not built for this use case, and I'm not sure if this approach will scale. Might need to maintain a queue for each instance.
https://redd.it/dyjz62
@r_devops
reddit
Sending message to all service instances.
Hello, I have a situation where a message has to be sent to all the instances of the service running in the cluster(messages are for increasing or...
Tell me why I should not use kubernetes
Hi r/devops. Lately my company has been looking into possibilities to launch our service across the globe (using cloud providers) closer to end customers. We are now running several Rails apps with Redis and Postgres on our own servers in datacenter. Everything is managed by Ansible and some Terraform scripts.
Some parts of our app are containerized (and running on AWS ECS currently). We are pretty happy with it. But given toolkits available and availability of managed kubernetes (AWS EKS, GKE, Digitalocean etc..) we started looking into it. It looks promissing so now we are evaluating if we should switch to it.
Please give me as many reasons as possible not to switch to managed Kubernetes and stick to traditional VMs running rails, ngunx, redis, postgresql etc managed by terraform and Ansible
https://redd.it/dyiejk
@r_devops
Hi r/devops. Lately my company has been looking into possibilities to launch our service across the globe (using cloud providers) closer to end customers. We are now running several Rails apps with Redis and Postgres on our own servers in datacenter. Everything is managed by Ansible and some Terraform scripts.
Some parts of our app are containerized (and running on AWS ECS currently). We are pretty happy with it. But given toolkits available and availability of managed kubernetes (AWS EKS, GKE, Digitalocean etc..) we started looking into it. It looks promissing so now we are evaluating if we should switch to it.
Please give me as many reasons as possible not to switch to managed Kubernetes and stick to traditional VMs running rails, ngunx, redis, postgresql etc managed by terraform and Ansible
https://redd.it/dyiejk
@r_devops
reddit
Tell me why I should not use kubernetes
Hi r/devops. Lately my company has been looking into possibilities to launch our service across the globe (using cloud providers) closer to end...
I wrote a script to create a working ECS cluster on AWS using the AWS CLI.
It creates the ECS cluster and all the required elements; a task definition, service, EC2 instance, security groups, load balancer and target group. The end result is a simple Hello World app.
There is a clean up script too to take it all down afterwards. Hopefully it'll be useful to someone trying out ECS
[https://gist.github.com/gordonmurray/259eb3c52e66188ea4b0e3b420a6ccd8](https://gist.github.com/gordonmurray/259eb3c52e66188ea4b0e3b420a6ccd8)
https://redd.it/e0q7c9
@r_devops
It creates the ECS cluster and all the required elements; a task definition, service, EC2 instance, security groups, load balancer and target group. The end result is a simple Hello World app.
There is a clean up script too to take it all down afterwards. Hopefully it'll be useful to someone trying out ECS
[https://gist.github.com/gordonmurray/259eb3c52e66188ea4b0e3b420a6ccd8](https://gist.github.com/gordonmurray/259eb3c52e66188ea4b0e3b420a6ccd8)
https://redd.it/e0q7c9
@r_devops
Gist
Create an AWS ECS Cluster
Create an AWS ECS Cluster. GitHub Gist: instantly share code, notes, and snippets.
What is a good script to spin up a Windows EC2 to freeze python app to create an EXE?
https://www.reddit.com/r/Python/comments/dylrif/what_is_a_good_script_to_spin_up_a_windows_ec2_to/
https://redd.it/dylrqv
@r_devops
https://www.reddit.com/r/Python/comments/dylrif/what_is_a_good_script_to_spin_up_a_windows_ec2_to/
https://redd.it/dylrqv
@r_devops
reddit
What is a good script to spin up a Windows EC2 to freeze python...
Currently looking into packer, vagrant-aws, or perhaps terraform apply and terraform destroy. Any suggestions would be helpful.
what's the difference between Prometheus and grafana? Can someone please ELI5.
Let's say that I have to monitor CPU utilization and create alert lifestyle based upon some thersold values , or reading some DB to get live time metrices of an app and create alerts .
https://redd.it/dygjhq
@r_devops
Let's say that I have to monitor CPU utilization and create alert lifestyle based upon some thersold values , or reading some DB to get live time metrices of an app and create alerts .
https://redd.it/dygjhq
@r_devops
reddit
what's the difference between Prometheus and grafana? Can someone...
Let's say that I have to monitor CPU utilization and create alert lifestyle based upon some thersold values , or reading some DB to get live time...
So I've create my first gitlab+docker pipeline. What next?
hi. Finally created my first gitlab pipeline. gitlab-ci.yml looks like this:
image: docker:dind
services:
- docker:dind
stages:
- build
- deploy
build:
stage: build
script:
- docker build -t repo/project .
deploy:
stage: deploy
script:
- docker run -p 8088:8088 -d --name projectname -t repo/project
And here is the Dockerfile:
FROM openjdk:11-jdk
VOLUME /tmp
COPY . .
RUN apt update -y && \
apt install maven -y && \
mvn install:install-file -DgroupId=com.oracle -DartifactId=ojdbc7 -Dversion=12.1.0 -Dpackaging=jar -Dfile=ojdbc7-12.1.0.jar && \
mvn clean package
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","target/project.war"]
Now how do I keep things from redownloading again? Why are there many more images than just the one I create are being created on the machine running gitlab-runner? Is there a dynamic way to get rid of my previous container and deploy a new one on it's place? What's the further reading you would recommend to work on my pipeline?
https://redd.it/dyg3sv
@r_devops
hi. Finally created my first gitlab pipeline. gitlab-ci.yml looks like this:
image: docker:dind
services:
- docker:dind
stages:
- build
- deploy
build:
stage: build
script:
- docker build -t repo/project .
deploy:
stage: deploy
script:
- docker run -p 8088:8088 -d --name projectname -t repo/project
And here is the Dockerfile:
FROM openjdk:11-jdk
VOLUME /tmp
COPY . .
RUN apt update -y && \
apt install maven -y && \
mvn install:install-file -DgroupId=com.oracle -DartifactId=ojdbc7 -Dversion=12.1.0 -Dpackaging=jar -Dfile=ojdbc7-12.1.0.jar && \
mvn clean package
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","target/project.war"]
Now how do I keep things from redownloading again? Why are there many more images than just the one I create are being created on the machine running gitlab-runner? Is there a dynamic way to get rid of my previous container and deploy a new one on it's place? What's the further reading you would recommend to work on my pipeline?
https://redd.it/dyg3sv
@r_devops
reddit
So I've create my first gitlab+docker pipeline. What next?
hi. Finally created my first gitlab pipeline. gitlab-ci.yml looks like this: image: docker:dind services: - docker:dind stages: -...
Jenkins Migration
Can anyone help me out How to Migrate from Cloudbee's Jenkins to EC2 Instance based Jenkins - What are the steps , plugins needed ???
https://redd.it/dxyl36
@r_devops
Can anyone help me out How to Migrate from Cloudbee's Jenkins to EC2 Instance based Jenkins - What are the steps , plugins needed ???
https://redd.it/dxyl36
@r_devops
reddit
Jenkins Migration
Can anyone help me out How to Migrate from Cloudbee's Jenkins to EC2 Instance based Jenkins - What are the steps , plugins needed ???
Zoho (paid own domain) mail delivery issues to outlook.com addresses
[ /r/devops is maybe not the right place for this, but hopefully close enough, mods delete if disagree ]
TLDR; is anybody else using (paid, own domain) zoho for email having trouble getting mail to outlook.com addresses?
---
A month or so back I moved all my email to zoho.com (my own domain with a paid account that is, not a free zoho account).
A week or so ago there was a day where mail was being temp failed to outlook.com addresses (including my own), Zoho logged them as fail and retried and eventually (many hours) they got through.
Today, I did another random test email to my personal (free) gmail, no problem, and an email to my personal (free) outlook.com which just vanished.
The Zoho mail logs SAY it was delivered. The zoho support chat, which seemed a bit weird, like they couldn't find the message-id I gave them, and did a screen-share (which seemed REALLY weird for an email issue for which I've given the log details, but whatever) said, after a lot of "investigating" that it was delivered and that it must be Microsoft's problem.
No, it is not in my outlook.com junk or any folders, my (zoho driven) address is specifically even whitelisted as a safe sender. I have previously sent to the outlook.com address through the zoho driven address fine, it's not a typo.
So not sure where to go from here, I guess it *could* be a problem at Outlook's end, it *could* be a problem with just my (again, free) outlook.com address, but.... ehh....
I've tried several times, same result, zoho says delivered, never appears in outlook.com account.
https://redd.it/dy0k3j
@r_devops
[ /r/devops is maybe not the right place for this, but hopefully close enough, mods delete if disagree ]
TLDR; is anybody else using (paid, own domain) zoho for email having trouble getting mail to outlook.com addresses?
---
A month or so back I moved all my email to zoho.com (my own domain with a paid account that is, not a free zoho account).
A week or so ago there was a day where mail was being temp failed to outlook.com addresses (including my own), Zoho logged them as fail and retried and eventually (many hours) they got through.
Today, I did another random test email to my personal (free) gmail, no problem, and an email to my personal (free) outlook.com which just vanished.
The Zoho mail logs SAY it was delivered. The zoho support chat, which seemed a bit weird, like they couldn't find the message-id I gave them, and did a screen-share (which seemed REALLY weird for an email issue for which I've given the log details, but whatever) said, after a lot of "investigating" that it was delivered and that it must be Microsoft's problem.
No, it is not in my outlook.com junk or any folders, my (zoho driven) address is specifically even whitelisted as a safe sender. I have previously sent to the outlook.com address through the zoho driven address fine, it's not a typo.
So not sure where to go from here, I guess it *could* be a problem at Outlook's end, it *could* be a problem with just my (again, free) outlook.com address, but.... ehh....
I've tried several times, same result, zoho says delivered, never appears in outlook.com account.
https://redd.it/dy0k3j
@r_devops
reddit
Zoho (paid own domain) mail delivery issues to outlook.com addresses
[ /r/devops is maybe not the right place for this, but hopefully close enough, mods delete if disagree ] TLDR; is anybody else using (paid, own...
There's no such thing as CI/CD!
Recently, I became aware of a trend that has been growing since too long: conflating CI and CD into the same word group - CI/CD. When this is done by marketers, this is business as usual, a mix of buzzwords and hype that shouts "Look at me!". But when this is repeated by professional software engineers, I start to worry. And this is exactly what’s happening right now. This post is meant to be a snapshot of my thoughts that I can refer others to in order to dispel the confusion I believe is surrounding CI/CD.
Read further <https://blog.frankel.ch/no-such-thing-as-ci-cd/>
https://redd.it/e10utq
@r_devops
Recently, I became aware of a trend that has been growing since too long: conflating CI and CD into the same word group - CI/CD. When this is done by marketers, this is business as usual, a mix of buzzwords and hype that shouts "Look at me!". But when this is repeated by professional software engineers, I start to worry. And this is exactly what’s happening right now. This post is meant to be a snapshot of my thoughts that I can refer others to in order to dispel the confusion I believe is surrounding CI/CD.
Read further <https://blog.frankel.ch/no-such-thing-as-ci-cd/>
https://redd.it/e10utq
@r_devops
A Java geek
There's no such thing as CI/CD!
Recently, I became aware of a trend that has been growing since too long: conflating CI and CD into the same word group - CI/CD. When this is done by marketers, this is business as usual, a mix of buzzwords and hype that shouts 'Look at me!'. But when this…
Continuous deployment
So I have Express backend and React frontend and basically what I wanna achieve is: write code, push to git repository and then on the server automatically build react app and serve it via nginx and start/restart the backend.
What would I need to achieve this?
https://redd.it/e0z6ez
@r_devops
So I have Express backend and React frontend and basically what I wanna achieve is: write code, push to git repository and then on the server automatically build react app and serve it via nginx and start/restart the backend.
What would I need to achieve this?
https://redd.it/e0z6ez
@r_devops
reddit
Continuous deployment
So I have Express backend and React frontend and basically what I wanna achieve is: write code, push to git repository and then on the server...
Network as Code
Is anyone here using Ansible with physical firewalls and switches to implement Network-as-Code? At work we're moving towards having source control be the "single source of truth" and doing away with all other change control. We're doing well with IAC for most things, but network configs are still manually done then backed up remotely for fail-back.
I'd like to set up an git+ansible+jenkins network-as-code solution. I'm wondering if anyone else has done this and would be willing to share their experience? What hardware did you use? Did it work ok?
https://redd.it/e123un
@r_devops
Is anyone here using Ansible with physical firewalls and switches to implement Network-as-Code? At work we're moving towards having source control be the "single source of truth" and doing away with all other change control. We're doing well with IAC for most things, but network configs are still manually done then backed up remotely for fail-back.
I'd like to set up an git+ansible+jenkins network-as-code solution. I'm wondering if anyone else has done this and would be willing to share their experience? What hardware did you use? Did it work ok?
https://redd.it/e123un
@r_devops
reddit
Network as Code
Is anyone here using Ansible with physical firewalls and switches to implement Network-as-Code? At work we're moving towards having source control...
How do you create index in ElasticSearch ?
I have shipped log files from logstash to ElasticSearch.
I can open Kibana also.
Now I'm stuck how to create index.
Is it possible to create index from Kibana ? (if Yes, How)
https://redd.it/e13052
@r_devops
I have shipped log files from logstash to ElasticSearch.
I can open Kibana also.
Now I'm stuck how to create index.
Is it possible to create index from Kibana ? (if Yes, How)
https://redd.it/e13052
@r_devops
reddit
How do you create index in ElasticSearch ?
I have shipped log files from logstash to ElasticSearch. I can open Kibana also. Now I'm stuck how to create index. Is it possible to create...
A general-purpose, git-centric, audit-friendly quality gating tool
I wrote [Red Light Green Light](https://github.com/atgreen/red-light-green-light) last year in an effort to simplify quality gating for CI/CD pipelines. The idea behind it is that we should decouple the process of evaluating test results away from the underlying testing tools themselves, in a way that is:
* centrally managed
* version controlled
* auditable
* customizable
* protected with authentication/authorization mechanisms
The goal of all of this is to enable auditors to easily answer the following questions as they relate to any artifact promoted through a CI/CD pipeline:
* who presented test results for evaluation?
* what were those test results?
* what policies were they evaluated against?
* who defined the policies and when?
I'm also the author of an opensource library called libffi, and Red Light Green Light was critically important to helping me qualify travis-ci test results. I just released a new version of libffi and wanted to share how useful Red Light Green Light was in the process.
[https://github.com/atgreen/red-light-green-light](https://github.com/atgreen/red-light-green-light)
[https://rl.gl](https://rl.gl)
https://redd.it/e144sj
@r_devops
I wrote [Red Light Green Light](https://github.com/atgreen/red-light-green-light) last year in an effort to simplify quality gating for CI/CD pipelines. The idea behind it is that we should decouple the process of evaluating test results away from the underlying testing tools themselves, in a way that is:
* centrally managed
* version controlled
* auditable
* customizable
* protected with authentication/authorization mechanisms
The goal of all of this is to enable auditors to easily answer the following questions as they relate to any artifact promoted through a CI/CD pipeline:
* who presented test results for evaluation?
* what were those test results?
* what policies were they evaluated against?
* who defined the policies and when?
I'm also the author of an opensource library called libffi, and Red Light Green Light was critically important to helping me qualify travis-ci test results. I just released a new version of libffi and wanted to share how useful Red Light Green Light was in the process.
[https://github.com/atgreen/red-light-green-light](https://github.com/atgreen/red-light-green-light)
[https://rl.gl](https://rl.gl)
https://redd.it/e144sj
@r_devops
GitHub
atgreen/red-light-green-light
A git-centric policy management and enforcement tool designed to accelerate your CI/CD pipelines. - atgreen/red-light-green-light
Kubestone - Benchmarking Operator for K8S and OpenShift
https://www.reddit.com/r/kubernetes/comments/dxz9ez/kubestone_benchmarking_operator_for_k8s_and/
https://redd.it/dxzhpf
@r_devops
https://www.reddit.com/r/kubernetes/comments/dxz9ez/kubestone_benchmarking_operator_for_k8s_and/
https://redd.it/dxzhpf
@r_devops
reddit
Kubestone - Benchmarking Operator for K8S and OpenShift
[Kubestone](https://kubestone.io) is an Open Source benchmarking Operator that can evaluate the performance of Kubernetes & OpenShift...
Does anyone know if Chef/Berkshelf has a way to tar up a directory full of chef cookbooks? My project lead says it was a thing but I can't find it.
Thanks for the help :).
https://redd.it/e18vib
@r_devops
Thanks for the help :).
https://redd.it/e18vib
@r_devops
reddit
Does anyone know if Chef/Berkshelf has a way to tar up a directory...
Thanks for the help :).