Is a raspberry pi a good platform for learning docker?

Hi all.

Pretty fresh in the devops space. Just passed my AWS Certified Cloud Practitioner test on Wednesday, and looking to continue to build on my knowledge. I figured docker is a logical next step.

Would a raspberry pi be a decent platform to play around with linux containers, or do I need more powerful hardware? I'm not looking to do anything crazy. Just have like one apache web container and one mysql container (i think i said that right). Just trying to get my hands dirty.

I have fairly limited linux knowledge, but I do have some enterprise experience patching/updating Oracle Linux and MySql.

Thanks!

https://redd.it/dy5zt9
@r_devops

Can someone ELI5 Service Meshes (Consul, Istio, etc)



https://redd.it/dy7ecv
@r_devops

Using Datadog just for centralized logging?

I'm at a pretty small shop and we generate \~3GB (120mil events)/mo, which at Datadog would cost $230 for 15-day retention...but with \~100 hosts it looks like you're *required* to pay for DD's infra management at $18/host? So it would be a $1800/mo charge just to turn on Datadog, then an extra $230/mo to add logs?

For comparison 3GB/mo is $108\*3 at Sumo Logic, no infra monitoring fees.

Anyone else use Datadog just for outsourced log management?

https://redd.it/dybghe
@r_devops

Gitlab integrated with Phabricator - Protected Branches (push to master)

Hi everyone,

I'm not entirely sure if this is the right place to post, please let me know if there's somewhere else more relevant.

So I'm in a project where we use Phabricator for Code Reviews and Gitlab to host the project. As it is right now, once the code has been approved, one can 'arc land --onto master' and push to master from Phabricator. As of now, the master branch is protected on Gitlab but everyone has push permissions. When I set it to no one, pushing from Phabricator becomes impossible.

Is there any way to ensure that no one can push directly to master in Gitlab, but can from Phabricator (onde the review is approved)?

https://redd.it/dy31fd
@r_devops

Perf testing ci build?

Do you run perf tests as part of a CI build? Sure I'd like the code to be as near production ready as possible before it's needed to main ( trunk based development) - but worried that developers will not be happy if CI builds take too long. I'd appreciate any recommendations. Thanks VK

https://redd.it/dyaovn
@r_devops

Docker in Vagrant

Hi,

is it a common use case to setup Docker Containers in Vagrant VMs?
And would you do it in production environments?

https://redd.it/dyajer
@r_devops

Two services, same domain on GCP?

Hey folks.


I'm facing a problem that I can't find ways to easily solved it on Google Cloud.

Today, I have an AWS Account with several systems running on a Kubernetes cluster, Lambda functions and Elastic Beanstalk. For accessing them, I use an API Gateway like this:

Domain xpto.com
|----------------> xpto.com/login -> Lambda Function
|----------------> xpto.com/users -> Kubernetes Cluster
|----------------> xpto.com/employee -> Elastic Beanstalk

So, I have the same domain with different paths and each of those paths is redirected to one specific place where a service is running.

The problem is, I didn't find a way to "translate" this API Gateway rules to GCP.

Is there a native way to it? And if isn't, how do you guys recommend a way to solve this (DNS, reverse proxy, ...)?

Thanks in advance for the help.

https://redd.it/dy6m4l
@r_devops

iptables in container as a service, go or not go?

Following the containers best practices, that a single service should be a single running container, how would you approach the following requirement?

In the current environment, non containerized, I am running a single Logstash process. In same machine, I have set up the iptables to forward the UDP traffic from one port to another on same machine (interface).

​

Should be moving the iptables to its own container better approach?

​

If so, I need to took care of updating the iptables rule in case of scaling the Logstash.

​

Or, is it there a better way to forward the traffic? All other approaches seem like an overkill to be honest.


Yes, I am on AWS.


NLB is not the best choice as I have a lot of traffic and it could introduce higher operating costs.

https://redd.it/dy8w2s
@r_devops

Continous Deployment - Where to start?

Hello guys,

recently I already asked about this but my questions were too vague I guess so I want to try it again as I am pretty sure that there are more than enough experienced people who could give me some pointers.

I need help with the following:
I got a GitHub repository, a DO Fedora droplet, and a docker-compose file as well as a Dockerfile for my backend.

My questions are:
1. Where do I build my Dockerfile for my backend?
2. Where do I place my environment variables?
3. Is creating my own Jenkins for this already overkill?

I wanted to make use of GitHub actions but I am kinda confused by the workflow. Would I need to write a script which opens up a ssh connection to my server, clones the repo, builds the image and launches docker-compose?

Thanks so much!

https://redd.it/dy8iwt
@r_devops

Resources and learning path?

Hi all,

I'm looking for good resources (books, courses) to learn devops. Something that starts off with the logic and design and goes on to implement projects with the proper standards and tools in the industry, bonus point if it uses Python because I like the language.

Thanks for your help!

https://redd.it/dy2fnm
@r_devops

Terraform with multiple environments and workspaces

Hiya,

Thought this article might be of interest: [https://www.mooreds.com/wordpress/archives/3244](https://www.mooreds.com/wordpress/archives/3244) It covers how to use s3 based storage for open source Terraform for two different environments that might differ in terms of parameters. It also uses two different buckets to minimize the blast radius. As noted in the post, I didn't get a chance to implement this in production, but would love to get feedback on if this approach makes sense.

https://redd.it/dxz89d
@r_devops

What blogs, newsletters, or twitter accounts do you follow for your DevOps and technology watch?

Technology watch can sometimes be time-consuming because we don't always know where to look. So, I am giving you here the [9 sources (blogs, newsletters, twitter accounts) that I follow](https://www.padok.fr/en/blog/devops-blogs-newsletters-follow) daily. I'm interested to know which ones you follow too, tell me in the comment!

https://redd.it/dylj29
@r_devops

How easy is it to move out of devops (back to dev)?

Hi all, I'm a software developer with a couple years of experience, considering the move into devops. Given that I'm early in my career, I'm concerned about getting pigeon-holed into the field if I decide I don't like it as much as development. Is it reasonable to think that after a year or so of working in a Devops Engineer role I would easily be able to get a job in development again?

https://redd.it/dymj9r
@r_devops

Aws separated,segregated environment, how?

Google cloud has projects within an account, and all of its resources are separated, unless you setup vpc peering , there are no connections between the projects (environment)

I'd like to achieve this in AWS.
* Every env has its own vpc
* Completely disconnected from others.
* Has its own user access key that cannot view or list other environments

Can I achieve this in AWS with the above requirements? How?

https://redd.it/dymzbq
@r_devops

Question Re: Bitbucket Pipelines and setting up django tests

Notice: I'm not a devops engineer - just a django dev who is trying to automate more and learn more devops as I go. When it comes to CI/CD I'm very new and know very little. If you mention Docker, while I am familiar, we don't utilize Docker where I work so I am not a Docker Pro (entry level at best).

In Bitbucket, I have a 'project' repo that contains my django project level files, and each app for said project is in its own repo.

* `project_repo`
* `app_repo`
* `other_app_repo`

I want to ensure that my tests are passing (automatically) and I think Pipelines will allow me to do that.

I'm trying to configure Pipelines so I can run tests automatically when pushing up to an app repo to ensure tests pass, coverage is acceptable, etc.

Currently, to test an app here are my steps:

* clone in the project
* cd into project directory
* `virtualenv venv`
* `source venv/bin/activate`
* clone in the app `git clone https://bitbucket.com/myapp`
* cd up one level (back to the project directory)
* `pip install -r requirements.txt`
* `pytest myapp`

Is Pipelines capable of doing this or am I wasting my time?

https://redd.it/dyq15a
@r_devops

Looking for part-time DevOps related work

Hi guys. I am looking for a part time job/role (10-20hrs remote) during the weekends for some extra income. I am currently employed, but i have spare bandwidth on the weekends. High level overview of my skillsets are:

1. Terraform
2. Docker
3. Ansible/Puppet/Chef
4. AWS (VPC, EC2, ECS, ECR, Lambda, IAM, RDS, Cloudfront, Route53)
5. Elk (Log transformation/Ingestion/Graphs)
6. Jenkins
7. Python/Bash/Ruby/Java/C

Please send me a message if theres any interest. Thanks for reading

https://redd.it/dyr2sn
@r_devops

Application for route json messages to the multiple services

So i have a question if there is an application that can do this or if we have to build one on our own.

>We have a service which runs in multiple containers in ECS it can connect to A, B, C, D etc system depending on what is received in a json message. It will start a client module connected to the B system and do the action. All well and good the problem is that after that has happened we would like to route all new messages to the same container since it already has an active client connection.
>
>
>
>The messages are today consumed directly so i can't really use a loadbalancer unless i'm missing something it will just place the json messages in the container it Algorithm chooses and then we will end up with wasted resources since there will be client module B connection on all of our containers.

We have looked a bit at RabbitMQ or some other Message Broker which i think is the solution but i'm not sure if they can go down to such a detail or something else maybe anyone have a similar use case?

https://redd.it/dymifm
@r_devops

New release: CloudFormation CLI (open source) and support for third-party resource providers

This has been rumored to be coming for a while. But AWS has finally pulled the trigger. A new native open source CLI for CloudFormation and support for native third-party resource providers (not custom resources). Also, there's a CloudFormation registry out of the box now for future third-party providers to be hosted.


https://aws.amazon.com/blogs/aws/cloudformation-update-cli-third-party-resource-support-registry/

https://redd.it/dywrqe
@r_devops

Organizing Hashicorp Vault secrets?

Are there any best practices to organizing Vault secrets? Right now it seems like it’s the Wild West with secrets in different places.

I have AWS secrets here, client secrets there and I’m just a bit overwhelmed.

Any helps would be much appreciated!

https://redd.it/dyv3xd
@r_devops

Untested changes merged to our develop branch end up in production. What should we do?

We work by having a develop branch and a master branch. The develop branch can be deployed to a dev or test environment. The master branch can be deployed to QA and production. When we start work on a feature, we create a feature branch based on the develop branch, and then merge back into develop, when we want to start sending the feature through our TEST environment(where it gets verified by the business supposedly - rarely ever happens). In QA Operations are meant to verify that the deploy went okay.

Today we experienced that some old untested changes merged to the develop branch were automatically deployed as another team had merged their changes into the develop branch and merged it into master and deployed to production. We want to avoid this from happening.

My thoughts on this are the following:

\- We could drop the idea of having a develop branch and maybe ensure that we can spin up the test and QA environments. I'm not sure why it matters to have both a DEV, TEST and QA environment. When I ask I get some very vague answers. Instead when we're done with our work we merge directly to master.

\- To assume master is okay, we set up good automated test coverage.

\- We split up services into microservices, so we don't run into the issue of multiple teams working on the same service all the time. Many of our services could easily be split up.

\- We add feature toggling for new features, so they by default are disabled when deployed.

Myself and the company I work for are very far from working in accordance with all the DevOps guidelines, but I'm trying to push the ones that seem to solve our problems. I'm not too sure about our branching strategy as above though. Thoughts? I'm thinking in must be a very common problem that's been solved a million times.

https://redd.it/dys8rj
@r_devops