wrap.sh: an experiment in devops UX

My team has been looking into ways of improving the devops experience for a while now, and testing our theories through the spontaneous creation of a bunch of small tools.

One theory we examined goes as follows:

1. The projects with bad CI experiences have poor, slow and low-signal feedback from their CI pipelines.

2. The most direct signal you can get is from direct access to the broken version of the project

3. So if a project lets you connect directly to the pipeline when a test fails, that should come with a good CI experience.

This led to one of the more promising experiments: Wrap.sh - an attempt to increase feedback and visibility in CI pipelines. It's also open-source.

We don't fully know what to expect from this thing; it might be useful to someone, maybe for OSS projects using end-to-end tests. Or maybe not.

In any case, we'd love to hear your questions and feedback regarding the tool (or the theory it's based on).

The links:

https://wrap.sh

https://github.com/layer-devops/wrap.sh

https://redd.it/lbdmgo
@r_devops

Containers monitoring solution

Hello Folks,

I am working on a task for comparison on different monitoring tools currently out there for containers/k8s. I understand Prometheus has some native support like service discovery for Kubernetes, but i would like to understand what are the close competitors to look for. I am looking for about 6-7 tools for a deeper evaluation. A quick Google search returns many tools but would like to hear from the community experts about your preference on monitoring k8s clusters. Any useful inputs on this is highly appreciated.

Thanks

https://redd.it/larf53
@r_devops

Solving ArgoCD Secret Management with ArgoCD-Vault-Plugin

Hi everyone, i wanted to share an ArgoCD plugin that i have been working on that allows for connecting to Vault in a simple way that does not require an Operator or CRD. The plugin is in its early stages and only supports a couple backends but we look forward to any contributions/suggestions or ideas you may have!

https://werne2j.medium.com/argocd-secret-management-with-argocd-vault-plugin-539f104aff05

https://redd.it/lbpcpp
@r_devops

How do you automate AWS AMI updates?

I currently manage most of our infra with terraform. I have a module that returns the latest AWS AMI for a particular service (EKS, ECS, etc). This means that whenever we run a terraform plan for a project that uses the service, the plan will include an AMI update if AWS has released a newer AMI. This has worked fine but I'd like to make this a little bit more stable. I'd like to have the latest AMI run for a while in our non-prod environments and then have some sort of approval process so that production gets updated later. Any ideas on how to make this work? Or any ideas for an alternative approach?

https://redd.it/lbrfhs
@r_devops

A guide to the best SRE tools

A beginner's guide to common SRE/DevOps tools and incident management automation (monitoring, oncall, IaC):

https://www.getcortexapp.com/post/a-guide-to-the-best-sre-tools

https://redd.it/lbtomt
@r_devops

transfer thousands of files of any size with optimization

We have been doing a mix of manual process and some scripts to transfer files of various sizes from one system to another. Basically there are shares where people may dump hundreds or thousands of files of varying sizes. We then move these files to another location.

We want to use a tool that would automatically optimize speed/perf based on file size and amount and transfer the files. (nifi maybe?)

https://redd.it/lboo0s
@r_devops

Keeping track of the infrastructure

Hi there,

the cloud application my company is developing did not start with orchestration in mind (and it's to late to do so :D).

We have some hosted components in Azure (fixed set) together with some managed machines in OVH (scalable).

For two purposes:

\- dynamic topology

\- monitoring

to be able to have a service where every component can "check in" and that other applications can use to reliably get info on the current online components.

Do you guys have any suggestion?

Thanks a real lot,

https://redd.it/lbnw5t
@r_devops

Prometheus exporter to retrieve the DockerHub rate limit counts as scrape target

This exporter allows you to retrieve the DockerHub rate limit counts as scrape target for Prometheus as Gauge metric.
Multi arch docker images are available (arm/arm64/amd64) with a complete docker-compose example.

I hope you find it useful.

Docker Hub Rate Limit Exporter Github Link

https://redd.it/lbigg8
@r_devops

What do you use to manage on-call alerting on AWS?

Hi, we have a current system where we use cloudwatch and ms teams notifications for alerting if something happens in production.

However, management requires a 24/7 support. Therefore I'd like to use a system with scheduling for 24/7 support and ability to call/alert developer in case severe incident with production happens.

​

What are the best tools for the job?

https://redd.it/lbldob
@r_devops

Help me with setting this up

Hey guys, I'm working on a project which has a production database hosted in AWS RDS. We work on a separate local Postgres database from inside a Docker container. The APIs are to be uploaded to Lambda. This is where things get complicated. Whatever changes or migrations we make to the development database are not reflected in the RDS. I want the final changes to be applied to the RDS automatically through some pipeline. Is there any guide that helps with this kind of problem?

Sorry if I sound noob, this is my first time working with large techs. Thank you.

https://redd.it/lbl04s
@r_devops

Need help from someone with AWS CI/CD and VPC experience

I hope this does not violate community guidelines, but I really need some help with an AWS project. I am working on a MENN App in AWS and we are having a lot of issues with CI/CD from CodeCommit --> CodeBuild --> CodeDeploy for Lamdas. I am also having no luck connecting Mongo Atlas to a Lambda. We are all full stack devs and I have a background in systems administration so I was able to setup VPC peering, but can't get IAM authentication to work for connecting to Mongo Atlas from Node.JS Lambda. I don't have a lot of money, but would be willing to pay if anyone could help.

https://redd.it/lc4b6q
@r_devops

Help with specific metrics around platform for increasing headcount

Unsurprising story: ask for head count fails because DevOps / platform / infrastructure isn’t easily quantifiable like revenue of an external product / feature.

Really need someone to share either/both:
1. specific measures they use that are effective when talking to executives
1. how these are scrapped / generated

Disclaimer: I’m aware of Accelerate metrics but this is one of those challenges beyond that. A product team that makes money can show they’re making more money by adjusting those metrics; it is difficult if you’re 1-2 deviations off.

https://redd.it/lc2q8y
@r_devops

How does manual testing fit into CICD and trunk based development?

Struggling how to understand how people do CICD and trunk based development with or without manual testing. Surely you can go straight to prod if you pass all automated regression tests - but is anyone ever still performing manual tests? And at what point in the development cycle/process is manual testing performed? Is it a gate before release is actually "deployed"?

https://redd.it/lc10mf
@r_devops

Career Advice I want to move from Civil Engineering to DevOps engineering?

I'm currently doing my bachelor's degree in Civil Engineering technology in South Africa and when I graduate I may become a civil technologist/engineer. However, I want to branch into DevOps. What is the best route for me to become a DevOps engineer ? Is there a bridging honours or masters I can do to become a DevOps Engineer?

https://redd.it/lc3u1z
@r_devops

Declarative API's

I am wondering whether there's actual use case or its an advanced users feature that is a nice-to-have.

​

Will declarative API's, infra as code capabilities affect your decision when choosing a tool/platform?

View Poll

https://redd.it/lc1qnb
@r_devops

Looking for simple local build system

I'm looking for some kind of simple generic build system that will run entirely locally on my Windows machine (not Docker) that will basically do 4 things:

Execute a sequence of commands
Capture the commands and output
Collect generated files from a build and put them somewhere
Maintain the history of builds, logs, and files

Even better if it could automatically do a lot of things a CI/CD system would do, e.g.

Checkout a Git revision (from a locally hosted Git repo, or a Github repo)
Setup environment variables
Run tests
Generate some reports
Generate a manifest
Identify and collect artifacts

https://redd.it/lby4ta
@r_devops

Which job should I pick?

I am a middle level DevOps engineer. I am familiar with all general DevOps tools, and have spent quite some effort on AWS (I have 3 certs already, 1 is Specialty), but not much real life experience.

Currently I am receiving two job offers (first of all, the salaries and company sizes are the same):

* Job 1:
* AWS
* Serverless
* No K8S (yet)
* Website and mobile app
* Possible working from home 60%
* Quite a distance from home
* Job 2:
* Azure
* K8S
* IoT
* Possible working from home partly
* 1/2 distance from my home, comparing with the job 1

Which one should I pick, or is there anything I should consider?

https://redd.it/lbwa5w
@r_devops

Can I bulk upload epics and features to a backlog?

Basically the title. I've got about 20 epics with multiple Features cascading under them. I want to be able to bulk upload everything.

https://redd.it/lbvzwd
@r_devops

CI puppet code using docker image

We're developing puppet code to automate configuration for VMs shipped our customer.
For the moment a simple pipeline is set up to check code and synchronize modules in foreman. Each time we want to check the result, we need to connect to the VMs run the puppet agent and analyze the output.

I would like to setup a pipeline using customize centos/debian docker images (with systemd enable) running puppet server and agent to test new development.

I assume the result should be the same as if I was deploying the manifests into VMs.

Am I right to think that it would have the same effect in productive VMs? Does someone already tested?

https://redd.it/lbtdiv
@r_devops

Need tips on package managers

So my environment has Linux nodes windows nodes and docker images running on both virtual and physical servers all are in the same network
I would like to create a local repository to host windows packages, Linux packages, docker images and packer VM/iso templates in one location.

I believe Linux, docker and packer templates should not be a problem, but I am wondering about windows.
I would like everything to be in one virtual node
Does anyone have a ideas/ tips on what I can explore??

I am open to anything (open sourced of course)
Thanks in advance

https://redd.it/lced8d
@r_devops

Looking for some good rules of thumb

Hi!

I'm a web app developer and when I have to deploy stuff I always choose the smallest tier, because I have no idea of what specs what traffic/requests can hold approximately.

So if someone with experience can help me in either of the 3 following things, that would be amazing:

1. For a basic JSON API backend server that let's say executes 1 database operation when gets a request (average speed framework for everything - it shouldn't be that big of a difference), how should I think about when choosing hardware, like if I'm expecting at max 5.000 requests/second what hardware can handle that and what about 10.000 req/sec, 20.000 req/sec and so on
2. The same for a basic static file sever that serves static html + css + js. Here again like if the sum of all is for example 3MB and I have X req/sec how should I think
3. Server Side Rendering HTML Server (React SSR or any MVC framework). This one is the hardest, but if someone has a lot of experience there's a chance that there are good rules of thumb for this one: how much heavier it is than a simple JSON server that executes a DB operation

If someone can help me with any of it or link me some good resources I would be very thankful!

https://redd.it/lcfayp
@r_devops