Maintaining configs for each repository when there are many repositories?

The problem

There is a slowly growing number of repositories and each one contains a configuration describing its pipeline. Sometimes we need to change the configuration and this requires going to each repository now. One of the problems going through each repository is that not all branches are going to be updated automatically. Configuration files are not identical even though some parts are. Going through each repository and updating it does not seem scalable so I'm thinking of a solution or a way to automate it.

Possible solutions

So far I thought about maybe including a step in configuration to download script from some bucket and run it - this is why we need to update in 1 place and all branches will get the update right away.

But what happens if I want to remove or add steps in the configuration file that is used to download the script? Sed and commit from script? Seems fragile but maybe I'm mistaken or this is something I should figure out in a different way.

Maybe centralize all configuration files in one repository and push them to corresponding repositories on push? Does not solve updating branches but makes managing configuration easier I guess.

Any thoughts?

https://redd.it/rd4wu1
@r_devops

Health check on VM app that runs interactively

I am looking to automate the deployment of an app that runs on Windows and needs to be run as an interactive login - it doesn't run as a service. I am planning to put the app within the startup profile of a service account and use Sysinternals Autologin to run the app.

I'd like to be able to run an Azure VM health check so that if the process were to fail, it would just delete and redeploy it. The app however doesn't open an inbound port so I cannot monitor it.

Is there a way I can achieve this? Any utility around that will create a dummy inbound service that mirrors the availability of another process?

https://redd.it/rd4dda
@r_devops

Best small database?

I want to use the choreography cloud pattern on Kubernetes. As part of this I expect each microservice/service would have its own database.

The service is only going to write into it and later something might query on specific fields accross databases.

If deploying on aws or azure I know what I would use but I want to develop this at home on minikube so choosing something than can use minimal ram/cpu is ideal.

Does anyone have any suggestions?

https://redd.it/rd4aog
@r_devops

I got an AWS Role

Hello People,I applied for a job as an AWS Solutions Architect. I feel bad because I know I can not do the job. I lied my way through all 4 interviews.I can speak well and bold enough to say things I can not do.The job pays so well and I am so afraid they will figure out that I can not do anything. Please what do you advice I do? I need the money badly

https://redd.it/rd1ip1
@r_devops

Parallel terraform init

I'm running multiple deployments in different environments which means I have separate backends. How do I run terraform init in parallel? At the moment if I attempt to do this, they will all try to write to .terraform which is not ideal.
Is it possible to change the folder terraform caches data to?

https://redd.it/r9scub
@r_devops

3-year programming degreE

I'm currently enrolled in a 3 year software degree (CPU+Math+Algorithms+Architectures+Paradigms) but I realized I'm also interested in Linux/CLI/Cloud so I was told to look into "DevOps" which was basically told to be Sysadmin + little scripting + cloud certs

That's when I thought I may be better off dropping the degree and just getting certs and stuff but then I read this

>You need to understand data structures, algorithmic complexity, IPC, multiprocess execution models, how code runs inside an operating system (how userspace interacts with the kernel, how memory is allocated, how shared libraries work, etc).

>If you don't understand this stuff, you'll be at best a mediocre Dev/Ops/DevOps person. You can't design efficient system or debug a problem on a live system if you don't understand how all the pieces fit together.

Is the degree worth the time investment then? I'm going to start next year for the record and I may want to certify for redhat in the future..because..it sounds cool.

https://redd.it/rda5aw
@r_devops

Automate, Customize and Codify AWS PCI DSS with HCL and SQL

We released a guide on how to codify AWS PCI DSS compliance with open-source CloudQuery policies which works by running standard SQL on a PostgreSQL.

CloudQuery itself is an open-source cloud asset inventory that extracts all your cloud assets configuration and loads them into vanilla PostgreSQL. https://github.com/cloudquery/cloudquery

https://www.cloudquery.io/blog/running-aws-pci-dss-with-cloudquery-policies

https://redd.it/rdb7pz
@r_devops

Manual CI (continuous integration) Pipeline?

Im just getting into devops (emphasis on CI/CD) and having trouble understanding what i would actually be automating. How does one start understanding what they are actually automating?

I dont know how someone who is a non-dev could automate a build process if they rarely, if ever had to build software.

Thanks.

https://redd.it/rdeugj
@r_devops

Critical RCE Vulnerability: log4j - CVE-2021-44228

https://www.huntress.com/blog/rapid-response-critical-rce-vulnerability-is-affecting-java

Many WAF vendors are already offering mitigation.

If you have anything JVM based, you look into this immediately. Things like Jenkins, Sonarqube, Neo4j, etc may rely on log4j and may be vulnerable.

https://redd.it/rdfq2r
@r_devops

An open source app stack for Kubernetes with sane defaults combined with developer self-service and GitOps patterns

Otomi makes developers self-serving and helps DevOps teams to guarantee application security and availability at the earliest stages in the development lifecycle when using Kubernetes while strongly relying on GitOps patterns, where the desired state is reflected as code and the cluster state is automatically updated.

Check out the GitHub project: https://github.com/redkubes/otomi-core

https://redd.it/rde7xy
@r_devops

What swag item is your favorite to get at conventions?

*Asking for DevRel research purposes

View Poll

https://redd.it/rdgsty
@r_devops

How to develop good security instincts?

I think as a developer, I have good readability/understandability/composition instincts. I think I have fair or at least acceptable instincts when it comes to stuff like design, efficiency, scalability, extensibility, automation, and even non-security related resiliency problems. But I'm pretty sure my security instincts are just crap possibly because it's sort of antithetical to readability (obscuring the system to an attacker is kind of the point). My mind doesn't naturally drift that direction when I'm thinking about systems and problems. This seems pretty bad for any developer much less one targeting a devops career path. Is there a good way to train my brain to think about problems in this way?

https://redd.it/rdg7oc
@r_devops

At what point do you bring up "comp" when talking to recruiters / hiring managers?

​

I've been trying to be really candid and send all recruiters a google document about myself that also happens to mention my current comp and comp expectations before we even have a phone conversation, and this has helped me avoid awkward phone calls and time wasting.

​

But I've had a few instances where the recruiter doesn't have solid details on comp bands or says its flexible or "conversational" once you go through the interview process etc.

​

I was asked to do a virtual onsite for a company I'm really interested in, but we hadn't really had the comp conversation. I obviously don't want to spend 5 hours interviewing for a job that doesn't pay enough, so I emailed and asked to clarify before we set up the full interview. The recruiter responded saying in somewhat abstract terms that we are in the right range, but I felt tacky even asking like that.

What's the right approach?

https://redd.it/rdlj0f
@r_devops

Capacity of blue/green environments

For folks using blue/green deployment for the production environments: Do each of these environments have enough capacity to independently support all your users?

https://redd.it/rdmjx3
@r_devops

Best Build and Deploy Tools for Enterprise ios and android mobile apps?

Apologies in advance for this newbie-ish question. As a project manager, I have been tasked with getting my team to build and deploy mobile apps (ios and android versions of the app). What is the best enterprise tool out there for this? We use BitBucket instead of Github. Something that is all inclusive and really makes it simple, with workflows, and even hooks into the app stores. Thx in advance and apologies again for this newbie-ish question.

https://redd.it/rdm6hl
@r_devops

Viewing the Test or Fact creator on a build pipeline

Hi

Is there a way to view the first creator of a test or fact on an azure build pipeline, when using the test report tab?

Example.

A person creates a unit test using an IDE like Visual Studio 2019. The name is displayed above the test, but gets updated to a different person when the test or solution is updated.

The tests get merged into master and the build tests run and pass. On the test tab of the build execution, all the executed tests are filtered and displayed for further analysis.

Is there a way to include the test creator, and not the editors since it’s creation on Azure repo or build test?

Thanks.

https://redd.it/rdlgt3
@r_devops

How to run dependabot locally on your projects

Hey r/devops \- I thought I'd share an article I just finished this morning.

https://mikebifulco.com/posts/run-dependabot-locally

This is a follow-up to a primer on Dependabot which I wrote in 2019 - for whatever reason, that article still gets loads of traffic from people setting up Dependabot on their projects. I've also gotten quite a few DMs on twitter and reddit from folks asking how and if they can run dependabot locally. I finally got off my butt to write a quick article to share with dev teams interested in self-hosted-dependabot configurations. If this sounds interesting to you, give it a look!

https://redd.it/rdg0ds
@r_devops

Best learning path for SRE/DevOps?

For someone who has been in the software testing space for over a decade and wanting to move into the SRE/DevOps space… what would be the recommended order to learn the following subjects:

- Google cloud
- Kubernetes
- Terraform
- Docker

https://redd.it/rdrc92
@r_devops

Help: First time deploying

I've just finished a Django web app and I'm trying to deploy it to AWS.

Here's what I've done so far:

1. Dockerized using docker-compose to create 4 services: app, database, celery and redis.
2. Pushed app to amazon ECR using docker push

What I'm trying to do:

1. Get all the services working using AWS Lambdas (ideally using zappa)

I've looked all over but can't seem to find a way to get this working, can anyone help?

https://redd.it/rdakp7
@r_devops

Who else is still up right now working on log4j - CVE-2021-44228

It's the all nighter life for me.

https://redd.it/rdvhs0
@r_devops

Env vars and Docker differences between dev, staging, and prod

Hi r/devops,

Although my specific example involves Django, Docker, and Heroku, I believe these are pretty general testing/QA questions.

I have a dockerized Django app tested in dev with Selenium confirming that my static files are being served correctly from my local folder (EXPECTED_ROOT = '/staticfiles/'). This app is deployed to Heroku and I can see (visually and in the dev tools) that the static files are being pulled in from CloudFront correctly as well. I want to formalize this with the same test I'm using in dev. My first question is related to if/how environment variables are used for tests:

Do I add for example `EXPECTED_ROOT = 'https://<somehash>.cloudfront.net/'` as an env var to Heroku and use it in the Selenium test?

Also, to run this test in staging I would need to install Firefox in my Docker image like I do in dev. Perhaps this is ok in staging, but in prod I believe I should be aiming for the the smallest image possible. So the question is about differences between staging and prod:

Do I keep Firefox in my staging image, run the tests, and then send to production a replica of that Dockerfile, but now without firefox?

Any help is appreciated.

https://redd.it/rdzu7k
@r_devops