Puppet CIS Benchmarks

Hi all. I'm looking to push CIS benchmarks via Puppet to our infrastructure. I did see the CIS Compliance Service. Can anyone speak on it's success? Or, have you found success through a different route? Cheers!

https://redd.it/k5fmjg
@r_devops

Runbook Automation

Hi,

Have you considered automating runbooks? If so, which tools did you use to do so? I'm exploring Rundeck but some of their features like PagerDuty, Okta integrations come with a license fee of at least $20k.

Thank you.

https://redd.it/k5ilyw
@r_devops

Running vault in k8s vs dedicated vm

In my previous company we ran vault on dedicated hardware, so we had a few VMs in separate regions - per environment.

In my current company, I have an environment running one k8s cluster with a few nodes for services, but 2 nodes specifically dedicated to vault, running in separate regions, and am running 2 vault pods there.

Since my region spread is identical, are there any reliability gain from dedicated VMs over my current setup?

I have an urge to say dedicated VMs would be more reliable, but I don't have any evidence why. In fact hashicorp's own website provides a [k8s reference architecture](https://learn.hashicorp.com/tutorials/vault/kubernetes-reference-architecture) in addition to [standard reference architecture](https://learn.hashicorp.com/tutorials/vault/reference-architecture)

Also - could there be performance gains from VMs over pods in k8s?

https://redd.it/k5kowe
@r_devops

AWS - change file system on ec2 ebs root block device

Hello.

I've been looking for a way to change the file system in an EBS root block device with no luck. I'm looking to create a new EBS backed AMI with btrfs on the root block device. I'd prefer to do this as part of an Image Builder pipeline, but any method (except Packer) would help.

https://redd.it/k5quny
@r_devops

I'm looking for your feedback! (experiment about VR and TravisCI)

I am working on a very early idea: applying virtual reality in software engineering. My first try is this. I represent in VR the data that can be retrieved from [TravisCI.org](https://TravisCI.org) using its API v3. Your feedback will be very valuable to me. Thank you very much colleague

The experiment [is here](https://rv.dcc.uchile.cl/)

https://redd.it/k5ozve
@r_devops

debug a Docker container on aws Elastic beanstalk, still up but not working, no printing any logs or errors

Pretty much the title.

I have a simple python app doing some stuff in a main loop (while True: ...do stuff... sleep 5 sec). It runs in the official python 3.8 slim-buster image. I run the container on AWS Elastic Beanstalk (EC2 t3a.micro).

Now the app stopped working last week. I logged in the AWS console, nothing weird. I ssh'd directly into the EC2 instance, the container was still up with an uptime of 2 months. When I displayed the logs from the container, it indeed stopped logging last week. But what's weird is that it stopped in the middle of the algorithm (very basic stuff, nothing CPU/IO/time intensive), in the main loop, without logging any errors. It just stopped there. On the side of the EC2 instance, CPU usage is low, volume almost empty (I didn't think about checking the memory usage.. but if it was close to 100% I wouldn't be able to ssh there. edit: 50% so far but it has been up for only 30 min. Need to wait to see if it increases. edit2: doesn't seem to leak).

The same issue happened 2 months ago, so that is the second time in like 5 months that I've been running that app. If we exclude these 2 dates, the app did its job, 24 hours a day, 7 days a week.

**So I tend to think the problem is not in the code or in python otherwise there would be some errors and it would probably happen more often. So I guess it's Elastic Beanstalk or Docker.**

Any idea what it could be or how could I find the source of the issue? How to debug it?

Should I just force it to restart automatically like on a daily basis since it seems it happens after running for weeks? I don't mind the downtime, it's stateless and takes a few seconds to restart.

https://redd.it/k5k2cl
@r_devops

Sonarqube: What it is and Why to use it?

[SonarQube](https://www.loginradius.com/blog/async/sonarqube/) is a universal method that has become more or less the industry standard for static code analysis. With SonarQube, keeping the code clean, clear, and easy to read is also much easier. I found this basic guide to get a better overview of sonarqube. Let me know your thoughts in the comment :)

https://redd.it/k5t7r6
@r_devops

How to pass arguments to shell files and manipulate them inside the file. Also, ask user to insert certain input like username and password.

Arguments gives the bash program more details about what you need to do. Sometimes, it comes as a form of sub-command like yarn add <packageName> where "add" tells the program more about what to do, and the packageName is what you data that you what to pass the program.

You can also pass inputs in the middle of the working program. It could be asking for confirmation, authentication, etc.

Check out the video for hands-on implementation:
https://youtu.be/4bcUii6HDYo

https://redd.it/k5tu57
@r_devops

How would go about creating an isolated environment of ~60 .net (4.x) micro services?

So.. I'm a the only DevOps guy in the company, inherited 3 environments from my predecessor: test,qa and ofcourse prod.
Each environment is comprised out of multiple servers hosting a magnitude of ~60 types of different micro services that make up the entire system (a trading platform) .
Few of these services have multiple instances, some are IIS hosted services, some are windows hosted services.
Most can bind to the same IP address with different DNS bindings, but some require binding on entire scopes, so they have their own IP addresses.

Currently we use a rather complex but very customizable and generic build and deploy system, that was built in house on top of a commercial build runner, very similar concept to Jenkins I believe, but windows based and running from an IDE (with a Web-UI for developers to run and deploy services on these 3 environments).

I was tasked with finding a way to "pack" the entire trading platform - with all ~60 micro services and the relevant infrastructure (rmq, ) into something that can be deployed on demand per developer, so we could spin up entire environments in matter of minutes
and have a separate "dev" environment for each developer, created and destroyed on demand.

What could be a good way of achieving this?
Right now I am perusing the idea of a single windows machine, hosting all these services on it, that can be cloned on re-configured with a custom dns name via scripting.
Still not 100% sure this is the best way to go around this, and it does have some challenges
(For example: many if not most of the services should be accessible from outside the self-contained environment, so that the developers can access their APIs and test them with external tools, there are many IP addresses internally, trying to figure out a way to only expose only a single IP address on the host network and keep the rest internal to the "all in one" server.

I have 0 knowledge and experience with containers, not entirely sure if that's the best idea here.
I would love to hear your throughs on this, and how would you achieve this goal and be able to spin complete environments up/down on demand.

Thanks!!

https://redd.it/k5svoc
@r_devops

Scanning artifacts for tokens, passwords and other secrets

Hello all

In our company we want to scan artifacts that are uploaded to Nexus repository for passwords, tokens and secrets. We want to avoid situations that someone upload artifact with secret by mistake or make sure that this person made in on purpose.
I'm looking for tools to check packages for such things. It doesn't need to one swiss army knife to do it all, it can be set of tools to scan different standards like npm, mvn etc.

I'm aware that it may contains many false positives and false negatives but it may improve quality of our products

https://redd.it/k56uy1
@r_devops

Elasticsearch on Docker - file permission issues

I am trying to test LDAP in elasticsearch/kibana by running it in Docker on Windows 10.

I pulled the image down:

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.10.0

And ran it:

docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.0

I did the same with Kibana and everything came up as expected. I then upgraded the elastic licence and started the 30 day trial to get the LDAP features in X-Pack.

I opened a shell to the container and updated the elasticsearch.yml file with my LDAP information and set the path to the role\_mapping.yml file. I also updated role\_mapping.yml file with the LDAP informaiton. I followed the instructions here: [https://www.elastic.co/guide/en/elasticsearch/reference/7.10/ldap-realm.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.10/ldap-realm.html)

Now when I restart the elasticsearch container I get the following error then the container halts:

uncaught exception in thread [main]
java.lang.IllegalStateException: security initialization failed Likely root cause: java.security.AccessControlException: access denied ("java.io.FilePermission" "/role_mapping.yml" "read")
at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.base/java.security.AccessController.checkPermission(AccessController.java:1036)
at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408) at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:747)
at java.base/sun.nio.fs.UnixPath.checkRead(UnixPath.java:810)
at java.base/sun.nio.fs.UnixFileSystemProvider.exists(UnixFileSystemProvider.java:524)
at java.base/java.nio.file.Files.exists(Files.java:2514) at org.elasticsearch.xpack.core.XPackPlugin.resolveConfigFile(XPackPlugin.java:361) at org.elasticsearch.xpack.security.authc.support.DnRoleMapper.resolveFile(DnRoleMapper.java:86)
at org.elasticsearch.xpack.security.authc.support.RoleMappingFileBootstrapCheck.create(RoleMappingFileBootstrapCheck.java:47)

I overwrote the elasticsearch.yml file by copying a version with the LDAP config commented out so I could start the container. When I did this I checked the file permissions and both elasticsearch.yml and role\_mapping.yml have the same:

-rw-rw---- 1 elasticsearch root 199 Dec 2 15:51 elasticsearch.keystore
-rwxr-xr-x 1 root root 568 Dec 2 18:36 elasticsearch.yml
-rw-rw---- 1 elasticsearch root 2301 Nov 9 21:28 jvm.options
drwxrwxr-x 2 elasticsearch root 4096 Nov 9 21:32 jvm.options.d
-rw-rw---- 1 elasticsearch root 8451 Nov 9 21:35 log4j2.properties
-rwxr-xr-x 1 elasticsearch root 502 Dec 2 20:15 role_mapping.yml
-rw-rw---- 1 elasticsearch root 197 Dec 2 20:10 roles.yml
-rw-rw---- 1 elasticsearch root 0 Nov 9 21:32 users
-rw-rw---- 1 elasticsearch root 0 Nov 9 21:32 users_roles

I even tried CHMODing the role\_mapping.yml file to 777 but I still get the same result.

I feel like I'm missing something obvious...

&#x200B;

edit: formatting

https://redd.it/k5y3y2
@r_devops

Flagr – a feature flagging, A/B testing, and dynamic configuration microservice

I thought the r/devops subreddit might be interested in this project I just found!

https://github.com/checkr/flagr

If you like this, [I do a weekly roundup of open source projects that includes an interview with one of the devs you can subscribe to.](https://console.substack.com/)

https://redd.it/k614vs
@r_devops

Major Pagerduty outage

Status page - https://status.pagerduty.com/incidents/tpvmgn9nnnfb

Event Investigation Issue Affecting Global Routing Keys

https://redd.it/k665k7
@r_devops

Keycloak alternative!

I'm looking for an open source Auth server, Keycloak seems pretty good but doesn't support MongoDB database, I also found Gluu but its system requirement is a little bit high, do you guys know any good alternative that support MongoDB?

https://redd.it/k6747i
@r_devops

GitOps CI/CD tools for non-Kubernetes workloads, like Spark Structured Streaming?

Hello,

So, I'd like to have some declarative way to manage Spark Structured Streaming deployments: rollbacks, healthchecks, logs, etc. I've read about Airflow, but it seems very finely tuned to run batch applications only.

Most of deployment tools like ArgoCD, FluxCD, etc do provide a very nice workflow, but are extremely specific to Kubernetes... Do you guys know of any alternative? How are streaming jobs usually managed?

https://redd.it/k69rnp
@r_devops

gitlab-ci.yml pipeline merge branch

I'm not finding any documentation on how to do this? But maybe I'm not wording my question correctly for google. This is what I'm trying to do....

Repo branches

* dev
* master

When I run my pipeline on dev I would like to do this...

* validate
* This will do the needed checks on stuff
* If this passes, merge the dev branch with the master branch
* build
* Do build stuff in master branch

Is this something that is possible?? If so are there any examples someone could point me to?

https://redd.it/k670pk
@r_devops

Is there a good AWS tutorial that covers all the aspects and development and devops?

I started to look at Stephane Maarek AWS DevOps course. After I bought it, it suggested that I needed to have some AWS experience.

I could do both his courses. If I study something in the DevOps course and notice I need more exposure I can refer to Stephane's Developer course.

I was curious if there was a class/course/tutorial that covered building a product in AWS that taught a person developer & devops skills.

https://redd.it/k670i2
@r_devops

Learning contains AWS (basicly DevOps) with CloudAcademy or LinuxAcademy or aclodguru

Hi I wanted to learns DevOps skills and I wonder which subscription should I buy. I want to have courses with a lot of labs included (hands on labs- learning by doing).

Which one do you recommend and why?

Does this certificates matter in job market?

Thanks!

https://redd.it/k65rvr
@r_devops

Authenticated docker pulls with selfed hosted gitlab

I was wondering if anyone can point me in the right direction, we use self hosted GitLab CI/CD. (We manage the runners in EKS) With the recent Docker pull limits implemented we started occasionally hitting limits. We are planning to purchase a Docker team account but I am unsure how to actually make sure the runners are using the team account since they are currently pulling as anonymous.

I was looking into https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ but not sure if this is the right direction...

FWIW We stand up our EKS cluster with terraform and deploy the Gitlab runners with a helm chart

https://redd.it/k61muy
@r_devops

CICD for WordPress to deploy custom plugin + knockout js, using GitLab

Hi,

Where can I find someone professional to help me set this up? I prefer to hand over the task, instead of hacking it together and experimenting myself.

I have a WordPress site which contain 2 custom things which is version controlled:

\- Some regular files such as a custom WordPress plugin and some other things. In total 5 folders and subfolders.

\- One of the folder contain a custom knockout script which needs to compile. Developers should have as easy workflow as possible, to save time.

We want to figure out a better workflow and automate as much as possible. Looking for suggestions who to hire and approximate number of hours needed?

&#x200B;

\*\*Background:\*\*

Current setup is not optimal:

>! \- Developers develop fetch latest git version, and develop in local workstation. They upload it to a "dev-server" (different environment than live server) and inside of that server, they run a set of commands (to compile knockout etc), and then they check their work. If OK, then they commit the compiled code to staging branch, which using cicd script will rsync the select folders to staging server (same environment as live server). Check everything again, and repeat with master branch which rsync to live serer. !<

https://redd.it/k5y63v
@r_devops

Beginner Question: Deploying server application with Terraform and Packer

I've been developing most of my applications to run using Docker but want to have a better understanding on some other deployment patterns. I'm looking for a guide or some clarification on how to effectively use Terraform and Packer to deploy applications *without* relying on Docker. Is this possible? What is the pattern.

From my basic understanding (using AWS) is Terraform can help me make the virtual server instance (EC2) and can be defined using an AMI which can be used to create an image with a copied binary that has been compiled for deployment. But is that it? Will the application start on its own or do I need something else like Ansible to start the application?

https://redd.it/k5x7e8
@r_devops