Collect inspec test results from multiple hosts

So far, I was able to integrate inspec with ansible to run the profiles across all of our 30+ hosts in our environment. The result being the json report is present in individual host. Is there any way we can get data from those reports and publish it to show compliace level per host?

I have read [https://www.reddit.com/r/devops/comments/7ycwaw/what\_tool\_can\_i\_use\_to\_aggregate\_serverspecinspec/](https://www.reddit.com/r/devops/comments/7ycwaw/what_tool_can_i_use_to_aggregate_serverspecinspec/) which doesnt give any answers

I tried geekdave/prometheus\_inspec\_exporter and it gives me a total tests run/skipped/failed but the results doesnt match the output json file.

Chef compliance server is decommisioned where we could push all these reports and get a summary view and Chef automate is a paid tool.

Are there any other option for collecting and showing these reports ?

https://redd.it/fody9n
@r_devops

The manager wants to start using SALT with Terraform. WHY?

What added functionality will adding SALT to do when we already use Terraform?

https://redd.it/fojyzn
@r_devops

HTTP request queueing mechanism question

We have a Kubernetes cluster with around 30 microservices running. In front of the cluster we have nginx acting as a front end web server/reverse proxy to Ingress nginx within the cluster. What is the best way set request rate limit based on the URL to each of the services? For example we need to queue 100 requests to a certain service after 30 concurrent requests, and drop all the requests if the queue of 100 is full. Also set a timeout for requests in the queue say for 30 seconds.


I know that nginx has built-in module to set rate limits but it is very limited and restrics only based on the requester IP address and not URL. Also it doesn't have queues.

Developers may implement this into the services themselves, but this approach may become complex to manage and fine tune after they are deployed.

Please share your experience on this.

https://redd.it/foa89c
@r_devops

DevOps contract roles: pros and cons?

Hello,

I lost my job back in February due to an acquisition related re-org. Perfect timing right? Locally lots of positions are drying up, but I'm seeing an uptick in contract roles coming in from recruiters. I was hoping the community might have some insight on the potential gains or pitfalls of accepting a contract gig like this, aside from the lock in that comes with a contract. Thanks!

https://redd.it/fo9scv
@r_devops

A Prometheus fork for cloud scale anomaly detection across metrics & logs

Gavin from Zebrium here. We've just published/open-sourced a Prometheus fork with the following features:

1. Metrics are streamed in near real-time.
2. Uses very little network bandwidth (testing shows 500x reduction over raw).
3. It does not store metrics locally (no TSDB).
4. Every sample scraped locally will reach the remote server (i.e no dropping of metrics because of timestamp ordering issues).
5. Adds extra labels, that can be used to join these collected metrics with the logs. Please see our fluentd log collector which adds similar labels [here](https://github.com/zebrium/ze-kubernetes-collector)

Details in [this blog](https://www.zebrium.com/blog/a-prometheus-fork-for-efficient-cloud-scale-autonomous-monitoring) and [this Github](https://github.com/zebrium/prometheus) repository.

https://redd.it/fo8gtb
@r_devops

Java / Spring - 2020 Application Deployment?

Hi All, I've been dumped back into Java world after a 10+ year hiatus.

In terms of shipping applications, what is the go to in 2020 for shipping monolith applications?


In a previous life I would just spin up Jenkins, build and scp JAR files around. Yuck, but it got the job done. I don't have K8s handy, and I really couldn't be bothered spinning up a cluster for for a single application + database that doesn't need to scale.

Infrastructure will most likely be VMs in the public cloud (AWS, Azure or GCP). Something that could handle all three would be great.


Help / pointers greatly appreciated.

https://redd.it/fpq9cu
@r_devops

Beginners Luck!! (Looking for help)

Hey there,


Let me tell you about my background a bit. I am someone who has pretty good exposure in the development area and knows and comfortable with Java, C#, Python, etc. I have worked as a game development using unity3D.

Currently, I am a student of Network and Security field. I like networking as much as I loved development and I am constantly looking for something to merge them. From what I researched I think DevOps is the way to go further. However, for something as big as DevOps I am still a beginner and have basic exposure working with Python, Linux or Cloud tech. I wonder if there is something pretty basic that I need to know.


Looking for a list of courses for beginners. (Free courses preferred(As there are so many things going free during these quarantine time)/paid welcomed) Also, it would be a great help if someone can guide me further for my said path. How right is it and how can I get good at it.

P.S. Feel free to private message me if you want to discuss it.

https://redd.it/fppmzk
@r_devops

GitLab CI CD deploy CloudFormation Template

Team is looking to switch from Jenkins to GitLab CI/CD and I am doing some exploration. We have cloudformation template in git and use Jenkins to run aws cli to deploy the cloudformation template and rolling update. What is the best way to do this with GitLab CI/CD?

https://redd.it/fpnf24
@r_devops

Kibana dashboard question. I don't get it.

I am attempting to build a dashboard using Elasticsearch + Metricbeat where I have successfully sent data from metricbeat to the elasticsearch server.


What I think is simple is not, at least for me, and would like a quick understanding.

I am attempting to get a dash of Numbers reflecting the current CPU usage on each machine. This way at a glance I can tell if our CPU spikes after a push.

https://redd.it/fpltdv
@r_devops

Is there a way to deploy sqlite3?



https://redd.it/fpjgqt
@r_devops

Do distributed file systems partition a file across nodes?

In HDFS, a file is partitioned into blocks which are distributed across several nodes.

I am wondering if that is also true in the following distributed file systems: NFS, Andrew File systems, Bayou, Coda?

Thanks.

https://redd.it/fo64ew
@r_devops

Speed up Docker image build (> 2GB image)

Hi guys!

​

I'm a juniur DevOps member and i received a project. The project is that need to reduce deploy time.

One of these are the build, what is taken \~ 12-13 minutes. In the dockerfile has 27 layer now. Included app copy, node and composer install and push to dockerhub.

How can i reduce build time? I finded about docker cache but at this point i don't know how can i implement.

​

Thank you very much any advice, help.

https://redd.it/fo4wxp
@r_devops

is Ansible "exact_count" provisioning the same as Terraform provisioning ?

hi guys, I havent used both tools enough, but when i was looking at the [AWS provisioning feature](https://docs.ansible.com/ansible/latest/scenario_guides/guide_aws.html#provisioning) of Ansible...i find it exactly similar to Terraform's declarative feature.

It basically gets you to the desired state. isnt it pretty much the same as terraform then ?

https://redd.it/fo3zs5
@r_devops

Install Hashicorp Vault independently without Consul

Hi, my organisation is looking forward on installing Hashicorp Vault as our K/V manager to provide secrets to our K8s cluster and local development securely.

Most of the examples and Terraform files were bundled with Consul. Do I really need Consul? I don’t think Vault will not function without Consul.

All help will be appreciated.

And also maybe you can point me to some tutorials or Git repos. Thanks!

https://redd.it/fo0weo
@r_devops

cloudformation or terraform

Hi guys,

what do you think is a better choice to study as a DevOps beginner?
cloudformation or terraform?


thanks, any further comments on the topic much appreciated

https://redd.it/fo2j4e
@r_devops

PoP: You Don't (Always) Need Kubernetes

Episode 87 of the [Practical Operations Podcast](https://operations.fm/) is out! We're talking about Kubernetes again and trying to pull apart the fads from the facts. Is Kubernetes the right environment for everyone? Do you really need Kubernetes in your startup? Listen to find out if you don't (always) need Kubernetes!

[https://link.chtbl.com/dDlLnNPR](https://link.chtbl.com/dDlLnNPR)

We really appreciate hearing from our listeners. If you have comments or suggestions about the show we'd like to hear from you! Maybe you do always need Kubernetes -- let us know right here on Reddit.

This episode sponsored by [42 Lines](https://www.42lines.net/devops?utm_source=practical_operations&utm_medium=podcast&utm_campaign=sponsorship-2020-03) specializing in DevOps consulting including observability, cloud migrations, cost control, security practices and team mentoring.

https://redd.it/fpx2lm
@r_devops

Building DevOps Team for StartUp

Hi DevOps,

I am working for a Startup and We are building the DevOps team, I need some input to on the below.

\- Enterprise DevOps tools - We are planning to go for partnership with Enterprise DevOps tools, Basically our intention is to get the business from the tools. Please suggest which are the tools we can consider as enterprise DevOps tool tools like Splunk, Sumo Logic etc.


\- Best SIEM tools as part of DevSecOps for Enterprise.

\- APM tools.

​

Please suggest. Thanks.

https://redd.it/fpy1gs
@r_devops

Nightmare on the way to SecDevOps

I was a cyber / jr systems admin hopping my way through various jobs in the IT industry, with the goal of trying to get to SecDevOps / DevOps one day. I have an eclectic background with a graduate degree in cyber security, Linux training, Full Stack Web Dev training and experience with ELK, LAMP, and MERN stacks. I ended up landing this job in the financial sector with potential for growth and an excellent boss. There was a lot of work to do but the C-Level seemed receptive and willing to grow. One day the company announced it was going to sell to another company. They ended up not consulting with my boss and kept him in the dark the whole time. This led to him quitting and getting another job as a senior director at a much bigger place. The C-Level came to me and asked if I could do what he did with the aid of all our vendors. I was taught to never lie in IT so I said no, but I told them with the right vendors it’s possible. They seemed to half listen to me, promoted me to IT Manager and aren’t listening to any of my recommendations now. I feel like a doctor who was suddenly asked to become a lawyer. The company still has a small business attitude and is on the verge of some very bad decisions. The director that left called me one day and told me he would like to pull me over into their environment. They have a huge IT team with every department possible, including a full DevOps team! I said yes, with the actions of the C-Level I’m going to leave no matter what now, but I still feel guilt in the back of my head. I know I’m not responsible for a company making bad decisions but I was curious about others DevOps stories. How did you become what you are, was your path as crazy as this; or was it worse?

https://redd.it/fpwzd7
@r_devops

automate setting up and provision Windows build PC

I do automation at a gaming studio - we have a build farm consisting of several consumer gaming PCs (lets call them slaves) that we compile and test games on. Each slave has an SSD (this can be clean formatted) and specific CPU/GPU combo, virtualization of any kind is absolutely out of the question, we need this running pure and on-the-metal, as this is how games are played in the wild.

I need to install a specific version of Windows 10 on a slave, add it to our Windows domain, install software like Visual Studio and various SDKs in a process that is fiddly with lots of user interaction, and then run some shell scripts to register the slave with our build master server so it's ready to work. I would like to be able to redo this process periodically to update Windows or build tools as needed.

Right now, I'm doing this by maintaining a single clean slave, that is not on our domain so it has no domain ID. I keep Windows and software updated normally, and snapshot it periodically with Clonezilla. I then manually go to each running slave, and from a USB key boot into Clonezilla and use it to completely overwrite the local OS drive with the latest snapshot. I re-add the slave to the domain using its fixed name, add local drivers if needed, and run a shell script to register with the build master.

I would love to automate this process, at least more than what it is now, as we could in theory scale up a lot of slaves to cover various hardware profiles, console devkits, etc etc. Is there a better way of doing this?

https://redd.it/fpvd2k
@r_devops

Using Vsphere customization specs with Terraform

I've been pouring over the terraform docs and the internet all day and I cannot for the life of me figure out how to actually apply a customization spec to a vm clone by terraform. All I can find is how to turn it off, not apply a specific one.

We use customization specs to support older infrastructure so I'd like to not have to build separate templates for use with TF, I will if I need too.

Anyone out there using Vsphere's customization specs with Terraform and can point me at the right option to call to use it? I'm fairly new to Terraform, but I've been using vagrant for about 3 years.

https://redd.it/fq7shh
@r_devops

Optimizing UE4 Project Builds With Cloud Native Infrastructure And Containers

Holistic view on the application of general computing cost-effective platforms like Kubernetes for Any-A and any kind of studio workloads.
[blog post](https://www.gamasutra.com/blogs/JoseMoreira/20200327/360277/Optimising_UE4_Project_Builds_With_Cloud_Native_Infrastructure_And_Containers.php)

https://redd.it/fq7gsj
@r_devops