Thoughts on Splunk observability tool?

I know people have strong feelings either way on Splunk, especially in the DevOps world, but I've heard good things about Splunk's integration of SignalFx and Omnition into a new observability / tracing / monitoring tool. I believe the idea is to eventually fold in capabilities from Plumbr (seems to be some niche APM capabilities) and Rigor (digital experience monitoring) into the offering as well.


Has anyone looked at it / used it? They are also including a logging solution integrated into the suite called Log Observer.

https://redd.it/n6ccfl
@r_devops

Is it possible to describe tests with xUnit so that the description is displayed in DevOps?

Standard CI/CD pipeline that's running tests on every release. Does anyone know if it's possible to add descriptions to those tests so that they are displayed in DevOps?

https://redd.it/n5b7b7
@r_devops

Cloudflare: I'm using it wrong?

I've been using Cloudflare for a few years now, I manage a few websites and management applications, with therefore restricted areas that contain personal data of many people.

At the moment all this data goes through Cloudflare, so Cloudflare could theoretically see and log all this personal data.

Should this concern me? Should I handle the data communication in some other more confidential way?

What are your experiences and thoughts about it?

https://redd.it/n6xl5x
@r_devops

Fluent-bit syslog input doesn't provide syslog Client hostname/Addr?

Idk if this is the right sub (as there doesn't seem to be a standard fluentd/bit sub) but I am working on log aggregation and filtering of physical devices and I have decided upon using fluent-bit as the syslog aggregator of these devices (which natively can forward their syslog to a pre-defined host/port). I am able to parse the syslog messages emanating from these products with the following basic config:

[SERVICE]
Flush 5
Daemon off
Log_Level debug
Parsers_File parsers.conf

[INPUT]
Name syslog
Alias xsync_syslog
Parser syslog-rfc3164-local
Listen 0.0.0.0
Port 514
Mode udp

[OUTPUT]
Name stdout
Match *
however I see the syslog client connection's address is not included in the event logs:

[0] syslog.0: [1620233326.000000000, {"pri"=>"185", "time"=>"May 5 16:48:46", "ident"=>"notifyd", "pid"=>"1979", "message"=>"In Sync"}]
I've used logstash in the past and it seemed to automatically add the client's hostname to the event however I cannot find any way to do so with fluent-bit (even after reading the documentation unless I missed something). My current solution would be to forward the syslog messages to another service, such as syslog-ng which can add the host info to the messages, and then to fluent bit however i'd like to use as few service as possible. Any suggestions/recommendations?

https://redd.it/n7512e
@r_devops

Face Palm

Today I learned 90% of my company's multi-million dollar revenue generating steams rely on 3 database backups :(

https://redd.it/n73y8w
@r_devops

How do you organize/track your maintenance needs?

Most of my background is strictly in software engineering, but I just joined a new company and our team is hybrid software/ops engineering.

We've got a ton of out-of-date stuff and it's hard to keep track of all the different things that need to be updated/migrated/maintained - OS versions on our cloud VMs, kubernetes version, old language versions (Ruby 2.1, Python 2.7) still in use, old/unsupported Docker images, etc.

Googling around, I haven't come across any industry best-practice thought-leadership kinds of solutions for this, so I figured I'd ask here. How do you keep track of all the stuff your team has to maintain? What works? What doesn't work?

https://redd.it/n74fjq
@r_devops

Has anyone used the O'Reilly book "Python for Dev Ops" for self study, if so would you recommend it to a newb with intermediate python and linux/bash background? Any other materials you'd recommend? I have an O'Reilly account through school if that matters..

Python programmer looking to get into devops, have spent some time learning Linux fundamentals and Bash Scripting. Looking for a good book or video course to follow. I also looked at Kode Kloud but couldn't tell if it was bloat (nearly every course has redundant sections, same videos being used over and over in multiple courses in the learning path etc).

https://redd.it/n70v83
@r_devops

What tools does your organization use for storing binaries?

What tools does your organization use for storing binaries?

How is your experience with it? Any complaints?

https://redd.it/n72406
@r_devops

Load Testing Tool

Can anyone recommend a good Load Testing tool? Not too complicated. Fairly straightforward to install. Preferably free.

https://redd.it/n70qfn
@r_devops

Is there a metrics endpoint for thanos querier which shows all the metrics from the stores and not just thanos itself?

I tried /metrics at end of the thnaos querier url, but it only shows the thanos metrics not the stores metrics that are connected to it

https://redd.it/n6yozh
@r_devops

managing anaconda environments python

I need to distribute my conda environment to 30+ boxes.

There are 2 ways to create an environment. First, conda env create env.yml. Second is create a environment locally, tar and ship to to my hosts. The downside of the former approach is all machines need an installation of anaconda/miniconda. I like the second approach but is there a downside to it?

https://redd.it/n6xpbb
@r_devops

Benthos/Bloblang and Alternatives

I recently saw our data science team started using Benthos. I am wondering how popular is Benthos specifically its bloblang (mapping language). Also, I am wondering what are the alternatives to its bloblang.

https://redd.it/n7z6js
@r_devops

Advice sought: I want to quit my devops job, but scared of outlook

Will be intentionally vague...

- I'm only devops person (doing security, infra, cicd, alerts, sre) in small startup.

- Equity is near zilch (nothing life-changing)... Will quit before 1 year period before it vests (been several months, no change in culture)

- I have enough saved for 6 months, financially, to quit

- Being the only has worn me out. No energy to interview either.

Question: if I quit, how much harder is it to get a similar job coming back (senior+ devops, devops manager) 4-6 months later. I need to quit to take care of myself, but afraid I'll be doing my career a disservice: I'm sure my employer will be dissatisfied (but probably expected otherwise they'd give more equity I presume before I joined if they knew what the role was)

https://redd.it/n87pt3
@r_devops

Using GitHub actions to deploy docker images

My GitHub action using docker image push workflow is successful executed.But image isn't in my docker hub . Where it could have been ? Where exactly image ran when building it ?

https://redd.it/n8i4ck
@r_devops

Working with kubernetes - how do you create development / debug environment?

Infra is in kubernetes and we have 4 services and 4 teams each is responsible for its service.

We have infrastructure as code.

Are there good tools / ways to allow each developer to be able spin up development environment in timely fashion, easily and without huge costs?

Requirements: ability to spin up production replica to develop / debug locally while still connected to environment that replicates one of ours (production for example).

Minikube seems to slow computers too much and somewhere someone is always having some exotic problems as we're working with linux / mac / windows.

Docker-compose requires separate maintenance as its not easy to set it up as our deployments in the cloud. Its also not that reliable and many devs had issues with it.

https://redd.it/n8n5gi
@r_devops

Diagram/document generation tool for Kubernetes cluster.

I was wondering if there was any open source tool that would generate architecture diagrams [or documents\] of a running Kubernetes cluster. I found this tool but it's not free. If you know any open source tool that would facilitate in documenting the architecture of a Kubernetes cluster, please share!

https://redd.it/n8e846
@r_devops

What does it take to do stress testing on a website?

I am a developer and just start to learn things in DevOps and Testing area. So this question might sound too simple, but it's really hard for me.

I am reading articles about load and stress testing to find out the upper limit of my website. I haven't done any tests so far, but I'd like to get some ideas about the following question:

If I'd like to do a stress test on a website running on a cluster with x cpu cores and y Gigabytes memory in total, how many cpu cores and memory do the test device (or test cluster ???) have to have? Please give me some real world data.

PS: I really have no experience about this, any comments are appreciated. Thank you guys!

By the way, is Jmeter the most popular tool in this area? Any other quicker and easier tools?

https://redd.it/n8e29s
@r_devops

Deploy Knative functions as a developer

I’m new to DevOps and Platform Engineering. My team has tasked me with setting up a serverless solution in our k8s clusters. After looking through some options Knative came up and it works well with our existing istio setup.

I have no issue deploying simple stuff with a Knative Service resource but how would this be done by a developer? How would our client engineers deploy a serverless function without interacting with k8s?

https://redd.it/n8g440
@r_devops

Applying library updates - how do you do that?

Hello, DevOps People!

Have a couple of newbie-ish questions regarding the libraries update.
Hope to get some help from the experienced DevOps:


1. Do you update libraries to patch security vulnerabilities, and if yes - where do you find the information about what needs to be updated?
2. When you installed the updated packages - do you reboot the whole server or restart the services?

Thank you in advance for your help to the beginner!

https://redd.it/n8fk1r
@r_devops

Any GoCD users that want Gitea Oauth?

Hi all,

I recently was investigating how to integrate GoCD login with Gitea Oauth and came up with this package https://github.com/DPr00f/gitea-oauth-authorization-plugin

For anyone interested feel free to use and test, I've simply changed GoCD gitlab package to fit Gitea Oauth.

Posting here for visibility only but if someone wants to manage that package let me know.

https://redd.it/n8c5qm
@r_devops

What country to choose for both working CDI and Freelance (taxes, salary, social life)

Hello everyone, I'm a DevOps engineer with 3 years experience. I'm from the middle east. But I'm searching a job on another country. I want to find a city where I can:

* work full time (CDI) as devops engineer and I can do freelance in the night and weekend. I mean when the law allows me to do both and without so much taxes.
* good salary comparing to life's costs. So I can save a good amount of money
* beautiful city to live in (social life, friendly people, transport...)

By the way, I speak English/French and basic German but I'm open to learn new languages too.

For now I have an offer in Paris. Some friends suggest : Zurich(Switzerland), Amesterdam, Portugal , Sweden.

Can you suggest some countries or cities please and tell me why you choose it?

Thanks

https://redd.it/n8p9p4
@r_devops