OpenTelemetry and everything included

I do a lot of work with OpenTelemetry, and I got tired of not having a good way to test out theories, experiment, and learn. By it's nature, OpenTelemetry alone and in isolation is pretty useless. You need an application to generate telemetry, and the observability tools to analyze it.

So I created [otel-shopping-cart](https://github.com/trstringer/otel-shopping-cart). This is everything:

* Microservices application that generates telemetry (traces, metrics, and logs)
* Traffic generator (an idle application doesn't generate telemetry)
* Quality-of-service interrupter (if all traces are <5ms that can be pretty boring)
* OpenTelemetry
* Multiple backends, including the whole Grafana stack, Jaeger, Elasticsearch

Now I can quickly get a meaningful OpenTelemetry implementation up and running and experiment.

Hopefully others can find some use in this tool. [Here's](https://trstringer.com/openetelemetry-sample-application/) some more longer-winded info on this, but all of the relevant info is in the GitHub repo.

https://redd.it/1bo4ks8
@r_devops

Started making YouTube videos on DevOps related topics, looking for constructive criticism

Hello, I’m looking for some constructive criticism on some of my videos. I took a hiatus between making videos long ago about saltstack and getting more into other videos now. I want constructive criticism so I can improve my skills and provide/promote better videos. Thank you!

Below are two of my latest on dagger and pulumi. I’ve also added my channel link. Again thank you!

Cloud infrastructure-as-code with a real language - Pulumi
https://youtu.be/kvKo0dQqn1Y

CICD As Code - Dagger
https://youtu.be/0I6-gdHiOJY

https://youtube.com/@Code-Militia?si=DSLvzqTDCFYckxXl

https://redd.it/1bo47o9
@r_devops

aws S3 system account: where to put the credentials in the development?

Hello,

I am developing a small JAVA app that requires S3 "CRUD" permission...

I am very confused where to put the credentials (ID and secret) that will allow the app to access S3...

anybody can help?

https://redd.it/1bo557s
@r_devops

The Ultimate Docker Compose Cheat Sheet

Hello DevOps Community!
As a university lecturer, I know the importance of sharing knowledge. That’s why I’m excited to share a comprehensive Docker Compose Cheat Sheet that I’ve created. It is available as PDF and PNG. It’s completely free, with no email signup or any other requirements.
This Cheat Sheet is part of my detailed blog post about Docker Compose. Here is the link: https://devopscycle.com/blog/the-ultimate-docker-compose-cheat-sheet/ It walks through the essentials of managing multi container application with Docker Compose. You will also learn about accessing running containers and how to persist data. The post also includes a public GitHub repository. It comes with all the source code, available for anyone to use and learn from. It is an excellent entry point for an internal hackathon at your company or university. If you struggle with learning Docker, we have a blog article and video series for that too.
If you like the blog post, the Docker Compose Cheat Sheet or the GitHub repository, I would appreciate it if you share it. Your feedback and experiences are always welcome. Let’s keep learning and growing together!
Cheers,
Mr_LA

https://redd.it/1bo8s8b
@r_devops

Fluent Bit 3.0!

📷 Exciting News: Fluent Bit 3.0 is Here! 📷 A serious advancement for cloud native #observability teams, with new features and enhancements to power your telemetry needs and keep you in control of your data. #OpenSource


Read more on the u/Calyptia blog: https://calyptia.com/blog/fluent-bit-v3

https://redd.it/1bo7zeu
@r_devops

Transitioning from CS Undergrad to DevOps: Seeking Advice!

Hey everyone,
I'm currently wrapping up my CS undergrad and am deeply fascinated by the world of DevOps. I understand it's typically not considered an entry-level path, but it's where I see my passion and career heading. I'm aiming for postgrad studies in the US to help bridge this transition and am actively seeking advice on how to best position myself for a career in DevOps. Given that I have some time before I dive into the workforce, I want to use it wisely to build a solid foundation.
Here's where I'm at:
I have a solid CS background and am comfortable with programming and software engineering principles.
I'm open to recommendations on certifications, projects, or any resources that could help make me a viable candidate for a DevOps role in the future.
What are the key skills and experiences I should focus on acquiring? Are there specific technologies or tools I should prioritize? Additionally, how valuable would postgrad education in the US be for this career path, and are there particular programs or courses that are well-regarded in the industry?
I'd greatly appreciate any advice, insights, or resources you could share. Also, if anyone has made a similar transition or is willing to share their journey into DevOps, I'd love to hear about it!
Thanks in advance for your help :)

https://redd.it/1bo9c0e
@r_devops

Seeking Advice: Transitioning from DevOps to Developer Role - How to Make the Move?

Hey everyone,

I'm currently working in DevOps with around 5.5 years of experience under my belt. My work involves scripting in Python, Golang, and extensive involvement with Artifactory plugins, which includes code from Groovy and Java. Additionally, I have experience working on Java middleware.

I'm contemplating a career transition into a developer role and would appreciate any guidance or advice on how to make this switch effectively. Given my background, I'm curious about:

1. What steps can I take to position myself as a strong candidate for developer roles? And what should I be preparing?
2. How can I highlight my relevant skills and experience from DevOps during the job search and interviews?
3. Are there any specific certifications or additional training programs that would be beneficial for me to pursue?
4. Any tips or strategies for networking within the developer community and finding job opportunities?

I'm eager to hear from anyone who has successfully transitioned from DevOps to development or anyone familiar with the process.

https://redd.it/1bodgpl
@r_devops

How to apply for Entry-Level DevOps Jobs as a Fresher?

I'm seeking advice on entering the DevOps field and navigating the job application process. Specifically, I'm curious about the commonly asked questions in entry-level DevOps job interviews and what salary package I can expect.
I'm a 2023 B.Tech graduate in CSE with an 8.4 CGPA.
I know AWS, Linux, Docker & Containers, K8s, Bash Shell, and Ansible.
I'm preparing for the AWS Solution Architect Exam and studying Infrastructure as Code (IaC) like Terraform and other CI/CD tools like Jenkins and ArgoCD.
However, I don't have enough confidence to apply for jobs. Please help me get through this. Also, I know basic Python, but I don't have any interest in programming.

https://redd.it/1boen3w
@r_devops

Impulse v.1.10 released: Real-time threat detection & integrity monitoring for Linux platforms

Hello everybody,
Deploying the next-generation of security telemetry technologies to production is hard, so I’m building a fully automated SIEM/XDR platform that leverages tools like Osquery and eBPF to provide real-time threat detection & integrity monitoring for Linux servers and workstations.
It detects malware from behavioural patterns rather than signatures and enables deeper visibility than legacy tools. It can be deployed on any device or VM running Linux such as cloud VMs in VPC networks, VPS servers or personal workstations and IoTs.
In terms architecture, it is organised around a self-hosted, manager-sensor model that provides traditional SIEM capabilities like centralized log storage, indexing and normalization, but also automated log-correlation and real-time threat detection via 2 open-source sensors (one called “light” for host IDS, and “heavy” sensor with network IDS capabilities).
It’s designed to be fast and easy to use; installs in 5 mins on as little as 1.5 gb RAM, 1-core VM.
Repository: https://github.com/bgenev/impulse-xdr
Please feel free to share any requests for further development.
Main features include:
• Security Analytics: Ingests telemetry data from its fleet of monitoring sensors and provides security analytics & insights.
• Integrity monitoring: Provides integrity monitoring for every aspect of your environment - files, processes, connections, ports, users, authentications, installed packages, kernel modules, etc. every variable that could be an indicator of compromise is tracked and added to “IOCs History” database.
• Network Visibility & IDS: Monitors network flows, detects intrusion attempts and automatically blocks offenders with active response.
• File Integrity Monitoring: Tracks changes on the filesystem tree and notifies you about file or permission modifications.
• Security Policies: Monitors system configuration settings to ensure compliance with preset core security policies.
• Active Response: Blocks suspicious IPs, stops processes, closes ports and quarantines files.
• Fleet Firewall: Fleet firewall blocks offenders across the fleet.
• Threat Intel: Integrates with high-quality threat intelligence providers to enrich your context data.
• Vulnerability Scanning: Discovers installed packages and associated CVEs.
• Self-Hosted & Open-Core: Data never leaves you servers.


https://redd.it/1boe375
@r_devops

Who has the nicest docs?

Thinking primarily about charts and containers, who are the people with the best documentation?

Whose Github repo has the best Issue and Pull Request templates?

Looking for inspiration about how to do things the right way. Ease of maintenance and clean formatting. Good ci integration.

https://redd.it/1boh2z3
@r_devops

Datadog Question

Hello,

My company started using Datadog recently and we've run into a problem. For context I am a SWE putting on a devops hat for this one. I appreciate any help that anyone can provide.

So, the problem is that we log our requests and the one application has requests around 700 key:value pairs. But, seems to be too large for datadog because they truncate it. Size wise they are about 20kb which is well under the 256KB limit with datadog agent. I can't seem to find any information of how to actually handle this. Datadog just asks why we have such a large JSON object and to respond it is what another team structured their api. Do I need to fight for a change in how large these request objects are or has anyone faced this and have a nice clean solution for logging such "large" objects?

https://docs.datadoghq.com/logs/log\_collection/?tab=hostThis portion below:

The HTTPS API supports logs of sizes up to 1MB. However, for optimal performance, it is recommended that an individual log be no greater than 25K bytes. If you use the Datadog Agent for logging, it is configured to split a log at 256kB (256000 bytes).A log event should not have more than 100 tags, and each tag should not exceed 256 characters for a maximum of 10 million unique tags per day.A log event converted to JSON format should contain less than 256 attributes. Each of those attribute’s keys should be less than 50 characters, nested in less than 10 successive levels, and their respective value should be less than 1024 characters if promoted as a facet.Log events can be submitted with a timestamp that is up to 18h in the past.


Edit: If this sub doesn't seem like the right place for you and have a good idea for another sub to post this in that would also be appreciated.

https://redd.it/1bojkrs
@r_devops

Best place to find Devops Engineer interested in hourly work?

I'm working on an application that uses Django and interacts with a Next.js application using the django-nextjs package (https://github.com/QueraTeam/django-nextjs). There is also a Rust service.

I've been looking for someone to help me create a simple containerized deployment and set up CI pipelines, but I've been having trouble finding someone.

Any ideas of websites or devops communities where I may be able to find someone?

https://redd.it/1bokssb
@r_devops

Self Service Models

Looking through an article about the DevOps Maturity Model, I noticed that the top level gets to self service. What does that look like in your organization? How many people are there?

https://redd.it/1bol1p7
@r_devops

A new guy might be faking his experience. What should I do?

I’m a DevOps contractor from a consulting company working a staff augmentation project and we just got a new FTE on our DevOps team. It seems one of my consulting teammates thinks that the new FTE is faking some of his past experience.

The FTE says he worked at the same company, at the same time more than 5 years ago, doing the same job, and that they know some of the same people, but our fellow consultant doesn’t remember this FTE. The company we’re talking about is a very large company, but our consultant has reasons to believe the FTE is lying.

On the other hand, this company has an intense interviewing process, and they do employment checks, so if the FTE is here then he passed all of that. The FTE also seems to have some decent skills, seems to do good work, and is very eager to learn.

Personally, I don’t even like my colleagues, they’re assholes. And I think they’re just jealous and hating on this guy & acting like petty hall monitors.

What should we do? Should we report our consultant’s concerns to management, or just let the FTE be?

https://redd.it/1bonrvv
@r_devops

So Many Hats

Is anyone else in a role/shop where half working code gets thrown over the fence and you have to make it work, all while hearing that the infrastructure is to blame? How does everyone deal with that mental beating? Also, feel free to feel me that I am just doing things wrong and your shop is much better than this 🤣.

https://redd.it/1bopg2o
@r_devops

What is the best platform for ops automation

For general ops tasks (think database migrations, running hotfixes, etc) what is the best platform?

The only platform i can think of that is dedicated to this kind of stuff is rundeck.

https://redd.it/1bovk40
@r_devops

Running Tekton Pipelines on Kubernetes at Scale

Running Tekton CI/CD pipelines on Kubernetes/OpenShift at scale: https://piotrminkowski.com/2024/03/27/running-tekton-pipelines-on-kubernetes-at-scale/

https://redd.it/1bowqxj
@r_devops

Senior Devops doesn’t know anything. It’s really frustrating

So, I recently started working as a DevOps engineer at an online shopping platform, and boy, let me tell you, it's been a rollercoaster ride already. The interview process was insanely hectic, and there were very few openings available.

Here's the kicker: my senior doesn't seem to know much about what we're actually working with. We're using GCP and Terraform for infrastructure as code (IaC), but they're clueless about even the basics. I'm talking about not knowing how a simple Cloud Build file is created or how it functions.

In our meetings, it feels like we're discussing nonsense most of the time. They're adamant about creating our own Google Cloud logs because apparently, using Logs Explorer is too expensive. but then they stumble when it comes to knowing which machine types are suitable or understanding the differences between them.

It's frustrating because there are countless talented, unemployed freshers out there who could bring so much more to the table. And don't even get me started on CI/CD pipelines – they're a foreign concept to my senior.

Has anyone else experienced something similar? How did you cope with it? Any advice would be greatly appreciated. Thanks for letting me vent.

https://redd.it/1boxx67
@r_devops

how are all of you testing your ansible?

I'm currently undertaking a project to switch our chef to ansible piece by piece, for chef we are leveraging the spec tests which do a decent enough job.

in looking at testing ansible most people seem to mention Molecule but in trying it I can't really seem to find many examples as to how the default verifier "ansible" is actually meant to be written. I've noticed though that many have opted to use testinfra instead.

For local testing in chef we use test kitchen and Molecule seems to emulate that part completely fine.

Curious as to how all of you are managing.

https://redd.it/1boyqdj
@r_devops

Fullstack dev tasked to do some devops : looking for pointers to learn what i should know.

Hello r/devops and sorry if my title is too vague, i'll enter in the details.

So, I'm part of a small dev teams, 5 people, and until today we were self hosting all of our tools : Jira for out ticketing system, Jenkins for our Continuous Integration and unit testing, perforce -soon to be git- repo. We're tech-savvy and can rtfm so we set up the vms to host these services ourselves, but it's in no way our job or our domain.

2024 is the year of the migration to the cloud for us as asked by the management. We already migrated Jira and our repo should be done by the end of the year.
I've been asked to look for solutions to do the same with our CI. The complexity seems a level higher as CI is not "static" in the sense that it needs to pull files, to have sdk installed to compile them (we do .Net6), to run unit tests on the result and copy it to a place where we can access it, all while keeping the interactions between Jira and our CI for version release for example.
And of top of that, there's the security which I also worry about.

I'm not tasked to do the actual migration, just to look for potential solutions, but I don't know where to look at.

Is it common to have cloud-hosted CI ? If so, are there resources you advise me to read ? Do you have an opinion on how I should do?

Thanks a lot.

https://redd.it/1boysrj
@r_devops

developer deleted heroku server and the frontend is not loading

so the developer I was working with was probably mad, and he deleted the heroku server without finishing the transition, now some parts of the frontend are not loading... i'm not technical and in the process of finding another developer. Is this easy to fix?

https://redd.it/1bp1t74
@r_devops