How to make sure that Lambda triggered by an event only executes when the event has been trigger by the right origin?

Hi, I am recent to DevOps and trying to learn on my own, but sometimes have questions around the best trade off, and thought this might be a good forum for it.


I currently have an AWS setup where I have multiple Lambdas that are triggered by events. The idea is that after Lambda #1 runs a service or makes status a change that will send an event and trigger Lambda #2 etc...


The challenge is that the event that triggers Lambda #2 might happen even without Lambda #1 has run.


How do I make sure that Lambda #2 only executes if Lambda #1 has been the trigger for the event that made Lambda #2 run?
Is using a store parameter that is set by Lambda #1 a reasonable solution, or are there better design patterns?



Thank you

https://redd.it/10ug5y4
@r_devops

Question about Health in IT

I been a DevOps engineer for 3 years, been in IT for 6 years. I've studied over 10,000 hours - I've been kinda logging them/keeping track every week to hit my study goals.

I've been noticing lately, even if I have my glasses on- sometimes I can't see. Like everything is kind of a blur. I visited an Optometrist last week and they said my eyesight 20/20, but they give me a very weak prescription with UV light protection lenses. So it seems like my eye sight is fine... it's just after staring at a computer screen 12 hours a day all week it feels like I can't focus on anything anymore. My brain is foggy, and it feels like my soul is forever enslaved by the computer. Sometimes people will talk to me and I can't hold a conversation, my brain is just buzzing/empty or full of IT related stuff and I don't even hear what people are saying to me.

I feel like I need to take a month off to just not look at my phone, not look at any TV's, computer screens, etc.

I try to eat as healthy as possible and exercise, any thoughts?

https://redd.it/10uhtda
@r_devops

What are thoughts about AWS CDK Pipelines?

I saw a demo about this library 2 years ago and thought it was neat to manage pipelines and cloud state via a code first framework.
Is it popular amongst Devops people?
Any experience with it?
Is it production ready in terms of stability and community?

https://redd.it/10uhoao
@r_devops

RAID10 on 8 x Enterprise NVMe drives - Terrible Performance - Help Needed

Hi,
8 x NVMe Samsung pm9a3 with software RAID10 we are getting a terrible performance of around 80k IOPS where we get around 1 million IOPS on a single drive.
The OS is Debian and CPU is Epyc.

What's your experience? Any help really?

Thanks

https://redd.it/10ugz5u
@r_devops

Are you running databases on Kubernetes?

Hello, I came across this Redhat survey claiming that 80% of the workloads deployed on Kubernetes were databases or data cache, based on the answers of their respondents.

I was intrigued by this because I always thought that Kubernetes workloads (i.e. containers) should be ephemeral and stateless. I tried searching on the internet and YouTube, but I'm struggling to find answers to my questions.

Are you running databases on Kubernetes? How does that work? What are you storing? Do all the pods connect to the same disk containing the database itself, or are all pods instances of the same HA database that is replicated between the pods?

https://redd.it/10umist
@r_devops

Are detached DEV cloud environments secure (and normal) for DevOps?

My organization is considering having cloud development accounts, that are fully managed by developers. Developers alone would have root/admin privileges. The environments would be Internet accessible, but not accessible from the VPN

The developers would be setting up permissions, networking policies, etc. Before bringing an environment back on the VPN, the developers would review the Terraform infrastructure code with security and networking team.

The problem is that our developers are not trained in security which could lead to unintentionally insecure Dev environments.

Have you seen development environments managed in this way before? Is it secure? Any issues?

https://redd.it/10uniyz
@r_devops

I'm getting tired of Terraform and want to give Pulumi a try. Looking for some suggestions

Coming from a TF background, I am starting to just get sick of all the nonsense surrounding making HCL work. I've used Terragrunt to keep code dry, and obtained my HTFCA.

The more I want to develop personal projects, the more I feel like coding in HCL is just slowing me down. All the little oddities and such.

So some questions I have for you Pulumi users:

1. Did you do this, and what were some of the pain points?
2. Did you end up going back?
3. How did you structure your repo(s)?
4. How did you break out your infrastructure files so that they made the most sense?
5. What language(s) did you use?
6. Edit: How did you implement policy-as-code such as linting, security checks, etc?
7. Any other gotchas I should look out for?

https://redd.it/10uoidy
@r_devops

Zero downtime hosting via horizontal scaling

Hey there

I‘m having trouble coming up with an idea for a shared hosting infra architecture.

My job is basically hosting highly customized web projects with an open-source CMS at its core.
We have a decent amount of customers paying only for shared ressources.
This is unmaintainable since we have to inform every single customer before updating the servers (dozens per server).

Our goals are: easy maintenance and high reliability/„failsafe“ and further horizontal scaling via snapshots


Would it be viable to create


Web servers:
- 1 cluster containing 3 web servers with a load balancer in front (easiest part)

Database:
- 1 database cluster containing 3 db nodes with a load balancer in front (easy but can get quite costly)

Persistant data:
- Self hosted minIO cluster (amazon s3 is ridiculously expensive)


What are some ways to make a session not disconnect in case the webserver goes down?


Maybe someone could point me in the right direction or give some helpful insight, that would be amazing

https://redd.it/10ur55j
@r_devops

CI/CD tooling choices

I'm always curious about the **why** when it comes to others' tooling choices. I'm a public cloud consultant, and prefer certain things between AWS/Azure native, as well as third-party. Lately I've been wondering about CI/CD after seeing a couple comments on Azure DevOps and GitHub Actions. My role in consulting for a while has pretty much been support what is currently in use, with some greenfields here and there. I'm less concerned with the agnostic approach because of flexibility of switching clouds, it happens less than acquiring/merging companies who are on a different cloud; but even in that instance one company's tooling just goes away in favor of the other, so unless there are a ton of A&M's, consistent tooling between clouds isn't a deal breaker.


Below are a few pointed things I'm curious about, if you could completely redo your CI/CD tooling with zero push back and costs were of no concern:

* If you're an AWS shop, do you actually like CodePipeline/CodeDeploy/etc over third-party?
* What specifically do you like/dislike about it to justify it?
* If you're an Azure shop, do you actually like DevOps over third-party?
* What specifically do you like/dislike about it?
* Why are you not using GitHub Actions in a greenfield? I could understand it's not a direct replacement for matured shops, at least at this time.
* Do you mix and match some things, such as using AWS/Azure for build but then use something like ArgoCD, Octopus Deploy, or a number of others for CD?
* Same question for CI?

https://redd.it/10ushi6
@r_devops

A No Code Terraform Tool

Hey guys! What do you think about having a visual Terraform editor? 👋😊

It looks like No Code tools are becoming quite popular so I was thinking how awesome it would be if you could create your Terraform configuration with a No Code approach!

Would anyone be interested in using such a tool? 🤔

https://redd.it/10uv31w
@r_devops

Let share pros and cons for Data base management in Container Environment and Virtualization

Could you please share this with me as the title?

https://redd.it/10uzov3
@r_devops

Securing Admin access to Apache APISIX

API Gateways are critical components in one’s infrastructure. If an attacker could change the configuration of routes, they could direct traffic to their infrastructure. Consequences could range from data theft to financial losses. Worse, data theft could only be noticed after a long time by mirroring the load. Hence, protecting your API Gateway is of utmost importance.

In this short blog post, I’ll list a couple of ways to secure your Apache APISIX admin access.

Read more

https://redd.it/10v0fmy
@r_devops

Surf CLI - New Feature: Fuzzy search DynamoDB (even encoded data)

**DynamoDB:**

[https://github.com/Isan-Rivkin/surf#aws-dynamodb-usage](https://github.com/Isan-Rivkin/surf#aws-dynamodb-usage)

​

**TLDR**

* surf ddb --query "my-text-\*" --table "\^prod" --out json
* Pattern matching inside objects
* Additional Supported formats: JSON, Protobuf, Base64, Binary

​

**Supported Platforms**

* surf <platform> -q <some text>
* AWS Route53, DynamoDB, ACM, S3, Opensearch
* Elasticsearch
* [Logz.io](https://logz.io/)
* Hashicorp Vault, Consul

&#x200B;

**Overview**

SURF is built for Infrastructure Engineers as a CLI tool that enables searching any pattern across different platforms. Usually, the results are returned with a direct web URL.

The search process depends on the context, for example: if you're searching in Vault it'll pattern match against keys. Instead, if you're searching in Route53 AWS a DNS address it'll return links to the targets behind it (e.g Load balancer).

https://redd.it/10v119c
@r_devops

Best book for terraform/azure?

I am currently working through Terraform Up and Running copyright 2019 based on v0.12. It's ok but I'm concerned about the age. What's the best current book for learning tf?

Also, Up and Running is AWS based. That's fine for learning the general syntax but my company is actually in Azure. What's the best book/resource (with examples etc) for learning the Azure provider and deploying resources in Azure?

Amazon.com is full of outdated books and fake reviews. :(

https://redd.it/10v2jvv
@r_devops

OpenLDAP Docker Container issue

Hi, I've used the osixia github repository to spin up a docker container act as a LDAP Slave and I've successfully used the custom configuration and spin up the container. When i checked the logs of the container i saw the following as the output when it tries to sync with the Master.

TLS: peer cert untrusted or revoked (0x102)
TLS: can't connect: (unknown error code).

I don't understand why this issue pops up. Then i blindly searhed in the internet and find some solutions but none of them worked.

TLS_REQCERT demand

Tried the above one but no luck.

Please help me on this TIA.

https://redd.it/10v2dwu
@r_devops

Most Used REST API Authentication Methods & Strategies

Maximize the security of your REST APIs with the help of the Link given below Stay informed on the latest technology and best practices.

Read now!

https://mojoauth.com/blog/rest-api-authentication/

https://redd.it/10v42e0
@r_devops

How much can you really get out of a 4$ VPS?

Many people around here are probably used to facing cloud bills in the six figures, and relying on sophisticated tools like kubernetes and terraform to scale their operations. However, many companies do not need to have large scale systems, and could rely on a few VPS to handle their traffic.

I did a small write-up, in which I explore the extreme example of using a single very cheap VPS. After load testing a dummy application with K6, I found out that a single 4$/month VPS could handle a couple hundred query per seconds without experiencing issues.

Full article here

https://redd.it/10v4lru
@r_devops

Learning Golang

Hi everyone, I wanna learn Golang for DevOps purposes. Now I am using Python and Bash scripting as automation, CI/CD flows, and general scripting. But I am observing Golang's popularity and capability, so it means Golang is more of a preferred language for DevOps usage. Could you recommend for me good sources for starting? Doesn't matter whether they are free or paid sources. I don't need deeply learn Golang at the Software Engineer level. I will use it just for DevOps scripting and integrations.


Thanks for advance

https://redd.it/10v6n34
@r_devops

Android and iOS app development

Hi !

First of all my apologies if this does not fit the sub, I'll delete if it is the case, I'm not sure where to post this.

As part of a project with a friend I need to develop a mobile application a bit like Uber eats. Basically, there will be a customer side, and a manager side (so that everyone can fill in the data related to their business).

I have no experience in app development, we are 2 data scientists with a technical background but almost exclusively Python oriented. I've heard about low-code or no-code tools and I'm a bit lost. I've seen things like bubble, glydeapp etc but there are too many choices!

My question is this: Have you had to deal with similar tools and what did you think of them? Do you think the no code approach is good, especially for a first quick PoC/mvp?

So here it is, sorry if it's a bit vague, it's because I don't really know where to start!

Thanks for your feedback if you have any !

https://redd.it/10v693a
@r_devops

fluentd failing when TLS added

I am adding TLS config to Fluent (working on HTTP), when I add the TLS Config and restart the service it crashes although the config is parsed okay.

There is no passphrase on the cert key (generated from vault).

<source>
@type forward
bind 0.0.0.0
port 24224
tag "host_logs"
<transport tls>
cert_path /etc/pki/tls/certs/fluentd.crt
private_key_path /etc/pki/tls/certs/certs/fluentd.key
</transport>
</source>

service output:

systemctl status td-agent
● td-agent.service - td-agent: Fluentd based data collector for Treasure Data
Loaded: loaded (/usr/lib/systemd/system/td-agent.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Mon 2023-02-06 15:06:22 GMT; 20min ago
Docs: https://docs.treasuredata.com/display/public/PD/About+Treasure+Data%27s+Server-Side+Agent
Process: 17157 ExecStop=/bin/kill -TERM ${MAINPID} (code=exited, status=0/SUCCESS)
Process: 19107 ExecStart=/opt/td-agent/bin/fluentd --log $TD_AGENT_LOG_FILE --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS (code=exited, status=1/FAILURE)
Main PID: 3390 (code=exited, status=0/SUCCESS)

Feb 06 15:06:22 prometheus.server systemd[1]: td-agent.service: control process exited, code=exited status=1
Feb 06 15:06:22 prometheus.server systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Feb 06 15:06:22 prometheus.server systemd[1]: Unit td-agent.service entered failed state.
Feb 06 15:06:22 prometheus.server systemd[1]: td-agent.service failed.
Feb 06 15:06:22 prometheus.server systemd[1]: td-agent.service holdoff time over, scheduling restart.
Feb 06 15:06:22 prometheus.server systemd[1]: Stopped td-agent: Fluentd based data collector for Treasure Data.
Feb 06 15:06:22 prometheus.server systemd[1]: start request repeated too quickly for td-agent.service
Feb 06 15:06:22 prometheus.server systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Feb 06 15:06:22 prometheus.server systemd[1]: Unit td-agent.service entered failed state.
Feb 06 15:06:22 prometheus.server systemd[1]: td-agent.service failed.

logs:

2023-02-06 15:06:20 +0000 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf"
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.1.4'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-flowcounter-simple' version '0.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-kafka' version '0.17.3'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.0.2'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-s3' version '1.6.1'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-sd-dns' version '0.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.5'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-td' version '1.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-utmpx' version '0.5.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-webhdfs' version '1.5.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluentd' version '1.14.3'
2023-02-06 15:06:20 +0000 [info]: brokers has been set: ["kafka.server:443"]
2023-02-06 15:06:20 +0000 [warn]: For security reason, setting private_key_passphrase is recommended when cert_path is specified

https://redd.it/10v9gwu
@r_devops

terraform-repl : A terraform console wrapper for a better interactive console experience

terraform-repl is a terraform console wrapper that aims at providing a better interactive console for evaluating Terraform language expressions.

https://github.com/paololazzari/terraform-repl

If you've used terraform console then you will be familiar with its limitations. This programs allows you to do everything you could normally do with terraform console, plus you can:

- Create new local variables on the fly
- View resources without having to specify the whole identifier
- View command history
- Clear screen

https://redd.it/10v9iv5
@r_devops