Are detached DEV cloud environments secure (and normal) for DevOps?

My organization is considering having cloud development accounts, that are fully managed by developers. Developers alone would have root/admin privileges. The environments would be Internet accessible, but not accessible from the VPN

The developers would be setting up permissions, networking policies, etc. Before bringing an environment back on the VPN, the developers would review the Terraform infrastructure code with security and networking team.

The problem is that our developers are not trained in security which could lead to unintentionally insecure Dev environments.

Have you seen development environments managed in this way before? Is it secure? Any issues?

https://redd.it/10uniyz
@r_devops

I'm getting tired of Terraform and want to give Pulumi a try. Looking for some suggestions

Coming from a TF background, I am starting to just get sick of all the nonsense surrounding making HCL work. I've used Terragrunt to keep code dry, and obtained my HTFCA.

The more I want to develop personal projects, the more I feel like coding in HCL is just slowing me down. All the little oddities and such.

So some questions I have for you Pulumi users:

1. Did you do this, and what were some of the pain points?
2. Did you end up going back?
3. How did you structure your repo(s)?
4. How did you break out your infrastructure files so that they made the most sense?
5. What language(s) did you use?
6. Edit: How did you implement policy-as-code such as linting, security checks, etc?
7. Any other gotchas I should look out for?

https://redd.it/10uoidy
@r_devops

Zero downtime hosting via horizontal scaling

Hey there

I‘m having trouble coming up with an idea for a shared hosting infra architecture.

My job is basically hosting highly customized web projects with an open-source CMS at its core.
We have a decent amount of customers paying only for shared ressources.
This is unmaintainable since we have to inform every single customer before updating the servers (dozens per server).

Our goals are: easy maintenance and high reliability/„failsafe“ and further horizontal scaling via snapshots


Would it be viable to create


Web servers:
- 1 cluster containing 3 web servers with a load balancer in front (easiest part)

Database:
- 1 database cluster containing 3 db nodes with a load balancer in front (easy but can get quite costly)

Persistant data:
- Self hosted minIO cluster (amazon s3 is ridiculously expensive)


What are some ways to make a session not disconnect in case the webserver goes down?


Maybe someone could point me in the right direction or give some helpful insight, that would be amazing

https://redd.it/10ur55j
@r_devops

CI/CD tooling choices

I'm always curious about the **why** when it comes to others' tooling choices. I'm a public cloud consultant, and prefer certain things between AWS/Azure native, as well as third-party. Lately I've been wondering about CI/CD after seeing a couple comments on Azure DevOps and GitHub Actions. My role in consulting for a while has pretty much been support what is currently in use, with some greenfields here and there. I'm less concerned with the agnostic approach because of flexibility of switching clouds, it happens less than acquiring/merging companies who are on a different cloud; but even in that instance one company's tooling just goes away in favor of the other, so unless there are a ton of A&M's, consistent tooling between clouds isn't a deal breaker.


Below are a few pointed things I'm curious about, if you could completely redo your CI/CD tooling with zero push back and costs were of no concern:

* If you're an AWS shop, do you actually like CodePipeline/CodeDeploy/etc over third-party?
* What specifically do you like/dislike about it to justify it?
* If you're an Azure shop, do you actually like DevOps over third-party?
* What specifically do you like/dislike about it?
* Why are you not using GitHub Actions in a greenfield? I could understand it's not a direct replacement for matured shops, at least at this time.
* Do you mix and match some things, such as using AWS/Azure for build but then use something like ArgoCD, Octopus Deploy, or a number of others for CD?
* Same question for CI?

https://redd.it/10ushi6
@r_devops

A No Code Terraform Tool

Hey guys! What do you think about having a visual Terraform editor? 👋😊

It looks like No Code tools are becoming quite popular so I was thinking how awesome it would be if you could create your Terraform configuration with a No Code approach!

Would anyone be interested in using such a tool? 🤔

https://redd.it/10uv31w
@r_devops

Let share pros and cons for Data base management in Container Environment and Virtualization

Could you please share this with me as the title?

https://redd.it/10uzov3
@r_devops

Securing Admin access to Apache APISIX

API Gateways are critical components in one’s infrastructure. If an attacker could change the configuration of routes, they could direct traffic to their infrastructure. Consequences could range from data theft to financial losses. Worse, data theft could only be noticed after a long time by mirroring the load. Hence, protecting your API Gateway is of utmost importance.

In this short blog post, I’ll list a couple of ways to secure your Apache APISIX admin access.

Read more

https://redd.it/10v0fmy
@r_devops

Surf CLI - New Feature: Fuzzy search DynamoDB (even encoded data)

**DynamoDB:**

[https://github.com/Isan-Rivkin/surf#aws-dynamodb-usage](https://github.com/Isan-Rivkin/surf#aws-dynamodb-usage)

​

**TLDR**

* surf ddb --query "my-text-\*" --table "\^prod" --out json
* Pattern matching inside objects
* Additional Supported formats: JSON, Protobuf, Base64, Binary

​

**Supported Platforms**

* surf <platform> -q <some text>
* AWS Route53, DynamoDB, ACM, S3, Opensearch
* Elasticsearch
* [Logz.io](https://logz.io/)
* Hashicorp Vault, Consul

&#x200B;

**Overview**

SURF is built for Infrastructure Engineers as a CLI tool that enables searching any pattern across different platforms. Usually, the results are returned with a direct web URL.

The search process depends on the context, for example: if you're searching in Vault it'll pattern match against keys. Instead, if you're searching in Route53 AWS a DNS address it'll return links to the targets behind it (e.g Load balancer).

https://redd.it/10v119c
@r_devops

Best book for terraform/azure?

I am currently working through Terraform Up and Running copyright 2019 based on v0.12. It's ok but I'm concerned about the age. What's the best current book for learning tf?

Also, Up and Running is AWS based. That's fine for learning the general syntax but my company is actually in Azure. What's the best book/resource (with examples etc) for learning the Azure provider and deploying resources in Azure?

Amazon.com is full of outdated books and fake reviews. :(

https://redd.it/10v2jvv
@r_devops

OpenLDAP Docker Container issue

Hi, I've used the osixia github repository to spin up a docker container act as a LDAP Slave and I've successfully used the custom configuration and spin up the container. When i checked the logs of the container i saw the following as the output when it tries to sync with the Master.

TLS: peer cert untrusted or revoked (0x102)
TLS: can't connect: (unknown error code).

I don't understand why this issue pops up. Then i blindly searhed in the internet and find some solutions but none of them worked.

TLS_REQCERT demand

Tried the above one but no luck.

Please help me on this TIA.

https://redd.it/10v2dwu
@r_devops

Most Used REST API Authentication Methods & Strategies

Maximize the security of your REST APIs with the help of the Link given below Stay informed on the latest technology and best practices.

Read now!

https://mojoauth.com/blog/rest-api-authentication/

https://redd.it/10v42e0
@r_devops

How much can you really get out of a 4$ VPS?

Many people around here are probably used to facing cloud bills in the six figures, and relying on sophisticated tools like kubernetes and terraform to scale their operations. However, many companies do not need to have large scale systems, and could rely on a few VPS to handle their traffic.

I did a small write-up, in which I explore the extreme example of using a single very cheap VPS. After load testing a dummy application with K6, I found out that a single 4$/month VPS could handle a couple hundred query per seconds without experiencing issues.

Full article here

https://redd.it/10v4lru
@r_devops

Learning Golang

Hi everyone, I wanna learn Golang for DevOps purposes. Now I am using Python and Bash scripting as automation, CI/CD flows, and general scripting. But I am observing Golang's popularity and capability, so it means Golang is more of a preferred language for DevOps usage. Could you recommend for me good sources for starting? Doesn't matter whether they are free or paid sources. I don't need deeply learn Golang at the Software Engineer level. I will use it just for DevOps scripting and integrations.


Thanks for advance

https://redd.it/10v6n34
@r_devops

Android and iOS app development

Hi !

First of all my apologies if this does not fit the sub, I'll delete if it is the case, I'm not sure where to post this.

As part of a project with a friend I need to develop a mobile application a bit like Uber eats. Basically, there will be a customer side, and a manager side (so that everyone can fill in the data related to their business).

I have no experience in app development, we are 2 data scientists with a technical background but almost exclusively Python oriented. I've heard about low-code or no-code tools and I'm a bit lost. I've seen things like bubble, glydeapp etc but there are too many choices!

My question is this: Have you had to deal with similar tools and what did you think of them? Do you think the no code approach is good, especially for a first quick PoC/mvp?

So here it is, sorry if it's a bit vague, it's because I don't really know where to start!

Thanks for your feedback if you have any !

https://redd.it/10v693a
@r_devops

fluentd failing when TLS added

I am adding TLS config to Fluent (working on HTTP), when I add the TLS Config and restart the service it crashes although the config is parsed okay.

There is no passphrase on the cert key (generated from vault).

<source>
@type forward
bind 0.0.0.0
port 24224
tag "host_logs"
<transport tls>
cert_path /etc/pki/tls/certs/fluentd.crt
private_key_path /etc/pki/tls/certs/certs/fluentd.key
</transport>
</source>

service output:

systemctl status td-agent
● td-agent.service - td-agent: Fluentd based data collector for Treasure Data
Loaded: loaded (/usr/lib/systemd/system/td-agent.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Mon 2023-02-06 15:06:22 GMT; 20min ago
Docs: https://docs.treasuredata.com/display/public/PD/About+Treasure+Data%27s+Server-Side+Agent
Process: 17157 ExecStop=/bin/kill -TERM ${MAINPID} (code=exited, status=0/SUCCESS)
Process: 19107 ExecStart=/opt/td-agent/bin/fluentd --log $TD_AGENT_LOG_FILE --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS (code=exited, status=1/FAILURE)
Main PID: 3390 (code=exited, status=0/SUCCESS)

Feb 06 15:06:22 prometheus.server systemd[1]: td-agent.service: control process exited, code=exited status=1
Feb 06 15:06:22 prometheus.server systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Feb 06 15:06:22 prometheus.server systemd[1]: Unit td-agent.service entered failed state.
Feb 06 15:06:22 prometheus.server systemd[1]: td-agent.service failed.
Feb 06 15:06:22 prometheus.server systemd[1]: td-agent.service holdoff time over, scheduling restart.
Feb 06 15:06:22 prometheus.server systemd[1]: Stopped td-agent: Fluentd based data collector for Treasure Data.
Feb 06 15:06:22 prometheus.server systemd[1]: start request repeated too quickly for td-agent.service
Feb 06 15:06:22 prometheus.server systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Feb 06 15:06:22 prometheus.server systemd[1]: Unit td-agent.service entered failed state.
Feb 06 15:06:22 prometheus.server systemd[1]: td-agent.service failed.

logs:

2023-02-06 15:06:20 +0000 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf"
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.1.4'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-flowcounter-simple' version '0.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-kafka' version '0.17.3'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.0.2'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-s3' version '1.6.1'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-sd-dns' version '0.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.5'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-td' version '1.1.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-utmpx' version '0.5.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluent-plugin-webhdfs' version '1.5.0'
2023-02-06 15:06:20 +0000 [info]: gem 'fluentd' version '1.14.3'
2023-02-06 15:06:20 +0000 [info]: brokers has been set: ["kafka.server:443"]
2023-02-06 15:06:20 +0000 [warn]: For security reason, setting private_key_passphrase is recommended when cert_path is specified

https://redd.it/10v9gwu
@r_devops

terraform-repl : A terraform console wrapper for a better interactive console experience

terraform-repl is a terraform console wrapper that aims at providing a better interactive console for evaluating Terraform language expressions.

https://github.com/paololazzari/terraform-repl

If you've used terraform console then you will be familiar with its limitations. This programs allows you to do everything you could normally do with terraform console, plus you can:

- Create new local variables on the fly
- View resources without having to specify the whole identifier
- View command history
- Clear screen

https://redd.it/10v9iv5
@r_devops

cloudtrail-event-fuzzy-viewer: cli tool for searching cloudtrail events using fuzzy search

I built a cli tool that allows you to easily search and view cloudtrail events.

The program fetches cloudtrail events with the aws cli lookup-events cli call, and then gets you in an interactive fuzzy search command line (fzf).

For whichever event you look for, the body of the event itself is displayed on the right.

Here's how it looks:

https://github.com/paololazzari/cloudtrail-event-fuzzy-viewer/blob/master/doc/demo.png

You can find it on github:

https://github.com/paololazzari/cloudtrail-event-fuzzy-viewer

https://redd.it/10v9e2e
@r_devops

Is it possible for ticket statuses to trigger pipeline builds?

I am new to Devops but in my dream development world there is no more need for merging of code. Previously I build a lot of custom bat files, stored procedures and SQL jobs that would deploy all of our database code using a combination of SVN and msbuild. This took an enourmous amount of time, but it is now very stable and our QA team only needs to execute one stored procedure to deploy all of our SSDt projects (i.e. SQL, SSIS, SSRS and SSAS). We did not use TFS and that is a long story but we were stuck with old SVN and doing manual merging between different branches, DEV, QA and Production.

What I would like to do in Devops (if it is possible) is that all code is associated with a ticket, whenever the status changes on a ticket, that triggers a pipeline execution. (Release to production is probably more complicated and would require some manual intervention.) So for example, let's say you have ticket statuses of:

Created,In progress, Pending Release to QA, Ready for QA, QA In progress, QA Complete, Pending Release to UAT, Ready for UAT, UAT in Progress, UAT Complete

So certain changes of status would trigger a pipeline, for instance, Pending Release to QA would be set by the developer after development is done, this would trigger the pipeline release to QA, taking only code from the repo associated with tickets with that status. After the pipeline completes successfully then the status for all the tickets are automatically set to "Ready for QA". The QA team member ticket owner then manually changes to "QA In Progress". "Pending Release to UAT" is set by the QA member which then triggers a pipeline that deploys to the UAT environment etc.

So theoretically then there would no longer be a need for merging, since the statuses determine what code is deployed to what server. The only issue would be reverting code, but I was thinking that if each pipeline release automatically takes a snapshot, then you could then just deploy from the previous snapshot to "revert", although the code itself would just have it's status changed back to "In Progress".

Is this possible? Does it even make sense? Thanks.

https://redd.it/10vdr0v
@r_devops

A nice tool to render pretty markdown in the terminal

Pretty Markdown rendering in the Terminal:

https://www.youtube.com/watch?v=h9JJjyiHOAw

https://redd.it/10v5bqo
@r_devops

(Windows) Containers for People in a Hurry!

Hey folks,

I enrolled myself on a mission of trying to understand the inner-workings of Containers and, along the way, created 15 InfoGraphics that cover various topics from "What is Container" all the way to "How is it distributed". Since I can't upload images here, I'm sharing a PDF from my Google Drive: https://drive.google.com/file/d/13vyoHrc3bvAG480GTFDl4Xs1mfJP5PqP/view?usp=share\_link. If anyone wants the actual images, just let me know and I'll send you PDFs.

Why I think this is useful? Because I deeply believe that even in 2023 many people don't really "get" containers and yet they usually have "better" things to spend time on than dealing with Containers. And I'm saying this from the first person view :)

My hope is that this series of graphics may be of use to anyone who wants to learn containers and yet has no time to do so :)

Let me know what you think!

P.S. I hope this is in accordance to the rules?

https://redd.it/10vgy7n
@r_devops

What type of companies should i be targeting that usually follow best devops practices as a junior?

I'm considering getting into Devops now for the **nth** time after doing software dev (99% webdev) for a few years now. Naturally, I'm already familiar with some of the Devops related tools such as Docker, CI/CD and doing linux related stuff in the CLI and others.

From what I've read a lot of companies out there are doing Devops wrong which then causes stress and headaches for the Devops employees such as all of you people!

So my question is, what type of companies/industries should I be specifically targeting that is known to follow good Devops strategies/practices so that in 3 years I won't be a burned out miserable f*ck wishing that I never got into Devops in the first place.

Thanks Guys!

https://redd.it/10vi390
@r_devops