Feature Flags: How to implement "sticky" variants?

I'm trying to wrap my head around how to best solve a problem my team is running into with regards to feature flags. We are using [LaunchDarkly](https://launchdarkly.com/), but I don't think the problem we're dealing with is platform-specific.

My team is responsible for conducting a variety of A/B/N experiments for our company's SaaS product. Very often, this involves the creation of feature flag variants that should only ever be enabled for new, prospective customers... In other words, anonymous users who do not yet have an account with us. For example, consider the following scenario:

We offer five different plans that potential customers can choose from. For 10% of prospective customers, we want to instead offer them a single, streamlined plan (with the hope of increasing conversions).

Accomplishing this goal is simple enough. Within our feature flag platform, we create a flag with a rule that says:


* Only enable for anonymous users.
* Only enable for 10% of visitors. Show the other 90% the five plans that we normally show everyone.

There's a problem, though. Once a prospective user has been assigned the experimental variant, we need that variant to stick with them after they have registered. That does not currently happen, because according to our rule - the experimental variant should only be made available to anonymous users, and our user is no longer anonymous... they have an account and are signed in.

After having wrestled with this for a few days, I'm left thinking that there is no simple solution to this problem that does not involve us having to maintain additional state on our end. Has anyone else dealt with this before?

See also:


* [How to Implement "Sticky" Treatments for an Experiment](https://help.split.io/hc/en-us/articles/360051389331-How-to-Implement-Sticky-Treatments-for-an-Experiment)

https://redd.it/z7zgee
@r_devops

Is Kafka the Key? The evolution of our company's event ingest pipeline!

Would love for y'all to share thoughts here! We've made some changes to our ingest pipeline recently and we're pretty proud of what's come of it.

https://www.highlight.io/blog/scalable-data-processing-with-apache-kafka

https://redd.it/z83xs9
@r_devops

Need advice for deploying a web app on a single server: Docker compose VS direct install + systemd?

I'm new to Docker and the DevOps world in general, so I apologize if anything I say is naive or clueless. I would appreciate any guidance.

I've built a web app consisting of a backend written in ASP.NET Core, and a frontend built with SvelteKit. I use PostgreSQL as the underlying DBMS and plan to use Nginx as a reverse proxy. So essentially this is my stack:

PostgreSQL
ASP.NET Core (.NET)
SvelteKit (Node.js)
Nginx

Nothing complicated so far.

I'm planning to deploy this whole thing on a single Ubuntu server (a VPS purchased from some cloud provider), I would also like the process of re-deploying (whenever I make a change to the source code and push to master/main) to be fully automated. So I'm basically trying to engineer an optimal "deployment system", given these requirements.

Now, there seems to be countless ways of accomplishing something like this, and I'm not really sure which one's best (for my use case in particular, and also perhaps more generally) — I've simplified things down to 2 approaches for now, just to not make this super long and bore you to death:

## Approach #1:

Ansible + Docker + Docker Compose + GHCR (as the container registry) + GitHub actions

In this approach, both my frontend and my backend projects will have their own Dockerfiles, and then I'll have a GitHub Actions workflow that builds the image off of those Dockerfiles on every push to master and upload them to GHCR.

I can also write an Ansible playbook to do some initial configurations on my server and install Docker, and also to copy a special `docker-compose.yaml` file that includes 4 services (PostgreSQL, my backend, my frontend, and Nginx) over to the server, and then simply run `docker compose up` to get the whole thing up and running.

Then on my backend and frontend GitHub repos, I'll have another GitHub Actions workflow that uses something like appleboy/ssh-action to simply SSH into my server, pull the new images, and re-start the containers, whenever someone pushes to master.

This approach seems reasonable, but what I'm worried about most is whether or not it's okay to actually run Postgres as a Docker container. I've read several articles and also Reddit comments from various people saying that it's a bad idea to run a database in a container, and that databases should generally just run on the host machine directly. I'm still not sure about the reason behind this, but it's a dilemma, so this is one of my central questions here: Provided that you configure the volumes and everything properly, is there any downside to running your production database inside a container? Especially from a performance standpoint?

## Approach #2:

Ansible + Git + Installing PostgreSQL, Node, .NET, and Nginx directly on the host machine + systemd + GitHub actions

This approach is more bare-bones and doesn't use Docker.

I can write an Ansible playbook to configure the server and install the necessary software (Postgres, Node, .NET, etc.) — Ansible is optional here of course, I could perform all these steps manually but I think it makes sense to use something like Ansible to automate them and make them reproducible.

Then, I clone the frontend and backend repositories on the server, and then create the necessary systemd units (in order to run my backend and frontend as background processes) to start the app.

On the other side, I'll have a GitHub Actions workflow for my frontend and backend repositories, to SSH into the server, pull any changes in both of the cloned repos, and restart the corresponding service(s).

I really don't feel like I'm knowledgeable or experienced enough in this particular area to be sure that I'm not making any fundamental mistake here, so if I am, please let me know, thank you very much.

And as for the main question, with all this, which of these two approaches would you recommend I go with? Or should I go with a completely different solution altogether?

Also

considering that I'm not going to be using this server for anything else other than this particular website, so I wouldn't necessarily benefit from using Docker in this regard, although there are other niceties that Docker offers here, such as not having to deal with installing the various pieces of software required to make everything work on the host machine's operating system, and so on.

Thank you in advance.

https://redd.it/z845d1
@r_devops

Engineers at large scale orgs (100+ engineers): how do you manage/pay for central tooling?

Imagine you're at a large scale organisation, with multiple distinct projects that have their own distinct costs. You run some tools along the lines of Jenkins "on prem", and you also consume SAAS offerings like GitHub Actions, where there is a distinct dollar cost to each CI/CD run.

If you're at that kind of organisation, I'm really curious to know how you pay for shared tooling. Do teams that consume that tooling pay a "License Fee"? Do they "pay as they go"? Do you even use shared tooling, or does each project use their own tooling?

https://redd.it/z881w8
@r_devops

Central config repository for ansible, terraform and kubernetes

In order to avoid having to manage the same credentials, host variables etc. in multiple places, I'm currently running everything through ansible and use jinja2 templates for the terraform and kubernetes files, but this makes working with the individual tools a nightmare.

Is there a simpler solution to share variables and secrets between these different tools?

I know that something like Vault could provide with somewhat of a solution, but the overhead and complexity makes me want to see if there isn't a simpler solution out there.

How do you all manage this?

https://redd.it/z885cn
@r_devops

LoadTesting as Code

Hello,

What are you all using to define web (HTTP/1.1, 2, 3, gRPC) load tests?

Stuff that can live in version control, is diff-able

(Bonus points if it can load tear arbitrary protocols)

https://redd.it/z8ak15
@r_devops

How would you set up your work laptop differently if you had to do it again?

After much saving and researching I finally bought a MacBook Pro M2 chip coming in today. I’m starting a devops boot camp in January (I know, I know, devops isn’t a position, it is an environment, process, or whatever you want to call it) but right now I’m learning Python, web development and taking various courses on code academy like learning the command line. I don’t know much else, but I’ve read the phoenix project and to be as specific as I can, I’m aiming towards working in a position similar to Bill. Understandably, after being part of this sub for a while, I know I will end up in a position completely different especially at first.

I know this might seem like a minuscule thing to ask, but do you have any tips or advice on how to set it up and how to use this new lap top after unboxing for someone who has only used laptops for Microsoft office and Netflix before? (What default settings to apply, programs to delete or download, habits to get into, etc.) Being a considerable investment, I’d like it to be optimized and dedicated to the max specifically for what I am going to be using it for right from the get go.

https://redd.it/z8ao7d
@r_devops

New DevOps please learn networking

I know the current meta is f college and lets become DevOps engineer after watching few YouTube videos… but please add some networking videos to your playlist… I interviewed more than 20 “DevOps” engineer in the last few weeks and the lack of basic networking knowledge is nuts…

https://redd.it/z8fanh
@r_devops

S3 to sns when files uploaded

AWS SNS - post message Using a URL

I am trying to figure out how to send a message on to a SNS topic on clicking a url ?

How do I go about creating the url and how do I associate with the SNS topic ? My files are html integration testing results

https://redd.it/z8j43e
@r_devops

What are the main differences between OpenSearch, CloudSearch, and Kendra

All of them are used to build search engine from your data. I could see Kendra uses machine learning, while OpenSearch and CloudSearch does not. What exactly are the main differences between the three?

https://redd.it/z8elqh
@r_devops

Is KodeKloud's annual membership worth it?

Thinking of whether to subscribe. I have passed all Kubernetes cert

https://redd.it/z8m1ij
@r_devops

AWS CloudHSM

Has anybody tried to do code signing with AWS CloudHSM recently?
It seems borderline impossible to use. The documentation is poor at best. At worst it's just plain wrong.

It seems that they've tried to rewrite the CLI utility but it's just half finished but they've already released it.

The funniest part... The user management utility isn't even codesigned!

https://redd.it/z8940z
@r_devops

Need help using cpanel certificate on nginx with digital ocean

So I have a cpanel account with a wildcard domain I have been using for a while, I created a front-end app with would be on the cpanel, but my backend is hosted on digital ocean( I have a paid plan from a company and I would have to pay to build node on my cpanel).

Is there a way to use the wildcard certificate or install it on my digital ocean server

Say my domain is yes.com, since it's wildcard, can I have something like backend.yes.com on digital ocean?? While using same certificates?

Do I need to change the nameservers, need help with this, thanks.

https://redd.it/z8nxhd
@r_devops

What do you guys think about WASM + Docker?

The tech is still in preview, but might that just be the future of containerized apps?

https://redd.it/z8ojev
@r_devops

Terraform and Kubernetes deployment samples

Hello all,

i wanted to share my previously deployed templates for Terraform and Kubernetes. It should be easy to understand at all level developers with a little knowledge in the field.


https://github.com/tanerjn/terraform-deployment


https://github.com/tanerjn/kube-deployment


Feel free to star or fork to check it.

https://redd.it/z8ubn4
@r_devops

Build AWS AMI on GP3 with Packer 1.8.4?

Hello....I'm trying to figure out where the Packer experts hang out. I ask questions on the Hashicorp forums and they go unanswered, seems like there's little activitty there in general.

I'm trying to build AWS AMIs using GP3. GP2 works fine, but I can't get the syntax right for GP3.

This is what I have:



"builders": {
"type": "amazon-ebs",
"launch_block_device_mappings": [{
"volume_type": "gp3",
"volume_size": "3",
"device_name": "/dev/nvme0n1p2" },

But I keep getting an error that /dev/nvme0n1p2 is an invalid device name. /dev/nvme0n1p2 is the device root is being built on. I have also tried /dev/nvme0n1, /dev/nvme0, and even /dev/sda all with the same error.

I've also read that as of 1.7, GP3 is supposed to be the default volume type, but I'm still getting GP2 when I look at the AMI Storage tab.

Any help appreciated.

https://redd.it/z8ujdr
@r_devops

Is hosting Sentry ourselves worth it?

As our usage is growing, we have to pick whether to byte an expensive bill (over 3k/month), use sampling a lot, or self-host. The primary cost driver is the use of transactions.

What has been your experience self-hosting Sentry?

https://redd.it/z8v967
@r_devops

Seeking recommendations for code coverage and documentation presentation from Github actions.

Hello everyone,

**tl;dr** What's the best solution for maintaining unit test coverage HTML output and auto generated from code HTML docs for every open branch in a repository that's triggered by GitHub actions or a similarly automated update?

I'm looking to create a CI/CD pipeline from my teams growing set of GitHub repositories and I'm looking to start off with suggestions for how to present code coverage from unit tests as well as automatically generated documentation from code comments.

Firstly I would like to say I've seen codecov and I really like their offering but their $10/person/month price tag is just not going to work for us and it's a private repository. I am perfectly willing to self-host this solution and would love to hear if there are free self hosted packages that do what I need.

I am hoping to achieve the following goals with my CI:

* HTML code coverage generated by the project is available linked to from the README.md or PR comment in a way that is specific to the current branch. This could be also done by annotating code views in GitHub if necessary but I would like this to be specific to each branch rather than having one HTML coverage artifact for the whole project which would need to be like the main branch or something
* Documentation generated from the build is placed somewhere that's easily linked to from the README.md
* Code coverage and documentation HTML is removed when a branch is deleted so that every live branch has artifacts but storage isn't building up forever.

There are some solutions I've considered that I don't love:

1) Auto generate documentation and HTML code coverage in a GitHub action and post it to the the GitHub pages for the repo. This has the disadvantage of there being only one coverage and documentation artifact for the entire repo, not one per active branch. But it's pretty easily done.

2) Check in the code coverage and documentation into each branch and view the HTML by navigating to the code. Something about this just weirds me out from a developer perspective. I don't love build artifacts cluttering the repo and we don't need to maintain historic changes for the HTML coverage report. Also I cringe at the thought of this causing merge conflicts that might have to be dealt with.

3) Build my own web host that has an API where in GitHub actions can upload the build results of code coverage and documentation to the server I create and provide a few simple links for users to go look at it. While this is exactly what I want, and also not exactly a very difficult project, I feel like something like this probably exists and I don't want to maintain this in the future if there's a good solution with a community.

https://redd.it/z91y9q
@r_devops

Containerizing devops tools with docker compose

Recently I've been thinking about containerizing our devops tools in my current work place. The dev and devops teams currently have everything installed locally to do deployments. We use terraform, terragrunt, ansible, packer, kube etc.

I have been toying with docker compose and feel it would be a good fit for us as we can control the versions of the tools at a repo level. I can get individual docker compose services running fine e.g. just terraform. When I introduce multiple services with the same docker compose file, things start to fall apart.

We use terraform and terragrunt together, packer and ansible so having the 2 related tools within the same docker compose file makes sense.

Does anyone have experience of this type of situation or would suggest any material?

https://redd.it/z90nv3
@r_devops

Access variables in Jenkins trigger section

I am trying to use a shared library pipeline (if that's the correct way to put it) from two different Jenkins files. One to do nightly builds and one for deliveries. This may not be the best setup (i'm just learning) but I am having trouble accessing a ?parameter? passed in. I want the nightly build to get scheduled for a certain time and the delivery to only be run if started manually. The rough outline is:

shared lib

**pipe.groovy**

def call(body)

triggers{

cron(env.target == 'nightly' ? '00 8 * * *' : '') // env.target is null?

....

stages{

stage{

steps{

sh('printenv') // env.target is set



**Jenkins.nightly**

pipe{

target = 'nightly'
...


**Jenkins.delivery**

pipe{

target = 'delivery'
...

I seem to be able to access the ?build in? env vars in the trigger section but not this way. I've tried many combinations of trying to do so.

Any suggestions?

https://redd.it/z96891
@r_devops