Handling developer feature branches in cicd?

How is your organization currently handling pipelines for feature development? Is there an automated Jenkins pipeline that spins up temporary namespaces? Are you using cloud hosted options like Azure dev spaces? Are you using open source tools like scaffold?

https://redd.it/kwll45
@r_devops

Remote Debugging in AWS

Hi all,

As developers, we are losing our ability to debug after push our application to cloud. There are several workarounds to solve this issue. I wrote a blog post about this and wanted to discuss more here. On how are you debugging the compute on cloud? What's the practices that you've been following?

https://thenewstack.io/remote-debugging-in-aws-the-missing-link-in-your-debugging-toolset/

https://redd.it/kwc7z3
@r_devops

Who is running on bare metal?

Why?

How are you managing your data center?

What are the top five problems you face?

https://redd.it/kx0qy0
@r_devops

advice for local testing

I'm using MacOS catalina for local testing. I'm testing SDK's used for a NOSQL database.

I have a database cluster running in docker containers. This is working just fine.

I need to test SDK from 6 languages (Java, C, nodejs, python, C#, Go). Unfortunately I cannot simply run the code from my machine because there some test cases that won't work. Let's say I have two query nodes and need to run queries against both nodes. I have to open port 8093 which is used by the query service. Due to the way docker for Mac works, I can't expose the same port on multiple containers (otherwise I get a port is already allocated error).

What I've done for now is build my own docker image with runtimes for 6 languages. This is also working. However this image is large (almost 2GB). I feel like I'm doing it wrong. Shouldn't container images be small? How would you folks approach testing?

By the way, the only reason I'm using docker for the database as is it's very fast to spin up a cluster (I have a bash script that does this). I suppose one simple solution is to use vagrant instead of docker (then I wouldn't have port conflict issue), but it would be slow and I'm not sure how to do this. If vagrant (or anything else besides docker) is a simpler/better solution I'm willing to explore it.

https://redd.it/kwz312
@r_devops

What is Infrastructure as Code

Hello,

I wrote a blog post "What Infrastructure as Code", discussing Infrastructure as a Service, Infrastructure as Code, Infrastructure as Data, and a few others.

Only definitions, no value judgments.

Feedback more than welcome.

https://dominik-tornow.medium.com/what-is-infrastructure-as-code-30c112ec440f

(On medium, not paywalled)

https://redd.it/kv64v0
@r_devops

Is there a "delta" post from Kubernetes to Azure Kubernetes (AKS)?

I will use in AKS, in a few weeks and start / create a new cluster.

​

I´m pretty familiar with Kubernetes / ADM /kudectl etc. Creating and maintaining Bare metal k8s clusters. But is there a post / tutorial / page whatever about Kubernetes AKS?

​

Gotchas
What to look out for?
Differences in patching / Maintaining Kube
Networking

etc?

https://redd.it/kx5a8g
@r_devops

How to Publish Artifacts on Maven Central with GitHub Actions

A concise step-by-step guide on how to publish your first artifact to Maven Central via Github Actions: https://medium.com/codestory/how-to-publish-artifacts-on-maven-central-24342fd286cd

https://redd.it/kx6cgt
@r_devops

SQLite Databases With Python -Mega Course | Complete Course |Creating a Database and Running Queries

https://www.youtube.com/channel/UCuAjoGTTf7iaMixpiEdKFZQ

https://redd.it/kxa6ih
@r_devops

We launched a free cloud security and compliance tool

Hey fellow DevOps bois. I'm sharing this free product we launched today hoping it's actually helpful and not just shilling. I got involved here because I have nightmares about the 500 row spreadsheet we got from compliance a few years ago, and I never wanted to go through that again. I'd love any thoughts from you guys.

https://redd.it/kxah4u
@r_devops

HTTPS works for the frontend, but not for the API in the same domain.

Hey guys, I've built a microservices architecture with the following structure:

* frontend built with Nuxt.js
* api gateway built with FastAPI
* service 1
* service 2
* service 3

I used Docker Compose to manage all the services. The API gateway takes all API calls from the frontend, then routes them to the appropriate microservice. I deployed it on Digital Ocean and I was able to access the frontend on [https://mydomain.com](https://mydomain.com/), the API on [https://mydomain.com:8001](https://mydomain.com:8001/) and they were able to communicate with each other.

Everything worked until I tried to set up nginx with an SSL certificate provisioned by Let's Encrypt to enable HTTPS. I set up nginx and HTTPS following [this tutorial](https://danielwachtel.com/devops/deploying-multiple-dockerized-apps-digitalocean-docker-compose-contexts) and now the frontend is listening properly on [https://mydomain.com](https://mydomain.com/) but apparently the API is still on [https://mydomain.com:8001](https://mydomain.com:8001/), so whenever the frontend sends a request to the API I get a 'Mixed Content Error'.

I thought that once the SSL certificate was enabled for the domain, HTTPS would be enabled for the API as well. What am I missing here? I should change the nginx configuration or something is wrong with the API configuration?

https://redd.it/kxanx2
@r_devops

How to manage a large number of AWS Infrastructures?

Hello guys,

I hope someone with experience or a similar situation can give some help here. So we have a custom microservice App running on ECS with a lot of AWS infra parts and we manage this with Terraform ( Deployment, maintenance, improvements, patches etc ). We deploy App with Terraform to client's AWS accounts. So that's a lot of AWS accounts to manage given that we have separated AWS per env, prod, and nonproduction.
My question is how you guys manage or how you would manage let say 20 clients X 2 env = 40 AWS accounts/deployments/infrastructures? Let say I need to deploy some infra fix or feature with Terraform to all of them, how would you do that, as manually one by one is impossible?

I was thinking to use maybe AWS Systems manager, but I'm not really sure that it supports Terraform, maybe we should switch to CloudFormation?


Any help is appreciated :)

https://redd.it/kxehcs
@r_devops

Monitoring AWS ECS on EC2 - more than cloudwatch

I am looking for ideas or best practices on how to monitor the EC2 which provides the docker containers for ECS. Fargate is more expensive in our use case.

We are running some python Backends but also regular some proprietary software, which we can't adapt.

As cloudwatch has a) very limited metrics and b) a delay of up to 5minutes till you see the metrics, we want to use prometheus (node-exporter) and/or icinga agent.

We have both running for the rest of our infrastructure. Combined with ELK and grafana, this gives us great data. Just not for everything running on ECS.

ECS is configured with LoadBalancer and autoscaling, so discovery is necessary.

Obviously there is the option to add icinga-agent and node-exporter into the container itself or add it as a side-car. Updating the clients or configs or certs would potentially require a deployment of the app, which involves other Dev teams.

We kind of want to split monitoring from the actual app and be able to update configs or versions without touching the app container.

No idea if this is the right approach, we didn't find any solution, hence I am hoping for some friendly tips or better approaches to Monitor our apps.

https://redd.it/kxj0g4
@r_devops

Anyone here make a browser-based multiplayer game? I have a newbie question about sockets and want to ask someone with experience.

I'm trying to decide between using Unity and ThreeJS to make a browser-based multiplayer game. I intially thought they each had their own way of setting up and handling multiplayer functionality. (I was asking between the two, which one would be able to handle 600 players in one arena easily without necessarily customizing the underlying engine for performance enhancements.) I was told they both use websockets/ socket.io. Are all browser-based multiplayers using this same technology or is there another way to have multiplayer functionality?


Also, how many players at once can websockets handle? Could it handle a multiplayer arena of 600 players?

https://redd.it/kxhr2t
@r_devops

Automating an install of an application



Just wondering if some one know of a tool with which i can automate the installation of a windows software, i am not looking for cloning the machines etc, the application i am trying to install is kinda like the office application where each client has a different serial number, etc.

Also i need to be able to have custom location for the install, and i have few text to enter such as serial number, user name, company name etc.

I do not have much experience with Auto-IT and not sure if i can achieve this with it

Thanks

https://redd.it/kxeipp
@r_devops

API Documentation- Postman or Swagger?

I'm on a small team currently using Postman for our customer facing API but have a couple minor problems that we think could be fixed by swapping to Swagger, just wanted to find out if anyone else has had experience with the two. Doing some research, I found a couple reviews and comparisons but really I'm looking for more personal experiences and hopefully even discussion.

The main issues with Postman we'd like to solve:

* Theming- it'd be much nicer for the company if the API docs were consistent with theming when compared to the rest of the online presence
* Custom domain- it'd be preferable having a custom domain pointed at our docs, with a separate domain for our internal docs. This is possible with the premium tiers of Postman, but that still doesn't solve the other two issues
* Hosting control- My boss would prefer having control over the server itself, I think mostly for analytics purposes

https://redd.it/kx4zsc
@r_devops

Terraform and IAM policies documents

Hello everyone,

Can you give me some insights about what you're using for managing your IAM policies documents (JSON) with Terraform ?

Do you prefer using plain JSON or maybe the TF dedicated object (which is pushed by Hashicorp) https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam\_policy\_document

Thx

https://redd.it/kxa6a0
@r_devops

Question about the new AWS RDS CA

When updating the database to use the new CA, you get this popup:

"Before scheduling the CA certificate rotation, update client applications that connect to your database to use the new CA certificate. Not doing this will cause an interruption of connectivity between your applications and your database."

Does anyone know if this is required? We didn't have to import the CA originally, so why would we have to now? If it's a matter of the new root CA not being part of the trust chain of older distros, does anyone know if Ubuntu 18.04 has the new CA in its ca-certificates package yet?

Thanks!

https://redd.it/kx8lhl
@r_devops

Can I customize all my operating systems that I will install on kvm/qemu with Virt-customize?

**For my private cloud provider, I want to create a virtual machine from the operating system iso I want using kvm / qemu (since I was looking for an alternative to cloud-inite, I chose to customize it with virtual-customize) but I could not customize freebsd and openbsd images with virtual-customize*

https://redd.it/kx2nvo
@r_devops

lOOKING FOR A GIT SAAS IN EU (WITH FAILOVER)

I'm a bit confused that this doesn't seem to exist? Are my Google-fu skills just rusty?

We want to spend money to not host our own Gitlab so first we wanted to move to .com (US only)

https://www.stackhero.io/en/services/GitLab - Preview and no failover/HA

https://gitlabhost.com/ - Single tenant

Bitbucket / Github etc also all are US.

(Sorry for CapsLock title)

https://redd.it/kx2eor
@r_devops

Bit annoyed with everyone thinking their way is better than our current process (contractors, new starters etc.)

This is a trend I've noticed a lot over the years and frankly, does my nut in. Anytime anyone joins a project, they nit-pick. It might be subtle... 'oh... Why did you choose to do it this way?' or just blatant rude.. 'ah well I'd have used terraform cloud for this instead of managing statefiles in this way'. Would you? Glad you're a fucking psychic who can tell my bosses and BA's requirements from a single look at a repo!


First of all, code is 50% more difficult to read than write. That's a proven established fact. If my scripts look over complicated to you because you didn't write them. That's probably why.


And okay great you wrote this fantastic yaml pipeline which does a build in an amazing way, but is that way relevant to my use case?


And fine, my terraform modules are a little bit messy now. This is 18 months into a cloud transformation project and frankly sometimes you have to sacrifice code cleanliness for functionality because you need to get it working and the provider you use doesn't have the updated options, and changing provider will give you weeks of work, and you're a month into the crunch and things aren't going to schedule and that's why you've hired a contractor in the first god damned place.


In a perfect world we're all perfect engineers, but come into a project and understand that you're coming into something that's been on a journey you'll never quite understand fully. Come into it with an open mind and think about that time where you've been stuck and had to throw in a hack to make it just bloody work this time.


Stop being a judgemental prick please.

...Sorry.... Been a long few weeks.

https://redd.it/kxr39t
@r_devops

QUESTION Testing PRs across multiple repositories in a single build?

Current setup:

SCM Manager: GitLab

Build Server: Jenkins

Branching Model: Trunk-Based Development (in each repository)

I have the challenge of figuring out how to set up a way to trigger builds on Jenkins when PRs are created for multiple repositories. While this in itself is not a challenge, I want to be able to test them in a single run on Jenkins, since some of the repositories are tightly-coupled in such a way that our developers often work in multiple repositories to fix a bug / develop a new feature.

This leads to the issue. How can I create a PR on each repository and then be able to fetch those in the Jenkins build? I can set up a quiet period in Jenkins so the build waits for a duration before building - This should be enough time to create the PRs in all the different repositories.

By the looks of it, GitLab assigns each PR an incrementing number as name. These PR names might not always match, so I was wondering if there is a way to do some pattern matching to get the branches created from the different branches (labels, created by developer)?

Furthermore, is this even the right approach to take? We are currently on the steps of setting up versioning for each repository, but versioning the repositories will also slow down development time, since we'll have to wait for the packages being available before even being able to test if they work properly in other repositories.

https://redd.it/kx1nw4
@r_devops