Release pipelines -- smooth as silk or still a pain (sometimes?)

Asking for a friend.

deployment pipelines to dev/staging/prod are super stable for you or sometimes (rarely even?) break. but a super pain when they do break?

Or smooth sailing all the way?

https://redd.it/10eoms9
@r_devops

Any good contents/course for serverless framework

My company infrastructure is based on serverless and I definitely want to get involved in that. Does anyone have any good contents they know related to this?

https://redd.it/10elf1d
@r_devops

Free version of SonarQube broken??

I've been wasting entirely too much time trying to get Jenkins and SonarQube to work together. Has anyone been able to get the two to work recently? I am currently stuck on the SQ quality gate returning a 401 error but the credentials work for the actual scan.. I absolutely hate SonarQube at this point so I am open to other open source static analysis tools as well.

https://redd.it/10eojgx
@r_devops

How to master in Devops

I recently moved from IT help desk ( 5 years experience in it ) to Devops engineer via Internal movement in my company. I learnt Devops mainly via YouTube & online ( Learnt Cloud Platform : AWS/AZURE , along with Jenkins, Docker, basic Unix / Python, Ansible ). how can I expertise in Devops / Devsecops ?

https://redd.it/10eno0t
@r_devops

Noob CICD Question

In the CI portion of the pipeline, once a build is triggered by a change in a repository, does that source code typically get delivered to a Dev Environment, where it gets built, and then delivered to a Test Environment, where unit testing occurs? Also, are these Dev and Test environments provisioned users themselves(for instance, ec2 instances), or do tools like Jenkins somehow have these environments already set up?

In other words, with a tyical CICD tool like Jenkins or Bamboo, where does the building, compiling, and testing take place?

https://redd.it/10ez5ka
@r_devops

What kind of self service tools did you build for your dev teams?

We are looking to build more automation and self service tools for developers to use so DevOps doesn't block them. I've been trying to brain storm what this looks like. Anyone mind sharing useful tools that have been valuable and time saving? Where does the request happen at?

I went to re:Invent in November and went to a conference where they showed off 'DevOps as a service', which I thought was interesting.

https://redd.it/10ed1l1
@r_devops

TypeScript CI Tool to Identify Nested Loops

I'm looking for a CI tool that can identify certain code patterns in TypeScript and enforce that they have unit tests written. For instance, enforcing that any function with nested for loops has unit tests before it passes CI.

Do any tools like this exist? I'm only newly familiar with ESLint, but this functionality seems outside the scope of it.

Thank you!

https://redd.it/10f22ee
@r_devops

Architectural concerns about our CI/CD pipeline

Hi all, Hope you are having a great week :).

So i just arrived at a new position as the only devops here, and i am having some concerns about the decision that were made in the past around here, let me explain.

We are in an environment with 3 microservices : A, B and C. All of which is built and deployed using Gitlab CI on EKS, using Karpenter as an autoscaling solution.

In order to assure that the devs can test their environments without having to disturb others, it has been decided that every branch would be deployed on the cluster so it can be tested. This means that if Microservice A has branches A1,A2,..,An; then all these branches will be deployed. Moreover, for each one of those branches, we will deploy a B:latest, and C:latest so we can assure that the WHOLE ecosystem is up and running. Now apply the same thing I said about A to B and C, in every possible combination.

My first Question is : Why ? why can't we just validated a PR and then, when it's merged, deploy the "Developpers" version on the cluster ? adding more replicas/changing the rollout strategy would be enough to assure availability for the devs to not disturb each others right ?

I wish this was the end of my troubles; Those microservices are in React.

And in order for microservice A to work properly, it needs the FQDN of B and C DURING RUNTIME. and since A,B and C are deployed PER BRANCH, all the resources and ingresses and namespaces are a composition of the branch name.

For example :

Dev created branch called "feature-234-foo" for microservice B, than the namespace would be called "feature-234-foo" and the ingress would be called "feature-234-foo.something.sth.com". And A would need to have that DURING runtime.

​

The solution they came up with ? Rebuilding microservice A, after B and C are deployed and have their ingresses "generated".

​

If you are having headaches reading this, it's fine, i already found some solutions for these using javascript files as config map where i would inject directly the React environment variables for it to be consumed by A.

​

I feel like the team before me tried to solve all the solutions they had using gitlab CI, which turned it into a clusterFuck of a code that i need to live with / completely change.

https://redd.it/10efa9a
@r_devops

Question: How do companies document their business processes?

For example, do companies keep who is responsible for what processes, how it’s started, what department is doing processes? Etc

Is there a standard/single source of truth for the managers or C suite that I’m not aware of? Or is it just memorized mainly by heart by a few employees?

https://redd.it/10ejxsq
@r_devops

hosted Plastic SCM and AWS CodePipeline

We're researching possibly going from hosting our pipeline building on-site to hosted with AWS as they have some packages specifically built around game development. We currently use the cloud hosted package for Plastic SCM. What I'm wondering is can AWS communicate with this service easily or will something custom have to be built or host the whole repository and service on AWS? I wasn't able to find anything on Plastic's documentation or AWS'.

https://redd.it/10ejsl2
@r_devops

We created a Docker tool to monitor other active Docker containers, from a container

Hi r/devops,

My team and I have recently released an early build of a containerized application to monitor Docker container metrics, DockerWatch. We've been using it primarily to monitor application resource utilization prior to release to production. We would love to hear your feedback for this project as we add additional features for developers. This application primarily supports Mac hosts but we will be adding Linux and Windows support in the future.

https://redd.it/10ee52v
@r_devops

tools for creating installers

what tools do you use to create installers for deployments by a customer? or is everyone doing containers and nothing else these days?

Windows is the target here.

https://redd.it/10f4zmy
@r_devops

What is the cheapest place to host a small (7 pods) Kubernetes deployment?

I've been searching around for the cheapest place to host an Kubernetes cluster with a cloud vendor.

Requirements:

* The vendor must have support for Terraform.
* Zero management of the control plane
* Preferably a good amount of additional services in case I'll need peripheral things.

I would really appreciate your thoughts and opinions. As it stands GCP (my preferred vendor) is costing me around £150 p/m for a 3 node cluster (without reservation).

https://redd.it/10f4zk3
@r_devops

Wrote an article about deploying a container into AWS EC2 with ECS

Hi everyone, recently I have wrote an article about explaining how to deploy a container into AWS EC2 with ECS. I'm wondering your thoughts on this piece, hopefully I'm not missing something out.

https://www.permify.co/post/how-to-deploy-your-container-into-aws-ec2-with-ecs

https://redd.it/10fb3pd
@r_devops

how many CICD pipelines per git repository

One thing that is not discussed a lot is the amount of seperate CICD pipelines for a GIT repository. In GitLab CI you are limited to 1 Pipeline but in Azure Devops and GitHub you can have multiple pipelines and or workflows per repository. How many pipelines do you have, on average, per repository in your organisation? And why?


I like to have 3 Pipelines, one "main" pipeline which compiles, tests, scans and deploys to a feature environment.
One release pipeline which changes the version of your dependency management tool, like Maven or NPM, compiles and deploys it and tags a commit. And one deploy pipeline which only deploys artifacts.


Sometimes these 3 are combined into one pipeline which does everything but if you would describe that pipeline with a function name it would be like:

compile_and_deploy_to_feature_env_or_only_make_a_release_or_only_deploy_artifact()

whereas seperated it would be

compile_and_deploy_to_feature_env()
build_release()
deploy_release()

Also with the last option you see fewer parameters in a CICD job

https://redd.it/10f70l0
@r_devops

Team is trying to build an entirely custom service to calculate DORA metrics for our company. Is this normal or are we over-stretching ourselves?

We are a team of 4 cloud engineers managing the platform of a small-mid sized company. We investigated Sleuth as a SaaS solution but it didn’t quite fit our needs. So we thought we’d have a crack at building something ourselves. I’m a little worried though that something like this goes into the realms of complex algorithm design and is beyond the skillset for a team of this size. I have no idea how it’s been done in other places though, so can anyone share their own experience with using DORA metrics and thoughts on our situation?

https://redd.it/10f93v4
@r_devops

What's the difference between GitOps and Ci/CD

It just sounds like fancy market talk.

Specifically this link here from github - https://github.com/readme/featured/defining-gitops

https://redd.it/10faw7u
@r_devops

An application agnostic remote agent

Does anyone know of a generic remote agent tool? By this I mean some kind of agent software that can be installed to a remote system (likely on a different network outside of my control) that can be instructed by a control server to execute arbitrary tasks, preferably containers. I know of several application specific task handlers like GitHub's self-hosted runners, but I can't find anything that is application or language agnostic.

If there isn't such a tool, what level of interest do you think there would be in one?

https://redd.it/10fl0up
@r_devops

What are Terratests good for?

So I'm working on a new team that has a requirement to run terratests on all terraform modules. But the way they are currently implementing them is by checking that all the resource names match the expected output and they have all the right tags.


Looking at this, it seems kinda pointless. Terraform creates what we tell it to, and I don't really feel a need to test that terraform is creating what I laid out in the module. The only real benefit I see in running the terratest is seeing that the apply finishes without an error, but after that, I don't really need to check all the resource names. We use interpolation for some of the naming conventions, but they aren't that complex and creating terratests just for that naming convention seems like overkill.


I can think of some better uses for terratest, like to test the functionality of some of the more complex conditional logic we sometimes use in modules. But I was curios if anyone here has uses terratest for something more useful or if I'm missing what the main point of testing terraform is all about.

https://redd.it/10fbrcl
@r_devops

Keda with kafka scaler won't scale from zero

We have multiple K8S clusters and in each cluster we have a kafka cluster. I want to use keda to scale up and down pods due to topics lag. The issue is when a new cluster is being created or if we recreate the topics, and the pods are on 0 replicas, the keda does not recognize that there's a new lag. It's happening because it can't identify the consumer group, since it was not created yet. Is there any solution for that? Thanks

https://redd.it/10fgtwb
@r_devops

Fighting Slow and Flaky CI/CD Pipelines Starts with Observability

Observability in production is great, but what about our own CI/CD? Here's a tutorial on how to build intelligent data collection, dashboarding and alerting over Jenkins pipelines with open source tools like Prometheus and OpenSearch.

https://redd.it/10fhejs
@r_devops