What to use for mass deployment with default configs?

Hi everyone. I made Python scripts using Paramiko and Selenium(SSH is disabled by default on the switches) for mass deployment of networking gear. The configs are exactly the same for every single switch and router which means management IPs are the same for the switches, etc.

My Python script updates firmware first, then adds the configuration so that I don't lose the connection to the hardware. I'm trying to make the script better by making it a CLI tool or using a different tool which is what I'm asking here for.


Ansible, Netmiko, or stick with my current scripts?

I want to add real concurrency (Go?) instead of using starmap from the multiprocessing library.

https://redd.it/1gelwdz
@r_devops

Are there IaC security scanning tools that are not noiser and allow you to select what rules to scan?

The default choices are too noisy like checkov kicks tfsec

https://redd.it/1geniim
@r_devops

secret variables in GitHub actions - more than 100 env vars

I have been working on the containerization of our existing application and the applications uses a lot of env vars/keys to work, there are about >100 vars for each environment. Also, we do not want to push our .env config file to github. As per GitHub, You can store up to 1,000 organization variables, 500 variables per repository, and 100 variables per environment. The combined size limit for organization and repository variables is 256 KB per workflow run.

So what would be an alternative for it? and considering the vars changes based on the environment, what would be the best and efficient way to tackle this?

https://redd.it/1geoh7g
@r_devops

I want to learn a scripting language

I have been using Go for scripting for 6 months, but I would like to learn a more suitable language for scripting, like Python or Bash. Which scripting language would you recommend me to learn and why? It would also be nice if you shared any resources to learn the language.

https://redd.it/1gepl6x
@r_devops

Why Cloud Engineering & DevOps Are Essential for Modern Business Growth

The Technological landscape in today’s fast-paced world is changing. Businesses constantly seek ways to optimize their efficiency, scalability, and innovation. The rise of Cloud Engineering and DevOps has played a significant role in the changing dynamics of businesses. Many businesses’ successes involve having cloud engineers and DevOps departments in their company.

Learn More: **Why Cloud Engineering & DevOps Are Essential for Modern Business Growth**

https://redd.it/1gep5hq
@r_devops

Any Advantages to running nginx in a docker container?

Typically I run this with apt install nginx and then configure the config files. As the title suggests, are there any advantages with 'docker pull nginx' and running nginx separately in a docker container on my VM.

I haven't had any issues with it running globally, but assume if it crashes then the whole machine goes down, whereas with docker only the container would?

Thanks.

https://redd.it/1ger10o
@r_devops

advise for creating listening process in aws ecs

i have an application in EC2 with laravel to server as listener queues to standby receive any queue available in SQS to process. It is working fine with supervisorctl in a EC2 instance. Lately i try to dockerize it and run with ECS runTask by define the artisan queue command in the docker command to hang the session. But i notice it i have a new version of ECR how can i restart all the listener queue task i run in ECS ? roughly we have 21 listener queue so is impossible to run manually 1 by1.

https://redd.it/1ges7id
@r_devops

Should I go for GCP Ace or AWS associate developer?

So i just got into gcp cohorts where they will provide some discount or free cert for ACE of i qualify. So i am going to start my internship in January for Devops and company is AWS and Azure centric. I already have some experience with AWS so I don't think getting associate developer will take time. So any idea what should i do? I am too confused 😕

https://redd.it/1gesxaa
@r_devops

Jenkins jobs logging solution needed

Hi All,
I have around 200 Jenkins jobs running for a bunch of projects. Not all of them are deployment jobs only a very few & this query is not regarding them. I have other 170+ jobs which are created to run certain functionalities within few applications. They're like cron jobs (or batch jobs) you can say.

So these batch jobs are like taking file input from various SFTP servers & then executing them one by one.

Issue is that these jobs give success message even if one of the file from any SFTP server is not fetched. Let's say each job is fetching 10 files from different SFTP's and they miss out on 1 file but successfully execute 9 of those so it's still a success. Now it's not possible for me to go into console log of each job & see which of these executed all 10 file; if I'd do that then it will be very time consuming.

Is there any solution for cases like this where I can have a dashboard or anything which collects the logs from all specified jobs & I can check them all in as minimum time as possible? I was thinking something like ELK?

Thanks in advance.

https://redd.it/1geso87
@r_devops

Why do we need automated regression testing in CICD piplines?

Somke tests,integration,end to end. I am trying to grasp the whole picture. Why do we need regression testing? How it should be implemented? What are the pros and cons? Blog post or books on this,would be welcome.

https://redd.it/1gevfvz
@r_devops

Secure deployment on client's system

Hi,
I have an application which runs on multiple EC2 instances and around 10 Dockers running in that. It process some sensitive data.
Now what is the best method to deploy this on a client's AWS account? I need to protect my logic, sources code and some other data.
Since it's client's account, they can login to EC2 and see the contents. How can I prevent this? What are some best Industry practices?

https://redd.it/1gewgyz
@r_devops

GitLab CI/CD Pipeline with multiple ArgoCD clusters

We're migrating our azure pipelines over to GitLab, and so far the build process is fairly straightforward.

The problem is we're using multiple instances of ArgoCD, so the same application is deployed to multiple clusters with ArgoCD set up on each cluster.

I can't find a good way to reflect the ArgoCD sync status in the pipeline, and have this 'gate' the subsequent stages of the pipeline.

The pipeline looks something like this:

install dependencies
build/ push the image to Azure ACR
write the image tag to a git repo -> webhook from the repo to ArgoCD to trigger sync
ArgoCD picks up the change to the image tag and syncs

The problem is that in the final step we've effectively 'handed off' to ArgoCD, and the only way around this I can see is to have a stage that sleeps and periodically calls each ArgoCD endpoint to get the sync status, and once all apps have synced to mark the stage as successful/failed.

This seems quite a janky workaround, and the only 'good' alternative would be to move away from Argo and have GitLab update the kubernetes manifests - this is a big change that's not really feasible for us at this time.

Does anyone have any suggestions or a similar setup?

https://redd.it/1gexuhc
@r_devops

Terraform related query

Hi everyone, I just got certified as a CKA and I am super excited. As part of my next thing to learn I am giving a thought to learn terraform. I just want your thoughts on whether I should pursue, if not what should I pursue next.

Some background
I don't have any experience with it at all. Maybe I tried something here and there but not to the extent to crack the associate exam.

If you recommend me pursuing it, please drop the links to courses that will help in learning.

Thanks

https://redd.it/1gezwdh
@r_devops

Unveiling Jaeger V2, the next major release of the popular open source distributed tracing tool

After about 7 years and over 60 minor V1 releases, Jaeger is finally reaching V2 🥳
Great seeing this distributed tracing tool that started as an internal tool at Uber evolving into a mature open source under the Cloud Native Computing Foundation (CNCF), and now undergoing re-architecture to leverage the OpenTelemetry Collector framework (another CNCF project), which promises more flexibility, performance, extensibility and ease of use.

Here are some insights from Jaeger creator Yuri Shkuro:
https://medium.com/@horovits/be612dbee774

https://redd.it/1gf2g2c
@r_devops

WebSockets load balancing

Our setup: Client browser -> Cloudflare -> AWS ALB -> Ingress NGINX -> Node.js WebSocket server deployment with multiple replicas and autoscaling.

We're struggling with WebSocket load balancing. Some pods have more connections than others. When autoscaling adds new pods, new connections are often sent to existing pods rather than the new ones, further disbalancing the average connections per pod.

Has anyone tried to load balance WebSockets?

https://redd.it/1gf1ud4
@r_devops

AI agent, Multi-Cloud Support: AWS (using SageMaker) GCP (using Gemini)
Azure (using Copilot)

I'm building an AI assistant to guide the setup of cloud resources in a secure manner. 

Example prompt Setup a production-grade AWS foundation with the following requirements :
\- VPC in us-east-1 with 3 availability zones
\- Private and public subnets- Network segmentation for different workloads
\- Implement security best practices- Enable encryption for all services
\- Setup CloudTrail for audit logging
\- Configure AWS Backup for critical resources
\- Implement WAF and Shield for protection
https://medium.com/@rasvihostings/ai-agent-to-set-up-a-foundation-in-a-public-cloud-secure-manner-4a555d8fdd84

hashtag#googlecloud hashtag#aws hashtag#azure hashtag#gemini hashtag#SageMaker hashtag#Copilot hashtag#python hashtag#terraform

https://redd.it/1gf5m0b
@r_devops

Building a devops lab (and deliberately not posting this in /r/homelab)

Using cloud services is too expensive when a business class enterprise workstation plus 64 gigs of RAM ordered from crucial comes in at pretty close to a grand and then this machine lasts for 3 years. Now granted you don't learn cloud services when you do this but there are still an awful lot of tools and techniques you can focus on. It looks like the current optiplex machiens will even do 128 gigs of RAM and it's not THAT expensive to do that.

My dilema is if I'm better off using the macbook pro as a daily driver and spinning stuff up on this machine remotely and using something like proxmox, or if I'm better off running Ubuntu on it (and a lot of containers and VMs) or using Windows as Hyper-V is a decent platform along with WSL.

Curious what others here might do in this situation.

There are a lot of pros and cons to each choice. Focusing on using the mac and just using this box as a resource has some advantages, but using the desktop with either windows or linux might be an opportunity to get a nice new dual monitor setup.

https://redd.it/1gf7fcn
@r_devops

Made a list of free DevOps learning resources that offer digital badges

This is more for you to learn the tools, gain confidence to try more complex projects. So, if you don’t know where to start, here you go:

https://github.com/catinahat85/GitGudAtCloudNative/blob/main/learning-resources/README.md

https://redd.it/1gfdq7f
@r_devops

How do you set up your active directories?

What are the active directories set up you did that allows you to have a network drive to store files and shared across with everyone securely?

https://redd.it/1gff0t9
@r_devops

Learning DevOps

Hey guys I am a student and don't have money to spend on courses so please help me with some free resources to learn DevOps.

https://redd.it/1gffp7b
@r_devops

Flatcar open source project is accepted into the Cloud Native Computing Foundation (CNCF) - a community-owned cloud native Linux OS

Pretty amazing, this is the first time the Cloud Native Computing Foundation (CNCF) has adopted an operating system distribution.

As Chris Aniszczyk rightly put it: “A secure community-owned cloud native operating system was one of the missing layers of the CNCF technology stack".

Flatcar provides a lightweight Linux OS (derived from CoreOS) specifically tailored for hosting container workloads.

Here's the CNCF announcement for more details:

https://www.cncf.io/blog/2024/10/29/flatcar-brings-container-linux-to-the-cncf-incubator/



https://redd.it/1gfgqh0
@r_devops