Can someone please help with a difficult azure pipeline YAML issue I'm facing?

I've been hitting my head on this for 4 hours, I can't seem to see what is wrong with this code. It seems to follow the MS documentation and LLMs also tell me it is correct.

However, when running the pipeline, I'm getting this error:

> The pipeline is not valid. ... references service connection $(azureSubscription) which could not be found.

The variables I have defined at the job level are not being recognised, they are being treated as literals, i.e. it is trying to find a subscription called "$(azureSubscription)".

My pipeline code is this:

trigger:
- main

extends:
template: deploy-template.yml
parameters:
buildConfiguration: 'Release'

The template code is this:

parameters:
- name: buildConfiguration
type: string
default: ''

stages:
- stage: buildstage

pool:
vmImage: ubuntu-latest

jobs:
- job: buildjob

variables:
azureSubscription: 'name of our subscription'
buildConfiguration: '${{ parameters.buildConfiguration }}'

steps:

(couple of tasks here)

- task: AzureRmWebAppDeployment@4
displayName: 'Deploy to slot'
condition: succeeded()
inputs:
ConnectionType: 'AzureRM'
azureSubscription: '$(azureSubscription)'
appType: 'webApp'
(etc)

All the documentation says this should work, i.e. '$(azureSubscription)' should resolve to 'name of our subscription', but it is not and is apparently being treated as a literal. This is also happening with other variables (not shown here for simplicity) so the entire "variables:" section doesn't seem to work?

Does anyone have any insight into why this is happening? Would really appreciate some help as I've been trying to work this out for hours. 😅

https://redd.it/1ic3kjm
@r_devops

Infrastructure diagram to Terraform code tool

Hello everyone, I’ve posted here a couple of times about a tool I created that lets you build AWS infrastructure using a visual diagram, similar to draw.io, and generate Terraform code from it.

Previously, there was a major limitation: you could generate infrastructure resources like databases or Kubernetes clusters, but you couldn’t integrate any application code. Now, I’ve developed a mechanism to address this.

For now, only AWS Lambda functions are supported. You can visually link them to other AWS resources, and all necessary IAM permissions and network configurations will be automatically created for you.

The tool is as before free to use and publically available,  if you are interested you I would love to hear any feedback: https://archformation.com/

https://redd.it/1ic4psh
@r_devops

Unified automated platform design

Hi All!
I’ve been working as a DevOps/SRE engineer for about two years now. When I started at my current company, there wasn’t much of a foundation for infrastructure, which made automating things pretty challenging.

I’ve always dreamed of building a “plug-and-play” platform that ties everything together, but sometimes I feel like my lack of experience or knowledge about tools limits what I think is possible.

The idea is to create a platform that brings together all the tools my team uses for code, infrastructure, secrets, and monitoring into one simple, automated system..

Here’s how it would work:

1. **Central Repository**: A main GitLab project uses Terraform to create and manage all other repositories with the same setup and rules.
2. **Automation for New Repositories**:
* Each new repository gets pre-configured pipelines for building, testing, and deploying code.
* Also Vault gets automatically set up to secure secrets like passwords and tokens for each project, it would create new storage paths in vault for that specific repository
3. **Service Integration**:
* New services register themselves with Consul for easy discovery. (I have one single job in Prometheus using consul for the discovery of services)
* Prometheus and Loki collect metrics and logs from these services.
* It creates a Grafana dashboards to monitor the metrics collected.

My plan gets up to this point.
Now, my questions are:
How reliable or feasible would that plan be?
Am I overthinking or trying to create the impossible ultimate unified system?
If it's possible, are there any other methods or tools I can implement to make it even more efficient?

Thank you!!

https://redd.it/1ic4gn0
@r_devops

Having trouble setting up DKIM record on Mailjet for domain with GoDaddy

I've set up the SPF record for Mailjet last week and that propagated within 5 minutes. But for the DKIM record, I keep seeing this:

There seems to be an error with your DomainKey record.

I've waited 48 hours twice now, although I've never seen anything take longer than 30 minutes to propagate in the past. Any ideas what the issue could be?

https://redd.it/1ic85r7
@r_devops

Best CI/CD tools for AWS ?

Does anyone have suggestions for a CDCI solution on AWS? Specifically, one that is scalable and works well for multiple deployment type/tech stack etc on AWS. Thank you in advance!

https://redd.it/1ic9w2t
@r_devops

SSL error incognito for 301 redirects

I have one domain, "example.com"

it's using cloudfront and serving the world through cloudflare.
so the A record is as below-
name: example.com
value: 4.4.4.4 (dummy ip)

and cname is-
name: www
value: cloudfront.net (which has ssl)

and I have set page rules as below-
example.com/\* 301 permanent redirect forward to https://www.example.com/$1
also, Always Use HTTPS is enabled
but, when i input example.com in my browser incognito, it is hitting http and giving --
example.com doesn’t support a secure connection
You are seeing this warning because this site does not support HTTPS and you are in Incognito mode.

but without incognito it redirects and works fine. It happens only in chrome and edge incognito, why?

https://redd.it/1icb8h8
@r_devops

Title: Need Suggestions for a DevOps/Cloud Project to Work on for the Next 10 Weeks (Will Put on Resume)

Hey everyone,

I’m a Computer Science Engineering student, and I’ve been tasked with completing a DevOps/Cloud project over the next 10 weeks. This project will go on my resume, so I want it to be meaningful, practical, and something that helps me learn key skills in DevOps and cloud technologies.

I’m looking for suggestions on project ideas that are:

1. **Beginner to Intermediate friendly** (I have some knowledge of DevOps tools and cloud platforms, but I’m not an expert).
2. **Relevant to industry standards** (something that recruiters would find impressive).
3. **Scalable** (so I can start small and add features as I learn more).
4. **Uses popular tools/platforms** (e.g., AWS, Azure, GCP, Docker, Kubernetes, Jenkins, Terraform, Ansible, etc.).

Here are a few ideas I’ve been considering:

* **CI/CD Pipeline for a Web Application**: Set up a CI/CD pipeline using Jenkins/GitLab CI/GitHub Actions to deploy a web app on a cloud platform.
* **Infrastructure as Code (IaC)**: Use Terraform or CloudFormation to automate the provisioning of cloud infrastructure.
* **Containerized Microservices**: Build a simple microservices-based app, containerize it with Docker, and deploy it using Kubernetes.
* **Cloud Monitoring and Logging**: Set up monitoring and logging for a cloud-based application using tools like Prometheus, Grafana, and ELK Stack.
* **Serverless Application**: Build a serverless app using AWS Lambda, API Gateway, and DynamoDB.

I’d love to hear your thoughts on these ideas or any other suggestions you might have. If you’ve worked on similar projects, I’d also appreciate any tips or resources that helped you along the way.

Thanks in advance!

**TL;DR:** Need project ideas for a 10-week DevOps/Cloud project that I can put on my resume. Open to all suggestions!

https://redd.it/1icbuoo
@r_devops

How do you handle your deployments in a multi-repo architecture?

Hi everyone,
 
I’m looking to chat with people handling deployments and GitHub administration in teams managing tens of repositories (often linked to microservices). I’ve built an internal web platform to make multi-repo deployments more manageable, and I’m trying to check if there’s interest in opening it up. The idea is being able to assess whether what I’ve built is “shareable” or too specific for my current business.

At my company, we manage around 50 microservices, each with its own GitHub repository, versioning, and release cycle. Sometimes, we need to coordinate deployments across multiple services, which led me to develop an internal web platform that:
- Aggregates into a single place changelogs from merged pull requests for all GitHub repos (services)
- Allows me to orchestrate tag-based releases and service promotions across environments with a single button.

As an example: if a feature being built impacts 4 repos, I will simply extract all the changelogs, present them and ensure with QA that they’re tested, and then promote the versions of the 4 given repos (you can’t just run continuous deployment on some changes, especially in regulatory environments if the change implies fiscal or regulatory topics for example).

These features bring me better visibility into what’s being deployed, which makes it easier to discuss release content with management. Through this platform I also enforce repository configurations that GitHub doesn’t fully support at the org level (and need to be enforced on each repo individually).

I’d love to hear from others who deal with similar challenges. How do you manage multi-repo deployments? Would a tool like this be useful in your workflow? If you’d be down for a quick chat that’d be awesome.


https://redd.it/1icd9ol
@r_devops

Slow work for new work?

Junior level here, boss is on parental leave so it is expediting the plans to have me be more independent / lead things as he mentioned this was the idea.

I am taking some initiative on things and also getting requests from developers. I assist when i can. Although I noticed with new initiatives, it takes me a while to get things rolling and i dont have a lot on my plate for the daily stand up - at least at the moment. I feel like crap cause when it is my turn, i do provide updates, but sometimes I get stuck for days on end. Googling things, etc. It went from a two man team to a one man team.

For example,, this new intiative, I have almost completed the POC, but I can’t get the damn authenticating working. Stuck on it for 2 days. Tomorrow is another day of updates where i dont have a lot to provide. Should I be happy with how far I’ve come alone, or drown in my damn wrk. I literally have no one else to talk to regarding work. Small shop, so principal/senior engineers, any advice?

https://redd.it/1iccigl
@r_devops

Can Keycloak be provisioned beforehand as opposed to using the UI?

I was wondering if there is any documentation or tutorial as to whether it is possible to provision a Keycloak container beforehand through some files?

I was planning to create some jinja2 template to create a file that can then be mounted into a keycloak instance and I can get the realms, users, groups and all the shebang configured without doing a lot in the UI, because I find the UI a bit tediuous.

I am using ansible and was wondering if someone knows what the provisioning file should look like

https://redd.it/1icdosv
@r_devops

Devops training

Hey all

Not sure I'm in the right place but have to start somewhere

So I'm currently working in tech as a app support analyst and would love to get into devops. My problem is I never went to uni I have no degree and no real formal education. Everything I know is self taught from powershell scripting to abit of python but like I say self taught

So to my question would anyone be able to recommend sites or courses that would help me achieve my goals? Like I say probably in the wrong place and if I am sorry

https://redd.it/1icgakv
@r_devops

𝐃𝐞𝐩𝐥𝐨𝐲𝐢𝐧𝐠 𝐃𝐞𝐞𝐩𝐒𝐞𝐞𝐤-𝐑𝟏 𝐃𝐢𝐬𝐭𝐢𝐥𝐥 𝐌𝐨𝐝𝐞𝐥 𝐨𝐧 𝐀𝐦𝐚𝐳𝐨𝐧 𝐄𝐂𝟐

Step-by-step guide: Deploying DeepSeek-R1-Distill-Qwen-14B on Amazon EC2 with Ollama

: https://community.aws/content/2sEuHQlpyIFSwCkzmx585JckSgN/deploying-deepseek-r1-14b-on-amazon-ec2

https://redd.it/1icljv9
@r_devops

DevOps freelancing

Hi everyone, I've wondered how have you guys proceeded to get DevOps freelancing contract online ?
It looks difficult in my case, I'm still a beginner and I'm in a third-world country in Africa and just want to know some of your paths.

Thanks in advance

https://redd.it/1icn2ie
@r_devops

Datacenter or Datacenter network

I am a bca grad from India . I am currently get trained in domain Datacenter Linux and Cloud in my company .

Now in my specialization field I have an option to chose between two domains

Datacenter network :
SDN fundamental
SD wAN fundamentals
Ansible Yaml Scripting
Aviatrix associate
Firewall introduction
Zscaler

Datacenter :

Terraform
Ansible
Docker & Kubernetes
Ansible yaml scripting
Cloud native (CI/CD)
Isaac Academy

I want to make career in devops which would be suitable to chose . I am asking this because I haven’t listen any tech stack mention in Datacenter network so I have no idea what to chose and can Datacenter network have more scope than Datacenter domain

https://redd.it/1icpyup
@r_devops

CKA Kodekloud vs Thrive?

Hi, I'm a software engineer with almost 15yoe and I've decided to trying to shift to a devops role, so I've been started to studying kubernetes.

I've already took the KCNA certification, I'm almost there to try the exam for CAPA and the next step will probably be CKA.

Linuxfoundation has started shoving thrive subscription up everithing and they're really pushing for it, but I keep reading good things about kodekloud to prepare for the exams (also, kk is 180$/y vs 360$/y for thrive)

have you tried thrive? is it really worth it?

https://redd.it/1ictpd1
@r_devops

Creating test environments and tearing them down

My data team is having a data processing flow that they need to test regularly.

Our current flow is as follows:

We run the processing job and save it in part to S3. After all parts are written we run another job that inserts all the data to our testing DB which is a single server with static DNS name.

Since out team is growing and we want to do multiple tests as such we are having a problem with the current flow and we would like to be able to have multiple DBs, one for each test.

However those tests are temporary by nature, after a few days (changes from test to test) we do not need this DB and it can be disposed.

I am looking for the best way to manage those DBs in the following way:

* Create them.
* Get their DNS name (maybe even give them one?).
* Keep them up as long as the experiment is being done.
* Easy way to remove them.
* Some system that the data engineers can see that there experiment is still on and they should mark it complete.

All of our servers are on AWS so please take it into consideration when answering.

https://redd.it/1icuuyr
@r_devops

Infra/DevOps Lead - Student Company

I’m leading the infra team at a student company next semester. Main goal is to show off my abilities for graduate devops/infra roles and for the juniors I’ll be leading. But… tbh all the student companies are small enough that we could get away using GitHub pages, runners, a couple big cloud VMs etc. so I’m wondering what I should select as our semester goal. It seems like anything would be overkill like IaC, monitoring, container setup etc. Does anyone have advice, or worst case what would be the best to show abilities for grad jobs? Thanks :)

https://redd.it/1ictjy4
@r_devops

How to run Deepseek R1 Locally

https://youtu.be/edbw6BZTqk4

https://redd.it/1icxq5g
@r_devops

Who are the top elasticsearch / opensearch / search-tech experts to follow?

There doesn't seem to be a go-to list of thought leaders and experts to learn from in the devops/platform engineers/search engineering space. So I'm interested to know, who are the top people to follow if you're into elasticsearch, opensearch or other search relevant technologies?
I saw that there's an initiative to put a list of "top voices" together here - https://pulse.support/top-voices so I guess you can nominate your favorite people there as well :-).
Thanks!

https://redd.it/1id100o
@r_devops

How can I port over 120 Env Variables in a k8s pod that are being consumed from a k8s secret as a volume mount?

We have a new team member that came in and their design is new for me. They have like a whole bunch of Env Variables (almost 120) and its being consumed from a k8s secret.

My ask here is, if I redo it and make the secret mount as a volume:

* Will it break the code (without modifying the code to consume from volume)?
* Will it make a difference if I make this volume mount change?
* Will this help me with the performance changes?


I am all ears as I never even saw this before.



https://redd.it/1id2mno
@r_devops

GitOps Fundamentals: A Modern Approach to Software Delivery

🚀 GitOps Fundamentals: A Modern Approach to Software Delivery 🔥

In today's fast-paced #DevOps world, ensuring faster, safer, and more scalable deployments is critical. That's where #GitOps comes in!

Read more in my latest blog post:

https://ruan.dev/blog/2025/01/28/gitops-fundamentals-a-better-way-to-deliver-software?utm_source=reddit&utm_medium=post&utm_id=1

https://redd.it/1id52e0
@r_devops