Where to run unit tests when building inside docker

Trying to determine what would the best approach in the following situation:

Currently I have a CICD pipeline in Jenkins for building a Next.js application. It consists of separate stages like:
1. installing deps
2. running unit tests
3. building the application and finally copying the build output into a docker container and pushing it.

Because steps like building also require a separate base image with node and pnpm installed this adds extra maintanance.

I’m looking into doing the building inside of a multi-stage dockerfile. This seems to work quite well with the exception of the test stage for the unit tests. I’m kind off conflicted if it’s wrong to add an extra stage and run the tests on building of the dockerfile or if it really should be done outside?

Curious to hear how others are approaching this issue.


https://redd.it/1egrmbs
@r_devops

15 Cloud-Native DevOps Tools You Should Know

[https://overcast.blog/15-cloud-native-devops-tools-you-should-know-36129057a15c](https://overcast.blog/15-cloud-native-devops-tools-you-should-know-36129057a15c)


These tools are particularly beneficial in several scenarios. They are essential when managing microservices-based applications, requiring continuous integration and delivery, needing robust monitoring and alerting, seeking to automate infrastructure management, and ensuring security and compliance in complex environments. Utilizing these tools helps maintain seamless operations and quickly adapt to changes.

* **Managing microservices-based applications.**
* **Requiring continuous integration and delivery.**
* **Needing robust monitoring and alerting.**
* **Seeking to automate infrastructure management.**
* **Ensuring security and compliance in complex environments.**

https://redd.it/1ehb0fb
@r_devops

Applying for DevOps positions as a East Indian-American

Disclaimer: this is a rant, but I'd like some advice, please.

Hey there, I know there's similar threads for job application strategies on r/cscareerquestions and r/jobs, but I wanted input specifically from other DevOps/Platform Engineers.

I was born and raised in America, but my ethnicity is Indian- and my name is pretty Indian sounding. I've been hearing SO many reports of (from Reddit) about rampant fraud happening from other Indians or people on a Visa, that I believe my application has a high chance of just being tossed because of my name.

I've never had this problem before, and my non-indian coworkers have been getting blasted with job offers and have a similar background as me. I don't want to blame it on my race, but I'm thinking it's crazy I can't even get an interview! I'd hate to do a bait and switch by changing my resume to an Anglo name just to test out my hypothesis, but I have bills to pay and my current position is destroying my mental/physical health.

Any advice?

My tech stack is .NET, Azure, Bicep (IAC), GitHub Actions, Octopus Deploy, Windows IIS, PowerShell and Bash.

I'm applying to jobs that don't have Kuberketes while I study for the CKA. I was a network engineer for 1 year before I ventured into DevOps 3 years ago. Any advice on how to make my resume stand out? I have a pretty nice looking one I made with LaTeX and has the standard jobs, personal projects, skills, and education.

Also, I'm a woman- Idk if that helps, but wanted to mention since there's literally no other women in my team of 40 engineers, so I'm either doing something right or I'm getting in because of my gender- which UGH.

https://redd.it/1ehax2n
@r_devops

Should I leave or stay?

I am feeling quite burnt out now. I am currently in a DevOps position in a startup and the hours + responsibilites are taking a toll on me. Being expected to debug application issues, manage kubernetes environments, manage cloud infra, CICD across 3 different platforms, supporting various projects as well, maintaining observability by creating reports from SLA metrics and usages. All without getting the support I need.

Its only been 1 year and 3 months. I currently have a 1 year contract offer for a DevSecOps position. Should I just take this role? Or stick around longer and toughen it out and hope it gets calmer?

https://redd.it/1ehcvsx
@r_devops

Transitioning from Jira Admin to Devops

Hello, I wil probably accept an offer as a "System Engineer" for which the main responsibilities are Jira administration and integrations. I currenty work a similar role and before that I have worked as a backend developer (Java/Spring) for 3 years. The company I join is a very big e-marketing one, and the team I would be in is actually DevOps.

The System Engineer job is described like this:

full stack admin of jira+confluence
maintain and optimize tools
automation of processes
develop enhanchements
support requests triaging
They require: jira/confl admin experience, MySQL/Postgres, Linux OS, nice but not necessary: API, Python, Groovy

The role is focused on Jira but I would like to evolve from there as a DevOps engineer. In the team here are people who take care of jira (2 + me if I join) and then 6 people who are more "pure" devops (I assume CI/CD, Infra, integrations, etc.).
One thing that made me successful during the interviews was my engineering background and Java/Groovy knowledge.

Here's my question - how likely do you think it is that I can evolve into a ""pure"" devops position? Don't get me wrong, I know there is no actual "pure" devops as it is a very generic term, but I mean that as taking care of many more technologies and automation rather than only Jira/Confluence. Can Jira get me closer to infra tools and CI/CD?

The company I currently work for is rather small and Jira is not integrated with CI/CD, it is not even used by developers, so it is probably very different in a company like the one I will join.

https://redd.it/1ehdpq2
@r_devops

10 Examples Why cURL is an Awesome CLI Tool

Hi /r/DevOps,

Today I published an article titled "10 Examples Why cURL is an Awesome CLI Tool" - Where I explore some of the lesser known or unexpected features of cURL, such as globbing, parallelization, output formatting, URL parsing and much more.

Here's the link: https://levelup.gitconnected.com/10-examples-why-curl-is-an-awesome-cli-tool-0b23223ec845

Feedback is very much appreciated!

https://redd.it/1ehhihj
@r_devops

AWS code build stuck at npm install and getting timeout with no errors, No error on code build logs as well as cloud watch.

Using CodeBuild On-demand
Running node v16.13.0

The same code works previously now I'm not able to build since last few days.

Since there is no error how do I debug this?

https://redd.it/1ehirh7
@r_devops

How hackerrank works across organizations?

I am back in market looking for open roles and started noticing firms sending out hackerrank tests to gauge candidates.

I am trying to understand how this works?

Basically will it have a standard questions (at different levels) or can each company set their own questions for the roles they publish out in market?

How does it work people of this group? 🙋‍♂️

https://redd.it/1ehih4u
@r_devops

Need Complex Final Year Software Project Ideas (No Hardware/E-commerce/Management)

Hi everyone, I'm in my final year of computer science and I'm looking for some innovative and challenging software project ideas for my final year project, which is worth 200 marks.

Here are some constraints and preferences for the project:

No hardware involvement: Purely software-based project.
No e-commerce solutions: I want to explore something different.
No management systems: Looking for unique and exciting domains.
Complexity: The project should be complex enough to include more than three modules with multiple features.
I'm open to various fields and would love to hear your suggestions on what could be a good project idea that meets these criteria. Any unique ideas or suggestions from your experience would be greatly appreciated!

Thanks in advance!

https://redd.it/1ehma7d
@r_devops

Devops vs cloud

Devops vs cloud

Can anyone help me out figure out what needs to be done first I am in 2nd yr i have done backend in js using node express mongodb I also know postgres though

I don't know the pre requisite to do these
If any skill or topic is needed to be done earlier pls tell me

Also should I continue doing backend I am not too proficient in that for now

What should be done first out of the two and what role does backend have in it pls help me clarify these doubts

And if possible pls share roadmaps on these too

https://redd.it/1ehnr1m
@r_devops

Telepresence but for the VM?

I used to work with telepresence a lot while developing k8 native apps. It's a great way to create a tunnel b/w locally running process and remote cluster. However, I am now developing apps that are deployed as rpm (jar wrapped as rpm) on remote machines (VMs to be specific). Everytime I have to test some code, I have to bundle up the app as jar then bundle that jar as rpm and run on the remote machine where other infrasture is also present.

Question: Is there something similar to telepresence for my setup where I can create a tunnel (bi-directional tunnel) so I can just run the java process locally (using intellij) and trick this process into behaving as it was running on the remote machine?

https://redd.it/1ehnpsz
@r_devops

Need Help with Resume - Applied to 1500 Jobs and Only 1 interview

I’ve been on the job hunt for a while now and I’m getting pretty frustrated. I’ve applied to 1500 jobs and only got 2 callbacks and one interview. I feel like my resume might be the problem, but I’m not sure what to change.

Can anyone take a look at my resume(attached below) and give me some advice on how to improve it? I’d really appreciate any suggestions or tips you have. I'm an international student and in my initial OPT.

Thanks in advance!

DevOps Resume

https://redd.it/1ehoo17
@r_devops

Which resource would you recommend to learn the fundamentals of Kubernetes?

I find Kubernetes to be the hardest when it comes to DevOps tools and I was wondering if anyone knows of a good reaource where I can learn the fundamentals of it, thank you in advance.

https://redd.it/1ehnlld
@r_devops

Terraform - ACR and azure container instance group, it seems to be trying to use docker images not ACR.

I'm defining a resource group, some fileshares, a container registry, and a container group with 3 containers in my terraform setup.

My setup script successfully builds and pushes docker images to ACR and pushes files to the shares which are to be mounted on the containers. But when terraform tries to start up the container group and containers, I get an error that is mentioning docker.io as if it's trying to use images from docker and not ACR.

I must be missing something, can anybody suggest what's wrong or missing with this main.tf configuration? I'm a bit new to terraform and kind of fighting through this a bit.

I don't think the fileshare volumes are quite right but that's tomorrow's problem. Need to get the containers fired up first.

Here's the core of the error:

An error response is received from the docker registry 'index.docker.io'. Please retry later.';'BadRequest':'InaccessibleImage':'The image 'xxxxx.azurecr.io/vs-agent-openjdk11-python:latest' in container group 'vs-agent-airflow-group' is not accessible

And this is main.tf:

# Define the resource group
resource "azurermresourcegroup" "main" {
  name     =
var
.azureresourcegroup
  location =
var
.azurelocation
}

# Define the storage account
resource "azurermstorageaccount" "main" {
  name                     =
var
.azurestorageaccount
  resourcegroupname      = azurermresourcegroup.main.name
  location                 = azurermresourcegroup.main.location
  accounttier             = "Standard"
  accountreplicationtype = "LRS"
}

# Define the storage shares
resource "azurermstorageshare" "postgresshare" {
  name                 = "postgres"
  storageaccountname = azurermstorageaccount.main.name
  quota                = 20
}

resource "azurermstorageshare" "agentshare" {
  name                 = "agent"
  storageaccountname = azurermstorageaccount.main.name
  quota                = 20
}

resource "azurermstorageshare" "airflowshare" {
  name                 = "airflow"
  storageaccountname = azurermstorageaccount.main.name
  quota                = 20
}

# Define the Azure Container Registry
resource "azurermcontainerregistry" "acr" {
  name                =
var
.azurecontainerregistry
  resourcegroupname = azurermresourcegroup.main.name
  location            = azurermresourcegroup.main.location
  sku                 = "Standard"
  adminenabled       = true
}

resource "azurermcontainergroup" "main" {
  name                = "vs-agent-airflow-group"
  location            = azurermresourcegroup.main.location
  resourcegroupname = azurermresourcegroup.main.name
  ipaddresstype     = "Public"
  dnsnamelabel      = "vs-agent-airflow"
  ostype             = "Linux"

  container {
    name   = "postgres"
    image  =
var
.postgresimage
    cpu    = "1.0"
    memory = "4.0"

    environmentvariables = {
      POSTGRESPASSWORD =
var
.postgrespassword
      POSTGRESUSER     =
var
.postgresuser
      POSTGRESDB       =
var
.postgresdb
    }

    ports {
      port = "5432"
      protocol = "TCP"
    }

    volume {
      name                 = "postgres-data"
      mountpath           = "/var/lib/postgresql/data"
      storageaccountname = azurermstorageaccount.main.name
      storageaccountkey  = azurermstorageaccount.main.primaryaccesskey
      sharename           = azurermstorageshare.postgresshare.name
      readonly            = false
    }

    volume {
      name                 = "postgres-initdb"
      mountpath           =

"/docker-entrypoint-initdb.d"
      storageaccountname = azurermstorageaccount.main.name
      storageaccountkey  = azurermstorageaccount.main.primaryaccesskey
      sharename           = azurermstorageshare.postgresshare.name
      readonly            = false
    }
  }

  container {
    name   = "vs-agent"
    image  = "${
var
.azurecontainerregistry}.azurecr.io/vaultspeed-agent-openjdk11-python:latest"  # Reference to ACR image
    cpu    = "0.5"
    memory = "2.0"

    volume {
      name                 = "agent"
      mountpath           = "/home/agent"
      storageaccountname = azurermstorageaccount.main.name
      storageaccountkey  = azurermstorageaccount.main.primaryaccesskey
      sharename           = azurermstorageshare.agentshare.name
      readonly            = false
    }

    volume {
      name                 = "agent-staged"
      mountpath           = "/home/agent/staged"
      storageaccountname = azurermstorageaccount.main.name
      storageaccountkey  = azurermstorageaccount.main.primaryaccesskey
      sharename           = azurermstorageshare.agentshare.name
      readonly            = false
    }
  }

  container {
    name   = "airflow"
    image  = "${
var
.azurecontainerregistry}.azurecr.io/airflow:latest"  # Reference to ACR image
    cpu    = "1.0"
    memory = "4.0"

    ports {
      port = 8080
      protocol = "TCP"
    }

    environmentvariables = {
      AIRFLOWCORELOADEXAMPLES = "False"
      AIRFLOWWWWUSERUSERNAME    =
var
.airflowusername
      AIRFLOWWWWUSERPASSWORD    =
var
.airflowpassword
    }

    volume {
      name                 = "main-share-airflow"
      mountpath           = "/opt/airflow"
      storageaccountname = azurermstorageaccount.main.name
      storageaccountkey  = azurermstorageaccount.main.primaryaccesskey
      sharename           = azurermstorageshare.airflowshare.name
      readonly            = false
    }

    volume {
      name                 = "main-share-start-script"
      mountpath           = "/startairflow.sh"
      storageaccountname = azurermstorageaccount.main.name
      storageaccountkey  = azurermstorageaccount.main.primaryaccesskey
      sharename           = azurermstorageshare.airflowshare.name
      readonly            = false
    }

    volume {
      name                 = "main-share-staged"
      mountpath           = "/staged"
      storageaccountname = azurermstorageaccount.main.name
      storageaccountkey  = azurermstorageaccount.main.primaryaccesskey
      sharename           = azurermstorageshare.airflowshare.name
      readonly            = false
    }

  }

  tags = {
    environment = "agent-testing"
  }
}






https://redd.it/1ehufje
@r_devops

Sad and feeling miserable

I've been in the DevOps space for about 8+ years.


Today I've just been sad and miserable. I feel like I don't know a lot of the newer technologies and feel really behind. I've been trying to catch up on learning Kubernetes and have made some progress. But there are so many other things I just don't know how work such as puppet, ansible, terraform, kubernetes (learning in progress), spinnaker. And I don't even know some if there are other things that I should know.

I'm good at programming and building things, automation etc. I can figure out some of the stuff even at work surrounding these technologies. But I don't have a deep understanding and feel behind and lost at times.

I feel like the best way I've learned is managing my own version of these technologies and doing some project(s). But I don't even know where to start. And when I do start (kubernetes has been a little nice to learn on minikube), I don't know the cost efficient way to do so. For example, I don't even know how to learn Terraform without a cloud provider and it being practical.

I don't know, my headspace is such a mess. I feel alone. I feel worried if tomorrow I lose my job, I'll be homeless. I don't think anyone would hire me or I could even get another DevOps job.

I don't know, just need some advice and help. Feel so hopeless and sad.

https://redd.it/1ehwd9s
@r_devops

Will people be interested in a super hands-on/practical data security + crypto key short course?

Hello reddit,
I'm a platform/security engineer. I do a lot of preaching on how standardized data encryption + crypto key management could work and how it could simplify platform engineer's life by not having to configure access/permission policies for every single data platforms and then figure out how to align these policies on various platforms. I was wondering if this is something that will be interesting to people where I can walk through the end to end process from creating a key, configure key access, adding it to the client, encrypt the data etc. show how different types of crypto keys could be applicable in different scenarios. I thought I could maybe just create 30 mins course w/ some terraform + data encryption code step by step. Will this be something people are interested?

Appreciate the feedback.

View Poll

https://redd.it/1ei2owa
@r_devops

How to get ready for a junior/entry level DevOps job

Hello people, I am about to finish my thesis as an electrical engineering student and I would like to get into DevOps. Having no experience in software development since I focused on telecommunications and robotics, I started about a month ago the IBM DevOps and Software Engineering professional certificate on Coursera. Can you help me lay out some goals about what skills to pursue or what certifications to pass so I can have a clear path in mind before I prepare my CV?

Thank you in advance.

https://redd.it/1ei3x48
@r_devops

Can't get Chef to play along nicely with API for certificate issuance (fine with Ansible though)

Maybe someone can explain this ... or has an idea

I have the following recipe

# Install openssl
package 'openssl' do
action :install
end

# Install jq
package 'jq' do
action :install
end

# Generate CSR
execute 'generate_csr' do
command <<-EOH
openssl req -new -newkey rsa:2048 -nodes -keyout #{key_path} -out #{csr_path} -subj "/C=#{country}/ST=#{state}/L=#{locality}/O=#{organization}/CN=#{common_name}"
EOH
not_if { ::File.exist?(csr_path) }
end

# Check CSR
execute 'check_csr' do
command "cat #{csr_path}"
action :run
only_if { ::File.exist?(csr_path) }
end

# Send CSR request
execute 'send_csr_request' do
command <<-EOH
curl --location '#{url}' \
--header 'x-api-key: #{api_key}' \
--header 'Content-Type: application/json' \
--data "$(jq -n --arg csr \"$(cat #{csr_path})\" '{profile: {id: \"#{profile_id}\"}, seat: {seat_id: \"#{seat_id}\"}, csr: $csr, attributes: {subject: {common_name: \"#{common_name}\"}}}')" \
>> #{cert_path}
EOH
action :run
only_if { ::File.exist?(csr_path) }
end

The certificate it creates is weirdly formated - it basically seems to be the full json format including headers - for example (gap is intentional obviously)

{"serial_number":"78A16E498xxxxxxxxx","delivery_format":"x509","certificate":"-----BEGIN CERT
FICATE-----\nMIIEdDCCA1ygAwIBAgIUeKFuSYuyqzly34Y7vExa00frLqswDQYJKoZIhvcNAQEL\nBQAwgYsxCzAJBgNVBAYTAlVTMQswCQYDVQQIE (...)


(...) c5LCeO5lueAmuYeEPZsPMkIWEK0wMG\nnHbfpg+ICIwsB4JA3seExi5J7/orrH5L73laWcRsebU
mu+h3wDuXL1SJP3bb9VVP\nyZYUqusTWHGUq2JX8qEd3OhokExj6AiMzsKyeif5K4lRlSOP4TnGTA==\n-----END CERTIFICATE-----\n"}

Even if I use some cmd magic to remove the header, remove the linebreaks and manually make it 'look' like a real cert - the cert is not valid... The characters are fine - so it seems all about formatting.

If I run the same as Ansible work book - for example

tasks:
- name: Install openssl
ansible.builtin.package:
name: openssl
state: present
- name: Install jq
ansible.builtin.package:
name: jq
state: present
- name: Generate CSR
ansible.builtin.command:
cmd: >
openssl req -new -newkey rsa:2048 -nodes
-keyout {{ certificate.key_path }}
-out {{ certificate.csr_path }}
-subj "/C={{ certificate.country }}/ST={{ certificate.state }}/L={{ certificate.locality }}/O={{ certificate.organization }}/CN={{ certificate.common_name }}"
args:
creates: "{{ certificate.csr_path }}"
- name: Check if CSR exists
ansible.builtin.stat:
path: "{{ certificate.csr_path }}"
register: csr_file
- name: Read CSR content
ansible.builtin.slurp:
src: "{{ certificate.csr_path }}"
register: csr_content
when: csr_file.stat.exists

The cert is just fine

-----BEGIN CERTIFICATE-----
MIIEdjCCA16gAwIBAgIUT8P6KVyWLnfhi8LFodI2rfV9NWswDQYJKoZIhvcNAQEL
BQAwgYsxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJHQTEQMA4GA1UEBxMHUm9zd2Vs
bDEOMAwGA1UEERMFMzAwNzUxIDAeBgNVBAkTFzE3MCBDb2NocmFuIEZhcm1zIERy
aXZlMRUwEwYDVQQKEwxSdWRsb2ZmIEluYy4xFDASBgNVBAMTC3J1ZGxvZmYuaWNh
(...)
ZdmaZwM8GSjj+CR7jZJquFK/w2DFn4vaaZWm3uik6VCwfF+VENf7G0W4F6BTIeYW
FKmrB5lEX3vD60pz+rLlTo3e+Mv7sc20sjUmOrdQrO0S7BJAZ8s7Vs+CHEgOiKIq
vOEXJ2p5MWVytZsevoXmHrV5QREKgFrVxXjpsq9N21d+KqL8nkglc4Ix
-----END CERTIFICATE-----


In fact I see the same issue with Puppet and Salt .. for now I just use a bashfile to issue certificates that are being run by Chef etc. - but it is puzzling that Ansible 'gets it right' - but the rest isn't ...

Any takers lol ?





https://redd.it/1ei5zfr
@r_devops

What do you use your developer portal's for?

In your company, what is the main use-case for developer portal (Like backstage, port, cortex, Roadie) ?


Is it the service template?
Incident management / On-call view?
Is it feature flags? ad-hoc permissions?
Deployment?

Or even security?

anything I'm missing? what do you think is the main use?

https://redd.it/1ei9z8d
@r_devops

Calculator for determining uptime required of dependencies in order to meet application uptime SLO

https://eason.blog/posts/2024/08/availability-dependencies/

Shows the relationship between an application's uptime and the uptime of it's dependencies. Post includes an interactive calculator you can use to determine what the dependency uptime has to be in order for the application to have a hope of hitting it's SLO. Curious if y'all have implemented policies that take this perspective into account and how that works at your company?

https://redd.it/1eicl70
@r_devops