How is the Delivery or Deployment process supposed to work with dynamic data (ie: environment variables)?

So I have some code for an application in a GitLab repository, and some of that code needs to read variables containing data like secrets, URLs, etc. These variables are in a .env file.

The system that ultimately runs the application would, once pulled, read the variables from that file into its environment where the application could then retrieve them (I recognize that another common method is to put the variables into a permissions protected file as opposed to using environment variables, and I think this question would cover both use cases).

In my pipeline, I have a job that collects this data (ie: HashiCorp Vault), but now what?

I was under the impression that a pipeline job would be able to take this data and write it back out to the appropriate file(s) in the repository. It seems to me like that would be a perfectly reasonable approach to making it so that a user or operator could checkout code and run it without having to do things like manual variable substitution; that seems counter intuitive to me.

And so I tried to build a job that did that, but while the file in the job had the correct data, the file in the repository didn't. While I was trying to look for answers as to why, I read a bunch of things that basically said "don't let your pipeline write back to your repository, that's bad".

If it's actually bad practise, what is good practise?

To be clear, keeping passwords and secrets protected isn't the important point of this question. The important point of this question is how to make it so people don't need to be involved in the deployment by having to edit variables in pulled code before running it.

https://redd.it/120lk3w
@r_devops

Question Deploying a Node JS app on a windows server

A bit of a dumb question since I’m not a devops expert.

I have an express js REST api, and a react app that sends requests to it.

All of the system will be run on a private network for internal use, the problem is that the machine currently used runs a windows server and IIS, serving more internal use systems.

Serving the react static files was not a problem, but running node was a bit harder.

### What I tried:

1) Running

 run start-prod

as a service. I didn’t like this approach since I don’t have control over the traffic in IIS and it was hard to restart the server if a crash happens.

2) Using iisnode. This didn’t work and required me to change my code which I’m trying to avoid to be able to easily develop and test my api locally.

What I already know and wanted to do is build the api as a docker container since I’m more familiar with configuring a linux machine, but then I’d need to run docker inside the windows machine which seems like adding an unnecessary layer to the production environment.

Is there a correct way to do things? All the methods I tried look dirty and incorrect and I’d love to get some tips from people with experience on this.
A really nice bonus would be to automate updating the code to include some what of a CI/CD environment.

TLDR: Have an express js api, need to run it on a windows server that is already serving other applications.

https://redd.it/120kvxm
@r_devops

Tell me your CI/CD process

I'd like to see how you set up your ci/cd in high level for developers. I like to get an idea if ever we upgrade. I'll start.

1. Users create a project in gitlab. They follow rules like specific json filenames for dev, stage and prod. These json files are for building the CD pipeline in Spinnaker. Then there is another json named pipeline.json which is meant for building CI in Jenkins automatically.
2. Once a developer git pushes their code with valid json files, a jenkins job will automatically be created. It runs their code for building their application inside a slave machine(triggered by master jenkins)
3. Once the build is complete, an artifact is created. It could be an rpm, deb, tar.gz etc and is uploaded to a central internal registry
4. Spinnaker is poke by the Jenkins job that finished running and will say "I'm done with my job"
5. Spinnaker has created a CD pipeline which will do several tasks, pull the artifact that was uploaded to the registry. It also builds the resources in AWS. We are in AWS. So if EC2 type was chosen by the developer, Spinnaker will provision an EC2 instance and it will install the artifact.
6. Done

Tell me your setup. Thank you.

https://redd.it/121aav8
@r_devops

Dataflow app I'm working on

Hi, I'm working on launching a little web app / service that helps people doing data engineering, operations, devops, perhaps other functions get data from where it originates and into the cloud. Curious if people have thoughts about this and might find it useful:

https://hexcloud.co/

https://redd.it/120jckl
@r_devops

Render.com vs DigitalOcean App Platform - PHP Benchmark

I'm benchmarking PaaS providers to determine which one will be my next, and I thought why not sharing the results with the community...

​

DO Instance type: PRO - 2GB/1vCPU (25$/month)

Render Instance type: Standard - 2GB/1vCPU (25$/month)

Both running PHP 8.2

​

Test script:

**https://github.com/lavoiesl/php-benchmark/blob/master/tests/memory.php**

​

DO PHP memory benchmark (1/2/1381ms):

^(1024 * 256) ^(1) ^(ms 0 B)

^(1024 * 1024) ^(2) ^(ms 100 % 0 B)

^(1024 * 1024 * 16) ^(1381)^(ms) ^(13800 % 0 B)

​

Render.com PHP memory benchmark (1/6/1870ms):

^(1024 * 256) ^(1) ^(ms 0 B)

^(1024 * 1024) ^(6) ^(ms 500 % 0 B)

^(1024 * 1024 * 16) ^(1870) ^(ms 186900 % 0 B)

​

I must say I'm a bit disappointed in Render's result, I was leaning towards them.

https://redd.it/121cm3j
@r_devops

DevOps speakers

I was put in charge of the agenda for our biweekly DevOps meeting for the entire IT team in our company. The idea was to find some internal/external speakers. I'm ok with the internal speakers, but I wonder how can I find people from outside of the company? Did anyone do anything similar? Thanks for any tips!

https://redd.it/121e142
@r_devops

DevOps and bombardment of tools

Hi everyone,

Does anyone feel like we are under bombardment of tools. When i've started working as a DevOps engineer 5 years ago, there aren't too many tools for various tasks for example:

zabbix for vm monitoring

ansible for configuration management

jenkins for all ci/cd. You can also combine ansible with jenkins.

graylog for logging.

and bash scripts for everything else.

But now cloud providers have their own tools for every single thing.

Think about k8s and related tools, everything become unbelievably complicated.

Every company have different tool set, if you change your job boom. your knowledge no longer works and you have to learn new tools for the same job. I really don't understand, all i want is deploying the code for example but it has become a rocket science.

Depending on cloud tools, is the worst thing based on my experience, i've seen that my company suddenly demand that migrate all our infra from aws to gcp couple of years ago.

What are you thinking?

https://redd.it/121dfc9
@r_devops

Is this helpful?

Hi, just wanted to know if you’d want to use a project like this :)

Mainly for quick prototyping?

https://youtu.be/B9r2koCpYm8

https://redd.it/121f829
@r_devops

Modernizing a PowerShell shop

I just started a new assignment as a DevOps engineer. One of the ambitions of the team is to become the AWS of the organization. A small portion of the users can use automated deployments of MS SQL Server. Most users are asking for custom deployments and sysadmin roles for the server.

The team struggles with legacy, custom deployments, config drift, and bad development practices. Basically, their standard workflow is to log on to a VM (the VMs are delivered by a virtualization team), download a git repo with 100s of PowerShell scripts, and deploy an MS SQL Server instance. If the script fails, they modify it at the VM but never include the change to git... Everything is on-prem (\~600 servers), with no IaaC in place or deployment orchestration.

As you can imagine, they are flooded with support tasks and having a hard time improving their stack.

I joined a week ago and have identified a bunch of improvement areas, some are probably completely wrong due to my limited experience in the team, but anyway.
How I'm thinking is to start small and get a better feel for their current process and introduce a better development workflow, evaluate how they can package their deployments better using tools like ansible, docker, and proper config management. Eventually, introduce API for deployments, pretty much inspired by the cloud providers.

My question is, have any of you been in a similar situation?
What would be an effective way of modernizing their setup and introducing better practices to the team?

https://redd.it/121h51u
@r_devops

Booz Allen DevOps Engineer

Hi Guys and Girls,

I am a transitioning veteran, (AD Army) looking to begin a career in the cloud. I have roughly 6 years doing enterprise level IT work, a bachelors in Cyber Security, Sec+, AZ-900, and a clearance.

I am at home familiarizing myself with technologies like Terraform, Docker, and messing around with some free credits I have with Azure. I haven't been exposed to any of these throughout my military career so I am just now catching up.

With that being said, I am going through a veteran transition program that helps active duty land internships/fellowships with companies with the goal of being a full-time hire.

I am certain I have my foot in the door with Booz Allen for a DevOps Role. They seemed to be okay with the fact that I don't have applicable experience in this area which is awesome for me, but also means I have a long road ahead, which I am fine with.

I didn't interview directly with the hiring manager, so I wanted to be transparent and ask what the starting salary was. I have a family of 4 and obviously supporting them outside the military is my main priority. So lets be honest this is why pay is important.

Anyone have insight as to what starting would be like for a Junior DevOps Engineer?

Whether you are a boozer or not please share your experiences.

Thanks for reading!

https://redd.it/121h44q
@r_devops

RESUME REVIEW - Jr. Devops - Toronto

Hi please offer your thoughts on my resume. I have previous co op experience in mechanical oriented jobs but for my new grad position i landed a software devops role at a bank.

https://imgur.com/vXpcRRW

https://redd.it/121dj8t
@r_devops

Comparing Github and Azure DevOps

I am currently evaluating Github and Azure DevOps with the use cases of CI/CD and version control. We are a fairly small org and will probably not need the scalability of Azure. Most of the collab tools of Azure are already available via other services and wouldnt really be used. Github has the disadvantages of being a bit tedious to manage (especially user groups and AD groups, as far as I am aware) and also maybe a bit too lacking in the reporting department.

I have to be perfectly honest so far it boils down to personal taste more than anything. I disliked TFS and have worked with Github a couple of times and found it to be elegant and effective.
Do you have any experience regarding these two in a small to medium org (50 Devs) with sometimes quite strict security concerns?

https://redd.it/121lb07
@r_devops

Good books or learning material for pulumi?

Anybody know of or have any good book recommendations for Pulumi? My searching doesn’t show any books. Outside of that, any preferred learning resources for somebody already experienced in IaC and programming? Thanks!

https://redd.it/121nchk
@r_devops

Hashicorp Vault Certification

Is it worth getting certified as HCP vault associate in 2023 for an entry level DevOps engineer?

https://redd.it/121o87o
@r_devops

I have 2 internships offers and I wanna be DevOps/SRE which one to take ?

So I have 2 internship offers at some different companies the first offer is a DevOps at a consulting company and the other one is a software developer (front-end) at a big N company , I lean towards DevOps but am not sure if it's okay to jump to this role without any experience building software before ? what do you guys think , can I take an internship as DevOps without ever working as Dev before?

Also the internship is a pre-employment one so I d likey get a return offer afterwards.

https://redd.it/121q65b
@r_devops

How does your company do secret management? AWS/GCP/Azure/Vault/CyberArk etc. thoughts?

It seems like every company has a different stack and setup and I'm hoping that y'all can shed more light and educate me (and everyone here really).

What does your company use for secrets management (stack, hosting, setup)?
What your thoughts are on AWS/GCP/Azure secret managers vs. Vault. Is Vault overkill or should AWS/GCP/Azure secret managers suffice even for most enterprise cases? Put differently, why would a company self-host Vault if they can use a native secrets manager from any one of the major cloud providers that already offers a ton of benefits like secret versioning, rotation, audit logging, scalability & availability, etc.
Does your company use one or multiple secret managers? If multiple, what for?
Do you set environment variables in platforms like CircleCI, GitLab, Netlify, etc. manually or do you manage to pull them in from secret managers? What's the typical approach here?

https://redd.it/121rorl
@r_devops

What type of programming do you do based on your role?

Let me explain… I’m a full time CS student who has roughly 4 years of operations experience and at my last operations role, I was automating the windows and linux end user workstation stack using PowerShell + bash and used PowerShell intensely with Active Directory and Exchange. I feel in love with automation.

The main three types of programming styles (correct me if I’m wrong) is Procedural; Object Oriented; and Functional programming. And I’ve noticed that the code I’ve written previously in my last job was strictly Procedural, very simple in my opinion and straight to the point.

Now in college OOP(Java to be exact) is kicking my ass and the concepts is extremely new to me so I’m learning/coding as I go but at this point I’m not too comfortable with the language, but I’m willing to learn.

So my final question is since I want into get into a DevOps role post college, is the automation more procedural or is it mixed with OOP and Functional Programming?

Curious to read your thoughts….

Thanks!

https://redd.it/121u50z
@r_devops

Code project help



Hi, so I created a project using azure, netlify, github, replit and notepad. The github readme for it:

"This project contains a simple chatbot interface that can be integrated with Microsoft Teams using the Direct Line API. The interface is built using HTML, CSS, and JavaScript, and it allows users to send and receive messages through the Microsoft Teams platform.

Getting Started

To use this chatbot interface, you will need to have a Microsoft Teams account and a Direct Line secret key. The Direct Line secret key can be obtained by creating a new Direct Line channel in the Microsoft Bot Framework portal.

Once you have your Direct Line secret key, you can update the index.html file in the chatbotsite folder with your secret key, and deploy the folder to a web hosting service like Netlify.

Usage

To use the chatbot interface, simply open the URL where you have deployed the chatbotsite folder, and start sending messages. The interface will send your messages to the Direct Line API, which will then forward them to your Microsoft Teams account. Likewise, any messages sent to your Microsoft Teams account will be displayed in the chatbot interface."

Now I think I've been able to get it created as ive done the azure stuff, deployed to netlify and configurated the bot using developer giving it permissions etc. The only issue is that i cant test preview it before i publish or even when i publish as i only have teams on my work laptop and i dont think my it admin has given me the necessary permissions to add any old app to Teams. Can someone suggest a way round this? Or even test preview the bot for me to see if what im trying to achieve can be done ?

I've spent the last three to four days on this hellish journey (i have no prior coding knowledge or experience at all) do through a combination of googling and chatgtp(literally feeding it errors line by line and correcting it myself on occasion), ive got to where i am now. I have no intention of just giving up-, need to see this through so any help would be appreciated.

https://redd.it/121v2ms
@r_devops

Alternative to alpine

What other small linux distro you guys use for docker image.

https://redd.it/121xq1m
@r_devops

How much Docker knowledge is enough to be hirable?

I'd like to get into DevOps but my current and past jobs mostly deal with Windows and AWS and doesn't touch anything Docker or Kubernetes. With this I'm left with self-studying them in the hopes of getting good enough to qualify for DevOps roles.

I asked ChatGPT what projects should an applicant do on Docker to showcase intermediate knowledge in it and it got me these:

1. Multi-service application: Ask the applicant to create a multi-service application using Docker Compose. They should create a simple web application with a front-end (e.g., React or Angular), a back-end (e.g., Node.js or Python), and a database (e.g., PostgreSQL or MongoDB). The applicant should create a Dockerfile for each service and a docker-compose.yml file to set up the entire application.

2. Dockerizing a legacy application: Provide the applicant with a legacy application or let them choose an existing open-source project. They should create a Dockerfile and any necessary configuration files to containerize the application. This will demonstrate their ability to work with existing codebases and adapt them to Docker.

3. Optimizing Docker images: Give the applicant an existing Dockerfile and ask them to optimize the resulting Docker image. They should focus on reducing the image size, using multi-stage builds, and improving build times. The applicant should also document the changes they made and provide reasons for their decisions.

4. CI/CD pipeline: Ask the applicant to set up a Continuous Integration and Continuous Deployment pipeline using a platform like GitHub Actions, GitLab CI/CD, or Jenkins. The pipeline should include building Docker images, running tests inside containers, and deploying the application to a container orchestration platform like Kubernetes or Docker Swarm.

5 .Container monitoring and logging: The applicant should set up a monitoring and logging solution for containerized applications. They can use tools like Prometheus and Grafana for monitoring, and Elasticsearch, Logstash, and Kibana (ELK Stack) or Fluentd, Elasticsearch, and Kibana (EFK Stack) for logging. The applicant should create a docker-compose.yml file to set up these tools and demonstrate how they can be used to monitor and analyze the performance and logs of the containerized application.

6. Networking and security: Provide a scenario in which the applicant should create a custom Docker network and secure the communication between containers. They should demonstrate the use of network segmentation, encrypted communication using TLS, and proper implementation of user privileges and secrets management.

I understand that I need to learn many other tools to get into DevOps but I'd just like to know if doing these projects could get someone a pass on Docker. If you have any additional suggestions or advice, please share them. Thanks!

P.S. I'd probably post another one for Kubernetes once I'm done with Docker.

https://redd.it/121tur0
@r_devops

Monitoring using Serverless

Has anyone used serverless workers (Cloudflare Workers, AWS Lambda, GCP Cloud Functions) or CI/CD service (GitHub Actions, Jenkins) for monitoring?

I'd like to monitor a service, and normally I'd make an exporter for it and have a server for Prom/Grafana or InfluxDB or whatever.

But I'd like to not have to manage a server, since this is just a hobby project.

One option is paying for some Prom-aaS hosted service, but I think it'd be interesting to run something on GitHub Actions or Cloudflare Workers, since those are both essentially free for paid users.

I don't need Grafana levels of visualization, honestly I could generate hourly/daily/weekly/monthly/yearly graphs with rrdtool and be good with that.

Has anyone tried this before?

What did you use for a your (time-series) database? A file-based db like sqlite in git, or maybe in object storage? How was performance like?

How would you display your graphs? In git? How do you make sure your git fs doesn't become bloated?

https://redd.it/12200dv
@r_devops