How does your application deployment pipeline look like?

I'm eager to learn what setups do you have because right now I have something what I always did , it was either setup before I joined the company or I recreated the same concept in a new job. I have a feeling of lack of perspective.

My current setup: code repository has 3 branches resembling 3 logically separated environments in the cloud: `dev`, `staging` and `prod`.

\- `dev` is where developers commit their feature branches,

\- `staging` is what we show to the client,

\- `prod` is a production environment.

Merging flow is like this `dev -> staging -> prod`. After a merge an automation deploys given branch to the cloud.

The downsides to this, that I see so far, are: the velocity of propagating a fix to production; it's hard to propagate just a single feature to production if `dev` branch contains multiple already merged (especially when they are not squashed 1-commit features); there is no way to prevent developers from hotfixing `prod` directly and leaving it as a mess.

Also, expect sharing your own setups, if you see any more downsides or can hint an improvement in this approach please share.

https://redd.it/yj4umq
@r_devops

How to certify myself?

Hi,

I am a 35yo male with over 10 years of experience in technical support (on-site and remote), network administration and Linux systems administration, but I have no certifications or formal job experience. When I was entering the market, I've met my wife (we are married for over 10 years now) and, since she already had a good paying career, we decided I would work autonomously. I should note I am not from the US.

Recently, I decided I want to leave my country for a chance of a better future for my daughter. I am currently learning a bit of German and have proof of proficiency in English (TOEFL B2, 2016 and EF SET C2, 2021). Yet, I have no certificates for my CLI/Linux/Windows sysadmin abilities.

Are LPI certificates a good start? I'm aiming to work with Python, Ansible, Jenkins, Docker/Podman and RHEL. I know the fundamentals of networking like OSI Layers, IPv4, IPv6, Servers (DNS, DHCP, NPT) etc.

Money is a problem. I can pay for some certification, but I don't really know which would be better for landing a job soon.

Thank you and sorry if this isn't the right board.

Regards, Gabriel

https://redd.it/yj6ojs
@r_devops

Monthly 'Shameless Self Promotion' thread - 2022/11

Feel free to post your personal projects here. Just keep it to one project per comment thread.

https://redd.it/yjdqwa
@r_devops

'Getting into DevOps'

What is DevOps?

[AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

Books to Read

The Phoenix Project - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
[The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
Google's Site Reliability Engineering - Google engineers explain how they build, deploy, monitor, and maintain their systems.
[The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book
The Unicorn Project - the "sequel" to The Phoenix Project.
[DevOps for Dummies](https://www.amazon.com/DevOps-Dummies-Computer-Tech-ebook/dp/B07VXMLK3J/) - don't let the name fool you.

What Should I Learn?

Emily Wood's essay - why infrastructure as code is so important into today's world.
[2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
This comment by /u/mdaffin - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
[This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.
Roadmap.sh - Step by step guide for DevOps or any other Operations Role

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

Please keep this on topic (as a reference for those new to devops).

https://redd.it/yjdscp
@r_devops

Alternative to cloudcraft

I have been trying cloudcraft to make network diagrams, but the free version has a very small tap, I like the design of the diagrams with the 3d version especially for projects in which the documentation has to be delivered to the business people.
Is there any other tool that is not as expensive as cloudcraft (pro versión with bigger grid 49/month)?
Maybe the best solution will do all the diagrams with draw.io 😭😭

https://redd.it/yjeznn
@r_devops

What's the best docker runtime for M1 Macs?

Hey folks! Just curious what everyone is using for their docker runtime on their M1 Macs?

I stopped using Docker for Desktop as it was annoying and bloated, and went to Rancher for Desktop. It can hog the CPU and Mem at times, so trying to see if anyone has any other recommendations?

https://redd.it/yjif5p
@r_devops

Did "DevOps" somehow become synonymous with "Deployment Engineering" in the job market?

When I first started getting into DevOps (that is to say, the DevOps philosophy, not any job title or team named "DevOps") it was all about providing developers with tooling, education, and guardrails on service ownership and operations. We would give them the keys to open cross-service firewall ports, scaling/autoscaling rules, building deployment pipelines and stages, machine size and resource allocation, and all the things an "ops" person would do for them. With those keys, we provided some guidelines and automatic checks for sanity. We would write linters for their terraform code and require someone (an SRE or senior developer) schooled in operational needs to approve their Terraform/Chef/Puppet/whatever code. We would write the common/sidecars needed to allow their service's containers to run.

Now I see job after job listing and recruiter after recruiter with "DevOps" and "SRE" roles all about deployment engineering. Speed up testing. Speed up deployment. Fast rollbacks. Very little collaborative interaction with service developers to help them understand how there service operates, but a whole lot of "here's a black box - push your code into it and now it's online."

What happened?

https://redd.it/yjp95b
@r_devops

Help me hone my focus. My goal is to transition into a SRE and/or Platform Engineer style role in the next year-ish.

Hi all,

I currently work on the operations side, however focus the majority of my efforts on automation. I've spent most of my time in a quasi-hybrid role, primarily around infrastructure configuration management and automation. Ansible, PowerShell, PowerShell DSC are my bread & butter right now.

I'd like to make a list of say 4-5 technologies to focus on over the next year to make myself attractive for roles related to platform engineering or site reliability.

I just recently passed my AWS CCP exam. I also work with AWS somewhat regularly, and so I have a good conceptual knowledge of the core services: S3, EC2, VPC, CloudFront, IAM. I also have a decent idea around API Gateway, Lambda, and SSM from my experience. (Note: I'm lumping in a bunch of the networking into VPC, but I have a decent idea about NAT Gateways, VPC endpoints, subnets, yadda yadda). I also have my Terraform Associate certification, and am very comfortable with Terraform / Terragrunt.

So my list over the next year is as follows:

1. HTML / CSS / JS. No way around it. I'm not that great at this, but I need to be better. At least proficient.
2. Python. I feel like my years and years of PowerShell has set me up for learning another language, but I don't think many places will look at PowerShell favorably. I can already muddle my way through, but I need to be able to actually understand what I'm building with Python.
3. Containers. Again, conceptual understanding, but I need to learn how to use it in AWS using ECS. Obviously a stepping stone to EKS.
4. AWS Database services. I know that DynamoDB exists, but beyond that have no idea how to really use it, or when it's preferred over something like RDS or PostgreSQL.

What are your opinions? Am I on the right track? This seems like a lot, but I could devote a few months to each and I feel like this would set me apart.

https://redd.it/yjierh
@r_devops

kxkn - Simple cli tool for switching between kubernetes namespace and cluster

This is small opensource tool that i have developed while learning rust. (Inspired by kubens and kubectx).

https://github.com/koolwithk/kx-kn-rust.git

Why kx and kn in rust?

Learning :)
small binary size

It does not have all the feature and proper error handling as in kubectx hence it has smaller binary size and perform faster :) You can give a try and report any bug/feature or contribute :)

As of 2n NOV 2022 it's faster than kubectx(by 1.5x) and kubens(by 2x) used `time command` on same cluster to calculate the performance.

Alternative tools:

[Kubectx](https://github.com/ahmetb/kubectx)
Kubie
[k9s](https://github.com/derailed/k9s)
kubeswitch

https://redd.it/yjjrz9
@r_devops

Any nginx expert

I am using nginx stream to use it as a transparent proxy ([https://nginx.org/en/docs/stream/ngx\_stream\_proxy\_module.html#proxy\_upload\_rate](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_upload_rate)). In a way, it is acting as a firewall. here is my nginx config [https://pastebin.com/xmVdnax1](https://pastebin.com/xmVdnax1). I am getting these errors.

2022/11/02 02:56:17 [info] 25278#25278: *2333 recv() failed (104: Connection reset by peer) while proxying and reading from client, client: 172.25.239.179, server: 0.0.0.0:443, upstream: "136.146.33.36:443", bytes from/to client:666/4737, bytes from/to upstream:4737/1111

If I reduce the connect timeout to 10 seconds, I dont get these errors. I am running a very big server with 64GB of Ram so it is highly unlikely that it does not have enough ram. using amazon linux. anyone got an idea? thanks

https://redd.it/yjv15s
@r_devops

Can I use Cloudfront as a single URL for multiple services?

Aloha colleagues,

To give you a bit of context, we need to deploy our application to our customers, and lots of them having proxy we need to provide them with a list of URLs to whitelist.
The problem is that we want to keep the list as short as possible, and we wonder if it is possible to have CloudFront serving as "router" for different services. I know cloudfront can be used as front for S3, but I could not find anything about ECR.
Is even Cloudfront the right tool for the job? We are yet not settle with ECR or S3 and could even go for a complete different stack.


Thanking you in advance for the help!

https://redd.it/yjy4pz
@r_devops

Development on Kubernetes Multicluster with Devtron

How Devtron (https://devtron.ai/) may simplify Kubernetes for developers? The article shows how easily run apps from a single UI to multiple clusters with Helm support: https://piotrminkowski.com/2022/11/02/development-on-kubernetes-multicluster-with-devtron/

https://redd.it/yk10d9
@r_devops

A question for GitHub Actions users

Are you running your tests on GitHub or any external service such as AWS etc

View Poll

https://redd.it/yjz0q4
@r_devops

Write docker image size and build date to a file and contain in the image

I want to be able to read the container image size and date from a file in the container after it’s published and when running.
I’m also working on a bash script to read the date on the file but having some issues.
Any suggestions or help greatly appreciated!

https://redd.it/yk33ls
@r_devops

I wrote an OSS tool to tunnel your IDE to Kubernetes

Since the day I started my DevOps journey, it was always a dream of mine to create an open-source devtool.
I co-wrote a tool called \#KubeTunnel which connects your local development environment to your Kubernetes cluster for debugging complex microservice architectures without deploying them locally, without waiting for a long CI/CD process and without any syncing mechanism to the cluster.
This achieves developing exactly as you would locally with the added benefit of getting full network access to and from your cluster.

Check it out here: https://github.com/we-dcode/kubetunnel


*Buy me a cup of coffee by leaving a star on Github🌟*

https://redd.it/yk2i5b
@r_devops

Different IaC environments on cloud

So I've been working with IaC (Terraform and CloudFormation) on AWS for awhile. I've touched on simple environment stacks where Dev, sit, UAT and prod are identical, this makes trunk based development very simple and easy.

However, I also touched on more complicated environments where the application stack uses different AWS services in different environments to save cost.

just as an example, Dev may only use EC2 instances to run the app, then UAT will include ASG. In prod it will use ASG + ALB...

I'm curious to know if this practice of using different services in different environments is normal? I find it very difficult to make an IaC change to say ALB where it only exists in prod.

In my opinion, UAT should be the exact same replica of prod, so testing can be done in UAT (non production) at the least... this still makes me think what branching and coding strategy is right for this type of infrastructure requirement?

Have anyone else here face similar challenges?

https://redd.it/yk3ppf
@r_devops

Datadog has OAuth Support Now

I'm a little surprised it took them this long but now I expect several companies will build on top of it. For example LambdaTest can show test results from within Datadog, https://www.datadoghq.com/blog/oauth/

It's not clear what endpoints are exposed yet but I imagine documentation will be forthcoming, and hopefully self-serve submissions too.

https://redd.it/yk6whi
@r_devops

How do you control images pulled from public image repositories like DockerHub?

We have a need to control what images a developer can source from DockerHub. Ideally we only want them to pull verified, approved images. But, how to ensure that only approved images are sourced?

For any images brought in, we want to have them scanned to ensure that they are safe to use. But are any other controls recommended to use?

I work in a highly regulated industry and our risk tolerance is very low. The more safeguards, the better. But we are new to container management.

https://redd.it/yk90ba
@r_devops

Guidance on provisioning QEMU VM images based on specific hardware products

## Description
I work for a company that mainly develops custom industrial grade Computer hardware. As a part of the Software, we ship the hardware with an Ubuntu Image with all the bells and whistles in it (think Docker, Linux Cockpit, necessary configuration, container images)

### Tools Used
- Cloud-Init (first-boot provisioning)
- Hashicorp Packer with QEMU Plugin for x86_64
- Ansible (post-processor provisioning)

### Resultant Output

I have `qcow2` images that are successfully push to our internal artifacts registry.


## Query

Since we have a couple of different hardware that we produce in-house, I would like to separate the provisioning on the QEMU virtual machine images based on the Hardware Product Family.

The only problem here is, in a QEMU virtual image, Ansible Facts generally do not work. We build the images in a CI system and then create the filesystem tarballs and boot them "manually" in post-production stage of hardware.

Is there some way I can create Ansible Roles, that can be according to the Product Hardware Family without actually provisioning on "actual hardware"?

### TL;DR

How to create ansible roles for diverse hardware products when trying to provision images virtually using qemu?
e.g.
Product A --> consists of APT packages x,y,z,docker
Product B --> consists of APT packages x,z,docker
Product C --> consists of APT packages y,docker

etc.

https://redd.it/ykfuf7
@r_devops

FREE Azure Data Factory for Azure Data engineer and DP-203 Exam

Free Course: https://www.udemy.com/course/azure-data-factory-for-azure-data-engineers-with-hands-on-labs/?couponCode=100\_OFF

https://redd.it/ykjd0g
@r_devops

DevOps for generated art?

Not sure if this is the correct subreddit to post in, but here goes. (feel free to point me to a more appropriate one)

I am getting into generated art, which is going in the way of AI. I want to deploy some sort of pipeline of AI tools/services. But, I don't know where to start? Where do I begin? What tools should I be using? What AI models are simple to deploy and use?

If anyone has experience doing this, I'd love to hear from you.

Thanks!

https://redd.it/ykqhou
@r_devops