strategy for building ec2 instances

I am new to aws. I am building few hundred large instances. Most will be the same expect for one. At the moment I am using a shell script to create the instances. Its laborious.

To create a AMI, I use a stock AWS image, login to it and run bunch of commands and create my personal AMI. To run bunch of those commands, is it better to use Ansible or some other tool? Once the AMI is done, I launch them thru Ansible ec2 module. It works great. I am wondering if there is a better way to do it. I want to make sure I am not overlooking anything.

https://redd.it/ew4cpk
@r_devops

Experience with RunMyJobs

Just wanted to share my experience, we recently bought ourselves a subscription on the "industry standard" cloud based workload automation solution, "RunMyJobs" (TBH, I had never heard of it before).

I'm genuinely impressed. This solution supports integration of a whole lot of platforms out of the box and has agents for pretty much every platform out there (ranging from HP-UX, AIX to Solaris to Linux etc) has support for pretty much every scripting language out there and support Java (called RedwoodScript in the product, which is basically Java with some magic added), Web-Services (SOAP sadly, not REST).

I have been using this for a couple of weeks now and converted the majority of my cron jobs, batch processes, Docker deployments and am currently working on migrating a bunch of SAP transactions to it .. so far, we're really impressed with its capabilities.

A downside, which we are struggling with is the limited alerting capabilities that are available by default. Only email seems to be there out of the box, however it should be possible to integrate / alert to other services through RedwoodScript.. still figuring this one out.

https://redd.it/ewa05m
@r_devops

Thoughts on CSPM tools??



https://redd.it/ewa0mb
@r_devops

Looking for insights on private domains, public domains and TLS on AWS.

Hello DevOps colleagues,

I am asking for insights about private domains, public domains and SSL on AWS.
Our entire AWS setup is completely internal - no public subnets, no public IPs. The VPCs/subnets are connected via TGW+VPN with our company network for access and internet. We currently have a private Route53 domain for each env, lets call it "env.my-private-domain". Each of the 5 envs as it's own AWS acc.

Our company DNS is forwarding "env.my-private-domain" to the corresponding AWS acc and it's working fine for now. But we will have to use TLS in the future. :)

We are running internal workloads on Fargate. Users will have to connect from the company network to webservices hosted on Fargate. Wether we run SSL end to end or terminate SSL inside AWS via LB or NGINX is not a problem yet. Because we need to have the option to use TLS first.


Going towards TLS leads us to two options:

A) Keep the private domain "env.my-private-domain"

But this gives us new problems.

* We could use AWS ACM to build a private CA, but it costs 400$ a month + for every cert. Each env is a seperate AWS account with would lead us to 5x400$ per month just for a CA in each env. -> This is too expensive for us.

* Host our own CA with Vault. -> We want to avoid the hassle of managing Vault and hosting our own CA.

B) Switch to our public company domain "mycompany.com".

I am not complete sure if this would work. We ask internal IT to allow us to use a subdomain like "department.mycompany.com". We create a Route53 domain for each env like "dev.department.mycompany.com". We ask our internal IT to forward domains like "dev.department.mycompany.com" to our corresponding Route53 Inbound Resolvers. From here we would like to use Let's Encrypt wildcard certs. I think it should work because we using a public domain and should be able to prove that we own mycompany.com. Also Let's Encrypt started supporting wildcard certs in 2018.

We think about such a setup. We use a wildcard Let's Encrypt cert for every env. Something like "*.dev.department.mycompany.com"
So we can use the same wildcard cert for multiple different services/versions in a single env:

Wildcard Let's Encrypt *.dev.department.mycompany.com

* v1-serviceA.dev.department.mycompany.com
* v2-serviceA.dev.department.mycompany.com
* v1-serviceB.dev.department.mycompany.com
....

Wildcard Let's Encrypt *.staging.department.mycompany.com

* v1-serviceA.staging.department.mycompany.com
* v2-serviceA.staging.department.mycompany.com
* v1-serviceB.staging.department.mycompany.com
....

Do you any remarks or insights for option B ? Thank you in advance :)

With best regards

https://redd.it/ew3twx
@r_devops

Automate Building Docker Base Images for Ruby and Rails applications

Automate Building Docker Base Images for Ruby and Rails applications

[https://medium.com/faun/build-docker-base-images-for-rails-2258ea2f11dc](https://medium.com/faun/build-docker-base-images-for-rails-2258ea2f11dc)

https://redd.it/ew20z6
@r_devops

University Concentration

Hi all,

Currently working part time in IT support (internal support + automation) and studying a Masters in IT. I've built some simple web apps, and have done a lot of scripting (python, bash, powershell) on the job with Azure InTune deployment.

With my background, my question is which should I focus on for a career in devops, a concentration in networking and infrastructure or software engineering?

https://redd.it/ew1oon
@r_devops

[Looking for Recomendation] Orchestrated Toolchains and automation

Hi all!

I was wondering what is the normal/standar for toolchain orchestration and automatic tasks.

the kind of Alarms -> Open Tickets -> Remediation -> Close ticket or escalate in case it doesn't do anything. and such...

I'm starting with learning the Flow Designer of xMatters. We are sending data from Dynatrace and CA UIM and testing creating a ticket in BMC Helix and executing remediations in our environment with CA RA and such.

Does anyone have something similar? What was the approach?

Thanks for any insight / examples or alternatives!

https://redd.it/ewffmw
@r_devops

PaaS to SaaS?

Quick question from a noob:

So until last week we had a elastic cluster with two nodes. From what I know, We had multiple VMs in azure (2 for data nodes, 2 for client nodes, 3 for master nodes).

Now my devOps head creates a cluster and applies a YAML with elastic configurations (elastic docker image). There is an ingress reverse proxy for this k8 cluster (not relevant here)

​

Can someone help me understand in easy terms what exactly happened? Did we just move from PaaS and SaaS? How did containerization work in this case.

​

I am sorry, this is very basic question, but I am trying to understand what exactly happened. I know just very basics about k8 and containers. Thanks

https://redd.it/ewfyb1
@r_devops

Help with understanding infrastructure changes

Quick question from a noob:

So until last week we had a elastic cluster with two nodes. From what I know, We had multiple VMs in azure (2 for data nodes, 2 for client nodes, 3 for master nodes).

Now my devOps head creates a cluster and applies a YAML with elastic configurations (elastic docker image). There is an ingress reverse proxy for this k8 cluster (not relevant here)

​

Can someone help me understand in easy terms what exactly happened? Did we just move from PaaS and SaaS? How did containerization work in this case.

​

I am sorry, this is very basic question, but I am trying to understand what exactly happened. I know just very basics about k8 and containers. Thanks

https://redd.it/ewh035
@r_devops

What level of CVE severity do you reject builds?

I’m a fairly junior DevOps at a very small company, and just added container scanning to our CICD pipeline. Ideally I’d like to make sure all of our images are perfectly clean with no vulnerabilities but it doesn’t seem feasible (at least right now).

Currently I set our tool to reject builds that have HIGH or CRITICAL level CVEs. Was hoping to get more input from the rest of you! Thanks!!!

https://redd.it/ewfl67
@r_devops

Can't convert to gigabytes from ansible?

"{{ ansible\_mounts | json\_query('\[\*\].size\_total') | join(' ,') }}"

i tried to add int filter but it gives me 0. I need to read it as human\_readable. The number is on bytes but i need to conveert to gb. I even tried to divide by 1000 but i am errors. Can you help what i need to do ? i tried so many things to work but it won't work.

https://redd.it/ewc22m
@r_devops

Best tool(s) for creating a consistent build environment across entire dev team

Apologize if this is the wrong place to ask, but I figured you guys could point me in the right direction. If I'm being honest, I'm a C developer who is incredibly unfamiliar with modern tools to help with development workflow.


I'm currently working on a project that has a handful of prerequisite programs that need to be installed (e.g. a specific version of cmake, a specific version of gcc, etc.). We've had a lot of problems onboarding people, because we essentially ask them to go through a checkbox of things that they have to download before our project can build/run on their machine. And, because people are imperfect, a lot of people end up downloading the wrong versions or forgetting a prerequisite.


To further complicate things, we have developers using Windows, MacOS, and Linux.


I'm looking for a tool where I can throw all of these prerequisites and have it _just work_. Docker seems to be kind of what I need, but getting it set up for Windows has been a huge pain so far, so I'm hoping there might be better options out there.
Thanks all!

https://redd.it/ewg8s9
@r_devops

What is the latest and proper way to create Kubernetes cluster on AWS?

In GCP it's very easy to deploy Kubernetes cluster and apps. Share your thoughts on AWS.

https://redd.it/ew13vo
@r_devops

Building k8 cluster on teams latop

Based on this article: [https://levelup.gitconnected.com/how-to-deal-with-computing-resource-cost-for-kubernetes-based-development-f723aef01660](https://levelup.gitconnected.com/how-to-deal-with-computing-resource-cost-for-kubernetes-based-development-f723aef01660)

​

Keeping multiple pre-prod clusters up can be very costly for a smaller companies. But most of the time peoples laptops are under utilised. If we can build a K8 master/worker node on teams laptop, We might be able to save some cost.

​

Not sure how much this is feasible or if someone already tried this

https://redd.it/ewp7c9
@r_devops

Tool to run Envoy as an edge proxy for Consul Connect services.

I've created an xDS server for Envoy that lets Envoy route traffic to consul connect enabled services. It has been a royal pain in the backend to find a reverse proxy that can discover services from consul catalog and route traffic to them, and configuring nginx and haproxy using consul-template turned out to be too brittle and finicky.

Here is [Flightpath](https://docs.flightpath.xyz/), it can discover services and routing information from consul catalog like Fabio or Traefik but it also registers itself as a connect native service and shares its leaf certificates with Envoy so that Envoy can communicate with upstream services via sidecar proxy.

It is a work in progress and I have a couple of features planned already but please feel free to try it out and give me your feedback.

[https://docs.flightpath.xyz/](https://docs.flightpath.xyz/)

https://redd.it/ewqgen
@r_devops

CAdvisor vs Experimental Docker Daemon Metrics for Prometheus?

I am wondering which one should I use for Grafana dashboard?

I am open for other solutions, but self hosted since I have small amount of servers to handle and free resources for logging.

https://redd.it/ewpphm
@r_devops

Wrote a small article about experiments with Pulumi

Spent a few days with Pulumi and created a small infrastructure stack. Here are [a few thoughts ](https://grem1.in/post/pulumi/) about it

https://redd.it/ewtc3g
@r_devops

How do you handle co-authoring of code? Check-in/check-out, or Live Share, or similar?

Hey all,

​

Just wondering what you do within your team on the operations side when it comes to co-authoring scripts, modules, and settings files. I've looked into VSCode Live Share paired with Github Desktop, I think it could be powerful but so far haven't had success installing the extension. Are there other options? Do you prefer to check things in and out instead?

https://redd.it/ewsynp
@r_devops

How do you guys build AWS AMI?

We use Packer and Jenkins to bake application AMIs. For testing we use ciscat and rspec. I was wondering what other tools are used on enterprise level for building and testing AMIs?

https://redd.it/ewtlwh
@r_devops

DevOps & DevSecOps certifications

Going to first say that peoples views differ greatly on the validity and value of certification. I understand that and am looking for certification guidance. I have the opportunity to get certification that I know will give me benefits within my organization

I'm looking for DevOps and ideally DevSecOps certification. I see a number from devopsinstitute.com but don't have any opinion or information of colleagues on the organization. I also found devops-certification.org which looks like a certificate factory.

Looking for advice in this area, thanks!

Edit: additional though, perhaps a dedicated security qualification would be more specialised and better recognized, something like CISSP

https://redd.it/ewt7lm
@r_devops

Get kubectl access to your private cluster from anywhere

Get kubectl access to your private cluster from anywhere

This tutorial shows you how to punch out your private [Kubernetes API server](https://kubernetes.io/docs/concepts/overview/components/) to the Internet, so that you can manage your cluster from anywhere, just like you would with a cloud offering. You can also use it to punch out from one VPC to another, if the hosting VPC doesn't allow inbound connections. You can also use the networking tools here to punch out a Raspberry Pi and other TCP traffic to other networks.

Does this sound useful for your DevOps tool-belt?

[https://blog.alexellis.io/get-private-kubectl-access-anywhere/](https://blog.alexellis.io/get-private-kubectl-access-anywhere/)

https://redd.it/ews2c8
@r_devops