Puppet tutorial/book recommendation

Hey guys,

So I just got an offer letter today for DevOps position (yay!) and the team uses Puppet for their configuration management. I am trying to prepare myself for the role as much as I can before the start date and wondering if anybody can share resources for learning Puppet.

I have been using Linux Academy to learn RHEL/Git/Ansible and I liked them all so far but I do not like their Puppet tutorials.

I would really appreciate your input!

https://redd.it/esib8z
@r_devops

Checking what an upgrade will break?

Hi, I'm looking to upgrade some software at my current job, but I want to do it with minimal breakage.
In this case, it's Jenkins, and it's a big jump forward(our version is 2.2xx.3, newest appears to be 2.7.x).
Is there an aggregator of change-logs that will tell me what this will break? I don't want to go in blindly since that would be a huge pain in the ass.

https://redd.it/escj7w
@r_devops

Does anyone know a way to pass a file to a remote instance using ssh forwarding?

I'm using Terraform to instantiate and instance and It always generates a new instance with a new hostname. I wanted to pass a DNS value from a file using SSH forwarding to automate this process. does anyone have an idea? I'm thinking something like this.

cat efs_dns.txt | ssh -A ec2-user@$$(terraform output bastion-public_ip) ssh master.openshift.local

https://redd.it/esgmr2
@r_devops

Here’s an idea: a 3D VR world of your Kubernetes cluster

Pods, containers, services, deployment, logs and live traffic visualized in a 3D space that anyone can move around in. Useful for debugging.

What do you think of this idea?

https://redd.it/esd6rb
@r_devops

Has anyone passed the Cloud Foundry Certified Developer Exam?

Any suggestion on study path? It is an affordable exam?

What should I study most and which platform could I use for make practice?

https://redd.it/esbzhi
@r_devops

jenkins approval workflow

I want to set up an approval workflow where a user can approve (or disapprove) a pipeline step. How does one go about setting this up? Ideally, I would like to get the user from ActiveDirectory.

https://redd.it/esb75v
@r_devops

Looking for social media recommendations: who should I follow on Twitter for the best kubernetes/backend-related posts?



https://redd.it/es9aqa
@r_devops

Prometheus Operator Federation Questions



EKS Clusters running in different aws accounts. Goal is to monitor different clusters at one place

Found Prometheus Federation is a good fit for the design and scaling

[https://prometheus.io/docs/prometheus/latest/federation/](https://prometheus.io/docs/prometheus/latest/federation/)

[https://www.robustperception.io/federation-what-is-it-good-for](https://www.robustperception.io/federation-what-is-it-good-for)

Master : I have a dedicated EKS for monitoring system where prometheus operator helm chart is running with prometheus and Grafana exposed behind load balancer service

Master prometheus server is designed to scrape other slave prometheus servers with the /federate endpoint and plot the Grafana dashboards for different clusters

One Grafana Endpoint to monitor everything

Slave Account 1:

Running prometheus operator helm charts in the other slave accounts where all the alerting is done through the alert manager in the slave account 1

Prometheus - Default scraping from prometheus operator + additionalscrapeconfig for istio services

Alertmanager - alerts related to slave cluster 1 integrated with Victorops

Slave Account 2:

Running prometheus operator helm charts in the other slave accounts where all the alerting is done through the alert manager in the slave account 2

Prometheus - Default scraping from prometheus operator ‘

Alertmanager - alerts related to slave cluster 2 integrated with Victorops

Now we have another use case to monitor the bunch of bare metal servers, aws ec2 instances and monitor different endpoints

Monitor Bare Metal, EC2 Instance - Installed NodeExporter for disk metrics and can able to scrape the metrics directly from master prometheus server

Endpoint Monitoring(Public & Private) - Installed blackbox exporter on the master eks cluster and probing different endpoints

Here are my questions

​

1. Is it a good practice to enable alertmanager on the master prometheus and do the alerting only for the last usecase(bare metal, aws ec2 instance, endpoint monitoring)
2. From the below article, I see alertmanager should be on the slave prometheus(reliability purpose), If so do we need to install prometheus on each bare metal again?
3. Can we use any existing loadbalancer for exposing prometheus, because in helm chart if we expose the service.type=loadbalancer it used to create new one(internet facing/internal)

https://redd.it/es4q2r
@r_devops

Why does vendor lock-in seem like an especially bad thing in tech?

Not sure if this is the best place to discuss this, but I thought I'd post it anyway.

Every discussion on cloud seems to lead to some talk about lock-in. Now, I understand the risk of lock-in. But vendor lock-in is not a problem that's unique to cloud. Most enterprises are "locked-in" to Microsoft office and windows. A good proportion of people are "locked-in" to the Apple ecosystem.

In fact vendor lock-in is a business risk in other industries too. Southwest only uses Boeing 737's. There's a risk that Boeing could decide to be nasty and raise prices in future. Sure, Southwest could buy Airbus planes (but it would create massive problems for them), and then Boeing's reputation would take a hit. Similarly, if AWS raises prices, it would take some effort to move elsewhere, but their reputation would also suffer.

So I guess my question is, why is vendor lock-in the scary bogeyman it is in cloud? I personally see it as a business risk, but not a avoid-at-all-costs thing.

https://redd.it/esvga0
@r_devops

How to find passwords in source code?

Hi everyone, I’m new to software development and have been tasked with finding any exposed credentials or keys in plaintext in our source code repos. Looking for any suggestions on how to go about doing this. I’ve tried searching “password” and “pwd” but I’m not turning up much. Any advice appreciated. Thanks

edit: I don’t have direct access to the repo aside from read-only as I work in internal compliance and not the dev team.

https://redd.it/esvb0p
@r_devops

Intro to Pulumi for Python

Hey y'all


As a Terraform veteren I am really liking Pulumi, I tired out the CDK and didn't love it especially with the synthesization to CloudFormation. Does anyone have any experience running Pulumi outside of just small dev projects I would love to hear thoughts as I go deeper into it!

For those that have not used it yet, I have been creating some Devops videos to share with some co workers and also for myself to learn some new technologies. This Pulumi video is the tutorial I wanted when I was starting with Pulumi so maybe it can be helpful to someone out there. [https://www.youtube.com/watch?v=60LYNRnmM5M](https://www.youtube.com/watch?v=60LYNRnmM5M)


*PS. I am learning that coding while talking is hard*

https://redd.it/eswe3d
@r_devops

Microsoft Customer support data breach

Seems like a misconfiguration lead to the exposure of personal data of customers that had contacted Microsoft customer support. Wooopsie

"Our investigation has determined that a change made to the database’s [network security group](https://docs.microsoft.com/en-us/azure/virtual-network/security-overview) on December 5, 2019 contained misconfigured [security rules](https://docs.microsoft.com/en-us/azure/virtual-network/security-overview#security-rules) that enabled exposure of the data. Upon notification of the issue, engineers remediated the configuration on December 31, 2019 to restrict the database and prevent unauthorized access. This issue was specific to an internal database used for support case analytics and does not represent an exposure of our commercial cloud services.

As part of Microsoft’s standard operating procedures, data stored in the support case analytics database is redacted using automated tools to remove personal information. Our investigation confirmed that the vast majority of records were cleared of personal information in accordance with our standard practices. In some scenarios, the data may have remained unredacted if it met specific conditions"

[https://msrc-blog.microsoft.com/2020/01/22/access-misconfiguration-for-customer-support-database/](https://msrc-blog.microsoft.com/2020/01/22/access-misconfiguration-for-customer-support-database/)

https://redd.it/estu79
@r_devops

Detailed Jenkins architecture guides?

I'm curious if there are resources out there for understanding Jenkins in excruciating, vaguely academic, detail, similar to Linux Insides. I'm not looking for user guides, but something more akin to tools from a software engineering and architecture point of view. I'm genuinely curious in nitty gritty details of DevOps tools in general, but I've been drawing blanks short of reading source code.

https://redd.it/esyujo
@r_devops

Devops approach to CMS sites?

The entire concept of one or more people on a marketing team doing live YOLO-OPS edits on a production website triggers the spidey senses, yet sometimes this is unavoidable. I have very limited experience doing this and in the past the approaches I've taken have been to limit the scope of the CMS site to decrease the potential blast radius and/or use the CMS only in dev, then generate static html content from that which is what ends up in prod.

I'm curious to know how others have tackled this overall, and specifically with DNN.

https://redd.it/esv342
@r_devops

How do I know if I know enough DevOps for a very junior position?

Hello guys, in the past year and a half I've been learning as much I as I could about DevOps and servers. I have a background in manual QA and would like to transit to a junior position but I don't know if I can qualify.

I've done the following courses, so I'm assuming I have basic knowledge about these topics:

Linux Administration Bootcamp: Go from Beginner to Advanced

Linux Foundation Certified SysAdmin (LFCS)

Network Protocol Fundamentals

Intro to Linux Shell Scripting

Learn The Linux Command Line: Basic Commands

Kubernetes Hands-On - Deploy Microservices to the AWS Cloud

AWS Concepts

AWS Certified Solutions Architect - Associate 2020

Ultimate AWS Certified Solutions Architect Associate 2020

Ansible Quick Start

And now I have these in progress:

DevOps Essentials

Mastering Ansible

Implementing a Full CI/CD Pipeline

Docker Mastery: with Kubernetes +Swarm

The company I work for, even though helps its employees in their career path it's not quite willing to move me to DevOps as they find me more valuable as QA, they keep finding reasons to not move me. How do I know I should look for a company willing to give me a mentor so I could learn more and switch to devops?

Everyday I spend around 3-4 hrs after work to learn more stuff so my commitment it's as high as it should.

Thank you.

https://redd.it/esuim1
@r_devops

Does this seem like a good DevOps position?

Role is at store chain with a few hundred locations in my country.

Preferred Skills: communication, Jenkins, cloud, blue/green development, datadog, New relic, Splunk, AppDynamics, dynatrace

Job title is junior DevOps engineer

If I got this job would I learn a lot? Would I struggle to move into a development career if I decided to switch in two years?

https://redd.it/esw8dn
@r_devops

How do you all handle IAM in your organization for system management?

This is a heavy topic and I’m looking for ideas and a discussion on how others handle the IAM topic. Putting the whole pets/cattle arguments aside, we as devops professionals have tons of services we need to create and manage. We have internal tools, external tools, third parties, infrastructure, databases, sometimes hybrid environments, the list goes on and on. Some have rock solid features. Others are crap and we are forced to use them. This topic is probably more security focused, but it effects many areas. How do you all handle IAM across your entire organization ? I work for a cloud service provider and we are struggling to find a solution that works well. I’ve been trying to find a one size fits all strategy. I know this is not realistic, but it’s at least a North Star.

Some background on some of our workflows:
- Deploying applications across multiple cloud providers (Azure,AWS) as well as private data centers
- Primarily automating and spinning up Windows and Linux servers as the infrastructure to host are applications we deploy to customers. We are moving some to containers/kubernetes, but that is a long way out. Many are apps developed in the traditional sense that will take a few years to be fully cloud ready.
- SSH and Windows RDP access is needed
- Access to services such as AWS consoles, Azure portal, MFA providers, ticketing systems, list goes on

I could expand on many of these topics. Suffice it to say, IAM is hard. Our goal is to have a centralized, non-discretionary, role based access approach that can be handled across all our various management solutions. We have looked at tons of solutions from endless providers, but nothing seems to fit the bill. For example, we would prefer to have Azure AD as definitive source for IAM, access, roles, auditing, etc, it works well for Azure focused and Microsoft items, but not well for things out side of it (other cloud providers for example).

Anyway, I’ll open the floor for discussion. How are others handling their IAM strategy? What challenges are you running into? What best practices do you follow? What is working/not working well?

https://redd.it/ess6t1
@r_devops

Ship logs to Elasticsearch over [unstable] internet connection

hi reddit,

I need to ship logs to a single central Elasticsearch over the internet, from a bunch of different locations, in some of them connection can be unstable. I plan to use filebeat installed in every location with reasonably large queue configured, so any connection glitches would not stop logs from eventually being shipped to Elastic.

A question - had you ever worked with scenario like this? Do you know anything better that just putting filebeat with large queue in every remote location? Any thoughts/ideas, please?

Any feedback/suggestion is extremely welcome.

Thank U!!!

https://redd.it/esvi1x
@r_devops

What kind of alerting do you use on your API's?

Trying to setup alerting to detect suspicious behaviour, abuse or anything else...

​

If theres any documentation you guys reccomend on what to alert on that'd be helpful aswell, as I'm not exactly sure what to look for at the moment

https://redd.it/estxbf
@r_devops

Want to learn enough to keep up

Want to learn enough to keep up

If you’re going to be working among the cloud, rather than specifically in the cloud, say in an executive or a marketing capacity, you should consider a Cloud Practitioner certification.

Upon gaining this certification, you’ll be able to keep up with your IT teams and understand cloud concepts sufficiently enough to think about how they can apply to your company or industry.

https://redd.it/est0bn
@r_devops

What is my next step?

Just finished my Comptia Linux+ Cert, and I am graduating with an AAS in CIT with a focus in Cybersecurity this spring; 3.5 GPA. I also have Comptia Network + and A+ certs. I have worked at a paid IT internship in a Windows environment for the past 18 months. I have done some automation but not a ton. I took some online Docker courses and have played around with Kubernetes. Coursework for my AAS included general IT, networking, Cisco, Windows server, Linux server, security management, Powershell, Vsphere, and a lot of other technologies I'm not going to list. I studied CS for 3 semesters at a state university and know my way around C++, Python, Powershell, and scripting, but my GPA was very low. My questions for you are:

​

1. Should I now apply for DevOps internships, a DevOps junior positions, DevOps engineering positions, or Linux Administration positions? What level do I currently qualify for? I feel like I should apply for everything, but I don't want to waste my time or anybody else's. I don't want to get a job I am overqualified for. I live in the Midwest in a big city, and can relocate to expand my search range. I do very well with people in interviews in person, but I look like I'm 19 (I'm 25m).
2. What certification should I go for next? I am finding a lot of AWS recommendations, but the Kubernetes, Ansible, and Docker certs seem pretty relevant and more general. Also I think I would have fun with the red hat certs.
3. I know completing a BA alongside my next job would probably be the right choice for me financially. I am going to wait to see how my next job pays and how heavy the work load is before I decide. I'm sick of college professors at the moment. This isn't really a question but I thought I would get it out there.



Anyway any input or reality checks would be greatly appreciated. I feel like there are a lot of options in front of me, but I don't know enough to accurately judge them.

https://redd.it/et46km
@r_devops