Intro to Pulumi for Python

Hey y'all


As a Terraform veteren I am really liking Pulumi, I tired out the CDK and didn't love it especially with the synthesization to CloudFormation. Does anyone have any experience running Pulumi outside of just small dev projects I would love to hear thoughts as I go deeper into it!

For those that have not used it yet, I have been creating some Devops videos to share with some co workers and also for myself to learn some new technologies. This Pulumi video is the tutorial I wanted when I was starting with Pulumi so maybe it can be helpful to someone out there. [https://www.youtube.com/watch?v=60LYNRnmM5M](https://www.youtube.com/watch?v=60LYNRnmM5M)


*PS. I am learning that coding while talking is hard*

https://redd.it/eswe3d
@r_devops

Microsoft Customer support data breach

Seems like a misconfiguration lead to the exposure of personal data of customers that had contacted Microsoft customer support. Wooopsie

"Our investigation has determined that a change made to the database’s [network security group](https://docs.microsoft.com/en-us/azure/virtual-network/security-overview) on December 5, 2019 contained misconfigured [security rules](https://docs.microsoft.com/en-us/azure/virtual-network/security-overview#security-rules) that enabled exposure of the data. Upon notification of the issue, engineers remediated the configuration on December 31, 2019 to restrict the database and prevent unauthorized access. This issue was specific to an internal database used for support case analytics and does not represent an exposure of our commercial cloud services.

As part of Microsoft’s standard operating procedures, data stored in the support case analytics database is redacted using automated tools to remove personal information. Our investigation confirmed that the vast majority of records were cleared of personal information in accordance with our standard practices. In some scenarios, the data may have remained unredacted if it met specific conditions"

[https://msrc-blog.microsoft.com/2020/01/22/access-misconfiguration-for-customer-support-database/](https://msrc-blog.microsoft.com/2020/01/22/access-misconfiguration-for-customer-support-database/)

https://redd.it/estu79
@r_devops

Detailed Jenkins architecture guides?

I'm curious if there are resources out there for understanding Jenkins in excruciating, vaguely academic, detail, similar to Linux Insides. I'm not looking for user guides, but something more akin to tools from a software engineering and architecture point of view. I'm genuinely curious in nitty gritty details of DevOps tools in general, but I've been drawing blanks short of reading source code.

https://redd.it/esyujo
@r_devops

Devops approach to CMS sites?

The entire concept of one or more people on a marketing team doing live YOLO-OPS edits on a production website triggers the spidey senses, yet sometimes this is unavoidable. I have very limited experience doing this and in the past the approaches I've taken have been to limit the scope of the CMS site to decrease the potential blast radius and/or use the CMS only in dev, then generate static html content from that which is what ends up in prod.

I'm curious to know how others have tackled this overall, and specifically with DNN.

https://redd.it/esv342
@r_devops

How do I know if I know enough DevOps for a very junior position?

Hello guys, in the past year and a half I've been learning as much I as I could about DevOps and servers. I have a background in manual QA and would like to transit to a junior position but I don't know if I can qualify.

I've done the following courses, so I'm assuming I have basic knowledge about these topics:

Linux Administration Bootcamp: Go from Beginner to Advanced

Linux Foundation Certified SysAdmin (LFCS)

Network Protocol Fundamentals

Intro to Linux Shell Scripting

Learn The Linux Command Line: Basic Commands

Kubernetes Hands-On - Deploy Microservices to the AWS Cloud

AWS Concepts

AWS Certified Solutions Architect - Associate 2020

Ultimate AWS Certified Solutions Architect Associate 2020

Ansible Quick Start

And now I have these in progress:

DevOps Essentials

Mastering Ansible

Implementing a Full CI/CD Pipeline

Docker Mastery: with Kubernetes +Swarm

The company I work for, even though helps its employees in their career path it's not quite willing to move me to DevOps as they find me more valuable as QA, they keep finding reasons to not move me. How do I know I should look for a company willing to give me a mentor so I could learn more and switch to devops?

Everyday I spend around 3-4 hrs after work to learn more stuff so my commitment it's as high as it should.

Thank you.

https://redd.it/esuim1
@r_devops

Does this seem like a good DevOps position?

Role is at store chain with a few hundred locations in my country.

Preferred Skills: communication, Jenkins, cloud, blue/green development, datadog, New relic, Splunk, AppDynamics, dynatrace

Job title is junior DevOps engineer

If I got this job would I learn a lot? Would I struggle to move into a development career if I decided to switch in two years?

https://redd.it/esw8dn
@r_devops

How do you all handle IAM in your organization for system management?

This is a heavy topic and I’m looking for ideas and a discussion on how others handle the IAM topic. Putting the whole pets/cattle arguments aside, we as devops professionals have tons of services we need to create and manage. We have internal tools, external tools, third parties, infrastructure, databases, sometimes hybrid environments, the list goes on and on. Some have rock solid features. Others are crap and we are forced to use them. This topic is probably more security focused, but it effects many areas. How do you all handle IAM across your entire organization ? I work for a cloud service provider and we are struggling to find a solution that works well. I’ve been trying to find a one size fits all strategy. I know this is not realistic, but it’s at least a North Star.

Some background on some of our workflows:
- Deploying applications across multiple cloud providers (Azure,AWS) as well as private data centers
- Primarily automating and spinning up Windows and Linux servers as the infrastructure to host are applications we deploy to customers. We are moving some to containers/kubernetes, but that is a long way out. Many are apps developed in the traditional sense that will take a few years to be fully cloud ready.
- SSH and Windows RDP access is needed
- Access to services such as AWS consoles, Azure portal, MFA providers, ticketing systems, list goes on

I could expand on many of these topics. Suffice it to say, IAM is hard. Our goal is to have a centralized, non-discretionary, role based access approach that can be handled across all our various management solutions. We have looked at tons of solutions from endless providers, but nothing seems to fit the bill. For example, we would prefer to have Azure AD as definitive source for IAM, access, roles, auditing, etc, it works well for Azure focused and Microsoft items, but not well for things out side of it (other cloud providers for example).

Anyway, I’ll open the floor for discussion. How are others handling their IAM strategy? What challenges are you running into? What best practices do you follow? What is working/not working well?

https://redd.it/ess6t1
@r_devops

Ship logs to Elasticsearch over [unstable] internet connection

hi reddit,

I need to ship logs to a single central Elasticsearch over the internet, from a bunch of different locations, in some of them connection can be unstable. I plan to use filebeat installed in every location with reasonably large queue configured, so any connection glitches would not stop logs from eventually being shipped to Elastic.

A question - had you ever worked with scenario like this? Do you know anything better that just putting filebeat with large queue in every remote location? Any thoughts/ideas, please?

Any feedback/suggestion is extremely welcome.

Thank U!!!

https://redd.it/esvi1x
@r_devops

What kind of alerting do you use on your API's?

Trying to setup alerting to detect suspicious behaviour, abuse or anything else...

​

If theres any documentation you guys reccomend on what to alert on that'd be helpful aswell, as I'm not exactly sure what to look for at the moment

https://redd.it/estxbf
@r_devops

Want to learn enough to keep up

Want to learn enough to keep up

If you’re going to be working among the cloud, rather than specifically in the cloud, say in an executive or a marketing capacity, you should consider a Cloud Practitioner certification.

Upon gaining this certification, you’ll be able to keep up with your IT teams and understand cloud concepts sufficiently enough to think about how they can apply to your company or industry.

https://redd.it/est0bn
@r_devops

What is my next step?

Just finished my Comptia Linux+ Cert, and I am graduating with an AAS in CIT with a focus in Cybersecurity this spring; 3.5 GPA. I also have Comptia Network + and A+ certs. I have worked at a paid IT internship in a Windows environment for the past 18 months. I have done some automation but not a ton. I took some online Docker courses and have played around with Kubernetes. Coursework for my AAS included general IT, networking, Cisco, Windows server, Linux server, security management, Powershell, Vsphere, and a lot of other technologies I'm not going to list. I studied CS for 3 semesters at a state university and know my way around C++, Python, Powershell, and scripting, but my GPA was very low. My questions for you are:

​

1. Should I now apply for DevOps internships, a DevOps junior positions, DevOps engineering positions, or Linux Administration positions? What level do I currently qualify for? I feel like I should apply for everything, but I don't want to waste my time or anybody else's. I don't want to get a job I am overqualified for. I live in the Midwest in a big city, and can relocate to expand my search range. I do very well with people in interviews in person, but I look like I'm 19 (I'm 25m).
2. What certification should I go for next? I am finding a lot of AWS recommendations, but the Kubernetes, Ansible, and Docker certs seem pretty relevant and more general. Also I think I would have fun with the red hat certs.
3. I know completing a BA alongside my next job would probably be the right choice for me financially. I am going to wait to see how my next job pays and how heavy the work load is before I decide. I'm sick of college professors at the moment. This isn't really a question but I thought I would get it out there.



Anyway any input or reality checks would be greatly appreciated. I feel like there are a lot of options in front of me, but I don't know enough to accurately judge them.

https://redd.it/et46km
@r_devops

How to purge Cloudflare cache via Jenkins?

I am trying to have it so when a certain Jenkins pipeline is ran that one of the steps in the pipeline is clearing the CloudFlare cache since it's not needed. I found the API though was wondering if someone else had a better solution than a simple bash script and a curl command.

https://redd.it/et3qdv
@r_devops

Running DevOps at a start up, looking for advice

I've been working in DevOps for about 7 years now. I have more Ops experience than Dev, but I'm leaning into the Dev side now. I've recently been approached to run devops at a small early stage startup. This is for sure something above me, and I'll admit I'm a little overwhelmed.

Tech stack is GCP with lots of data flows. Primarily Linux and open source software.

Does anyone have any advice or any materials to read? I've read The Phoenix Project already, but I'd love some good articles or blogs, if anyone has any!

https://redd.it/esxi2m
@r_devops

How do I inject secrets into an instance provisioned by terraform for use in userdata?

Hi all. Please excuse any ignorance, I'm a developer and not super experienced with devops.

I have the following setup currently:
- A custom AMI built using packer
- Terraform provisions an EC2 instance using the AMI and runs some scripts in userdata that requires private git access/credentials and npm credentials.

Problem: I am not sure how to get my credentials into the EC2 instance securely/not in source control so the userdata script can execute successfully.

Only thing I have been able to think of:
- Using packer to configure the credentials. I can look into this if there is no other way but I don't like the idea of baking my credentials into the AMI (even though it's private) and I assume there's a proper way to do this.

Other information:
- This is a windows machine and I'm using powershell (not sure if relevant)

Any help/suggestions would be great. Thanks all.

UPDATE: I found a solution that works for me:

Use `templatefile()` and populate the credentials using TF_VAR_ variables.

https://redd.it/eszypm
@r_devops

Opinion about Windows/Linux DevOps tools and migrating skills

I manage some .NET framework app which will eventually go to .NET core and become cloud-native. Here, PowerShell is my friend. I get the point, I understand the DevOps philosophy, I see the good and bad in company processes. However, I cannot see a lot of resources about Windows DevOps things and I cannot see a solution for some part of the apps and services that have so many restrictions to migrate to cloud. So to speak, it seems like Windows has a lot to do to catch up with Linux regarding every aspect and that would take some time (maybe years).

So I wanted to ask what do you think, will Windows ever catch up with Linux regarding DevOps tools?

The reason for my question is simple. Honestly, there is no so much job opportunities when working with Windows, so I thought taking my skills to Linux, to broaden my area of expertise, learn new skills and create new opportunities. I mean, I am not bad at Linux, I just haven't had a lot of chance to work with it. I am familiar with DevOps processes and same should apply to Linux, it is just different OS.

What are your thoughts and suggestions on this?

https://redd.it/esyx3p
@r_devops

python script to ansible playbook for excel

I need to write a python script that cleans up the output from ansible in to a excel spreadsheet.

i have the ansible written , I need a python script to import information into a spreadsheet.

https://redd.it/esy373
@r_devops

Kubernetes documentation no longer supports most recent GKE version?

Correction to title: "K8s documentation way ahead of GKE (and really not easy to navigate for older versions)"

Noob question here. Just started with k8s a few weeks ago.

As of writing, the default k8s version on GKE is 1.13.The default when using "release channels" is 1.14.Current k8s documentation is at 1.17.

I've been Googling and there aren't many results for this, but I see it as a pretty big red flag. In our case, we've been using k8s via docker-for-mac and just realized we have a versioning difference. Is this just something that is accepted? Is there a strategy for working with GKE which is so far behind?

https://redd.it/etbh6m
@r_devops

DevSecOps (Security in DevOps)

I have done implementation of security tools in DevOps process that helps to make DevOps more secure
Here is the github link
https://github.com/Rishabh-Tamrakar/DevSecOps

For now I have updated only 1st page that will explain process that I followed
Configuration of tools will be updated soon (most Probably this weekend)

Do tell me your thoughts about it and what more can be added, ideas are most welcome.

https://redd.it/etbe2k
@r_devops

Easiest way to deploy and host a running Docker Container?

I'm a bit overwhelmed by everything AWS has to offer, but am trying to find a way to deploy a Docker image to have the container running in AWS. The image is stored and updated in Docker Hub, and built via a CircleCI job.

Should I use Elastic Beanstalk, ECS, or something else?

Also, I was originally in favor of AWS, but if any of you have suggestions to keep the cost down, and improving the ease of maintainability, I'm definitely interested in hearing what's worked best for you!

Thanks

https://redd.it/eteld9
@r_devops

Containers that crash "by design"

Hi Everyone. I'm having a "discussion" with our dev team with regards to a container that was deployed to a non production k8s environment, which immediately went into crash-loop-back-off. I assumed the deployment had failed, but in fact, the pod was just waiting for a certain condition (a queue defined in a rabbitmq cluster) to become "true"

The conditions for the pod to continue running (and not crash) are then triggered by someone doing something somewhere in the UI of another service, which may happen 5 mins later, or 5 months later.

In the meantime, its crash-loop-backoff city for our pod.

The developers are claiming this is Kubernetes best practice to allow kubernetes to restart the container should it fail, but:

1) I can't find mention of this specific use case, and

2) It feels like its not in the spirit of what was intended when the best practices were laid out. E.g. "if your application fails due to some completely unforeseeable event, then kubernetes should restart it"... 100% agree with that, but this is "We're going to run code that we know will likely fail, and just let kubernetes restart it until the conditions exist that make it happy"

My concerns are that having a failing pod being a "normal situation" will reduce in alert spam, and eventually, people ignoring those alerts. Further, unless the devs tells us, how are we to know (and remember that those pods "might be in crash loop back off for some time" and this is ok)

My suggestion to the dev team is to have the container "sleep" (and reflect this state in the liveness probe) so that kubernetes will not direct any traffic to it, but similarly, it wont be stuck in crash-loop-backoff hell.

Thoughts?

https://redd.it/etcx9h
@r_devops

Completely random. Can someone tell me how kubernetes is pronounced?

Every time I see it, I read it as cue-bur-neets , but I overheard someone the other day and they pronounced it like cue-bur-net-ees

Please reply with a 1 for my interpretation or a 2 for the way I heard someone say it. If both wong let a man know lol.

Tia

https://redd.it/etfolj
@r_devops