How do you manage many resources with tools like Terraform?

We are looking at moving away from manually created infrastructure and going scripted. We have many projects and each has resources.

How does management of this work when using tooling like Terraform?

Should we just have many git repos? Should we use Terraform Cloud? Maybe we should use our deployment tool to assist? (Octopus Deploy).

This is new for us and there are so many options out there that it can be hard to understand the best route to take.

https://redd.it/k6vyqx
@r_devops

🦄The saga continues 12/8 at 1pm EST - AWS GameDay @ re:Invent 2020

Calling all devops enthusiasts. Come learn AWS hands-on and risk-free, network with other tech professionals, and have some fun! The Unicorn Polo League returns live to Twitch on Tuesday, 12/8 at 1pm EST. Claim your spot from the re:Invent [GameDay Site](https://virtual.awsevents.com/channel/GameDay/186983893?nc2=reinv20_m_hocgd) beginning 1 hour prior. Review this [FAQ](https://d1.awsstatic.com/events/AWS%20reInvent%202020%20GameDay%20FAQ.pdf) and see below for some more pointers. Additional video content is available at the GameDay site as well which will help you navigate the event. We are excited to see you there!

**Know-Before-You-Go**

* Event is free, we will provide your team with an AWS account to work in
* You need an Amazon.com account to complete a brief registration process and log in
* We will be streaming live on [Twitch](https://www.twitch.tv/awsgameday), with an internal team playing along with customers and sharing tips and strategy. You'll also have the chance to meet the game developers in exclusive interviews.
* No pre-registration
* No pre-formed teams, we'll assign you to a team of 4 after you log in
* First come, first served

https://redd.it/k71ycl
@r_devops

macOS taskbar plugin to monitor cloud costs

I am curious how do you monitor your cloud costs. Despite having credits, I have fear that I may exhaust it. AWS, GCP sends email invoices at the month end. Budgets and alarms is a solution. AWS has cost explorer API, but has limited information. GCP does not have a billing API.

How would you feel about a macOS plugin that displays your cloud expenditure and remaining credits in your taskbar. You can also customize it to show costs based on tags, resource type (EC2, S3, GKE). You can add any number of your cloud accounts from any cloud provider.

The user has to just enable billing exports and they see their cloud cost from menubar. Also, programmatic access to fetch their cost data so that they can integrate it with Zapier, IFTTT, webhooks.

https://redd.it/k6rnho
@r_devops

RHCSA 8 EX2OO

Hello guys,
Am preparing to take my RHCSA 8 ex200 by the end of this month,but am having difficulties getting a recent practice questions, please is there any recommendations,i just wanna practice the recent questions.. any help will be grateful appreciated..
Thanks

https://redd.it/k6rxjt
@r_devops

DevOps last mile: automating edge cases.

Companies chasing DevOps have an internal team dedicated to automation. Developers use the tools created by this team to run and operate their code. They have many names, but let's call them Platform, which most companies do. Platform teams create abstractions on top of the infrastructure. The goal is to increase developer speed and make systems reliable. They increase speed by simplifying infrastructure APIs and reliability by automating manual tasks. But a curious thing always happens. One type of task that exists from the start of the company is always left behind in the automation backlog.

**Why automate?**

We used to ship software by accessing servers and running commands inside boxes to pull new code. Now code goes from Git to servers without human intervention. Developers define what they need with code. Platform teams build the tools to make code changes become running systems. The goal is doing this for everything, from the business code to infrastructure. Networking, databases, queues, etc. But this is hard. Platform teams have a big backlog, they prioritize items demanded with higher frequency.

Developers make manual changes for things not yet automated. Some companies have compliance, regulations, and other constraints. This makes it hard for developers to get direct access to production. So they have the Platform team running these changes for them. This is what makes the higher frequency items get priority. Engineers want to write software, not run repetitive manual tasks. But this backlog is never decreasing. The business changes and adopts new technologies. Companies create new units and teams. Headcount grows, adding new items to the automation backlog. And this mysterious type of task is always left behind.

**One-offs.**

Sometimes a bug in software messes with a customers' money, time, health, or ego. They won't wait for three iterations of code reviews, tests, code analysis, and gradual rollouts. This takes time. Someone will access the database and update it. These are one-off scripts. They solve a problem for one or a few customers before the team creates a definitive fix.

One-off scripts have a bad reputation. When this happens too much, it's a sign that the software is not stable. In the ideal world, it would never happen. Engineers would spot such time-critical problems during design and review phases. Production issues should be light and wait for regular software delivery flow.

Almost every company lives under the illusion that one-offs should not exist. Or that they will stop happening at some point. Yes, one should not do this every day. But having a few senior engineers run manual scripts in production because it's an exceptional case is a mistake.

**This is a myth.**

One-offs won't go away, and companies need to embrace it. Avoiding them will drive the company to a bad path. Either centralizing execution with experienced engineers or creating a team dedicated to analyzing and running them. This isn't good.

Some companies solve this problem with a slow and manual Change Management workflow. Developers find the problem and add a script to a ticketing system. Someone from the operations team runs it without all the context of what she is doing. Avoiding one-offs is the shortest path to this model.

**It's hard**

One-offs are the hardest piece to automate. When you don't know what problems can happen, it's hard to build a solution upfront. Few companies had the courage to 1) embrace one-offs, and 2) try to automate them. We did this for one of the companies I worked at, and it was a big success. Developers were happy with the autonomy to build and run all solutions to their problems. Security and compliance were happier with audit logs. SREs were happy with fewer manual interventions in production. It was hard but paid off.

I enjoyed the solution so much that I decided to bring it to other teams. I created a company; it's called [RunOps](https://runops.io). We help teams automate one-off scripts within minutes. We see fantastic results with

the first few companies using it. It's early days; our landing page is not clear on how we do it, so feel free to reach out on [Twitter](https://twitter.com/andriosrobert) or here to learn mode.

https://redd.it/k6plye
@r_devops

How many DevOps Certifications are there right now?

Since DevOps is a widespread ideology, there is not a single inventor or authority to examine your caliber. There are various DevOps certifications in the market (more of which are based on its tools), but out of them, only a few highly recognized in the talent market are:

### A. Kubernetes Certifications

This exam tests aspirant’s knowledge & expertise on general Kubernetes features & is conducted in two types as:

* Certified Kubernetes Administrator (CKA)
* Certified Kubernetes Application Developer (CKAD) programs

### B. Docker Certified Associate

The exam aims at testing one’s experience of working with the Docker tool in an IT infrastructure.

### C. Puppet professional Certificate

The exam is named “*Puppet 206 – System Administration Using Puppet*” which approves your ability to operate system infrastructure using the Puppet tool.

### D. Microsoft Certified: Azure Administrator Associate

Examines your skills and expertise in maintaining storage, networking, and securing resources over Microsoft Azure.

### E. AWS Certified DevOps Engineer Professional Exam

The exam tests your technical skills and knowledge to provision, operate & manage distributed apps & systems across AWS.

### 7. Certified Jenkin Engineer

The exam tests your knowledge around the Jenkin tool’s use for building robust CI/CD pipelines.

https://redd.it/k6hcs7
@r_devops

Please help me decide between a DevOps or Microsoft 365 career path

Currently I'm working in a help desk position for a large gov. org. I have basic programming knowledge which I haven't been able to use in this role. I've impressed my bosses by showing potential and working hard. I've been able to shadow a couple BAs as well.

Right now we are introducing Microsoft 365 and no one on our team is an expert in that area. My manager has told me would would pay for my training for 365 so that I can be his guy in this area.
I assume a new role would be made for me with better pay eventually as well.

I was drawn to get into dev ops, because I thought there is more opportunities in that field, and the pay is much better.

I feel a little paralyzed because I can't make a decision. If I get into 365, is that a complete tangent to getting into Azure DevOps? Should I do both? Should I do 365? I'm just terrified I'll focus on too much and end up being stuck in help desk.

Any advice would be much appreciated.

https://redd.it/k6euf3
@r_devops

Exactly an year back, I was new to PagerDuty / DevOps and was suffering from anxiety in the new job. This group helped me a lot. Thank you my fellow engineers!

[Original post.](https://www.reddit.com/r/devops/comments/dj15cy/how_to_deal_with_oncall_pagerduty_anxiety/)

In fact after 1 year, what /u/lorarc [said has come true](https://www.reddit.com/r/devops/comments/dj15cy/how_to_deal_with_oncall_pagerduty_anxiety/f412gk1/). I am sitting here on a weekend (I am on-call this week), eagerly waiting for a PD alert! :-)

I would like to give my heart felt gratitude to this sub.

Thank you!

https://redd.it/k765aw
@r_devops

Build/Deployment guidance

I'm looking for some guidance from the devops community here.

I recently completed the MVP design of an architecture for my organization's backend data collection, processing and persistence pipelines, only we don't have our CI/CD strategy finalized. We're currently on BitBucket/Bamboo and are likely going to be making a change and it's looking like GitLab may be what we move to.

The architecture is multi-stage (dev, test, qa, prod), multi-layered (data collection, enrichment/processing, persistence) and multi-region with some layers being in more regions than others. Aside from the complexity brought on by being multi-regional and multi-layered, it's a pretty neat, clean and simplistic architecture, that provides HA and somewhat easy regional failover, however the deployment is a bit intricate and of course because each region and layers needs to know about the others, each stack has dependencies on outputs from other stacks, in other regions.

My goal is to allow the developers to create their applications and be able to plug into the architecture. So, I'm using an EventBridge in each region/layer/stage that the developers can easily create a rule for the EventBridge to route data to their component.

Problem is, I haven't solved for the dependencies that the applications and their deployments will have on the infrastructure. I automated the infrastructure deployment using CloudFormation, and the developers typically use Serverless, so theoretically I can have all of the stacks export everything and let the developers just import those values everywhere they need them, however:

\- That creates a pretty tight coupling that I've never liked and Cloudformation is known for getting in these weird states that can be hard to get out of.

\- I can foresee developers wanting to depend on resources in other regions and Fn::ImportValue doesn't allow that.

\- Some of those values (i.e. "touch points") are needed at deployment time and others would be more valuable being obtained at runtime (Depending on them at runtime would allow human intervention in the event the whole thing goes up in flames, change the values and let all the resources "autodiscover")

I had a vision in my head that these touch-points (resource ARN's, hostnames, etc.) would reside in some key/value store that would be maintained in/by the deployment environment. When something gets deployed, its outputs would get stored in this key/value store. Even better, if the data store were backed by something like a DynamoDB global table, the values could be depended on at both runtime and/or deployment time.

Am I off base in thinking along those lines? Does anything like that exist? I know GitLab has environment variables, but I haven't gotten a sense of whether they solve for what I'm thinking here. Should I just use Cloudformation Fn::ImportValue and cross the bridge of the above issues when I come to it?

Any guidance, pointer in the right direction, etc. would be much appreciated.

Thanks!

https://redd.it/k6ep0e
@r_devops

How do you interact with the API of internal services from CI systems?

I’m mostly thinking of the use case where you have something like Hashicorp Vault or Kubernetes running on EC2 in a private subnet. Right now, I have a bash script baked into an AMI that I trigger with the Systems Manager Run Command from CI. This bash script uses cli commands to configure internal services. Our infra is still in its infancy and I keep thinking “there has to be a better way!”. My main ideas were:
- switching to a self hosted CI solution, running the CI instance in the same VPC as internal services
- exposing the API of internal services to the internet, protect them with basic auth

https://redd.it/k78xfq
@r_devops

Any interest in sla management?

I used to work for a place that basically managed a bunch of programs between hospitals and hospital vendors. Followed up on contracts, found deals where the hospital could get free drugs for certain patients, etc. You're wondering where I'm going with this...

Before Thanksgiving when AWS went down it got me thinking. I saw a lot of people being very vocal about how it can be a pain to get credits and money back for failure to meet SLAs. I've had to do it myself, but usually I just felt like "that's why they pay me".

Is there any interest in a 3rd party managing your SLAs with your vendors?

I've got no idea exactly what *this* is right now, but I'd love to get some people's feedback on the idea, and if you're interested, [check out my website and sign up](https://slaslayer.com).

https://redd.it/k6co5d
@r_devops

Finding the IP/web host behind Cloudflare

Guys, I am new to Devops and I keep getting spammed by this web design company on my website. Constantly.

I tried to look up their host so I could complaint to them but they are behind cloud flare. Is there anyway I can find their actual IP or host?

https://redd.it/k7fquz
@r_devops

Development Project Overview

We run quite a few projects, and all of them are updating their information pages under confluence which.... well would like to say works but that is only for the ones that are high profile and in production . Im wondering if there is a tool that actually lets you browse the progress of project in DevOps terms (so not traditional Project Management). What I am after is something that lets me have a dashboard with pipeline data, service connections and uptime in sites and environments, resource utilization, and centralized linking to project information. I did see something that I base this kind of information from a DevOps consultancy firm (mockup? - dont know).


Does anything like this exist? Is it something that the ones have it build themselves??

https://redd.it/k783ps
@r_devops

Product preference between datadog, splunk, and sumo? And why?

21 y/o college kid here, and I am looking to spend my winter break learning about devops. Title says it all. Would welcome any sort of perspective from people who are knowledgeable/work directly with these platforms. Thanks!

https://redd.it/k7i2cu
@r_devops

Career Advice! Fullstack Engineer transition to DevOps Engineer

I was wondering if it anyone here has made the transition from a Full Stack Engineers to a DevOps Engineer. And if so, what was the transition like and what did you do to become a DevOps Engineer?

Currently I am a Full Stack Engineer with a solid understanding of Docker, and a pretty solid understanding of AWS from my professional experiences. I found these kinds of things more interesting than application development, and I'm seriously considering on making a pivot in my career.

https://redd.it/k717xi
@r_devops

HashiCorp Certified: Terraform Associate exam issue.

Hello everyone. Today I tried to pass my Terraform exam on PSI platform but face issue. After verification photos and 360 I got a message in secure browser that I must wait up to 15 minutes until proctor verified my identities. 10 minutes passed, 20 passed. I got message proctor not joined please restart your session. I did that. Again 5 minutes passed, 10 passed I wrote to chat: Hello Is anybody here? And suddenly proctor answered I'm here. I continue waiting because I thought that now everything will be fine but after 10 minutes of waiting I'm again got message that I should restart browser and bla bla bla. Same happened 2 times more and I decide not to waste my time. More than hour passed since appointment time.

Is anybody faced same issue. If yes how did you get free retake or refund? PSI live support always busy and never reply. Maybe anybody knows any Hashicorp or PSI e-mail addresses which could be useful to resolve this issue?

https://redd.it/k7jkk2
@r_devops

How to prepare for Kubernetes certification (CKA)?

I am learning k8s and have no real-world experience with it outside my homelab. I'd like to get certified with CKA. I'm currently halfway through Mumshad Mannambeth's CKA course with some nice labs in Kodekloud, but I'm wondering if that's enough? What else would you recommend? thanks!

https://redd.it/k7351v
@r_devops

What is Everyone Using Instead of Puppet Pipelines

We've been using Puppet Pipelines extensively in my company. Our application is hosted on a mix of Windows/Linux VMs in a co-located DC. We use Pipelines to do run through all the steps, adjust the load balancer, turn off services, sync files from artifactory, turn it all back on, etc.

When they announced it was going away we started looking at other tools, but many seem dedicated to the cloud, AWS, hyper-converged VM stuff, etc. Has anyone else found a replacement? It's interesting that Puppet didn't even attempt to recommend what customers try to migrate to.

https://redd.it/k7ly9s
@r_devops

Come join our Kubernetes focused discord server!

I'm posting this again as I've made some changes to the server. Discord offers a lot of things the Slack workspace doesn't such as:

• A channel just for Kubernetes memes.
• Voice chats you can cry about production issues in
• Less professional so we can take out our frustrations on each other!
• Music rooms
• Doesn't do that annoying aesthetic shit with hyperlinks

https://discord.gg/zSSRFB3G3h

https://redd.it/k6zt1u
@r_devops

K8s volumes

https://medium.com/swlh/k8s-volumes-claims-part1-138012b6f52

https://redd.it/k7p0vk
@r_devops