Multi-channel notifications

I need something that can send notifications and alerts to a series of devices based on escalation rules. For example, first message is a push notification (to Pushover or similar) and if that doesn't get a response of some sort then it would escalate to another channel, perhaps including Slack or Google Chat, SMS, telephone calls, etc.

I know that PagerDuty does exactly this and it's what I'm currently using. However, it's quite expensive for my use case, but worse it's overly complex. I don't need incident management, ultimately I just want a simple means of delivering a short message to one or more people with an escalation until it gets a confirmation. Surely this isn't an uncommon requirement? What do others do?

https://redd.it/k1hkd5
@r_devops

Trying to understand a new position

I have an interview with a Fortune 500 and I'm trying to understand what they're actually looking for. They're asking for a cloud engineer on paper, but are actually looking for a a solid network engineer who has automation experience. This is a big box retailer, they're using Google Cloud and AWS as far as I can tell. However, their main focus in my prelim interview was on BGP, EIGRP, and Ansible.

They did specify they wanted someone with BGP peering experience, specifically in GPC. Based on what limited info I have, does this position sound similar to what anyone else is doing? Is anyone working with similar technologies who can speak to the day to day work?

Thanks!

https://redd.it/k1h7o3
@r_devops

is: njrat, nuclear-rat and xtremerat a industry wide program?

I dont know any thing about the programs in the title, but when i was looking on shodan for sites i noticed quite a lot of sites had these programs.

https://redd.it/k1h387
@r_devops

Need some help understanding Terraform lifecycle rules

I'll try to give some background, we're deploying a Grafana instance via Terraform and GitLab CI/CD Pipelines.


The first time the pipeline runs the instance loads perfectly and we can access the grafana UI in a web browser. HOWEVER, if we then re-run the pipeline with changes, we will get a HTTP 500 error when trying to hit the grafana UI in a web browser again, every 'even' number run (2, 4, 6, 8, etc.) will cause this issue but the 'odd' number runs work fine.

​

I've found the fix to be to add 'ignore\_changes' block to the ASG, ignore changes to the loab\_balancers and target\_group\_arns - as is recommended by Terraform ([https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling\_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group))


However I'm struggling to understand what the implications of this change actually are, why does this fix the issue? I've had a Google to try and find some explanation but I can't say I understand any that I've read.


Could anyone help explain what adding these lifecycle rules to the ASG actually do?

https://redd.it/k1ctjg
@r_devops

Automate NGINX Unit with Ansible

Almost half a year ago, we found ourselves migrating from uWSGI to the NGINX Unit application server. And because there was no Ansible content available at the time to help us automate deployment, we had to write our own.

Our initial plan was to create a minimalistic Ansible Collection that only contained an installation role. But we had to change our course of action when we realized there is no such thing as a NGINX Unit configuration file. Therefore, we created a new Ansible Collection that contains a few modules for managing different parts of the NGINX Unit configuration. Luckily, we are experts in developing Ansible Collections, so this was a no-brainer for us ;)

And the best part? You can use it too since we decided to release it as an Open Source project. You can find it on [Ansible Galaxy](https://galaxy.ansible.com/steampunk/unit/). Or go read our [quickstart documentation](https://docs.steampunk.si/unit/quickstart.html) that contains everything you need to know to get up and running in just a few minutes.

We wrote a [blog post](https://steampunk.si/blog/why-and-how-of-the-nginx-unit-ansible-collection/?utm_source=Reddit&utm_medium=organic) that contains even more information about our development process. So if you are interested in how the Ansible Collection development process looks like, this might be just the thing for you.

https://redd.it/k1fpk7
@r_devops

ConnecTech - A podcast on infrastructure and technologies being used at various companies and startups

Hi Guys,

We started a podcast called **ConnecTech** where we invite brilliant developers to come and share how they solved specific problems or built infrastructure at their companies.

Spotify([https://open.spotify.com/show/0QkZAtQgg5yU0hKYGDYh8L](https://open.spotify.com/show/0QkZAtQgg5yU0hKYGDYh8L))

Pocket Cast([https://pca.st/g4qk0m1t](https://pca.st/g4qk0m1t))

Google Podcast([https://www.google.com/podcasts?feed=aHR0cHM6Ly9hbmNob3IuZm0vcy8zZDY1M2ZhMC9wb2RjYXN0L3Jzcw==](https://www.google.com/podcasts?feed=aHR0cHM6Ly9hbmNob3IuZm0vcy8zZDY1M2ZhMC9wb2RjYXN0L3Jzcw==))

We are a couple of episodes in now and we are looking for feedback from fellow Redditors.

Hopefully, the quality of audio and content will keep on increasing in the upcoming episodes.

https://redd.it/k1bfgp
@r_devops

Configure Windows registry and services during image creation using Packer and Ansible?

Hi guys,

Just trying to get some feedback on this request from a client who are starting to move Windows workloads to AWS.

We're in the process of creating a custom Windows AMI to be used as a base for all Windows builds in AWS, using Packer and Ansible. Has anyone come across configuring the Windows registry and services during image creation, with Ansible as the provisioner? They want the same configuration to match that of the on-prem golden image?

I would have thought that best handled by GPO, but really not sure?

Any help would be appreciated.

Thanks

https://redd.it/k1civ1
@r_devops

What's GitOps?

Hi all,

I tried breaking down what's GitOps, so here's how it turned out:

**GitOps** offers a way to automate and manage infrastructure. It does this by using the same DevOps best practices that many teams already use, such as version control, code review, and CI/CD pipelines.

Companies have been adopting DevOps because of its great potential to improve productivity and software quality. Along the way, we’ve found ways to automate the software development lifecycle. But when it comes to infrastructure setup and deployments, it’s still mostly a manual process.

With GitOps teams can **automate the infrastructure provisioning process**. This is due to the ability to write your infrastructure as code (IaC) with the use of declaration files. We can store them in a Git repository, exactly as we store application development code.

# How does GitOps work?

The GitOps concept was initially introduced by [Weaveworks](https://www.weave.works/), a Kubernetes management company. So discussions around GitOps are mainly in the context of Kubernetes. The transformation to microservices running in containers brought a need for orchestration platforms. Container-based applications can be complex and difficult for provisioning and management. GitOps helps in simplifying this by applying techniques proven in the DevOps world.

Nowadays the idea has become popular among DevOps enthusiasts, representing an upgraded model of the IaC concept. It revolves around 3 major components:

1. Infrastructure as code
2. Pull requests
3. CI/CD

*Let’s look at them separately.*

**Infrastructure as Code**

IaC is a practice of provisioning and managing infrastructure as **declaration files**, stored as code. By leveraging IaC and version control teams can optimize all operational procedures.

GitOps centers around the **declarative model of IaC**. This is why Kubernetes is a great example of implementation. Declarative means that configuration is more a declaration of an expected state, instead of a set of commands. For example, in Kubernetes, you can define the number of pods desired for a service in the manifest. The system will then take care of itself. No need for an engineer to write an imperative script that should get to the desired pod number.

Any cloud-native software that conforms to the declarative model can be treated as code. We use AWS CloudFormation, which is a declarative tool, to write AWS infrastructure. This means that **we can treat infrastructure itself as code**. Declare the desired state as code. The system applies the changes to achieve that state with automation.

With that said, declarative models are not a must to benefit in GitOps. You can do as well with imperatively defined environments.

**Pull requests**

The main idea behind the GitOps concept is that the version control system is a single source of truth. We use Git as a change management system for our application code. We can also use it for our infrastructure code. So the entire set of declaration files is in a single place where you can collaborate. This enables us to use the key concept of Git – the **pull request** for operational changes.

In an app development workflow, we use one main branch as a release branch. Developers create feature branches from the main branch. Develop a particular feature or story and when done create a pull request to merge it back into the main branch. This same approach is convenient for infrastructure code.

Creating a pull request enables the code to go through a process of code review before we integrate it into another branch of the codebase. Code reviews stop bad code from getting into test or production environments. This is even more important for infrastructure code. Having formal approvals in place via code reviews helps a lot with the auditing and troubleshooting.

**Git organization**

The deployment process in GitOps requires at least two repos: **the application repo and the environment configuration repo.** The first one contains the source code of the app together with its deployment manifests. The

second one contains the desired state of the whole system described using a declarative specification for each environment. You can describe your environments as dev, test, production in a code repository, containing the applications and infrastructure services that can run with a particular version of that environment.

In the case of infrastructure, the main branch can represent an environment. We can implement the changes in the feature branch. Then create a pull request to merge the changes in the main branch. With this, we enable collaboration, while being transparent of who performed which changes. This is also beneficial for issue tracking to the root cause since all changes are commits in Git.

GitOps works with any Git-based system, like GitHub, BitBucket, or GitLab. It is not dependent on any tool or technology.

**CI/CD**

To achieve a full GitOps implementation, you need a CI/CD pipeline. With automated delivery pipelines you can deliver infrastructure changes to designated environments, each time there is a change in the Git repository.

Pipelines are here to connect your Git pull requests to the orchestration system. When you trigger the pipeline with a pull request, the orchestration system executes the task.

There are two possibilities for a GitOps deployment strategy: **Push and Pull Pipelines**. The difference between them is in the way you ensure the deployment environment resembles the desired infrastructure.

**- Push Pipelines**

Many popular CI/CD tools are using this strategy. We store the source code of the application and its deployment manifests in one repository. The build pipeline triggers when a new update happens in the application code. The pipeline builds the container images and pushes the changes to the environment. This strategy brings more flexibility, as it can support any type of infrastructure. The disadvantage is that it gives the CI/CD tool access to write to your environment.

**- Pull Pipelines**

The community considers the pull pipeline approach a more secure practice for GitOps. With this approach, the operator is introduced. The operator is a component between the pipeline and the orchestration tool. It constantly compares the target state in the environment repository with the actual state in the deployed infrastructure. The operator changes the infrastructure to fit the environment repository if it detects any changes. Also, it’s possible to monitor the image registry to identify new versions of images to deploy. This is what makes GitOps so special.

In GitOps the environment updates happen only when there are changes in the environment repository. The system reverts any modifications made if the implemented infrastructure changes in any manner not defined in the environment repository.

For most applications, you’ll probably need more than one environment. GitOps allows you to create **multiple pipelines** that can change the environment repository. You can use separate branches in the environment repository to manage more environments. The operator can react to the change of one branch by deploying to production and react to another branch by deploying to test.

# What are the benefits of GitOps?

**Using DevOps best practices**

Since GitOps is a model focused on the pre-existing best practices of Git workflow, IaC, CI/CD pipelines, immutable servers, tracking, and observability, it represents a more advanced state of Kubernetes’ cloud-native application management. Therefore, the current stack and experience within the company can serve a lot.

**Continuous deployment—simplified**

Continuous deployment means deploying faster and more often. Due to different considerations such as statefulness of systems, downtime resistance, upstream/downstream dependencies, and many other organizational relevant processes and dependencies, proper continuous deployment has been very challenging.

GitOps allows you to do this without having to manage a bunch of tools as everything occurs in the version control system. It provides structure and automation, thanks to the deployment

operator.

This also increases productivity and faster MTTD (Mean-time-to-deployment). Automated continuous deployment ensures that the team can ship 30-100 times more changes every day, increasing average production performance 2-3 times.

**Lower MTTR (Mean-time-to-repair)**

MTTR is one of the key metrics DevOps teams should measure. Even small issues can be very challenging to repair in a microservice architecture. As GitOps keeps all changes in the version control system and the management is automated, it is possible to reduce MTTR significantly. You have a full overview of how the environment has changed and error recovery becomes quite easy.

**Simplified Kubernetes management**

Without getting to know Kubernetes thoroughly, developers can use familiar tools such as Git to handle Kubernetes upgrades and features more easily. Newly-embedded developers will get up to speed easily and be active in days rather than months.

**Improved standardization across the company**

You have transparent end-to-end workflows through the entire enterprise because GitOps has one framework for rendering applications, software, and Kubernetes add-on modifications. Git also fully reproduces your operations activities.

# How to prepare for GitOps?

\- **Establish a stable code review and testing process.** Carefully reviewing code changes can point out some obvious actions, like adding a global variable. It can prevent bad code from being released. You can then submit the validated code through pull requests, allowing no changes to be committed directly by developers. Once the pull request is reviewed and merged, you can trigger the pipeline. This is the first step in maintaining a high standard of code and subsequent stability of the system.

\- **Testing, testing, testing.** Incorporating GitOps means having high-level automation that requires thorough testing of pipeline-released applications. Even though GitOps allows you to roll back relatively easily, releasing good code that has gone through good test cases makes your process more reliable.

\- **Focus on monitoring.** GitOps allows repeatable operating processes, improvements in the traceable system state, rollouts, and rollbacks. Careful monitoring can help you identify and prevent any unintended drift and system changes in configuration. So, before starting with GitOps, review your monitoring skills and strengthen them in a way they can handle this change.

\- **Embrace the culture.** Conventional process constraints with lengthy release times can only hold you back. Living a DevOps culture means leveraging the best strategies that will help a team understand the value of both development and operation actions. At the same time, they have to collaborate together to create an overall stable infrastructure, execute applications more rapidly and smoothly, and manage the system effectively. The lack of DevOps culture will prevent you from enjoying the benefits of GitOps.

# Why GitOps?

GitOps is an extremely good workflow pattern that can help you efficiently handle cloud infrastructure. GitOps can provide an engineering team with numerous advantages, including better coordination, transparency, stability, and durability of the system.

Original article was posted here: [https://microtica.com/gitops-devops-for-infrastructure-automation/](https://microtica.com/gitops-devops-for-infrastructure-automation/)

https://redd.it/k1chb6
@r_devops

Laptop Options

Hey guys I was looking for a budget laptop to get started with developing. Below I will be pasting the links for my top 3 options at the moment and would like your input. I’m also wondering if they are compatible with Linux. Please let me know if you have any other suggestions under $500 as well. Thank you!

[HP Pavilion 14](https://www.staples.com/hp-pavilion-14-ce3065st-14-notebook-intel-i5-8gb-memory-256gb-ssd-windows-10-13q78ua-aba/product_24441411)

[Gateway](https://www.walmart.com/ip/Gateway-14-1-FHD-Ultra-Slim-Notebook-Intel-Core-i5-1035G1-16GB-RAM-256GB-SSD-Tuned-THX-Audio-Fingerprint-Scanner-Webcam-HDMI-Cortana-Windows-10-Home/828696636)

[Lenovo Ideapad 3](https://www.bestbuy.com/site/lenovo-ideapad-3-15-touch-screen-laptop-intel-core-i5-1035g1-12gb-memory-256gb-ssd-platinum-grey/6426712.p?skuId=6426712)

Edited to include better options

https://redd.it/k1bf2o
@r_devops

Botocove: Run Python functions against all AWS accounts in an organization concurrently

I found myself needing to interact with all of the accounts in our AWS organization often and copying chunks of code around to do it - doing it one account at a time was too slow, so I leveraged concurrency with async.io. I packaged it into a decorator that can run a Python function against a list of AWS account you can assume a role in, or every account in your organization!


https://pypi.org/project/botocove/


I use it for things like automation of inspecting all accounts for security and audit checks, or deployment of changes (such as updating IAM password policy, deploying a Cloudformation stack, and so on). By default it's opinonated to use standard AWS Organizations setup and roles, and generate a list of every account within an organization; but everything is configurable.


Any feedback would be great, hope it's helpful!

https://redd.it/k1330q
@r_devops

Destroy Every Version On Production Server - Meaning?

Apologies for the simple question, but I've seen t-shirts with "Destroy Every Version On Production Server" - I know it's not the origin of DevOps but what is it supposed to mean? Thanks!

https://redd.it/k166yy
@r_devops

Does anyone know if Gitlab Self Hosted is free?

I've installed Gitlab on Linux but I'm wondering if it's entirely free or I need to activate some license?

https://redd.it/k1wp1t
@r_devops

DevOps Introduction

DevOps tools are used basically to improve the efficiency of Developers. The DevOps monitoring tools are used to improve the productivity and efficiency of team members by using Collaborative tools. The presence of plenty of DevOps tools makes the choice difficult and therefore we have brought a consolidated list of the most used and popular DevOps monitoring tools. The list includes tenure and emerging tools, which are developed to help the Developers The following list includes the most relevant, reliable, useful, and secure DevOps tools and technologies. To know how they came into existence, you may check [DevOps Tool Wiki](https://en.wikipedia.org/wiki/DevOps_toolchain).

https://redd.it/k1bvrn
@r_devops

Since Atlassian terminates the support for self-hosted solutions, is Jira+Github a good cost-wise replacement for Jira+Bitbucket?

I know that this question is a very big "IT DEPENDS!!!", but I feel like every article I was reading that compares GH with Bitbucket were kind of purchased, or by people who didn't have to integrate it with many 3rd party systems (say CI/CD).

I use Jira and BB with almost zero plugins (I use one to connect it with Jenkins, but it was very light one) mostly for personal use / code backup / advanced kanban board. I try to keep repositories as small as possible so no, I do not push there any vendor, node\_modules, or zip archive with 4k graphic assets there.

I honestly hate this change, all BB was good for is keeping stuff away from organisations that can remove your repo under some pseudo fake DMCA policies. I am very lawful person, but I am also against storing my code away at big comps like Microsoft.

https://redd.it/k14old
@r_devops

What Every Scrum Master Gets Wrong

Have you wondered why doctors’ appointments are always delayed? It turns out there’s a key lesson for Scrum Masters: planning fewer tasks improves efficiency.

[https://www.codemotion.com/magazine/dev-hub/devops-engineer/scrum-master/](https://www.codemotion.com/magazine/dev-hub/devops-engineer/scrum-master/)

https://redd.it/k21xvp
@r_devops

How to install windows app store applications via Scoop?

I'm trying to install windows app store applications via scoop, specifically windows todo. How do I do this? Can I make an app manifest?

https://redd.it/k23rl3
@r_devops

Would you say that developers lack confidence?

Have been talking about it with some friends, and all the people we know in our field are friendly and approachable and successful but aren't particularly confident. Wonder why?

https://redd.it/k25hhn
@r_devops

Job offer, consultancy or not?

I have two job offers to choose from and wondered thoughts since they are very different job roles. I have not worked in a consultancy firm so far in my career so am obviously new to the consultancy game, if I were to take it.

Consultancy firm;

1. Offers a lot more variety, therefore better overall for experience
2. Could be a bit more stressful given the variety which may lead to org's that are so stuck with red tape & politics
3. Gives me more of an 'out' in terms of being less emotionally involved in the projects
4. Travel is limited post COVID but still an important thing, being a single person I don't have any specific reasons to \*not\* travel.
5. Remote mostly, 1-2 days in an office 15 minutes away by car on a route that is not overly busy at the best of times (unless needed on site by a client).

Insurance firm;

1. Single lead devops engineer leading Azure migration from on-premises
2. Interview with CTO that over ran showed me so good qualities - they seem to be eager for change, ready and willing to listen, put down some cash and focus
3. Being the single DevOps engineer = lack of collaboration since there is no 'team' but CTO \*says\* they will be looking to expand the team quickly once I am there and 'settled in'
4. Insurance is still the financial services, so may be a little slower than they say they are at change
5. Remote mostly, 1-2 days in an office 30 minutes away by rail (car traffic very bad as its a main route)

Both roles;

1. Seem to offer job growth in slightly different ways
2. Both gave great feedback in the interview, with zero negative points, and I felt a culture 'click' for both
3. Both have matched salary, benefits are very slightly different but nothing substantially different

So....anything you would think is worthwhile comparing/thinking about?

https://redd.it/k25687
@r_devops

DevOps 200k+! How do organisations calculate the compensation for DevOps engineers?

When hiring DevOps Engineers, how do organisations calculate the compensation and ROI (return of investment)? How are results measured or planned to justify investing in DevOps on a business case budget calculation?

This is a follow up question to [this post](https://www.reddit.com/r/devops/comments/jv7hzr/devops_entry_level_100k_usd/).

https://redd.it/k21h17
@r_devops