From @catops_chat

Happy Friday!

P.S. Facebook rebrended as Meta

Пару дней назад вышел новый Technology Radar от ThoughtWorks (https://a.e42.link/j1qLb).

На этот раз много пунктов относится к инфраструктуре и командообразованию, также немалое количество пунктов про удаленную работу:
— 4 ключевые метрики DORA перешли в Adopt и рекомендуются для применения всеми. Если у вас нет дашборда для их отслеживания можно периодически раз в квартал проходить DORA quick check: https://a.e42.link/j1qWj
— Платформенные команды также перешли в Adopt и рекомендуются как хороший подход. Важно отметить, что платформенная команда это не переименованные operations, а команда разработки точно такая же как и любые другие команды разработки — со своим product owner, продуктовым планированием, разбиением на фичи, работой с бэклогом, и т.д. Одним словом, платформенная команда — это команда разработки, которая пишет продукт для использования внутри компании другими командами
— Учитывание когнитивной нагрузки команд в проектировании архитектуры. Про это уже говорилось в предыдущем радаре и в книге https://a.e42.link/j1qW8 — кто еще не знаком с концепцией и подходом очень рекомендуем ознакомиться
— Remote mob-programming. Это как парное программирование, только больше чем вдвоем и не в одной комнате у доски, а удаленное. Парное программирование мы применяли с отличными результатами как раз через Zoom — оно хорошо подходит для случая когда не совсем ясно как именно и что писать, гораздо лучше чем параллельная работа с синками через каждые 2-3 часа.
— В блоке Assess появилось использование Kubernetes Operator для управления ресурсами за пределами Kubernetes. В предыдущих радарах уже упоминались инструменты для этого, теперь на радаре появилась и сама практика. Также в этом радаре появился и Crossplane (https://a.e42.link/j1qWT)
— В блоке Trial по-прежнему находится https://a.e42.link/j1qWz (инструмент для построения внутренних технологических порталов и витрин),
— Также в этом же блоке появились Clickhouse, Kafka REST Proxy, Kafka Mirrormaker 2.0, OPA Gatekeeper for Kubernetes и Sealed Secrets
— Из Assess в Trial поднялись GitHub Actions, K3s и Pulumi
— Написание скриптов командной строки на Clojure: Babashka (https://a.e42.link/j1qWY) — за счет использования GraalVM обещают, что он стартует мгновенно, а не как другие JVM-приложения
— ExternalDNS для синхронизации ингрессов с DNS-провайдерами появился в Assess
— Batect (https://a.e42.link/j1qWt) как способ настройки окружений локальных и тестовых
— Berglas (https://a.e42.link/j1qWm) для управления секретами в GCP
— Dive (https://a.e42.link/j1qWZ) — сканнер оптимальности сборки докер-образов. Может отслеживать неэффективность послойной сборки и вычислять «лишний» объем образа (например файлы создаются в нижнем слое, а затем удаляются в верхнем слое)
— Lens (https://a.e42.link/j1qWp) как UI для Kubernetes перешел в Trial
— cert-manager (https://a.e42.link/j1qWl) наконец-то появился на радаре
— Появились аж 2 инструмента для тестирования инфракода: Conftest (https://a.e42.link/j1qWB) и Regula (https://a.e42.link/j1qWn). Оба используют язык Open Policy Agent для написания тестов. Такие тесты могут использоваться, например, для автоматизированного тестирования Compliance
— Появился Cosign (https://a.e42.link/j1qWG) — инструмент для подписи и проверки подписи контейнеров
— Забавно, но в этом радаре появились и современные альтернативы командам из Coreutils (под именем Modern Unix commands) наподобие ripgrep, ag, jq, httpie. Большой список таких команд можно посмотреть на https://a.e42.link/j1qWx
— Mozilla Sops (https://a.e42.link/j1qWf) для безопасного хранения шифрованных секретов в гит-репозиториях (с расшифровкой например через AWS KMS)
— Pactflow (https://a.e42.link/j1qWC) — инструмент для тестирования контрактов
— Proxyman (https://a.e42.link/j1qWk) — прокси для отладки веб-приложений
— Telepresence (https://a.e42.link/j1qWe) — инструмент для подключения локально запущенного приложения к удаленному кластеру кубернетес. Может пригодиться например для песочниц разработки

There was a question in CatOps chat regarding the resources to learn the Go programming language from scratch with the background in other technologies.

So, here is a quick ad-hoc list of resources, we came up with:

Books:
- Go in Practice
- The Go Programming Language

Courses and tutorials:
- Practical Go Lessons
- Algorythms with Go
- Go by Example
- Effective Go

Blogs:
- Three Dots Labs

Of course, you can also find a great list of learning materials in the Awesome Go list

If you would like to add to this short list - welcome in the comments!

#programming #go

1Password promise two years free membership for their service for Open Source software maintenaners and their teams.

However, to get access you have to be a project lead or a core contributor for an active open source project that is at least 30 days old. We’ll also accept applications from the organisers of community meetups and events, as well as some conferences.

Open source projects need to use a standard open source license and must be non-commercial. Your project should not have paid support or pay contributors.

More details on the linked page.

#security #free_stuff

TIL:

AWS has much higher environmental impact comparing to Google Cloud and Azure.

Would it drive decisions on what cloud to chose? I don’t know. Probably not.

However, if you’re looking for a cloud provider for your pet project or a startup, this is something to consider.

#cloud #aws #gcp #azure

DevOps-ish community loves Go.

So, here's a short story about simple re-arranging fields in a struct that saved 1/3 of memory consumption.

#programming #go #performance

From time to time people bring the topic of testing during our voice chests.

Here Kent Beck, a creator of extreme programming, provides 11 properties for good tests:

- Isolated — tests should return the same results regardless of the order in which they are run.
- Composable — if tests are isolated, then I can run 1 or 10 or 100 or 1,000,000 and get the same results.
- Fast — tests should run quickly.
- Inspiring — passing the tests should inspire confidence
- Writable — tests should be cheap to write relative to the cost of the code being tested.
- Readable — tests should be comprehensible for reader, invoking the motivation for writing this particular test.
- Behavioral — tests should be sensitive to changes in the behavior of the code under test. If the behavior changes, the test result should change.
- Structure-insensitive — tests should not change their result if the structure of the code changes.
- Automated — tests should run without human intervention.
- Specific — if a test fails, the cause of the failure should be obvious.
- Deterministic — if nothing changes, the test result shouldn’t change.
- Predictive — if the tests all pass, then the code under test should be suitable for production.

#programming #testing

Some people use to think that a job interview is a one-way road. Like an exam: there’s a person or two who ask questions to evaluate your skills and you should excel in answering everything.

While in reality interviews are bi-directional process. It is important for a company to hire matching talent, but it’s not less important for an individual to look for a matching company.

Here Gergely Orosz, an author of The Pragmatic Engineer Blog, proposes a test to evaluate the engineering culture in a team (eng culture may be different from team to ten in the large companies). This tests consists of 12 questions, which you can ask at any stage of the interviewing process.

As a bonus, you can evaluate you current company as well!

#culture #hiring

I have already recommended CUE (or cuelang) in a few chats to validate Kubernetes manifests.

However, the language itself is capable of many more things than just validate some configuration files.

For example, you can write your configuration in CUE as well!

Here is a blog post, which describes the basic concepts of this language as well as some real world use cases.

#cue #kubernetes

​​Some useful resources for Kubernetes CKA exam preparation:
- Kubectl Cheat Sheet for Kubernetes Admins & CKA Exam Prep
- Useful bookmarks. You are allowed to use those
- Killer.sh - CKS, CKA, CKAD simulator

Good luck to those, who is looking forward to complete one of those exams!

#kubernetes #education

Upcoming features of Go 1.18

Yes, generics, but not only.

Also, here's the draft for release notes, but it's still in progress.

#go #programming

Nice overview of policy enforcement tools for Kubernetes by Viktor Farcic. Gatekeeper (OPA implementation for K8s) vs Kyverno specifically.

tl;dr: Kyverno won this one.

However, I would like to make a few personal additions in favor of Gatekeeper. The first important point - and Viktor mentions it as well - is that you can use OPA to enforce policies outside Kubernetes. For example, you can write policy checks for your Terraform code and why not. Also, you can use something like Conftest to check your resources even before they are applied in a cluster!

Another important thing I want to point out is that Rego is a real programming language, even though it‘s not the most obvious one. You can write tests for your constraint templates, which is very powerful in terms of keeping your policies in a good shape. With Kyverno you have YAML, which is easier, but you need to validate YAML somehow. With Rego you get tests out of the box. Here is a good article that helped me write tests for Rego back in a day.

#kubernetes #security #kyverno #opa #gatekeeper #rego

I have two recommendations of books bundles today. I was thinking if it makes sense to combine them into one message, but decided to push them separately.

So, the first one is a bundle of Python books by O'Reilly:
- Web Scrapping with Python
- Test Driven Development with Python
- Using Asyncio in Python
- High Performance Python
- Introducing Python
- Think Python
- Hands-On Unsupervised Learning Using Python
- Python Data Science Handbook
- Thoughtful Machine Learning with Python
- Flask Web Development
- Machine Learning Pocket Reference
- Hitchhiker's Guide to Python
- Elegant SciPy
- NLP with Python

As usual, you can pay at least €15.55 to unlock all of these books or pay less to unlock some of them. There's no upper limit, though. You can pay whatever you want and Humble Bundle will redirect your funds to charity.

# books #python #programming

The next book bundle is about security.

- Microsoft Azure Security and Privacy Concepts
- Hack Yourself First: How to go on the Cyber-Offense
- Security in the Cloud
- Security Compliance: The Big Picture
- Security for Hackers and Developers: Overview
- Threats, Attacks, and Vulnerabilities for CompTIA Security+
- Incident Detection and Investigation with QRadar
- AWS Cloud Security Best Practices
- Microsoft 365 Security: Threat Protection Implementation and Management
- Cisco CyberOps: Security Monitoring
- Cloud Security: Introduction to Certified Cloud Security Professional (CCSP(r))
- Linux Host Security
- Operationalizing Cyber Threat Intel: Pivoting & Hunting
- Security Awareness: Basic Concepts and Terminology
- Splunk Enterprise Security: Big Picture
- Threat Intelligence: Cyber Threats and Kill Chain Methodology
- Cyber Security Essentials: Your Role in Protecting the Company
- Security Management: A Case Study
- Security Awareness: Phishing - How Hackers Get Your Secrets
- Cyber Security Careers for IT Professionals

As usual, you can pay what you want. Minimum payment of €21.59 will unlock all 20 books.

#books #security

An article about some attack vectors for your AWS environments and ways to protect yourself against them.

There's a neat tl;dr section on top of the article, so you can have a quick overview before diving deep into the article itself.

#aws #security

From our subscribers.

Application Delivery Technical Advisory Group of CNCF released the v1.0.0 of GitOps specification.

You can find the specification itself on the GitHub.

Basically, a GitOps system should comply with 4 main principles:
1. Declarative: A system managed by GitOps must have its desired state expressed declaratively.
2. Versioned and Immutable: Desired state is stored in a way that enforces immutability, versioning and retains a complete version history.
3. Pulled Automatically: Software agents automatically pull the desired state declarations from the source. Agents within the system pull the desired state from the repository.
4. Continuously Reconciled: Software agents continuously observe the actual system state and attempt to apply the desired state.

You could kinda deduce these principles already, but now they’re formalized. Besides, you can adopt these principles and, well, GitOps not only for your services, but for IaC as well.

There are still open questions, for example, how to handle incidents in the immutable environment. However, I like the overall direction. Specifically the point that even though we switched to “cattle” servers from the “pet” ones, we still trat environments as “pets” and we need to stop that.

I see that demand for running dynamic environments increasing across the industry. So, this is definitely a valid point and an interesting area to explore.

#gitops #cicd #culture

Our tech stack differs from one company to another. However, there are certain things that almost everybody use. Like, for example, Git!

Here are some release notes for Git 2.34

This release introduces the use of sparse index for some of git commands.

You can read more about sparse checkout and sparse index here.

This is especially useful for monorepo users. Although, I haven't being working with one for more than 2 years now, I have some repos in mind, where I would like to test it.

As a bonus: An article about Git's data structures and their behavior. Commits are not diffs, folks!

#git

Ship / Show / Ask - A modern branching strategy

It's a branching strategy that combines the features of Pull Requests with the ability to keep shipping changes.

Changes are categorized as either:

- Ship (merge into mainline without review)
- Show (open a pull request for review, but merge into mainline immediately)
- Ask (open a pull request for discussion before merging)

From CatOps Chat

#github

Discount on courses + free certification on CKA, CKS etc + swag

https://training.linuxfoundation.org/cyber-monday-2021/

#education #courses

Our friends from Cossack Labs have released a new version of their Acra tool.

Acra is a database security suite for data protection. It provides application-level encryption for data fields, multi-layered access control, database leakage prevention, and intrusion detection capabilities in one suite. Acra was specifically designed for distributed apps (web, server-side and mobile) that store data in one or many databases. Basically, you can encrypt individual fields completely transparent for an application!

So, what's special about this release? A lot of features that previously were available only in the enterprise version now made their way to open source! Among them: database encryption, searchable encryption, and encryption-as-a-service API.

Apart from that, Acra allows to tokenize certain fields in your database to achieve anonymization. This is actually a cool feature! In one of my former companies we had to create our own tool to achieve that. Here you get it as a part of the package.

#security #databases #toolz

On our last voice chat we briefly discussed Kubernetes autoscaling and mentioned Karpenter - a cluster autoscaler backed by AWS.

This tool isn’t new. However, but AWS started to promote it recently. So, it’s probably “production ready enough” from their judgment. Also, it looks like Karpenter can work with spot instances, which makes it a super-interesting tool to follow.

You can read more about it in the AWS blog post.

If you are already using it or you have tried it, feel free to share your opinions in our chat!

#kubernetes #scaling #toolz