Azure DevOps Release and Build YAML Pipelines?

It does not appear that Azure DevOps has any official YAML Pipelines for Releases yet. I know they have multi stage Pipelines where you can put everything in one file. It does make sense to me to have everything in one file, but there are still some missing features.

In the meantime, are people getting by with multi stage pipelines or is anyone splitting it and creating 2 separate YAML files - one for Release and one for Build? I'm noticing one annoying detail in the latter approach.

https://redd.it/f618kc
@r_devops

HashiTalks 2020!

HashiCorp is hosting a free, all-day, 100% online conference this Thursday February 20. Literarily 24 straight hours, 48 half-hour talks, given by community members (no HashiCorp pitches), revolving around HashiCorp tools but touching on other subjects like Kubernetes, cloud security, CI/CD, etc.

I'm not posting the link because I don't know if that can be considered vendor spam, but you can easily find it with a Google search. Or I can post it in a comment if an admin says it's ok :)

Full disclosure: I'm **not** a HashiCorp employee but I am a local HashiCorp Users Group organizer and I will give a talk on Vault and Kubernetes.

https://redd.it/f5v6qq
@r_devops

Starting from zero.

I have the opportunity to take over a deployment that is entirely manual at this point.

Some of it is in the cloud and some of it is on-prem.

At the moment there isn't much code, although the process needs to be started and stopped and paused, what is a wonderful way to do this utilizing an existing tool like gitlab?

The basic process at the moment is data is loaded into a few servers and then the processing is started and may need to be paused at some point and then restart it.

https://redd.it/f63gzg
@r_devops

What is the best way to create a development server after you made a production server?

I've made a mistake of creating a production server without having a dev server.

I created a docker-compose file and then using Jenkins to build and deploy it to the dockerhub.

I've never created a dev environment, so I would like to get some tips on how to do it properly.

Do I just simply have to copy the docker-compose file and create a new Jenkins pipeline that pushes it to a different dockerhub repository?

Any tips would be great. Thanks in advance!

https://redd.it/f60tl9
@r_devops

Brainstorm DevOps Analytics

Hello folks, I am working on a product proposal for creating analytics features on a DevOps platform. Looking for ideas on what key metrics/KPIs to include for the user personas. Also how should I go around building my customer strategy on this? I have looked into Azure and Broadcom offerings but I have been asked to focus more on Oracle Visual studio angle.

https://redd.it/f5vcxz
@r_devops

Getting Started with Istio Using Docker Desktop - from the Docker blog

Cool blog post about setting up istio on Docker Desktop with kubernetes. I tried it, but when I run the setup I get EOF errors on a few of the installs and the kubectl checks after that don’t return what I want. Anyone else have any issues? It was my first time using Docker Desktop, I usually use it on Linux.

https://redd.it/f66jcd
@r_devops

Help configuring haproxy as ingress in kubernetes

Hi,

I'm learning kubernetes and right now I'm trying to configure an ingress with haproxy in kubernetes. I can't get it to work. As it is configured as a NodePort, I was expecting to see the ports being active on the worker nodes where the controller is deployed, but that's not the case. At first, I was having some issues as my worker nodes had IPv6 disabled and the haproxy-ingress pod kepts crashing as it was trying to bind to ::::80 and ::::443. So I've tried to configure the configmap so it only bind to port :80 and :443 (ipv4)...but I just gave up on that and re-enabled IPv6. Now, the pod won't crash anymore, but I still don't see the ports listening in my worker nodes. The logs of haproxy-ingress say this (idk how relevant this is):

2020/02/18 20:44:21 HAProxy Ingress Controller v1.3.2 8fed7bc
2020/02/18 20:44:21 Build from: [email protected]:haproxytech/kubernetes-ingress.git
2020/02/18 20:44:21 Build date: 2020-02-12T15:23:39
2020/02/18 20:44:21 ConfigMap: default/haproxy-configmap
2020/02/18 20:44:21 Ingress class: haproxy
2020/02/18 20:44:21 Publish service:  
2020/02/18 20:44:21 main.go:99: Default backend service: haproxy-controller/ingress-default-backend
2020/02/18 20:44:21 main.go:100: Default ssl certificate: /
2020/02/18 20:44:22 controller.go:97: Running with  HA-Proxy version 2.0.11 2019/12/11 - https://haproxy.org/
2020/02/18 20:44:22 controller.go:102: Starting HAProxy with /etc/haproxy/haproxy.cfg
2020/02/18 20:44:22 controller.go:115: Running on haproxy-ingress-54847454-8jgzx
2020/02/18 20:44:22 controller.go:69: Running on Kubernetes version: v1.15.1 linux/amd64
\[WARNING\] 048/204422 (20) : Can't open server state file '/var/state/haproxy/global': No such file or directory
\[NOTICE\] 048/204422 (21) : New worker #1 (22) forked
2020/02/18 20:44:27 controller.go:369: Confiugring default\_backend ingress-default-backend from ingress DefaultService
2020/02/18 20:44:27 controller-haproxy.go:152: HAProxy reloaded
\[NOTICE\] 048/204427 (29) : New worker #1 (30) forked


Here it is the configuration used to do the deployment (basically, copy/paste from official doc). [https://pastebin.com/ubAM98a7](https://pastebin.com/ubAM98a7)

I'm a little lost right now...maybe I'm expecting something that it is not to be expected yet. Could you please share any ideas of what I may be missing or were to look for it?

Thank you very much!

PS1: Please advice if this subreddit is not accepting this kind of posts, I did not see any community rule saying otherwise.

PS2: I guess my last resort would be to create a new image with the desired configuration as I'm not even able to properly make it work...leave alone using the configmap. At this point, any help is welcome.

https://redd.it/f5yv7x
@r_devops

Azure Active directory Integration with Hasicorp Vault

I am trying to setup Azure AD authentication for vault,
I refered so many document but still I can not figure it out.
Please help If some did this. Thanks in advance

https://redd.it/f5tu28
@r_devops

Blog Post: Running PowerShell Scripts in Azure DevOps Pipelines (1 of 2)

Hey guys, I just wrote a shiny new blog post you may enjoy.

Summary: Understand how PowerShell scripts work in Azure DevOps pipelines and learn how to run them in this informative tutorial.

https://adamtheautomator.com/azure-devops-pipelines-powershell/

https://redd.it/f5sloo
@r_devops

Securely Access AWS Services from Google Kubernetes Engine (GKE)

It is not a rare case when an application running on Google Kubernetes Engine (GKE) needs to access Amazon Web Services (AWS) APIs. Any application has needs. Maybe it needs to run an analytics query on Amazon Redshift, access data stored in Amazon S3 bucket, convert text to speech with Amazon Polly or use any other AWS service. This multi-cloud scenario is common nowadays, as companies are working with multiple cloud providers.

Cross-cloud access introduces a new challenge; how to manage cloud credentials, required to access from one cloud provider to services running in the other. The naive approach, distributing and saving cloud provider secrets is not the most secure approach; distributing long-term credentials to each service, that needs to access AWS services, is challenging to manage and a potential security risk.

In my [blog post](https://blog.doit-intl.com/securely-access-aws-from-gke-dba1c6dbccba), backed by open source project [doitintl/gtoken](https://github.com/doitintl/gtoken), I show how to securely access AWS services from Google Kubernetes Engine (GKE) without storing/sharing long-term AWS credentials.

https://redd.it/f5sdvk
@r_devops

Waiting times and context switches - how do you handle them?

If the developer user experience in a company is good, you can get to work, start working on your daily tasks, and enjoy a blissful flow state for hours while tackling useful tasks at your own pace. If the development environment is well-designed, the iteration times between making a change and getting feedback (through tests or on the screen) are very short and feel satisfying.

Now, compare this to the experience on the infra-side of things. Long waiting times are the norm. The usual workflow is to apply changes, do something else and check for the results every now and then.

More-or-less blocking workflows, big delays, the need to do busy waiting or frequent context switching seem to be widely accepted as a given.

How are you handling those rough, but only infrequently discussed daily obstacles to focused work?

* How do you handle regular waiting times and delays?
* How do you approach multi-tasking and forced context switches?
* Have you put special effort into taming your (AWS, Docker, k8s) workflows and have tips to share?

I'm looking forward to learn more and share fixes, or at least to join in on the venting.

https://redd.it/f5pjzr
@r_devops

How does a consumer receive a message from a message broker?

I have learned from *Data Intensive Applications* book that

> A widely used alternative is to send messages via a message broker (also known as a
message queue), which is essentially a kind of database that is optimized for handling
message streams [13]. It runs as a server, with producers and consumers connecting
to it as clients. Producers write messages to the broker, and consumers receive them
by reading them from the broker.

How do consumers receive messages by reading them from the broker? Does it work in the following steps?

- First, does the broker notify a consumer that there is a new message in the topic they have subscribed to?

- Does the consumer then request the broker for the new message?

- Does the broker then respond to the consumer with the new message?

- Does the consumer then send an acknowledge to the broker for having received the message?

Thanks.

https://redd.it/f5qg31
@r_devops

Announcing Bot Framework SDK for Go

Consequent to a great feedback to [BotKube](https://www.botkube.io/), an application for messengers like Slack and Mattermost to monitor and debug Kubernetes deployments, we have got interests from Microsoft Teams users for an integration. An integration, which would help them monitor their Kubernetes cluster from a Teams client. Since BotKube is architecturally designed to work independently of a messaging platform, it seemed a viable thing to integrate it with Teams.

Microsoft Teams uses the [Microsoft Bot Framework](https://dev.botframework.com/) to allow users to write bot applications. The Bot Framework is a set of tools, SDKs, templates and services to build conversational bot applications. The official SDKs are available for Python, C#.NET, Java and Javascript. Since BotKube is built in Go, there was a need of an interface to the Bot Framework services in Go, for a seamless integration of BotKube to Teams. This encouraged us to write a Go SDK that wraps the services of the Bot Framework.

Today, we are excited to announce the first release of the [Bot Framework SDK for Go](https://github.com/infracloudio/msbotbuilder-go); an SDK written in Go that enables users to build bot applications over the Bot Framework in Golang. The SDK enables application developers to build conversational bots for Microsoft Teams.

[Read the full blog here](https://www.infracloud.io/microsoft-bot-framwework-sdk-for-go/)

https://redd.it/f5pjb1
@r_devops

clientSet for in-cluster and out cluster

This is little old, but I wanted to share this. A `go module` for eliminating the kubernetes client initialization in cloud native app development. Give it a try and let me know your thoughts


[clientSet for in-cluster and out cluster](https://medium.com/swlh/clientset-module-for-in-cluster-and-out-cluster-3f0d80af79ed)

https://redd.it/f5mw3n
@r_devops

Multiple CI/CD on a single server?

Let's assume I have a single server which is heavily overspecced and I have 150 developers with at least one or more projects on gitlab.

I know its probably a very bad idea to use that one single server to handle the CI/CD of all the existing projects, but just for argument sake, how good would it work and what would stop this attempt from being effective? Or in case I'm actually completely wrong about my assumption, why would it actually work out just fine?

https://redd.it/f6bq8v
@r_devops

Mobile CI/CD

Anyone have any experience on building a CI/CD pipeline for mobile apps and can maybe give me some tips/pointers/tutorials?

My team is looking to build/test/release both an iPhone and Android app. Android seems much more straight-forward; download SDK & dependencies, run tests, and upload to Play store (found a gradle plugin that does this). iPhone seems much more difficult since it seems like you need a MacOS machine to compile iPhone code and no major cloud provider offers Mac VMs.

Currently looking at Microsoft AppCenter, which is a bit frustrating since I would prefer to use Gitlab which is where all our other code & pipelines run.

https://redd.it/f5mikn
@r_devops

What will be faster? Setting up my own mail server or mail service like sendgrid?

Above those what will be faster in delivering mails

https://redd.it/f6dii6
@r_devops

Bolstering infrastructure and services

I'd like to start using some of Hashicorps stack like vault, nomad and consul but want to start with Consul. We have existing infrastructure that is in need of some system-level but also application level monitoring. Currently we have Zabbix set up for this, but I really don't like it.


So my initial plan is to deploy consul next to Zabbix and get a cluster running first without any services registered. According to documentation I can monitor the node health on a hardware level and see the telemetry of the entire infrastructure in a non-arcane interface.

From there I'll start start integrating vault into the whole thing and once done will give me a nice demo to convince some old farts that plain text passwords and constantly having to intervene because shit died for days without anyone noticing is in fact, not ok.

Any feedback/suggestions/criticism?

https://redd.it/f6d6oq
@r_devops

Which CI/CD for infrastructure as code

I need to automate infra as code pipelines. Most of it is in Terraform, some of it is in CF, and although I am trying to get everybody to use Terraform for portability we still have some CF users.

Here's the issue: the team is growing a little and I've heard about people getting their TF/CF code under CI/CD. What CI/CD should I use (and why)? For some reason I keep hearing gitlab CI, haven't found out why

https://redd.it/f67nr0
@r_devops

Prometheus alertmanager api issue

the object I send is

{
"matchers": {
"value": "alert_name",
"name": "name",
"isRegex": true,
"startsAt": "utctimehere",
"endsAt": "utctimehere",
"createdBy": "swtrz",
"comment": "Patching silence for machine x"
}
}


alertmanager returns

{"code":602,"message":"comment in body is required"}

Im pretty sure that last line *is* my comment, what did I mess up here?

https://redd.it/f6gb85
@r_devops

Exploring plain bastion hosts, AWS SSM, EC2 Instance Connect, and combining them all with Terraform.

Being newer to networking, I felt pretty lost with all the recent posts about bastion hosts, SSM, and EC2 Instance connect.


I decided I throw one more resource into the pile, with the goal to be different by:
- Having complete terraform examples of the best practices
- Using examples that are closer to production ready security standards than other posts
- Starting from the beginning, and showing how bastion host access has evolved over time to explain why the current best practices are why they are.
- Show how to tunnel database access locally, which requires an explanation of how `ssh`s `ProxyCommand` option works


[Here's the Codelab, enjoy!](https://codelabs.transcend.io/codelabs/aws-ssh-ssm-rds/index.html#0)

https://redd.it/f6gvjs
@r_devops