How to use Azure Git with AWS EC2?

We are a new startup company that are trying to shift our workflow to DevOps. The code and new commits of our CMS application is stored on Azure Git. Whereas we plan to have our servers on AWS EC2 alongside with Terraform, Jenkins, Docker, Kubernetes, ELK stack etc.

We are unsure how to integrate Azure Git to this workflow. Does Jenkins allow us to retrieve the code from Azure Git then continue providing it to the process mentioned above on Amazon EC2.

Should we need to provide more information please let me know.

Sorry if my question is too basic or newbie. Any help would be highly appreciated. Thanks

https://redd.it/f586y5
@r_devops

Received email at work address about a local AWS certificate bootcamp - legit or scammy?

**EDIT: Consensus is that it's scammy. Thanks, everyone!**


Original post:


Subject is: "RE: AWS Solutions Architect Associate Course"

I've never corresponded with this person before, so it's not a real RE.

The email discusses the training course local to my city (a fairly large midwest city) and then says, "For more information and to register visit: thenameofthecompany.us"

It's not a link, they didn't put a comma after register, going to the domain with https results in a firefox "Unable to connect" page - it doesn't exist. Going to it in http redirects to a suuuper scammy sounding domain: https:// www . rebusbusinesssolutions . com/

I'm thinking....this is a scam. Anyone have experience with this domain?

https://redd.it/f5uijs
@r_devops

Automation of deployments, and blue/green

Traditionally, a blue-green deployment was described as a technique that reduces downtime and risk by running two identical production environments called Blue and Green. At any time, only one of the environments is live, with the live environment serving all production traffic. Because there is already another environment “ready to go” it’s possible to switch environments if something happens to the green environment or a new version needs to be released, and can be really handy in a disaster recovery scenario.

As times have changed, containerised technologies are now available, and it is possible to extend the description of blue green deployments to stacks or clusters of a containerised service, rather than just a traditional replica two environment blue green, as long as the tool you use supports some kind of connection draining, e.g. a load balancer.

[https://medium.com/@craiggoddenpayne/automation-of-deployments-and-blue-green-eae80f52494b](https://medium.com/@craiggoddenpayne/automation-of-deployments-and-blue-green-eae80f52494b)

https://redd.it/f56spu
@r_devops

What is the best way to manage a database that requires migrations to be run on it?

My problem: I don't want to deploy a codebase to my production server where my database isn't up to date.

My Goal: I don't want to manually run migrations on the database every time I deploy. It feels error prone. Ideally I have an environment where I can create a staging environment, sync the DB's, run migrations, run end to end tests, and then route traffic to the new version.

What do you think? Where should I look to get started?

Edit: My stack is django/postgresql/Reactjs and I am planning on integrating Kubernetes. Deploying to Google App Engine.

https://redd.it/f4ura3
@r_devops

How to handle integration/functional tests for an WebApi not accessible from pipeline(Azure)

I have been looking for the best way to handle tests for an Web Api. The service is only accessible from certain ip's and therefore I can't run the test (call the service) on the CI/CD agent.

How is this handled in the best way, I want to get the test results back to the CI.

​

I saw an old [post](https://www.reddit.com/r/devops/comments/bnh6fs/running_tests_in_k8s_like_with_dockercompose/) where u/forestjunk said something about kubectl exec to get the test results back, is this possible?

https://redd.it/f568g6
@r_devops

[QUESTIOIN]: About managed servers services

Hey y'all!

So I'm just wondering how do business like Media Temple and such offer managed servers services?

Do they have devops doing all the updates/upgrades/patches etc manually, or are the servers run with a bunch of custom scripts that do all the "managing" via cron?

I would think that having employees in charge of such duties will be super expensive...

Thank you!

https://redd.it/f54aos
@r_devops

Monitoring and APM solutions for Kubernetes


Hey all, I was wondering what your company/team uses for monitoring your kubernetes clusters along with how you monitor the applications running on them? At my organization we previously had the prometheus-operator helm chart and were in the process of getting an EFK stack up but are opting for an enterprise solution that can combine all those metrics into a single pane for consumption. They want something a little less tied to IAC and with a more user friendly process for rules and alerting (dozens of developers will use our monitoring solution). So far we've looked at DataDog, WaveFront and NewRelic, but are leaning towards DataDog as we have some other servers being monitored on there already. I've been shocked at how unfriendly DataDog is with Kubernetes. It seems as though DataDog basically only provides node metrics as if they were just another host and shows container metrics in separate panes. There seems to be little cohesiveness of a "kubernetes environment" in DataDog. Additionally, the default dashboard (note, singular) is literally trash and there's no place for community dashboards. I've found that their APM solutions are promising, but present some configuration complications when it comes to running the agents in kubernetes. Surely lots of you out there have been solving problems like this, and I was just wondering what you guys use, what you like about it and what some of the challenges have been.

https://redd.it/f5z7js
@r_devops

How do you work as a Agile team with DevOps and Microservices - from Feature split you o delivery?

Open for discussion. I’m interested in your way of doing DevOps and Microservices.

Today we have approximately 6 teams , each have a responsibility for a smaller area I.e. product. The competency balance between the team members within the same team is huge and therefore we end up with more “Frontend” developers than “Backend”.

Current way of working with the backlog items (PBI) is to horizontally split each PBI into Frontend and Backend with feature toggling capabilities.

Sure, we are working with Microservices and DevOps - but I would like to hear more about your way of working. How do you split the features into smaller releasable items and how do you work with CI/CD and releases.

https://redd.it/f5vr2x
@r_devops

How to develop your Lambda Functions like a rockstar - our firsthand experience

Hey all - [thought I'd share some learnings and experiences](https://medium.com/@joel.tbarna/how-to-develop-your-lambda-functions-like-a-rockstar-7d9422259d57) we've had getting up-to-speed developing our application with just AWS Lambda. It was pretty slow at first but we've created a pretty solid strategy around locally developing and testing that may be helpful to anyone taking on the challenge of Serverless development.

Let me know if you have any questions! Happy to help where I can.

https://redd.it/f5w5ot
@r_devops

Azure DevOps Release and Build YAML Pipelines?

It does not appear that Azure DevOps has any official YAML Pipelines for Releases yet. I know they have multi stage Pipelines where you can put everything in one file. It does make sense to me to have everything in one file, but there are still some missing features.

In the meantime, are people getting by with multi stage pipelines or is anyone splitting it and creating 2 separate YAML files - one for Release and one for Build? I'm noticing one annoying detail in the latter approach.

https://redd.it/f618kc
@r_devops

HashiTalks 2020!

HashiCorp is hosting a free, all-day, 100% online conference this Thursday February 20. Literarily 24 straight hours, 48 half-hour talks, given by community members (no HashiCorp pitches), revolving around HashiCorp tools but touching on other subjects like Kubernetes, cloud security, CI/CD, etc.

I'm not posting the link because I don't know if that can be considered vendor spam, but you can easily find it with a Google search. Or I can post it in a comment if an admin says it's ok :)

Full disclosure: I'm **not** a HashiCorp employee but I am a local HashiCorp Users Group organizer and I will give a talk on Vault and Kubernetes.

https://redd.it/f5v6qq
@r_devops

Starting from zero.

I have the opportunity to take over a deployment that is entirely manual at this point.

Some of it is in the cloud and some of it is on-prem.

At the moment there isn't much code, although the process needs to be started and stopped and paused, what is a wonderful way to do this utilizing an existing tool like gitlab?

The basic process at the moment is data is loaded into a few servers and then the processing is started and may need to be paused at some point and then restart it.

https://redd.it/f63gzg
@r_devops

What is the best way to create a development server after you made a production server?

I've made a mistake of creating a production server without having a dev server.

I created a docker-compose file and then using Jenkins to build and deploy it to the dockerhub.

I've never created a dev environment, so I would like to get some tips on how to do it properly.

Do I just simply have to copy the docker-compose file and create a new Jenkins pipeline that pushes it to a different dockerhub repository?

Any tips would be great. Thanks in advance!

https://redd.it/f60tl9
@r_devops

Brainstorm DevOps Analytics

Hello folks, I am working on a product proposal for creating analytics features on a DevOps platform. Looking for ideas on what key metrics/KPIs to include for the user personas. Also how should I go around building my customer strategy on this? I have looked into Azure and Broadcom offerings but I have been asked to focus more on Oracle Visual studio angle.

https://redd.it/f5vcxz
@r_devops

Getting Started with Istio Using Docker Desktop - from the Docker blog

Cool blog post about setting up istio on Docker Desktop with kubernetes. I tried it, but when I run the setup I get EOF errors on a few of the installs and the kubectl checks after that don’t return what I want. Anyone else have any issues? It was my first time using Docker Desktop, I usually use it on Linux.

https://redd.it/f66jcd
@r_devops

Help configuring haproxy as ingress in kubernetes

Hi,

I'm learning kubernetes and right now I'm trying to configure an ingress with haproxy in kubernetes. I can't get it to work. As it is configured as a NodePort, I was expecting to see the ports being active on the worker nodes where the controller is deployed, but that's not the case. At first, I was having some issues as my worker nodes had IPv6 disabled and the haproxy-ingress pod kepts crashing as it was trying to bind to ::::80 and ::::443. So I've tried to configure the configmap so it only bind to port :80 and :443 (ipv4)...but I just gave up on that and re-enabled IPv6. Now, the pod won't crash anymore, but I still don't see the ports listening in my worker nodes. The logs of haproxy-ingress say this (idk how relevant this is):

2020/02/18 20:44:21 HAProxy Ingress Controller v1.3.2 8fed7bc
2020/02/18 20:44:21 Build from: [email protected]:haproxytech/kubernetes-ingress.git
2020/02/18 20:44:21 Build date: 2020-02-12T15:23:39
2020/02/18 20:44:21 ConfigMap: default/haproxy-configmap
2020/02/18 20:44:21 Ingress class: haproxy
2020/02/18 20:44:21 Publish service:  
2020/02/18 20:44:21 main.go:99: Default backend service: haproxy-controller/ingress-default-backend
2020/02/18 20:44:21 main.go:100: Default ssl certificate: /
2020/02/18 20:44:22 controller.go:97: Running with  HA-Proxy version 2.0.11 2019/12/11 - https://haproxy.org/
2020/02/18 20:44:22 controller.go:102: Starting HAProxy with /etc/haproxy/haproxy.cfg
2020/02/18 20:44:22 controller.go:115: Running on haproxy-ingress-54847454-8jgzx
2020/02/18 20:44:22 controller.go:69: Running on Kubernetes version: v1.15.1 linux/amd64
\[WARNING\] 048/204422 (20) : Can't open server state file '/var/state/haproxy/global': No such file or directory
\[NOTICE\] 048/204422 (21) : New worker #1 (22) forked
2020/02/18 20:44:27 controller.go:369: Confiugring default\_backend ingress-default-backend from ingress DefaultService
2020/02/18 20:44:27 controller-haproxy.go:152: HAProxy reloaded
\[NOTICE\] 048/204427 (29) : New worker #1 (30) forked


Here it is the configuration used to do the deployment (basically, copy/paste from official doc). [https://pastebin.com/ubAM98a7](https://pastebin.com/ubAM98a7)

I'm a little lost right now...maybe I'm expecting something that it is not to be expected yet. Could you please share any ideas of what I may be missing or were to look for it?

Thank you very much!

PS1: Please advice if this subreddit is not accepting this kind of posts, I did not see any community rule saying otherwise.

PS2: I guess my last resort would be to create a new image with the desired configuration as I'm not even able to properly make it work...leave alone using the configmap. At this point, any help is welcome.

https://redd.it/f5yv7x
@r_devops

Azure Active directory Integration with Hasicorp Vault

I am trying to setup Azure AD authentication for vault,
I refered so many document but still I can not figure it out.
Please help If some did this. Thanks in advance

https://redd.it/f5tu28
@r_devops

Blog Post: Running PowerShell Scripts in Azure DevOps Pipelines (1 of 2)

Hey guys, I just wrote a shiny new blog post you may enjoy.

Summary: Understand how PowerShell scripts work in Azure DevOps pipelines and learn how to run them in this informative tutorial.

https://adamtheautomator.com/azure-devops-pipelines-powershell/

https://redd.it/f5sloo
@r_devops

Securely Access AWS Services from Google Kubernetes Engine (GKE)

It is not a rare case when an application running on Google Kubernetes Engine (GKE) needs to access Amazon Web Services (AWS) APIs. Any application has needs. Maybe it needs to run an analytics query on Amazon Redshift, access data stored in Amazon S3 bucket, convert text to speech with Amazon Polly or use any other AWS service. This multi-cloud scenario is common nowadays, as companies are working with multiple cloud providers.

Cross-cloud access introduces a new challenge; how to manage cloud credentials, required to access from one cloud provider to services running in the other. The naive approach, distributing and saving cloud provider secrets is not the most secure approach; distributing long-term credentials to each service, that needs to access AWS services, is challenging to manage and a potential security risk.

In my [blog post](https://blog.doit-intl.com/securely-access-aws-from-gke-dba1c6dbccba), backed by open source project [doitintl/gtoken](https://github.com/doitintl/gtoken), I show how to securely access AWS services from Google Kubernetes Engine (GKE) without storing/sharing long-term AWS credentials.

https://redd.it/f5sdvk
@r_devops

Waiting times and context switches - how do you handle them?

If the developer user experience in a company is good, you can get to work, start working on your daily tasks, and enjoy a blissful flow state for hours while tackling useful tasks at your own pace. If the development environment is well-designed, the iteration times between making a change and getting feedback (through tests or on the screen) are very short and feel satisfying.

Now, compare this to the experience on the infra-side of things. Long waiting times are the norm. The usual workflow is to apply changes, do something else and check for the results every now and then.

More-or-less blocking workflows, big delays, the need to do busy waiting or frequent context switching seem to be widely accepted as a given.

How are you handling those rough, but only infrequently discussed daily obstacles to focused work?

* How do you handle regular waiting times and delays?
* How do you approach multi-tasking and forced context switches?
* Have you put special effort into taming your (AWS, Docker, k8s) workflows and have tips to share?

I'm looking forward to learn more and share fixes, or at least to join in on the venting.

https://redd.it/f5pjzr
@r_devops