DevOps market in Canada advice

Hi, I want to know what market situation is in Canada as I want to move there an get permanent citizenship, I am English Russian speaking person and as far I know in some city's they speak France, What advice could you give me guys.

https://redd.it/f545ng
@r_devops

A Talk on distributed systems in college

\[Beginner talk\] Here is the recording of a talk I gave at a college in India: [https://youtu.be/9mEJZlD8c7k](https://youtu.be/9mEJZlD8c7k)

I try to take a simple application and discuss various places where it can fail and then come up with strategies to solve them.

Looking for some feedback from the community!

Cheers, and please do consider subscribing to the channel if you are benefiting from it.

https://redd.it/f4s9l2
@r_devops

Unable to recieve options requests on EKS

Hey everyone, I was hoping someone could link me or explain how to get a nginx ingress controller on an EKS cluster uisng a classic LB to allow CORS. I think I am having an issue related to CORS because my OPTIONS requests from my front end (axios) is coming back 405 method not allowed

Below are my ingress resource annotations that relate to the nginx issue.

​

kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
# needed to allow the front end to talk to the back end
nginx.ingress.kubernetes.io/cors-allow-origin: "https://portal.pageanengineer.com"
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, PUT, POST, DELETE, PATCH, OPTIONS"
# needed for aws elb
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"

This setup works fine on a KOPS cluster with the same type of LB created. I am having a hard time figuring out why my requests are being blocked

https://redd.it/f5k1tg
@r_devops

Running Things Locally

How do you guys solve the issue of allowing developers to develop services locally, and keeping the local environment in sync with dev/prod?

What does the workflow look like for running one of those services against external dependencies?

I'm using kubernetes. What I'm thinking is that while a given service is being developed, external dependencies can be proxied from minikube to localhost to allow that service to be run normally (with a debugger or whatever else).

Then, when that service is done, the container can be deployed to minikube for other services to use.

https://redd.it/f5j6ws
@r_devops

Docker-on-Docker vs. Continuous Integration?

I have been experimenting with tutorials and `jenkinsci/blueocean` image and related tutorials. It uses ***Docker-on-Docker*****^(†)**, but this is considered an anti-pattern (and dangerous). This is intended for local development use cases, such as building `Jenkinsfile` pipelines.

But what I was wondering, given that running Jenkins services in a container (such as Kubernetes) is quite useful, how is ***Docker-on-Docker*** avoided, but still have the ability to build and push docker images?

I imagine this is not just Jenkins problem, but any CI solution that requires build/push docker images, where the service is running in a container.

† ***Docker-on-Docker*** (in case anyone unfamiliar with the term) is running the docker client tools inside of docker, mounting the unix socket into the container, which allows you to build/push docker images, but also allows the container access to a daemon running with root privileges.

https://redd.it/f5ivfn
@r_devops

Automatically version and deploy your documentation via github actions

Hey guys,

I created two github-actions that allow you to automatically version and deploy your documentation.

You get the simplicity of keeping your documentation versioned in git, without the annoyance of having huge PR's due to auto-generated-doc updates.

Right now the only hosting provider implemented is netlfiy, however more are planned.

[https://github.com/bobrown101/version-docs](https://github.com/bobrown101/version-docs)

[https://github.com/bobrown101/deploy-branch](https://github.com/bobrown101/deploy-branch)

I would appreciate any feedback and stars on github.

Thanks,

Brady

https://redd.it/f5es79
@r_devops

Modern Alternatives to Rundeck?

Rundeck kinda sucks. Its interface is slow and clunky (even with the new coat of paint it got recently), its steps mechanism gets in the way so often that I generally just work around that with shell scripts, and I feel like there's got to be something that is more modern.

We're looking at bringing up a new Rundeck cluster in the cloud, so I feel now's a good time to look for alternatives. We need something that generally provides the same sort of functionality - defining self-service jobs that are easy to run for those who aren't necessarily developers.

I've used Jenkins in the past for this, but I'm not interested in dealing with that behemoth again. Is Rundeck still the best choice, or are there alternatives worth considering?

https://redd.it/f5cjpm
@r_devops

Unable to start helm start on KOPS cluster

I have a KOPS cluster which has two different namespaces in which application pods go, 'production' and 'development'. I want to have the consul helm chart only on nodes which match either production or development in their instance group names.

​

Below is

​

0/7 nodes are available: 1 node(s) had volume node affinity conflict, 6 node(s) didn't match node selector.

Two different instance group node annotations

​

Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/instance-type=t2.medium
beta.kubernetes.io/os=linux
failure-domain.beta.kubernetes.io/region=us-east-1
failure-domain.beta.kubernetes.io/zone=us-east-1c
kops.k8s.io/instancegroup=production
kubernetes.io/arch=amd64
kubernetes.io/hostname=ip-10-0-100-129.ec2.internal
kubernetes.io/os=linux
kubernetes.io/role=node
node-role.kubernetes.io/node=

-------------------------------------------------

Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/instance-type=t2.medium
beta.kubernetes.io/os=linux
failure-domain.beta.kubernetes.io/region=us-east-1
failure-domain.beta.kubernetes.io/zone=us-east-1c
kops.k8s.io/instancegroup=development
kubernetes.io/arch=amd64
kubernetes.io/hostname=ip-10-0-113-23.ec2.internal
kubernetes.io/os=linux
kubernetes.io/role=node

In my helm chart I am setting the node affinity using the below for production and exact same for development except the word production is development

nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kops.k8s.io/instancegroup
operator: In
values:
- production

Why am I unable to have consul start up on the proper nodes/at all?

https://redd.it/f5dw2o
@r_devops

What is the difference difference between Cortex and Serverless (the framework)?

I've been working with Serverless to deploy my python functions to Lambda. From looking at Cortex's GitHub, they seem very similar, except I can't find anything comparing them, which leads to the contradictory belief that they aren't even close to the same thing. Can someone clear this up for me?

https://redd.it/f5dcl2
@r_devops

Is there a Linux Live CD distribution for web development?

Hello guys,

Recently my home computer died. The only one that I have left is my work computer that I will use temporarily for personal projects, I don't want store personal stuff in it so I was wondering if there's any available Linux Live Cd distribution for web development?

Thank you :)

https://redd.it/f5d6tc
@r_devops

terraform: workspaces or file layout for multiple AWS accounts

With CI/CD in mind, and also wanting multiple AWS accounts to separate environments (among other reasons), are you using terraform workspaces? I like the allure of "one source" in combination with git branches, but I don't like the worry of being in the wrong workspace when running an apply. There's also the issue of state files having to all exist in same S3 bucket/account regardless of environment (at least it seems like an issue). Some of this could be alleviated by having the CI/CD system do deployments perhaps, but it's still a worry. I'm leaning towards file layout for different stages. Input appreciated!

https://redd.it/f5d33o
@r_devops

What do you need in a network scanner?

What is the one thing lacking from all network scanners?

https://redd.it/f5cpwu
@r_devops

[Student Group Project]: How do I integrate DevOps practices and tools in our software dev project?

We are essentially recreating our version of firehose.io (a semester course planning webapp)

Our project tech stack and rough architecture plan: https://imgur.com/cWB9zqG

Team: 4 Student Members (sophomore level)



Additional Info.

We are using GitHub as our code hosting. Will be using Firebase Hosting for our webapp. May not be important but we are using JetBrains' IDEs (PyCharm, Webstorm).

For the webscraping part, we are utilizing virtualenvironments.

**We want to adopt best practices as much as possible as long as within our scope.**

It'd be great if the learning curve is as smooth as possible and and if we can use as little tools as possible (such as GitHub Actions instead of a third party tool)

Looking for any open ideas and as detailed suggestions as possible. Thank you!!!

https://redd.it/f5aud5
@r_devops

kube-prometheus vs the hard way

I want to setup a metrics stack for my kubernetes cluster and contemplating my approach.
Currently I'm stuck on a quite old kubernetes version (v1.11.2), so it seems I can't use the latest kube-prometheus verison.
I've already tried using kube-prometheus before, but disabled it as it didn't provide any concrete value yet.
I liked all the things it provided out of the box, but I also feel like there are many pieces I don't really need, and much less understand how they work.

I've been considering starting something from scratch, simply installing prometheus on my cluster and configuring things as I need them.
I feel this approach will be much more educational and slim, but it also seems that very soon I would need things like kube-state-metrics, grafana, node_exporter with specific configurations for each one, that it quickly gets overwhelming.

Any recommendations, experiences would be appreciated.

https://redd.it/f58ft9
@r_devops

Centralized configuration management server (Lygeum)

A Very interesting tool for centralized configuration management which lets you handle applications and environments configuration in an intuitive way through it's web console or Command Line Interface. The tool is here, [Lygeum Centralized Configuration Server](https://kaiso.github.io/lygeum/)it provides a standalone installation but also a docker official image.

https://redd.it/f58d52
@r_devops

I Setup a CDN / Application Gateway and DDoS'd Myself | What other non-obvious can a CDN rollout fail? And how did you mitigate it?

TLDR; as an experiment I setup a CDN in front of a side project. It did not go as planned when the 40+ edge servers started hammering my little app.

Blindly putting a CDN in front of your app exposed three problem areas:

1. Lack of caching / last-modified calculation

2. No light-weight /health-check endpoint for the CDN to hit. It attempted multiple heavy page-loads from the home page as its health check. Ouch.

3. No rate-limiting on my reverse proxy.

4. Poorly configured CDN settings. 40+ edge servers making requests every 30 seconds seemed a bit much for the app I was using. It could have been more conservative.

Yeah, the full blog post is here:

https://www.simplecto.com/i-setup-a-cdn-and-ddosd-myself/?ref=devops

https://redd.it/f5873e
@r_devops

How to use Azure Git with AWS EC2?

We are a new startup company that are trying to shift our workflow to DevOps. The code and new commits of our CMS application is stored on Azure Git. Whereas we plan to have our servers on AWS EC2 alongside with Terraform, Jenkins, Docker, Kubernetes, ELK stack etc.

We are unsure how to integrate Azure Git to this workflow. Does Jenkins allow us to retrieve the code from Azure Git then continue providing it to the process mentioned above on Amazon EC2.

Should we need to provide more information please let me know.

Sorry if my question is too basic or newbie. Any help would be highly appreciated. Thanks

https://redd.it/f586y5
@r_devops

Received email at work address about a local AWS certificate bootcamp - legit or scammy?

**EDIT: Consensus is that it's scammy. Thanks, everyone!**


Original post:


Subject is: "RE: AWS Solutions Architect Associate Course"

I've never corresponded with this person before, so it's not a real RE.

The email discusses the training course local to my city (a fairly large midwest city) and then says, "For more information and to register visit: thenameofthecompany.us"

It's not a link, they didn't put a comma after register, going to the domain with https results in a firefox "Unable to connect" page - it doesn't exist. Going to it in http redirects to a suuuper scammy sounding domain: https:// www . rebusbusinesssolutions . com/

I'm thinking....this is a scam. Anyone have experience with this domain?

https://redd.it/f5uijs
@r_devops

Automation of deployments, and blue/green

Traditionally, a blue-green deployment was described as a technique that reduces downtime and risk by running two identical production environments called Blue and Green. At any time, only one of the environments is live, with the live environment serving all production traffic. Because there is already another environment “ready to go” it’s possible to switch environments if something happens to the green environment or a new version needs to be released, and can be really handy in a disaster recovery scenario.

As times have changed, containerised technologies are now available, and it is possible to extend the description of blue green deployments to stacks or clusters of a containerised service, rather than just a traditional replica two environment blue green, as long as the tool you use supports some kind of connection draining, e.g. a load balancer.

[https://medium.com/@craiggoddenpayne/automation-of-deployments-and-blue-green-eae80f52494b](https://medium.com/@craiggoddenpayne/automation-of-deployments-and-blue-green-eae80f52494b)

https://redd.it/f56spu
@r_devops

What is the best way to manage a database that requires migrations to be run on it?

My problem: I don't want to deploy a codebase to my production server where my database isn't up to date.

My Goal: I don't want to manually run migrations on the database every time I deploy. It feels error prone. Ideally I have an environment where I can create a staging environment, sync the DB's, run migrations, run end to end tests, and then route traffic to the new version.

What do you think? Where should I look to get started?

Edit: My stack is django/postgresql/Reactjs and I am planning on integrating Kubernetes. Deploying to Google App Engine.

https://redd.it/f4ura3
@r_devops

How to handle integration/functional tests for an WebApi not accessible from pipeline(Azure)

I have been looking for the best way to handle tests for an Web Api. The service is only accessible from certain ip's and therefore I can't run the test (call the service) on the CI/CD agent.

How is this handled in the best way, I want to get the test results back to the CI.

​

I saw an old [post](https://www.reddit.com/r/devops/comments/bnh6fs/running_tests_in_k8s_like_with_dockercompose/) where u/forestjunk said something about kubectl exec to get the test results back, is this possible?

https://redd.it/f568g6
@r_devops