[aws beanstalk] is it possible to configure application load balancer to return 403 based on path?

Hi,

I am aware that I can configure load balancer --> listeners --> rule and use path matching + action.

However, I am wondering if it is possible to achieve the same using beanstalk configuration?

I was reading

[https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-cfg-alb.html](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-cfg-alb.html)

and in the end of the link it gives us a example:

aws:elbv2:listenerrule:admin:

PathPatterns: /admin/\*

Priority: 1

Process: admin

​

From the code and also the beanstalk - configuration - load balancer UI page, it looks like it cannot return 403 and all action must tight to a "Process" (target group).

Am I missing something?

Thanks!

Jay

https://redd.it/ds8rud
@r_devops

What does your local development environment/setup look like?

I'm curious about how other companies and organizations setup their local development environment.

\- Do you perform most if not all of your development locally? Or on a remote host or virtual machine? e.g. AWS Workspaces/AppStream/Windows Remote Desktop

\- Are you calling your Cloud Provider's APIs using a full-fledged Admin role? Or a role with least privileges? Are you able to stay authenticated to a single role to perform all of your job duties/functions? Or do you switch IAM roles based on the services you are invoking/working with e.g. VPC, EC2, Lambda?

\- Do your developers write code locally? How are they invoking your Cloud Provider's APIs? Do they have their own isolated sandbox/dev accounts? Or do they all share the same dev account?

https://redd.it/ds3ud3
@r_devops

Building a CI/CD pipeline, suggestions?

Hi Guys, so I have 8 months of DevOps experience previously working as a desktop engineer for two and a half years.Starting a new role in the education sector, recently they brought their IT department which previously worked in different projects/faculties and now they have all been brought together.So this is where I come in to help dictate the application development process by designing and maintaining the infrastructure to build an efficient route to live regardless of the project. So this is essentially a blank slate.As a very high level overviewSo my ideal setup would be creating a production environment for whatever project, replicating the stack and using that as UAT. This would be done through Terraform and Ansible. The Dev estate I am not too sure how this is going to be setup yet but I was thinking abouta learning docker and k8s to bring up a dev estate.The way I would look into setting up a CI/CD pipeline using Jenkins and acted through Azure DevOps as this is what they use, I would ensure code pushed from the DEV branch is deployed to UAT and tested then deployed to replicate production servers and once tested DNS is switched over to the new servers and the old ones are brought down.The way how the code is deployed is through building the codebase on the same server Jenkins would be setup on, sending a zip'd file of the code to AzureDevops to be taken to the next environment. Through caveats include no changes or access will be allowed on production or UAT servers, any requested changes will be done through terraform and ansible to keep things consistent.Personally with my limited experience I have used Terraform and ansible and I have felt this provides alot of the work to have that infrastructure as code system in place. At the same time I understand there are tools like chef, puppet and salt which would probably make the job easier but that is where my question is what would you guys want and how would you want to implement it. I should finally mention we seem to be using Azure DevOps as the main tool to setup and manage the pipelines.

Would like some opinions on what could be changed and as mentioned how I could use the other configuratoin tools such as salt, chef and puppet to help manage the infrastructure better.

https://redd.it/dsdymp
@r_devops

Just need some clarity.

I’ve been asked to help setup a free ci/cd solution for a friend. At work we use Azure DevOps and we have on prem servers which we build, then install an agent on and the code gets deployed there etc. I started looking at azure pipelines as it’s free however I’m a tad confused. They offer CI/CD for free however I’m confused by the terminology. Do they also supply something for the code to sit on or would I need to setup a cloud solution with say Azure and build a server etc or am I getting confused because my brain is so baked with onPrem? If that’s the case what free solutions are out there so I don’t need to have a server. It’s c# code if that matters

Thanks

https://redd.it/dse80g
@r_devops

Migrating your app to Kubernetes: what to do with files?

While [almost?] everyone knows about 12factor app, it's still quite often that we have to explain our customers what we expect from their apps to migrate them to Kubernetes. [Here](https://medium.com/flant-com/stateful-app-files-in-kubernetes-d015311e5e6b) is our list of common workarounds for dealing with application's files (which we don't really recommend to use) and more practical solutions for that.

https://redd.it/dsfec3
@r_devops

Free event: All Day DevOps 2019 happening right now

Just want to let you guys know there's a 2 day DevOps conference going on and it's possible to watch for free:

https://www.alldaydevops.com/2019-schedule

https://redd.it/dsgfip
@r_devops

Deploying docker container to production - how to do versioning and have production pull new image? Using docker compose

I am creating a CI for production deployment using docker and circleci.


On production I am using docker-compose with:


services:

sp5: # Tomcat 6.0.39 /w 6 JRE w/o default apps

image: foo-registry/sp5-web-app-front:5

container\_name: 'sp5\_application'

I will have CircleCI build the Java application, build a docker image containing it, and push it to my docker registry. Then I will have CircleCI let production host know so it can pull down the docker image.


Question - how do I handle production pulling down the image? Meaning.... it does not seem efficient to use code to modify the docker-compose.yml file to change the version number in \` image: foo-registry/sp5-web-app-front:5\`. I could use 'latest' instead, but I think versioning would be better. Also, I have had issues with docker refreshing the image when it's 'latest'.... it's like it works off of cache and does not know latest changed.

https://redd.it/dshwq3
@r_devops

Alternative to Terraform for non-cloud

In my office we do not develop web applications/services, but rather embedded projects. We do have a complex setup for our custom CI/CD infrastructure (d as in delivery, since there is nothing to actually deploy) which includes homebrew web services, apps like gerrit and jenkins, and many many build machines that need specific environments. All of this is setup on our server cluster that we cannot move to cloud. Currently we employ the help of Ansible for configuration management of all these machines so that they can support our extensive CICD setup.

I read abit about terraform and i really enjoyed the idea of declaring the state of a machine you want to have, so that the IaC platform takes carr of figuring out what needs to be modified to reach that state. However from what I understand, Terraform(or similarly Pulumi) do not work with bare metal on premise setups, but only with (specific) cloud providers.

Is there an alternative for what I would otherwise do with Terraform/pulumi?

https://redd.it/dsgvkp
@r_devops

[mini-rant] Doing "DevOps" work, but not sure if others value it

It feels like sometimes it's more about "hey, just make DevOps do that so we don't have to".

I keep thinking to myself that if they truly valued and prioritized DevOps principles, that my job wouldn't exist, i'd just be another dev or sysadmin contributing towards sprint goals. instead, they've just created another silo to bury work that other people don't want to do, despite the fact that it's there to make their jobs and lives easier.

/rant

https://redd.it/dshfnx
@r_devops

Need help setting up Gitlab CI/CD to build docker images on new push to master, push these images to gitlab docker registry and then trigger a redeploy on an AWS EC2 instance with the new images.

I am new to devops. I need some help with understanding how I can set up my Gitlab CI/CD, is what I want even possible? I am thinking I need somekind of mix between using Gitlabs hosted shared runners and a runner installed and registered from my EC2 instance. Is that right? Where should I start?

https://redd.it/dsix8k
@r_devops

Artifact repository and other things....

My company and me are using .NET and npm most of the time. Actually we store our releases in a git repository in bitbucket (we also use Jira and Bamboo).

Now we are searching for an artifact repository where we can store our releases and in the future our container images. So, we need a piece of software which fits our requirements and is easy to use. We also want a feature called "package promotion".

At the moment we have ProGet installed and i'm testing Artifactory right now. But i'm not happy with any of these. I just found Azure DevOps, but i'm also not sure about it because it doesn't provide a container image registry. It has only a few package providers (artifacts). Because Azure DevOps provides pipelines we would also use them because Bamboo isn't that good :P

Maybe anyone has already the same experience and know which software is good? Maybe only a good artfact repository with atlassian integrations or a completely new DevOps Platform (sadly gitlab has bad support for package /artifact providers)

https://redd.it/dsi4p7
@r_devops

Seeking Advice with Separate Monitoring Stacks; Splunk vs Prometheus + ELK

My organization is trying to improve our monitoring and alerting capability across all of our new and existing applications. Across the board we have over 200+ applications and over 300+ VM’s we need monitored. Currently we have 85% of our infrastructure on primarily Windows machines in VMware and vSphere. This includes a large series of DotNet applications along with some of our message brokering systems like rabbitMQ and our Kafka Clusters.

The monitoring for these applications (which we consider legacy) includes Nagios and Splunk via WMI on the Windows VMs. These metrics are all exported to our Enterprise Splunk Cloud server. We’re importing about 50GB a logs a day (I know that’s next to nothing to many of you) and we expect that to grow to roughly 150GB as we enabled more application logging. Considering that we already have the license for Splunk Cloud, Splunk seems like the right solution when it comes to being able to just dump logs in any format and search through them.

This presents some challenges for us however. Right now our indexing strategy is garbage, where nearly all of these logs, OS metrics and other metadata for completely separate services and events are all sent to only a handful of indexes, making our 1000+ alerts on them all the more exhaustive. In reality I believe we need a more structured set of data being input into the system so that we can create simpler, less resource intensive queries to application metrics as well as infrastructure metrics (such as CPU, Memory over weeks for some applications).

To throw a wrench in the machine we’ve also been facing a lot of pressure to monitor the remaining (but growing) 15% of our tech stack, which is in kubernetes. I came from a background of monitoring kubernetes clusters using open source tools like prometheus, grafana and logging/monitoring solutions like ELK. I am trying to set a new standard for monitoring moving forward by using these solutions in our kubernetes environment but I’ve begun to face some pushback from others who are concerned about having all these metrics, dashboards and alerts all in one area, and that area they’re suggesting is Splunk. I totally understand the value of being able to centralize this information as it provides a lot of added value, but I’m wary as I don’t really think Splunk is the wisest choice for things moving forward considering how many of these open sources tools are being built around kubernetes while the kubernetes to Splunk exporter is fairly lack luster.

Have you guys ever worked in an organization that successfully had two entirely separate monitoring solutions for legacy vs present/future applications or for different tech stacks?

https://redd.it/dshg7i
@r_devops

New to Terraform: is there a way to get the object values of a existing resources in Azure. In powershell I can just query the Resource and get all its properties. It easier to get an ideas what property values a resource or an object pipes out.

Apologies for being naive..

https://redd.it/dsqx42
@r_devops

Any idea about the Cloud/DevOps market in China?

For the most years, we know the only leading name, 'Alibaba' when it comes to China's cloud market. But do you have any idea how it exactly is doing DevOps in China? Who are the players, and are Chinese organizations use Docker and Kubernetes, or they have something else that is employed to do the same job?

https://redd.it/dst1ba
@r_devops

What are you doing for load balancing Hashicorp Vault (HA cluster, open source license)?

I've been running a Hashicorp Vault 0.10.x HA cluster in AWS with an ALB in front of it for a couple of years. It works fine but I've decided to set up a new cluster using a new version of Vault, and start migrating to it.

The new cluster would replace Consul with DynamoDB for storage, which will simplify setup significantly. I'm also adding TLS termination at the node level so data is encrypted at rest and in transit.

At this point I'm wondering what to do for load balancing. I know Vault elects a cluster leader, and requests that you send to non-leader nodes are forwarded to the leader. I know this works fine with an ALB but am not sure if I need the ALB.

The individual nodes will have static IPs so I could do "load balancing" by adding the static IPs to a DNS "A" record and access Vault through that record.

I can also keep using an AWS load balancer, though since I'll be terminating TLS at the nodes, I could switch to NLB instead of ALB.

The latter option is what seems to be Hashicorp's recommendation in their [reference architecture docs](https://learn.hashicorp.com/vault/operations/ops-reference-architecture#load-balancing) (it doesn't mention AWS NLB specifically, just an "external load balancer", but the HAProxy example is similar to (if simpler than) the way NLB works.

Do you have something of the sort running and how did you architect it?

Bonus question: do you have any thoughts about using DynamoDB as the storage backend? I'm also open to MySQL or Postgres. Basically just want to get away from Consul.

https://redd.it/dsk736
@r_devops

What business outcomes have you made possible recently?

There's a lot of focus on the tools we use day-to-day, and technical details around that.

Let's ignore that part, and talk about the positive change, we as a profession make possible in the world. What outcomes has your work made possible that you're proud of? What was the impact on the business and the people involved (aka stakeholders)?

Don't mention the "how" in the top comment, only the results achieved and the why you are proud to have contributed to them. Looking forward to reading your stories!

https://redd.it/dsdjqt
@r_devops

Monitoring wall reccomendations

We have 6 monitors on our monitoring wall all of which have chrome open and revolve through tabs for New Relic and some for Grafana. How do others automate the opening of these?

Currently were using Windows for the PC powering the wall & Nircmd to open chrome and the tabs on the correct screen. but I'd prefer a GUI solution or something easier to manage.

https://redd.it/dsh4zv
@r_devops

Tutorial: Deploy a Kubernetes Cluster with Service Mesh on DigitalOcean Using K3s

I threw together a tutorial on how to deploy a Kubernetes cluster with a service mesh on DigitalOcean. It uses the [k3s](https://k3s.io/) Kubernetes distribution and [Rio](https://rio.io/). The cluster will be a single master, so no HA unfortunately. In the future I'll write a tutorial on how to deploy a k3s HA cluster as well, and how to deploy persistent storage.

I hope someone will find it useful or educational. :)

https://lunar.computer/posts/k3s-service-mesh-digitalocean/

https://redd.it/dsyfcz
@r_devops

Why choose Terraform over Chef, Puppet, Ansible, SaltStack and CloudFormation?

Hi all,

I have this post on [why choose Terraform](https://selleo.com/blog/why-choose-terraform-over-chef-puppet-ansible-saltstack-and-cloudformation) vs \*insert another solution from the title\* and I would really appreciate any comments / thoughts / experiences / possible improvements to it.

Which one, in your opinion, is the biggest Terraform competitor and why? Currently researching this as well.

Thanks in advance!

https://redd.it/dswxc0
@r_devops

Cloudformation not assigning ingress to security group

I have a ALB that I am trying to assign a security group and ingress to. I want 443/HTTPS open only. Currently it creates the SG and assigns it to the ALB but no luck on the ingress. I've tried the below as well as the ingress as it's own resource.

​

ALBSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: "Security Group for ALB"
SecurityGroupIngress:
IpProtocol: tcp
FromPort: 443
ToPort: 443
VpcId: <VPC ID>

https://redd.it/dsz52x
@r_devops