What sort of tools/microservices or scripts have you built?

I'm prepping to start interviewing for new jobs and my weakest aspect has always been programming, part of what drew me to DevOps was working more with systems and architecture than software dev.

I've built a few tools/microservices (healthcheck scripts, simple webservers, a microservice that receives AlertManager alerts and can then send JSON to another URL), but I'm hoping I can steal some ideas from you guys and make a few projects to practice coding :)


Thanks in advance!!

https://redd.it/nxzri3
@r_devops

Django using Gunicorn + Nginx in ECS Fargate

Hello Guys,

I'm trying to set up my backend using aws fargate currently i'm not using nginx for reverse proxy just using gunicorn to start the webserver, im almost ready to go live so i'm looking add nginx for it,

The problem is that i'm not very sure the approach using 2 containers in the same task definition, if i add manually added each container pointing to different ecr register ( one for the app and one for nginx) its not taking the nginx config i saw the logs and doesn't give me nothing useful

I'll try different task definitions but doesn't work either i maybe jumping one step

what i want to achieve is that my ALB -- point to my nginx container using port 80 and then redirect to my app container if anyone has done something like this i would appreciate your thoughs

Thanks!

https://redd.it/nx7eyp
@r_devops

Jenkins Pipeline As Code | Learn How To Create Jenkins Pipeline Job

Jenkins basics | Start your first Jenkins Pipeline code.

https://youtu.be/KtJCcqxKXbs

https://redd.it/ny5bnz
@r_devops

Anyone using TrendMicro- Cloud One for Image and container security?

Hello all,
we are planning to use Trend micro cloud one security tool for image and container vulnerability scanning and also for Runtime security. Currently we are using Prisma cloud but opting for a change. Any one has experiences with TM or can you share your reviews regards to performance of the tool, service and support etc..
Thanks

https://redd.it/ny4s8n
@r_devops

How to use Kubernetes Ingress to reduce resource utilization and cost

How Clumio’s Cloud Native Architecture Utilizes Kubernetes Ingress to Reduce Resource Utilization and Cost.

https://redd.it/ny99dn
@r_devops

Is there a considerable difference in DevOps which is implemented on premise vs Cloud?

When implementing DevOps, is there a considerable difference between On Premise and Cloud? Which is more easier to run and maintain?

https://redd.it/nyb7r6
@r_devops

Newbie Request. I want to learn about the build and deployment process in detail in traditional SDLC. Is there any resource which would be helpful in this

Iam new to DevOps. I want to learn about the traditional old school software build and deployment process, tools used for such delivery, so that I could have solid foundation before I start learning DevOps

https://redd.it/nydaxw
@r_devops

How to Jenkins/Terraform/Ansible correctly?

I am currently in the process of currently working with the tools mentioned in the title.

My plan(not sure if correct implementation or not) is to create a job with input variables from the user for server name, mysql version, magento2 version and then make jenkins call terraform which will spawn a cloud vm with the ssh key of the ansible user.

Then Ansible will come in and setup the server services and install a magento 2 with predefined configs for nginx, varnish and magento(details here don't matter i have already done this).

My problem is that in the way that terraform is meant to work is provisioning the whole cloud infra.

I thought about creating a job that copies the template for terraform so it's been saved afterwards for each cloud instance created and also. But it this the right thing to do? Also this way it does not properly provision the whole infra since it's gonna be a new directory for each client and not all the cloud instances created in time.

Maybe injecting a new resource into a main terraform file with a new variables file created before running? As i am typing this i thought about maybe including client .tfs? Is this possible?

I am kind of lost here. Any explanation or examples would be so so much appreciated!

https://redd.it/nye81f
@r_devops

Can azure devops be linked to gitlab?

Hi all,
My organization has bought an azure subscription recently. We as well use gitlab as our version control system. I understand gitlab is a repository management tool that is used for managing locally hosted repositories, which can only be accessed from within the organization's internet.

So, I just wanted to know if we can link this git server to azure devops? If not, then how can we use a vcs in azure? Does it come with its own git?

https://redd.it/nydse5
@r_devops

Getting "failed to instantiate provider" error with GitLab and Terraform CICD Pipeline. Any help appreciated.

Here is the full pipeline. Here is the portion of the CICD pipeline that is failing:

tf-apply:
stage: terraform:apply
extends: .base-terraform
variables:
STACK: "terraform/production"
PLANFILE: default.tfplan
script:
- cd "terraform/production" && terraform apply -auto-approve ${PLANFILE}
environment:
name: aws
rules:
- if: $CICOMMITBRANCH == "master" ||
$CIPIPELINESOURCE == "mergerequestevent"
changes:
- terraform/
- my.tfvars
when: manual
dependencies:
- tf-plan

Results in the following error:


│ failed to instantiate provider "registry.terraform.io/hashicorp/aws" to
│ obtain schema: unknown provider "registry.terraform.io/hashicorp/aws"

It works fine on plan and init so wondering what the issue could be...

https://redd.it/nydevr
@r_devops

Are there no modern artifact repos?...

Currently using JFrog on-prem and their cloud solution appears to just be their on-prem setup lifted and shifted to a SaaS model.
I'm going fully cloud and getting off on-prem where possible and I just can't find any artifact repo that is genuinely built for the cloud but also has a broad range of formats and features needed for a decently sized Dev operation.

If any of you can help I'm all ears! :)

https://redd.it/nyiarn
@r_devops

Cloud performance benchmarks

Would love to get community’s feedback on something. I’m interested to know how often you(or your organization) find the need to run cloud performance benchmarks (see examples below)? OR, if you currently do not run benchmarks but you would if you had the right tool to help with the process, how often do you think running performance benchmarks would provide valuable insights?

Example benchmarks:

- disk or network throughput of different VMs.
- network throughout between different zones, regions, clouds, etc.
- cpu performance of different VM types
- performance of different DBs such as MySQL vs Postgres. Or DynamoDB vs Cassandra
- CDN performance
- api response time
- container launch time
- data warehouse comparison
- etc.

Actions/metrics:
- compare contrast clouds, zones, VMs in terms of cost and/or performance
- compare metrics over time
- infrastructure recommendations based on collected metrics

View Poll

https://redd.it/nyjv2g
@r_devops

the dumbest problem (AWS Lambda)

I'm writing a Lambda function.

I'm clicking file > save.

When I refresh the page I'm seeing the example boilerplate

hOw Do I sAVe ThE fUnCTioN?

https://redd.it/nymtfr
@r_devops

Quick primer for a developer suddenly given a devops hat

Looking for some advice on current best in class tools to document and automate an inherited distributed application.

I've recently started a new role working for a B2C company that sells real world events to individuals. It's an established business that is currently doing a few thousand transactions per month, has 100 backoffice staff doing sales, fulfilment and customer support, in about 5 different countries, just to give an idea of scale.

On the tech side, there are half a dozen back office CRUD apps as well as the main customer facing website. This is all deployed as a mishmash of services in AWS. Most of the web apps and api servers are built on node and are deployed as ECS clusters, a bunch of Lambdas, a bunch of SNS queues, etc. There's also some stuff that lives in Azure. We also use Mongo, RDS, Elastic and Dynamo as data stores....

The original architect of the system was apparently a bit of a rough personality to deal with and he left a few months ago along with all of the original dev team. There was a brief handover period with the new dev team and some knowledge transfer happened, but there is minimal documentation and no real automation, aside from a bunch of buildkite pipelines that we have been using to deploy code updates and figure out how stuff holds together.

Im a reasonably experienced dev, been building apps since the 90s, but have not had much exposure to the devops world beyond using puppet to orchestrate our mostly monolithic apps in my previous job, and some docker-compose stuff for deploying ancillary services. Im comfortable with the aws cli tools, and have a pretty good understanding of what most AWS things do. However, I've not really paid attention to the CI/CD and microservices world, mostly glossing over it as that kooky stuff those crazy california kids play with over on HN. Now it seems I have to get hip with all that.

My question is, what do I start with to get a handle on all of this stuff. I honestly get a bit lost in the overlaps of things like Puppet, Terraform, K8S, Jenkins, Buildkite, github actions, cloud formation, etc. Ideally I want to create some sort of inventory of all our services, build a dashboard to start monitoring these services and smooth out deployments and orchestration of the various containers, and hopefully end up with something resembling a DR process, and some tools to manage these services that don't require us to log on to the AWS website and drill through a bunch of menus to restart whatever service is currently misbehaving. At the moment, any serious rearchitecting of the system is not an option - it's on the road map, but we dont have the resources for that as well as keep the current business flowing.

https://redd.it/nyl71k
@r_devops

Inheriting a cloud system with no documentation. What questions should I be asking?

Hey there, thanks in advance. Any guidance with this would be appreciated. I am currently a systems engineer with a sysadmin background; because of certain regulatory requirements, I have to take over a cloud system environment which is hosting an extensive web application and also expected to maintain a devops environment. All the documentation is in another language and does not translate well with Google translate.

What types of questions should I be asking for this environment? I know this is asking a lot but I am in the process of learning devops tools but I am still a junior in the devops realm. I know just enough to know I barely know anything with these tools. It's pretty fucky what they are asking as I have told them many times I do not know what I am doing if they will not take the time to train me. I do have an intern who speaks English and their language, but there seems to be a barrier as they don't fully understand what I mean. I am on the beginning of the devops path and this will be a good experience for my career. Basically I will be learning everything I can and leaving this toxic environment.

I will be working with the application developers and setting up their environments. From what I understand, we are using Prom/Terraform/Ansible/jumpserver/other tools, but as of right now not giving me access to these tools. I am expected to build out my own Jenkins for this environment, since they will not give me the code or access to the company Gitlab so I have a guide for this.

Thanks for taking the time to read this and appreciate any support.

https://redd.it/nyizca
@r_devops

Udemy & Resumes

Hey everyone! My wife and I have been going over my resume endlessly and we came to my certifications. I have the Azure Fundamentals and the AWS CP.

I am coming from a 15+ year career in Graphic Design. The only experience I have is the Udemy courses I have been taking over the past 2 years. My wife wants to figure out a way to put the Udemy courses on my resume, but I explained that my stating I have the knowledge to do these things kind of tells them about the Udemy courses. Also, I can specify that if I get an interview.

What are your thoughts?

https://redd.it/nyopva
@r_devops

Have you managed a separate “design” repository before? How did you make it seamless?

We’ve run into a problem at my company with a very large grails application where we have a ton of .css files that we’re trying to combine to be more uniform and only have a couple imports. The problem is, we initially broke out the overall css files into a good structure, but we’re having to import 7+ files every time we used it.

I have an initial solution for this by importing all the project design files into one like “main.css”, however that shouldn’t be the long term solution due to having to send a request for all the files anyway, and the files can’t be reused as we move to a micro service architecture.

So I want to move the design files into a separate repository and run a script on it when we build that will minify the project and copy it into the necessary folder in any service we run.

So having it minify in the builds is easy enough, my problem is, we use grails run-app locally, so having a separate design repo would either cause the developers to run a script to minify it when they make a change, or if we clone the repo into the project itself locally, there may be file name mis-matches on the imports.

How have you guys made this seamless at your company? I understand I might need to containerize the app, which will likely happen soon anyway, but even past that, how have you set up watchers to build those files on change? Any info helps.

https://redd.it/nyplic
@r_devops

Encryption and decryption of secrets while deploying it onto an EKS container.

I have a dockerfile where I am trying to copy everything in Github to dockerfile and build it as an image. I have a file called config.json which contains sensitive user data such as username and password. This will also be copied. The issue here is, since this contains sensitive data, I have decided to encrypt this file and store it in Github repo. I want the image to be deployed on to an EKS cluster in AWS and I want this encrypted file to be decrypted. I have a KMS key to be used for encryption and decryption. I had a couple of questions on the same:

1: At which phase am I supposed to decrypt such that the pod contains decrypted file. Should it be during the runtime or in the readiness probe? Or if there is another method to achieve this?
2: Is this method of using KMS the right approach? Or if there are other methods to achieve the same?

https://redd.it/nxz5eq
@r_devops

Digital Ocean Web Server HELP!

Hi Everyone,
I am currently stuck on what digital ocean droplet I will choose for my Web APP. My web app has an API built with laravel/lumen and frontend of Angular and the database is mysql. I want it to deploy on 1 server. But I don't know what particular droplet type will I choose. My system is a medium scale system which will host RSS Feeds, and will have around maximum of 100 users. I currently deployed it on a General Purpose Server which has 4 CPUS and 16GB Ram, and the feedback is slow, so I am planning to scale up, but I am stuck on what to choose, a CPU optimized or Memory Optimized.

https://redd.it/nyseq7
@r_devops

Getting Started with Step Functions for Serverless Workflows

https://towardsaws.com/getting-started-with-step-functions-for-serverless-workflows-8c030c2552fb

https://redd.it/nyuwb8
@r_devops

Application for inventory of enterprise applications that supports subscribing to an application to get notifications of ~configuration changes, etc

Say you're at a large enterprise with 50 or so enterprise applications (Active Directory, Peoplesoft, etc etc etc) and you wanted to undertake an initiative to implement a "portal" of some sort where people could go to see details on applications like version, add-ons (and versions), configuration settings, etc (basically, the things that one would need to know for integrations). Additionally, I'd like to have support for things like maintaining a history of major events (configuration changes, etc), as well as proposals for future events, which "subscribers" to the app would be notified of by email, something like:

On Sept 25, 2021, we are planning to make the following changes to Application XYZ: 1,2,3,4,5. If you have any questions or concerns, click here to review the details and open a ticket with any concerns or questions you have regarding this change.

Anyone know of any existing tools that have this sort of functionality (either as existing functionality in something like ServiceNow, or else dedicated standalone apps)?

https://redd.it/nypv1u
@r_devops