How do you implement API Gateway for a Microservice project?

I want to write an API Gateway for handling requests and routing them to proper services, it's kinda like a reverse proxy. But I want to add some functionalities like monitoring and analyzing requests before they reach final services. There are many people recommending tools like Kong, AWS API Gateway, Nginx while other people prefer writing one by themselves. So what are the pros and cons for using tools and writing your own?

Here are my concerns for this project:

* Scalability (easy to upgrade, add new features)
* Work well with Terraform and Heroku.
* Low cost, as much as possible (in terms of pricing and efforts)

Thank you!

https://redd.it/kbgczf
@r_devops

How do you do environment seggregation?

Currently we are running a separate Kubernetes cluster in a separate vpc for each of our envs, three dev envs, staging, sandbox and prod. There has been talk about collapsing into two vpc's and two Kubernetes cluster, just prod and non-prod. Then segregating environments based on kubernetes namespaces.

How do you divide environments?

https://redd.it/kaurvo
@r_devops

Change backend storage for hasicorp external vault

Hello,

I am using this terraform module to provision vault. Now, I want to change the backend storage. This seems to be not possible using the terraform that I am currently using ie https://github.com/terraform-google-modules/terraform-google-vault. If anyone can confirm this that will be helpful. According to the documentation we can use several backend storage storages. By default, it is currently using GCS as storage. I want to use MySQL.

https://redd.it/kaxup1
@r_devops

learn nodejs to write AWS Lambda functions

Hi,

I have a desire to learn nodejs so that I can write AWS Lambda functions. I use Python today but would like to learn node as I see plenty of AWS samples and examples in node. Looking for some pointers to simple tutorials where I can start.

Thanks

https://redd.it/kawjpf
@r_devops

how to uninstall argocd

how to uninstall argocd?

https://redd.it/kayuih
@r_devops

Do you think Kubernetes will keep on growing in DevOps land from 2021 on-wards?

Do you think Kubernetes will keep on growing in DevOps land from 2021 on-wards?

https://redd.it/kaqqxx
@r_devops

Do we have a consensus on where pipelines should stored?

Do you consider ci-cd pipeline as a part of the codebase?
Well, for public projects, especially open-source, I think it doesn't make much sense, yet Microsoft keeps Vscode build pipeline in the app repository. I am interested in the internal projects that you carry out in your company. Especially if you are using terraforming for immutable deployments. Do you keep stuff like terraform, packer, kubernetes manifest, dockerfiles and pipelines separately from your application repository?
Is there a consensus on best practice?

https://redd.it/kbnrd4
@r_devops

Thoughts on what belongs in a pipeline and what belongs in the build tool

Hello,

the question: how much should be done in locally runnable build tools vs build servers?

background:

I work on a project with an architecture similar to as follows:

- individual microservices are developed by semi-autonomous teams are part of a bigger project
- this bigger project has a provisioner which is used to install the project onto a cluster of VMs. It installs a variety of technologies such as docker, kubernetes, etc, to these VMs.
- this provisioner is released as a tarball with all necessary build artifacts, which all have fixed versions themselves. this is done because some of the install targets are not available through the public internet
- configurations which are included in the provisioner are versioned with the provisioner

this should give a rough idea of the approach, which I am sure is very common; individual sub-components product build artifacts which are orchestrated via a provisioner

the CI/CD challenge becomes how to automate things such that developers and operations engineers don't need to manually collect and version the provisioner each release. originally, this was how it was done, and it was a very slow, error prone and stressful process which resulted in lots of over time and sleepless nights.

I threw together some jenkins pipelines to automate most of it and it's been working well enough, but I'm not all that happy with it myself.

the issue is a lot of the build process exists in pipelines which are run in a jenkins server. these pipelines are under version control as well but they do an awful lot. building our compiled artifacts, dockerizing them, performing sonarqube and twistlock analysis and updating our provisioner all happens within these pipelines.

we use maven for some of our projects, and what I like about maven is it can be run locally by developers. there are plugins which can build docker images, perform sonarscanning and such. It's current quite a specialized task to update our pipelines, and I can't help but wonder had we done more in maven and less in jenkins if we might have been better off.

the issue with maven is it's got first-class support for Java, but kind of leaves our other projects out to dry. Is there a more language agnostic build tool which could be run locally that could be used to define our build pipeline? is this something other people have found is better than the Jenkins, GitHub Actions, TravisCI heavy approach that we're using now?

Interested to see what others think on this.

https://redd.it/kbr5wv
@r_devops

DevOps/SRE questions

I just wanted to share my experience. I went through couple of interviews for DevOps Engineer positions.

**Offer 1**
* Base: 89K
* Bonus: 7%
* Location: East Coast

**Offer 2**
* $61/hr on a 6 month contract
* Location: East Coast

**Offer 3**
* Base: 75K
* Bonus: 10%
* Location: South

**My Skills**
* AWS
* Python
* CICD

I did not take any offers because either they were less than what I make currently or not worth a swich because the increase is minimal. Have salaries taken a hit due to Covid? I have 2 YOE. I am hoping fellow professionals who have interviewed for similar positions can share their experience.

Stay safe everyone

https://redd.it/kbuigv
@r_devops

What factors do you use to determine your 2021 DevOps forecast ?

Example - Delivery Lead Time forecasted for the business in 2021

https://redd.it/kbuit8
@r_devops

Good Devops Resume Examples?

I graduated 2 years ago, went straight into a devops team at a SAAS, work with AWS/Terraform/Jenkins/etc. daily, but I’m being underpaid for my current position and looking to move into a higher salary devops role in Boston or remote.

I know my skills are in demand, but
since I haven’t had to put together a devops-specific resume before, is there anything important I should be adding? Or just make it clear I have experience in a modern stack and list some projects?

https://redd.it/kbzq8a
@r_devops

Resources for Python and Linux

Hello all,

I know Python and Linux skills are needed in the DevOps field but I don’t know which/what type of skills. Does anyone have resources or links to either docs/videos to learn Python and Linux specifically for DevOps? I’m trying to learn the necessities and the musts of Python & Linux just for the DevOps. Thanks in advance!

https://redd.it/kbvmw4
@r_devops

do you prefer deployments referencing specific docker images by some version/git-hash or using something like ":latest"?

I'd like to adopt a best practice with my new deployments. Currently I run with docker images in k8's where the deployment references image tags like prd or qa . Is it a better practice to deploy with specific tag names using git hash or version? When would you prefer one over the other?

https://redd.it/kbsweo
@r_devops

I'm trying to learn how to automate everything from development to production, care to chime in on how I'm doing and what to do next?

Hi r/devops, I hope all of you are safe.

I'm a software engineer that would love to transition to devops some time in the future. I figured that the best way to start is to learn how to implement the "devops" way of an application's lifecycle. As a precursor, I have developed a simple pipeline at work with Bitbucket that:

1. Upon pushing to remote, it runs the automated test suite and reports the result.
2. If something is merged to a branch of interest (like staging), the pipeline will SSH onto the relevant server, run git pull origin <branch>, and then restart Nginx.
3. It then pings a healthcheck endpoint that make sure services such as RabbitMQ or Redis are still functional.
4. A report of the whole process is then emailed to stakeholders

Nothing breathtaking, really. The servers are still provisioned and configured by hand, there's a ton of hardcoded (or not really, they're in Bitbucket's environment dashboard) stuff such as SSH keys that feels icky. But all in all, it gets the job done and I'm proud and happy to work on these kinds of solutions.

Now I have a side project in the works, and I want to use this opportunity to apply better practice with strong emphasis on automation. It is a non-SPA Django app with a Postgres database. I currently have the following things one:

1. Use Docker in development to make sure each dependency is consistent (i.e. I don't even have Postgres and the required Python version to run the app installed in my machine).
2. Use docker-compose to start both the app and the database in development
3. A simple Gitlab CI file that runs the app's test suite, utilizing Docker and docker-compose as well.

I now want to have a publicly available version of this app so that the client can test it. I added the following things:

1. A production docker-compose.yml that spins up nginx in front of the app. Also uses gunicorn to serve the app instead of Django's development server.
1. A Terraform configuration that spins up an EC2 instance and a bunch of other stuff, in which in the end I can access the instance via ssh and http
2. A couple of Ansible playbooks that:
- Install Docker
- Install Docker-Compose
- Copy the application source code to the ec2 instance using the synchronize module
- Rebuild the images and restart the container
- Collect the static assets and run the database migrations
- Create admin accounts if they are not existing yet

With this setup, I can consistently re-create the prod environment with terraform apply + ansible-playbook.

I know this setup is still pretty rudimentary so I have a bunch of questions:

1. After running terraform apply, I run terraform show to get the public ip of the created instance. I then update my /etc/ansible/hosts file. Is there a way to automate this?
2. When running ansible-playbook (or adhoc commands for that matter), I still need the instance's private key in order to connect: ansible-playbook ./initial-setup.yml --private-key=~/Keys/myprivatekey.pem. Is this a normal way of doing things? It just feels weird being tied to this key file, that I need to store it in a non-VCS storage for safekeeping.
3. What's my next step here? Do I integrate Terraform and Ansible into Gitlab CI, and run the commands above according to some trigger?

Thank you for your time to read my queries. All of this is new yet exciting to me, and I can't wait to hear your thoughts. Stay safe!

https://redd.it/kbswda
@r_devops

What does it look like to further your DevOps Career?

I see a lot online regarding how to become a DevOps engineer (it is the hot thing right now, after all). But what does it look like to grow your career in this field? What kind of roles would you move towards?

Are Senior/Lead DevOps engineer roles common? DevOps Architect?

https://redd.it/kbixqq
@r_devops

anyone use glowroot?

Just starting to use glowroot on cassandra servers for JVM monitoring, and I was curious if there are any command line tools for glowroot? I am seeing servers where my data.h2.db grows too big and would like to have a way other than going to the webpage and deleting the data and can't find any other way to do it.

https://redd.it/kbe92k
@r_devops

Verify mobile app layout changes on pipelines

On my mobile developer career it was always hard to control which layout changes was made before a release. Sometimes the changes isn't good enough to product team or has some info that mustn't be there. Do you remember of nissan developer busted for copying code from stack overflow? (https://www.theverge.com/tldr/2016/5/4/11593084/dont-get-busted-copying-code-from-stack-overflow)


The process to control the changes is very massive. Usually is like either open the app screen by screen and simulate an user or see screenshots of UI tests one by one and compare them. It is specially hard to small teams because this process waste too much time.


I'm creating the LayoutDiff to help my team control layout changes effortlessly and integrated with development process improving the accuracy of quality assurance and getting fast visual feedbacks.


To know more about it click on the below link:

https://www.layoutdiff.com


The first open source project is free!

https://redd.it/kc61ut
@r_devops

How to skip building a new container if only kustomization files have changed in Gitlab CI?

Recently discovered that Gitlab CI has [a few](https://docs.gitlab.com/ee/ci/yaml/#onlychangesexceptchanges) [ways](https://docs.gitlab.com/ee/ci/yaml/#ruleschanges) to skip a certain job if no files relevant to that job have changed.

I would like to use this to skip building a new container at each merge/push even if only my kustomization files have changed.

So I have one repo with app source, Dockerfile and k8s kustomization manifests.

Why one repo? Well it seemed like the only way to do a full deploy when my devs merge new code. I used to have separate Helm chart repos that would allow for separate Helm chart and source code development but I like kustomize better.

My problem is that my CI jobs use the current "$CI\_COMMIT\_REF\_NAME-$CI\_COMMIT\_SHORT\_SHA" as image tag so if I skip the container job this tag would not exist and kustomizations would result in failed image pull.

There are many ways to skin this cat so I'd love to hear suggestions on what I'm doing wrong.

* I want to push feature branches and see these changes in the staging environment. So relying on tags to avoid this issue won't work during development. Tags are only in use in prod.
* I wish I could make kustomize re-pull an image using a general tag like latest or the branch name, that way I could push two images one with short sha and one with just the branch name. And I'd still be able to make kustomization changes after the container image was built.
* I wish it didn't have to push a new container image every time I edit the kustomizations. Nothing is different with it from the last image pushed.

https://redd.it/kcadr1
@r_devops

What should be critical midnight-alerts?

It's my third year doing DevOps, but first doing On-Call shifts. And to be honest, it was terrible. It felt like a Russian roulette game. And often getting unlucky, meant getting really unlucky.

I took it with my manager, and this conversation had lots of tears being shed on my side, and understanding from my manager. He told me it's a matter of priorities and since it's so painful we'll prioritize it other than delivering features for the rest of the organization.

Now as I feel hopeful, I still have concerns, we have some alerts that seem to be like.."Something broke, good luck" which trigger PD's.


I wanted to ask you, have do you decide which alerts are critical enough to be handled at night, and how do you make them as minimal as possible.

thanks

https://redd.it/kca1p2
@r_devops

Difference between COPY and ADD in dockerfile

New article that helps to know the difference between COPY and ADD commands in dockerfile and the best practices for using these in dockerfile for building docker imagesDockerfile ADD vs COPY

https://redd.it/kcbrdz
@r_devops

Console #31 includes a CLI for easily managing secrets, that I thought /r/devops might be interested in :).

You can find the newsletter here:

https://console.substack.com/p/console-31

It also includes interviews with the developers of the projects and a new help wanted section, in case you're looking for open source projects to contribute to :).

https://redd.it/kca7p3
@r_devops