Need advice on microservices and database access.

I currently have microservices connecting to one database. i have every service connect to the DB with its own role with specific permissions. For example, the auth service has a DB role which has only SELECT access to the "users" table I'm using PostgreSQL btw.

Questions: Is there a better/easier way of doing this? Am I going the right direction here?

I ask because this might be overkill. On the flip side, some may think I'm not going far enough and that I need a different schema for each service.

I've read that to do ms properly, each service should have its own DB. I think that's impractical and a maintenance nightmare. At least for my project it is. So, i'm not going that route. But, I need to somehow isolate activities between the services regarding the DB.

I'm speaking only for production environments.

https://redd.it/kq67ix
@r_devops

Dev2Prod


Hey y'all!

I'm a self taught developer. I'm a college student. I've never been into an IT company and worked on anything from scratch.

I've developed a web app. I've written it on MERN stack and with some technologies like Redis, RabbitMQ, Firebase, Algolia and other stuffs. I've eight different services which talk among themselves using RabbitMQ. And Redis to store my active auth tokens in it.

I need some advice from you guys on DevOps. What is the process that happens from development to production? All my service aren't in a docker. I just have repositories for each and every services and my frontend. I am the only developer who developed this web app. So I didn't feel anything hard which coding. I've no CI/CD pipeline or any other sort of it.

Now I need to deploy my product in Heroku. What should I do now? Should I manually deploy every single service or is there any other way to do it? I will migrate from Heroku to AWS in a year mebbe. Just because of financial limitations, I use Heroku for now.

Also now it is in development environment. Should I stage it and then to production or dev env to prod env?

I need some advices from you people on how an IT industry develops a software from scratch to production release.

I'm going to release my app in publicly for everyone to use.

What are the checks should I look before it gets into production and how should I release my app in a production environment?

https://redd.it/kq8tvv
@r_devops

If your team does ML, what is your "MLOps" stack?

I'm getting more interested in/involved in machine learning, but the DevOps ecosystem around ML feels... rough, to say the least.

I'm looking for anyone with experience running ML in production. What does your MLOps stack look like? What platforms have you found that you love/hated?

https://redd.it/kqabgb
@r_devops

Release Dash - Dashboard for Visualising Commits in Pipelines

I've put together a simple dashboard for seeing what commits/changes are waiting to be released in your pipelines - https://github.com/lobsterdore/release-dash.

This dashboard is useful for teams that have multiple environments with manual gates in between, offering a quick overview of the pipelines of all registered services, I find it useful for preventing a buildup of changes and getting developers to stick to small releases.

The dashboard needs a Github PAT to read repos, each repo needs a YAML config file so the dashboard can figure out what tags needs to be diffed to construct the changelogs. Images are available on Docker Hub (https://hub.docker.com/r/lobsterdore/release-dash) and I've knocked a simple Helm Chart for pushing to Kubes (https://github.com/lobsterdore/release-dash-helm).

All feedback and suggestions are welcome, this dashboard is obviously not intended for lucky people that are releasing straight to production.

https://redd.it/kqb8tl
@r_devops

Create a Quick and Easy Prometheus Exporter

Create a quick and easy Prometheus exporter is a blog post I recently wrote. Prometheus is a really powerful platform, but it does require a strict metrics format for consumption.

This blog post shows how to quickly and easily write a Prometheus exporter. Let me know what you think!

This also might be helpful for those implementing AWS' new managed Prometheus service.

https://redd.it/kqak3r
@r_devops

Deployment Strategies Every DevOps Should Know

Hey guys i wrote an article today and I am interested if I've missed out on anything or if I could improve it since as you can see my articles are more of a reminder for myself, that's why they are not monetized. At least not yet. Let me know what do you think, cheers!

Article

https://redd.it/kqariv
@r_devops

why is my package manager missing newer versions?

So I do DevOps at a small company. Part of our build pipeline scans our docker images for vulnerabilities. When it does find them, more often than not I have to go into the Dockerfile and tell apk/apt/yum to upgrade to the version where it was fixed.


Why doesn't apk update && apk upgrade do this for me? My assumption is that some package versions are marked as stable or something so it will only upgrade so far, but I'm curious to know what the actual answer is.

https://redd.it/kqdv6f
@r_devops

I am a jr. web developer looking to move to Devops. Career Advice

I'm currently working as a jr. web developer mostly using html and css not so good with javascript or typescript. On my team I build the required webpages using html, css and my other team members will do the functionality part of the webpages using javascript. I am looking to move to devops as it involves much to little of programming and also I know how important devops is as part of a software development lifecycle so I am not worried about future in devops.

I wanted to move to devops because I am not so good with programming but can handle identifying and fixing bugs and devops mostly involves working using tools (this is what I heard from couple of my friends in devops role) correct if my understanding is wrong on this.

As most of the members here are on devops, it would be helpful for me if you could give me some insights on this.

https://redd.it/kqdgnk
@r_devops

IaaS providers in Europe

Hi All,

​

​

I'm currently looking for an IaaS provider which should be based in Europe and have European datacenters.

Would prefer not dedicated servers but at least a public cloud/VM-based solution.

The goal is for a small one-man startup launching a SaaS product.

So, based on the above, immediately out of scope are: Azure, Google Cloud, AWS, DigitalOcean, Vultr, Linode. As I said: European.

What I experienced myself until now:

\- OVH: have tried their public cloud last year - absolutely terrible with regard to stability and uptime.

\- Hetzner: tried a dedicated server there in the past, not impressed.

Others I've found until now:

\- Scaleway: looks as bad as OVH, would rather stay away from them.

\- Exoscale: looks ok I guess, can't really find a lot of info on their quality.

\- Cloudsigma: looks ok I guess, can't really find a lot of info on their quality.

\- Ionos: seems like a lot of negative info on them online?

\- Leaseweb: also looks like a lot of negative info on them online?

​

So, my 2 questions:

\- of those listed above (not OVH, not Hetzner, not Scaleway): anyone here has any positive experiences with them that they would care to share?

\- are there any other relevant European providers I missed and some here have good experience with?

​

Thanks!

https://redd.it/kqa4bc
@r_devops

Slack is down on the first work day of 2021

Looks like we're going straight into Episode 1 of the new season of 2020

https://redd.it/kqc3ga
@r_devops

What's your thought about AWS having downtime today again?

Seems like AWS had the second in a row downtime https://downdetector.com/status/aws-amazon-web-services/

Seems like like Notion, Slack, Zoom was affected by that. Seems like solutions like CAST AI https://resources.cast.ai/blog/when-aws-sinks-why-sink-with-it or not independent but still even Anthos might be a solution

https://redd.it/kqi7md
@r_devops

WebMap : A Python tool used to automate the execution of the following tools : Nmap , Nikto and Dirsearch but also to automate the report generation during a Web Penetration Testing

I am very excited to showcase my new python script : WebMap

https://github.com/Anteste/WebMap

All started from a small idea: how to automate a Web Penetration Testing .

I used to run every tool in his own terminal and it was taking a lot of time, but now with this tool you can execute all of them just with a simple command : ./webmap.py

This project is free and Open Source so use it as you want and if you have any suggestion you can submit a pull request 😉

https://redd.it/kqk645
@r_devops

DevOps Adoption

Has anyone here followed the Three ways from the DevOps Handbook by Gene Kim?

https://redd.it/kqlwx2
@r_devops

Official Salary Sharing thread for devops :: Jan 2021

Crediting this thread from /r/cscareerquestions that gets posted monthly December Salary Sharing Thread for Experienced Devs

I like to keep up to date with the current state of salaries/compensation across the world. Feel free to share your information below.

This thread is aimed at anyone from entry > Sr level DevOps/SRE/Infra engineers.

Please only post an offer if you're including hard numbers, but feel free to use a throwaway account if you're concerned about anonymity. You can also generalize some of your answers (e.g. "Biotech company" or "Hideously Overvalued Unicorn"), or add fields if you feel something is particularly relevant.

Education:
Prior Experience:
$Internship
$RealJob
Company/Industry:
Title:
Tenure length:
Location:
Salary:
Relocation/Signing Bonus:
Stock and/or recurring bonuses:
Total comp:


Note that you only really need to include the relocation/signing bonus into the total comp if it was a recent thing. Also, while the primary purpose of these threads is obviously to share compensation info, discussion is also encouraged.

The format here is slightly unusual, so please make sure to post under the appropriate top-level thread, which are: US High/Medium/Low CoL, Western Europe, Eastern Europe, Latin America, Aus/NZ, Canada, Asia, or Other.

If you don't work in the US, you can ignore the rest of this post. To determine cost of living buckets, I used this site: https://www.bestplaces.net/

If the principal city of your metro is not in the reference list below, go to bestplaces, type in the name of the principal city (or city where you work in if there's no such thing), and then click "Cost of Living" in the left sidebar. The buckets are based on the Overall number: Low: < 100, Medium: >= 100, < 150, High: >= 150. (last updated Dec. 2019)

High CoL: NYC, LA, DC, SF Bay Area, Seattle, Boston, San Diego

Medium CoL: Orlando, Tampa, Philadelphia, Dallas, Phoenix, Chicago, Miami, Atlanta, Riverside, Minneapolis, Denver, Portland, Sacramento, Las Vegas, Austin, Raleigh

Low CoL: Houston, Detroit, St. Louis, Baltimore, Charlotte, San Antonio, Pittsburgh, Cincinnati, Kansas City

https://redd.it/kqo29h
@r_devops

A structure for infrastructure repos

After working over many iterations of how to organize Infrastructure as Code in repositories, I've landed on the following general-purpose structure.

It took a lot of experimenting with different forms (many of them more complex and "fancy") to finally end on this. It's as simple as I could make it while also making it easy to reason about and solve some general problems of organizing content.

I've used it with many tools on very large-scale infrastructure, in monorepos and individual repos. I give Terraform as an example, but you can use it for deploying/maintaining all kinds of things.

# env/

This is almost exactly the same as a Unix system's etc/ directory. The difference is env/ describes environments. Know how the 12 Factor App says your configuration should be stored in "the environment"? That's this.

It's also hierarchical. Each directory is intended to "inherit" configuration from a parent directory.

The general structure I follow is env/{environment-label}/{region-label}/ .The {environment-label} typically contains the vendor, product name, and account name.Each directory is composed of json config files.

Example:

terraform plan \
-var-file env/aws-myproduct-nonprod/terraform.tfvars.json \
-var-file env/aws-myproduct-nonprod/us-east-2/terraform.tfvars.json

If you're deploying changes to just us-east-2, you can still inherit the variables that apply to all of nonprod. Your configuration is DRY, but you didn't need to do any "templating" shenanigans.

For regionless infrastructure, I use env/aws-myproduct-nonprod/all/{iam,route53,acm}/.You want to deploy regionless stuff separate from region-specific.

The default config file in a directory may be terraform.tfvars.json, and if needed provide overrides like override.auto.tfvars.json . Finally you have a root-module-name.tfvars.json file.In this way you have configuration for each of account-alias, nonprod, us-east-2, and root-module.

This way you can separate and re-use configuration at each level by just passing several -var-file options to Terraform.

To deploy all this, I recommend cd ing into a specific env/ directory and running a generic deployment command, like make plan-terraform or something. Your Makefile has the relative paths back to each config file. It would preserve those paths (like using readlink -f) , then change to one of the app/ directories below, and run terraform plan , like so:


frontend_conf=$(shell readlink -f webserver.tfvars.json)
region_conf=$(shell readlink -f terraform.tfvars.json)
account_conf=$(shell readlink -f ../terraform.tfvars.json)
plan:
cd ../../../app/tf-web-frontend/ && \
terraform plan \
-var-file $(account_conf) \
-var-file $(region_conf) \
-var-file $(frontend_conf)

The point behind this is so anybody can deploy anything without needing to actually know anything about how the deploy works. You just change to a directory and run make plan. (Ideally from a Docker container with pinned versions of tools for this repo)

Also, it's important not to reference configuration across hierarchies. Only reference configuration from your current hierarchical tree/level or below. Otherwise you get into dependency issues across environments/regions/accounts. If you have to reference something in some other environment, use something like a Terraform remote state data source.

# app/

This folder is used to store "apps". Think of them as a complete "application", like a Python module, or C program. They are intended to be "run" the same way you would "run" any other application. They have default configuration (that is overridden by whatever's passed in from env/), they take options/arguments/input, and produce output.

Basically this is where a Terraform root module would go - but not a sub-module (see below). All kinds of things can be "apps": Packer configs, Makefiles, Shell scripts, etc. Each directory should be its own complete component.

And no "apps" should depend upon other

"apps", so that there's no weird dependency issues, and you can easily reason about how each works. If you need to re-use code between multiple "apps", you put it in the next directory.

&#x200B;

# lib/

This directory is the same as your standard Unix "lib/" directory: libraries of code. No default configuration at all. It's intended only to be re-usable components that are used in "apps".

So this is where you put Terraform sub-modules, and anything else you need to keep DRY. You can depend on other lib/ directories from a lib/ directory, but keep it to a bare minimum. It's better to link to multiple lib/ folders from your app/ . This helps reduce dependency conflicts and makes it easier to reason about components.

&#x200B;

# bin/

Nothing fancy here. Shell scripts used as part of your CI/CD process, or wrappers to run tools with the above hierarchy. You can also keep them in app/ for more simplicity.

https://redd.it/kqfl5e
@r_devops

Cloudformation templates

I am using Cloudformation templates to build a Ec2 instance(https://pastebin.com/MYuc0UU1). Its very time confusing to stop and restart the Ec2 image afer I make a change to the .yml file. This is all for a AMI image creation.

&#x200B;

Is there a way to spin up my ec2 instance and execute the .yml file to ease my testing?

https://redd.it/kqli0l
@r_devops

Building an uptime dashboard for a distributed system

We have a product for which we would like to create a dashboard to show

the historic uptime and display any service outages or issues.

&#x200B;

Our service is a Rails app that is running on a handful of servers and depends on

other components like database, cache, S3 storage and some other supporting services

both internal and vendor provided

&#x200B;

Currently we are running health checks on rails app only and providing their results

in an html page. Since the system is constructed using many cooperating parts the

current html page many times says that the system is up and available but some customers

can be experiencing issues in production. It happens fairly frequently like a few times a week.

&#x200B;

We would like to build a better solution beyond polling port 80 alone and provide more

details like which component of the system is having issues and how customers are being

impacted like whether system is degraded and operating with limited functionality or

completely down.

&#x200B;

The system is running in AWS and monitoring individual components like EC2, EBS volumes,

RDS instances, Kinesis etc is very low level only engineers can consume that type of info.

&#x200B;

We like a dashboard that can be consumed by customers and executives who might not know

or care about all the components/services that the final product is built with but are very

interested in uptime.

&#x200B;

An open source solution is highly preferred and we are willing to invest engineering effort to put it together if there is nothing out there already.

&#x200B;

https://redd.it/kql7hf
@r_devops

Help:Automating installation of PostgreSQL & keycloak in Ubuntu & Windows


I am on my first job.My first task is this:Manager is asking me to automate installation & configuration of PostgreSQL & keycloak in Ubuntu & Windows.He asks for a design document.I am completely new to devops and job environment.I don't know where to start.I need to finish this today :(.
What tools are available for this types of tasks?
Should I write script (batch/shell) or should I use some tools?
Please help me get started!
Any link,advice will be very helpful! Thanks
--Noob devops guy

https://redd.it/kqrc70
@r_devops

Hosting options: Gatsby+Flask

Hi everyone!

I'm working on a Gatsby project with a Flask backend and I'm starting to think about the viability of this combination in terms of hosting options before it's "too late" to reconsider.

Heroku was the first option that came to mind and they do indeed seem to support Flask.

Even so, do you reckon it will be a smooth experience deploying a Gatsby+Flask website to Heroku?

Looking forward to hearing your insights!

P.S. The reason I've chosen Flask is twofold: learning something new and not having to reimplement the working Python web scraper I already have in JavaScript.

https://redd.it/kqqwm7
@r_devops

Solution for cloud assets inventory management.

Hi,

I am looking for solution that will help me manage the inventory of our cloud assets in Azure / AWS. R&D, QA, Solution architects, sales all open assets for POC \\ development \\ testing etc... some forget to terminate the resource after its not needed any more. i tried using tagging but its not clear enough, not centralized enough and I find it hard to enforce users who open new assets to follow the tagging policy (although i see now that AWS does provide such a thing), I thought of using a shared excel sheet where the user would fill the details, but i am sure there is a better solution

I to be able to know who is the owner of the asset

what is for? RnD / POC / Testing etc...

creation date

end date - does it have a date that it could be shutdown or stopped.

get alerts on assets the need to be closed

etc...

Thanks!

https://redd.it/kqu9sb
@r_devops