Reddit DevOps
270 subscribers
5 photos
31K links
Reddit DevOps. #devops
Thanks @reddit2telegram and @r_channels
Download Telegram
Private container registry

Hi all
I would like to use https://www.projectquay.io/ as a private container registry but
unfortunately I could not find any resources how to install it.

Is projectquay really open source? When I visit the site https://quay.io/, the provided version on premises is a trial version.

On github https://github.com/quay/quay, there is not any guide how to install it.

Would be nice, if someone could help.

Thanks

https://redd.it/l8vzns
@r_devops
Best practices surrounding password storage (hashicorp vault)

Hi,


I've been looking into vault lately, and I am trying to figure out best practices/most secure. We are not using kubernetes/aws/gcp. Currently we are just deploying docker-composes, and I was wondering what the best way to secure our passwords might be. With passwords I mean the application to database kind of credentials. Or credentials to another app.


Is there a good way to secure the authentication mechanism, so that not the whole token is in configuration files, or findable through config files? I get that getting a vault token removes the need of passwords in configuration, and that is pretty neat, but how do you prevent an attacker from using the same token to still get the passwords?


Or should you have the mindset that when an attacker gains access to a machine all passwords are "lost" anyway. Or are there some defined best practices?

https://redd.it/l8hhll
@r_devops
What are the best containers to use with Kubernetes

I read recently that docker was depreciated for Kubernetes. What container service do you use instead? Also, if you deploy your app and set up your CI/CD pipeline without containers, is it easy to put everything in containers later? Or do you realistically need to add containers at the very beginning?

https://redd.it/l8sc2p
@r_devops
Monorepo Build Systems (Bazel vs Pants vs Please)

This probably gets asked somewhat often, but these tools tend to change so I want to get an opinion from people who use monorepo build systems in their codebase.

I am looking for a build system to use with Javascript, Python and gRPC. I wanted to go with Bazel since a lot of big companies seem to be using it (or a version of it) to some extent (Google, Dropbox). The main problem I have with Bazel is the Python support just seems awful. From weird namespacing bugs to a somewhat fragmented Python ecosystem, Bazel seems to not have Python integrated well.

I wanted to switch to either Pants or Please, because they seem to support both Python and gRPC. Although they lack JS support. There is also Buck, but Buck has no gRPC support which is a deal breaker.

Would appreciate any opinions/recommendations.

https://redd.it/l8gmne
@r_devops
Transferring data between two minio servers

In my work place, we have deployed MinIO (I wasn't in charge when they did that, and the only thing I know is that it's not on Amazon and it's a self-hosted one). Then, I have to transfer the data to a newer service which has minio as well.

First one is a Linux server I have full access of. The second one is "Storage as a service". For now, I have no idea about how can I transfer the damn data between two servers.

https://redd.it/l8gfgs
@r_devops
How to calculate cycle time (process time) ?

Hello everyone, me and my team (one other guy) are working on a project, we have a Github and we decided to follow Devops principles to learn. We also created a Kanban board for the project directly on Github. Now, we'd like to use the github API to calculate the process time for each given task (issue), but it seems the Github API is lacking in terms of what issue is on any given column, so that we could get the issues in the todo progress and save their dates when they were first added in the column, so that we could use this date (minus) the date the issue is closed at to get the process time. Does anyone know how to use the github api to get the cycle/process time?

https://redd.it/l95iw9
@r_devops
How did you get into devops?



View Poll

https://redd.it/l9blqv
@r_devops
Decentralize Infrastructure As Code

As a developer I love infrastructure as code, especially collocated with my service code. For example, having Jenkins pipelines or k8s manifests in the same codebase as the source code for the service. One problem I face though is that I work with a couple of centralized repositories that manage things like secrets or Terraform definitions for my stateful backend pieces like storage (e.g. one large repository for all terraform).

There are some good reasons these things are centralized: auditing, human gating for cost and security reviews, as well as an ability to bootstrap it all in case a new environment needs to be spun up.

The downside as a developer is that it adds friction. I need to go make changes in a separate repository, possibly wait for team-external review, etc.

I'm curious if anyone has run into similar issues and find a compromise. I was hoping to allow service teams to "mount" (maybe use git submodules from the centralized repository) config related to their service, and get the benefits of having the service dev team be more independent.

https://redd.it/l9j5v0
@r_devops
I've got some Kubernetes diagrams to share

Hi folks!

As a Kubernetes freshman, I've been looking into ways to customize it. And, to my surprise, I came to the conclusion that the Kubernetes API plays a very important role there. Custom resources seem to be a very good design decision because they can be manipulated in the same manner as any other built-in resources, such as Pods, Namespaces, or Services. However, the documentation is a bit bloated and the API structure is far from being trivial. So, I ended up drawing this diagram with Kubernetes API structure. Then I turned to the Operator Pattern because apparently custom resources without code actually have very little use. And found myself reading tons of vague articles full of some marketing speech. So, when I finally figured out that operators are simply Pods with custom controller logic watching and manipulating custom resources, I ended up making an animation of one of the operator's logic (or full-sized GIF, but be careful, it's 16 MB). Since operators are actually control loops, I thought that it might be easier to grasp the idea by looking at a dynamic visualization, not a static diagram. And finally, I also wrote an introductory but concrete article about the Operators Pattern with some useful (in my opinion) links in the end.

Sharing my findings because I hope it may safe time for people on a similar journey!

https://redd.it/l9gp5b
@r_devops
Getting into DevOps from full stack web - what, where and how

Hey there. While I work as full stack, I've also worked with gscripts, python, java, aws s3, and other technologies before. I was told that DevOps kind of includes different technologies and languages in order to deliver a product. To be fair, I don't know a lot. What I know is that AWS certification is very important however. So where would you recommend me to start? Some udemy course I presume? I saw plenty about AWS solutions and Kubernets, but honestly I have no idea what they mean by that, nor do I know what's Kubernets. I just want to know what the best possible road I can take to reach a satisfactory level in order to take the AWS exams, and even which one of those should I take. Thank you.

https://redd.it/l9fd0q
@r_devops
Production-grade Kubernetes (self-hosted)

Hi guys,

I have been looking for production ready kubernetes on self-hosted environment. But could not really find anything that was for my need or was a bit suspicious of the security.

So far, i have only used kubernetes on local machine using Docker Desktop, which was pretty straightforward to use.

I have onpremise server and hosted server on Hetzner. Both has ESXi running and ready to be used.

I know that 3 masters is recommended, but i have 2 for now, and that is unfortunately the budget for now.

What i am thinking is:

2 masters = 1x master on each server

2 nodes = 1x nodes on each server

How can i build such an environment?

https://redd.it/l9f3rt
@r_devops
News AWS Re:Invent Re:Cap 2020 - Summary of the Latest AWS Features

I made a summary of the latest features announced by AWS from the Re:Cap session.

You can read the summarized version below:

https://kevinvr.medium.com/aws-re-cap-2020-summary-latest-amazon-web-services-features-290a9d5de718

Which feature are you excited about? I particularly like the 1ms billing granularity for lambdas, but the mac1 instances are also quite neat (except the cost which I believe could be improved, as summarised).

https://redd.it/l9n4qg
@r_devops
AWS: How to see S3 Bucket request metrics, egress and costs

Vantage posted this blog post on how to enable S3 Bucket Request Metrics (and showing associated costs which is really nice) and thought the AWS folks here might appreciate it: https://www.vantage.sh/blog/how-to-see-s3-bucket-request-metrics-egress-and-costs

https://redd.it/l9qkcd
@r_devops
MongoDB - set replication to DocumentDB

We're setting up a local MongoDB cluster - Locally, we'll have one primary and one node, and we want to have another node in AWS. Is it possible to have that node as the DocumentDB service instead of an EC2 instace?

Also, I know I must have an odd number of total nodes, is it possible to first add one node and then add another one?
Thanks ahaed.

https://redd.it/l9a3jr
@r_devops
SSH to Dell switch without password

When I SSH to a dell switch, it asks me for a password. I'm trying to set the SSH keys so it wouldn't ask me for that but I just can't get it to work. I couldn't find anything useful online or in Dell's documentation.

How do I edit the authorized keys so that I could SSH without a password?

Also, when I log in, I have to type en to start the shell. Can I disable the need for that too?

Model is PowerConnect 6248.


Thanks ahaed.

https://redd.it/l9bsnz
@r_devops
Run Python Code in a Shell Script

I recently wrote a blog post on how to run Python directly from a shell script. As DevOps professionals, we oftentimes have to mix complexity that is best done in Python, but we don't want to manage or maintain multiple files for the script functionality.

This post shows how you can embed your Python code directly in your shell script. Also it can show some things you need to consider. And finally it explains how to debug that Python code!

https://redd.it/la4x3t
@r_devops
Monthly 'Getting into DevOps' thread - 2021/02

What is DevOps?

[AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

Books to Read

The Phoenix Project - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
[The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
Google's Site Reliability Engineering - Google engineers explain how they build, deploy, monitor, and maintain their systems.
[The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book
The Unicorn Project - the "sequel" to The Phoenix Project.
[DevOps for Dummies](https://www.amazon.com/DevOps-Dummies-Computer-Tech-ebook/dp/B07VXMLK3J/) - don't let the name fool you.

What Should I Learn?

Emily Wood's essay - why infrastructure as code is so important into today's world.
[2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
This comment by /u/mdaffin - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
[This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.
Roadmap.sh - Step by step guide for DevOps or any other Operations Role

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

Previous Threads
https://www.reddit.com/r/devops/comments/koijyu/monthlygettingintodevopsthread202101/

https://www.reddit.com/r/devops/comments/k4v7s0/monthlygettingintodevopsthread202012/

https://www.reddit.com/r/devops/comments/jmdce9/monthlygettingintodevopsthread202011/

https://www.reddit.com/r/devops/comments/j3i2p5/monthlygettingintodevopsthread202010/

https://www.reddit.com/r/devops/comments/ikf91l/monthlygettingintodevopsthread202009/

https://www.reddit.com/r/devops/comments/i1n8rz/monthlygettingintodevopsthread202008/

https://www.reddit.com/r/devops/comments/hjehb7/monthlygettingintodevopsthread202007/

https://www.reddit.com/r/devops/comments/gulrm9/monthlygettingintodevopsthread202006/

https://www.reddit.com/r/devops/comments/gbkqz9/monthlygettingintodevopsthread202005/

https://www.reddit.com/r/devops/comments/ft2fqb/monthlygettingintodevopsthread202004/

https://www.reddit.com/r/devops/comments/fc6ezw/monthlygettingintodevopsthread202003/

https://www.reddit.com/r/devops/comments/exfyhk/monthlygettingintodevopsthread2020012/


https://www.reddit.com/r/devops/comments/axcebk/monthlygettingintodevopsthread/

Please keep this on topic (as a reference for those new to devops).

https://redd.it/la7j8w
@r_devops
Do DevOps Engineers need to know how to code?

I'm starting to work in DevOps Engineering and want to value add... I'm reading, listening and learning fast and furiously.

I see people coming from solutions architect, software engineer, sys admin... how much does a devops engineer need coding skill?

https://redd.it/laf6um
@r_devops
Automated deployment of ssh public key to 1000s Windows nodes

We use Chef for config management of 1,000s Windows 2012/2016+ servers. Have a need to deploy a public key into C:\\Users\\<user>\\.ssh\\authorized_keys.

However, while refining the requirement I recall that during a server's initial configuration a local user's profile is not created, so the C:\\users\\<user> folder doesn't exist. If I create in code it won't be associated with the user.

I am interested in how people have managed this in their environments. Perhaps I have missed a subtlety of ssh.

Servers have WinSCP and OpenSSH installed.

The Supermarket cookbook https://supermarket.chef.io/cookbooks/ssh\_authorized\_keys does not support Windows.

Regards

https://redd.it/la7n8h
@r_devops
What's cooking in your Kubernetes namespace?

As Kubernetes platform engineers, have you ever wondered what all things are running in the various namespaces that you have provisioned for your product teams?

We have developed a kubectl plugins to help answer that question. Check it out:

https://cloudark.medium.com/whats-cooking-in-your-kubernetes-namespace-9200be114f8

https://redd.it/lakvn7
@r_devops