Managed databases vs self hosted for dev environments

Hi /r/devops,

While managed databases are awesome and "just work", there is an issue with them and that is, of course, pricing. I totally get the idea that it is a managed service and you are saving time and that should come with a price. But how are you dealing with this when it comes to your dev/qa environments?

In my case, we have 5 different projects running in single VMs (so 5 VMs per environment) and each VM is running postgres. Running database servers on VMs along with the project itself is quite inconvenient - our infra can't be immutable in that sense. To completely recreate a project in dev environment, I would have to first backup the database somewhere and restore it after VM recreation.

And so we get to my final point - how are you guys handling this at your companies? I would assume it's a problem regardless if you are running your projects in VMs or Kubernetes.

Could I perhaps run a single managed database SERVER per environment (not per project) and host multiple databases on it, for different projects? Is that something that would make sense? The cost wouldn't be that high and it would provide me with greater flexibility when it comes to recreating VMs. Is that something worth considering or is running a single database server for different projects bringing some caveats?

https://redd.it/ox2q8y
@r_devops

Regarding Cloud Native buildpacks

A while back one of the users of Devtron opened an issue regarding the support for Cloud Native Buildpacks - https://github.com/devtron-labs/devtron/issues/543. For context, Devtron is an Opensource SaaS-like experience for an End-to-End Kubernetes Software Delivery. The issue raised is to create a workflow to deploy apps using Buildpacks adding relevant templates, CI/CD pipelines on top.

I have a few questions on buildpacks, It would be great to know what you'll think of it.

So, Are you using Buildpacks to deploy applications? If yes, How are you using it, meaning, do you customize it? do you use it at scale or move out of it at scale? Does it have more utility in some languages compared to others?

As of now, we don’t support it but would want to prioritize integrating it into Devtron, if there are enough usecases. Would love to hear from you on this.

https://redd.it/ox6ekc
@r_devops

Preferred CI/CD tools?

Hello. I work as lone devops/server guy at a smallish company, and I was wondering what you guys use for CI/CD? I've looked a bit at teamcity, but that seems, lacking in features? I was thinking something like CircleCI and ArgoCD, as they seem quite cloud native, but yeah, idk.

https://redd.it/ox7pur
@r_devops

How do you add devops experience on your resume if you don’t get any at work?

I work as an Ops Engineer at a startup and I only get to to press the gitlab buttons for deployment. How do I really add devops experience on my resume so that I can transition from Ops to DevOps role.

https://redd.it/ox68f5
@r_devops

How many of you work in an organization that actually adopts newer technology vs switching jobs every 2-3 years to keep up? What does that process for evaluating new tech look like? Do you get promoted periodically to stay on board?

I've grown extremely tired of having to interview every couple of years to have to switch orgs just because a new technology comes along that ends up pushing you out of the market if you don't know it. Also, of having to switch for a promotion. At least the former issue is solved at the new company it would seem, but I'm wondering how many of you get reasonable promotions as well? I'm talking like 10-20 percent to stay instead of job hopping assuming your performance reviews are also stellar.

https://redd.it/ox7jgz
@r_devops

Show /r/devops: The Cloud Cost Handbook

Today Vantage is launching "The Cloud Cost Handbook" which can be found at https://handbook.vantage.sh/ - a set of guides and best practices related to cloud costs for developers, engineering leaders and finance counterparts. We're hoping distill complex cloud cost terms and concepts into a set of "plain english" guides that anyone can access.

From speaking with our customers and community we've found that cloud cost knowledge is typically "tribal knowledge" held by a few key people at an organization but not well known amongst broader community of developers.

We're still early and we're looking for contributions and support (feel free to give the repo a star!) to build this up over time - please let us know what you think!

The repo backing this is here: https://github.com/vantage-sh/handbook

https://redd.it/ox93zq
@r_devops

Changing the node type in an Amazon Elasticache Redis (w/no down time)

Changing the node type in an Amazon Elasticache Redis cluster (cluster mode enabled)

Does anyone know if the cluster knows how to handle the instance type change without any downtime or without flushing the data?

Has anyone ever tried this on a running production instance?

https://redd.it/ox9c2g
@r_devops

not sure where to begin with deploying Docker containers, but otherwise competent at programming.

Competent enough in programming to write an elaborate script and have it run on schedule to reliably execute various tasks, but not sure where to begin with deploying it using docker containers, and all that other devops good stuff.


So at my place, we're dependent on a lot of various scripts that run on a dedicated EC2 machine on a scheduled basis to execute various tasks.

These are .py tasks that just run on a windows instance.

We're trying to modernize how we do Ops, which involves bringing things like Docker/Kubernetes into the picture, but I'm largely a newbie in that regard. Is there any good resource that holds the hands of someone just through the devops specific bits, if they are competent enough in programming and know their domain enough to write reliable scripts.

Most sources I find online tend to assume a lot of existing knowledge, or are trying to push an ancillary product, etc.

https://redd.it/ox59ud
@r_devops

Jenkins linux master and windows worker node set up

I'm looking to offload some Jenkins builds to a Windows worker node. How is this typically done? If someone could point me in the direction of a good guide, I'd appreciate it. Thanks.

https://redd.it/oxehon
@r_devops

NSA, CISA release Kubernetes Hardening Guidance

https://www.nsa.gov/News-Features/Feature-Stories/Article-View/Article/2716980/nsa-cisa-release-kubernetes-hardening-guidance/

sub doesn't allow cross-posts, but saw this in /r/cybersecurity and it's def relevant

https://redd.it/oxg2zi
@r_devops

Do Github actions have an IP address?

I tried to connect to a server through SSH, but I kept getting I/O timeouts. Tried doing it through a VPN server and somehow it worked, so I am wondering what's the IP of the Github action. Is there a way to easily find out? Is it a static IP or does it change with every action? How do you whitelist the IPs? Do you just enter all of the possible IPs in the AWS whitelist?

https://redd.it/oxe8t6
@r_devops

Kbrew: Install any complex app on Kubernetes with one step!

https://github.com/kbrew-dev/kbrew

As a Kubernetes engineer I wanted something that would install any complex app in one command assuming the context of my environment. Please checkout, and would love feedback!

https://redd.it/ox4yly
@r_devops

DevOps Role - Now a days

Hi folks,

I dont know whether this is my understanding, Now a days the line between DevOps, SRE and Prod Support is getting thinner n thinner. When I have started working in DevOps area, there are DEVs, DevOps and Prod Support team, all 3 are having the different role to play. Now, Org are expecting DevOps to do the On Call Support and Play role of SREs in case of issue.

What is your take on this?

https://redd.it/oxjsxe
@r_devops

What are some open source container orchestration tool?

Azure Kubernetes service is a good tool but it’s costly.
Is there any open source and reliable orchestration tool?

https://redd.it/oxegkg
@r_devops

Can you point the host to a microtik server in a Github action?

name: staging
on:
push:
branches:
- 'gh-pages'
jobs:
docker-compose-down:
runs-on: ubuntu-latest
steps:
- name: compose-down
uses: appleboy/ssh-action@master
with:
USERNAME: admin
HOST: vpn.presto.com
KEY: ${{ secrets.MY_KEY }}
port: 2247
script: |
cd /github.com/mydashboard
docker-compose down

Ok, so this Github action no longer works, and I am thinking it's because the DNS config for [vpn.presto.com](https://vpn.presto.com) was changed. If I ssh into it, it's a Microtik server and so I can't do any linux command like cd and ls. The reason why I think there was a DNS change is because in the Github action config you see ubuntu-latest when a Microtik server is a routing server that's not Linux based, or at least it seems so. Also, I can't find the folder /github.com/mydashboard inside of it.

https://redd.it/ox6ab0
@r_devops

Banzai Logging operator : Fluentbit > Fluentd > Loki 'entry out of order'

Hi Everyone,
Has anyone faced this issue with FluentD and Loki ?


Tried adjusting buffering time, timestamp, flush interval, upgrading fluent bit to 1.8.3 , upgrading Loki to version in master and etc


Nothing helps, part of the logs is lost in "out of order" and doesn't get pushed. From time to time, logs stop flowing into the Loki
Any ideas are welcome :)






2021-0x-0x 17:00:17 +0000 warn: #0 clusterflow:logging:all-logs-to-loki:clusteroutput:logging:loki-logging-output failed to write post to https://loki.xxx:3100/loki/api/v1/push (400 Bad Request entry with timestamp 2021-08-03 16:22:28.463529755 +0000 UTC ignored, reason: 'entry out of order' for stream: {, format="json",entry with timestamp 2021-08-03 16:52:28.463548525 +0000 UTC ignored, reason: 'entry out of order' for stream:

https://redd.it/ox82oc
@r_devops

Alternative to CD based on ssh

So, your CI creates artifacts and store them somewhere (e.g. Cloud storage), if you don't want the CI/CD tool to directly trigger the deployment via ssh, what are the alternatives? I can guess something like a cron job or Puppet can do it, but I'm curious to see what else is used (beside k8s-based approach).

https://redd.it/ox69en
@r_devops

Advice for a Junior DevOps Engineer.

Hey Everyone! I am 31 years old Junior DevOps Engineer (I know , I started the tech career pretty late).
Currently I am interviewing to a Senior role where I will be reporting to CTO. It will be expected a lot from me.
I am 90% sure that I passed the interview and they are going to give me an offer. And they are moving very fast.

My questions are: should I reject that offer even though it is a good opportunity to grow?

Yes, I admit. My resume looks good on a paper. However, I am not that good as my resume states.

Should I just reject the offer and save myself from an embarrassment?

Or they will accept me as I am and let grow in the company?

Thanks in advance!

https://redd.it/oxnog7
@r_devops

What do you think a Cloud/DevOps Architect needs to master?

As the title says, in your opinion what are the skills and knowledge to become a good Cloud/DevOps Architect. Both hard and soft skills

https://redd.it/oxo3ne
@r_devops

Ansible - Create a role that includes other roles

# SOLVED:


In `meta/main.yml` I can just have the following:

---
dependencies:
- role: role1
- role: role2


_______________________

I have organized all my playbooks into roles. Now that I have all the roles, I'm trying to create a new role that would include all the other roles.

In Ansible, is it possible to create a role that just calls other roles? If so, is it possible to do it as a list, like in a playbook:

---
- hosts: webservers
roles:
- role1
- role2
- role3

Thanks ahead.

https://redd.it/oxp1u8
@r_devops

Alerts Based on Dashboard



Hello Community,

I need some guidance on setting up the alerts, we have the 2 replicas of an application running in a namespace for which I need to set up alerts for the following conditions met,

At 60% threshold, a warning should be triggered

At 75% threshold, an high severity should be triggered

Please note we are fine with using Grafana alerting system, and I understand we need to first create a dashboard to set up alerts in Grafana. What is the query to create the dashboards for a specific pod and namespace cpu and memory utilization?

Thanks,

https://redd.it/oxnrxi
@r_devops