lOOKING FOR A GIT SAAS IN EU (WITH FAILOVER)

I'm a bit confused that this doesn't seem to exist? Are my Google-fu skills just rusty?

We want to spend money to not host our own Gitlab so first we wanted to move to .com (US only)

https://www.stackhero.io/en/services/GitLab - Preview and no failover/HA

https://gitlabhost.com/ - Single tenant

Bitbucket / Github etc also all are US.

(Sorry for CapsLock title)

https://redd.it/kx2eor
@r_devops

Bit annoyed with everyone thinking their way is better than our current process (contractors, new starters etc.)

This is a trend I've noticed a lot over the years and frankly, does my nut in. Anytime anyone joins a project, they nit-pick. It might be subtle... 'oh... Why did you choose to do it this way?' or just blatant rude.. 'ah well I'd have used terraform cloud for this instead of managing statefiles in this way'. Would you? Glad you're a fucking psychic who can tell my bosses and BA's requirements from a single look at a repo!


First of all, code is 50% more difficult to read than write. That's a proven established fact. If my scripts look over complicated to you because you didn't write them. That's probably why.


And okay great you wrote this fantastic yaml pipeline which does a build in an amazing way, but is that way relevant to my use case?


And fine, my terraform modules are a little bit messy now. This is 18 months into a cloud transformation project and frankly sometimes you have to sacrifice code cleanliness for functionality because you need to get it working and the provider you use doesn't have the updated options, and changing provider will give you weeks of work, and you're a month into the crunch and things aren't going to schedule and that's why you've hired a contractor in the first god damned place.


In a perfect world we're all perfect engineers, but come into a project and understand that you're coming into something that's been on a journey you'll never quite understand fully. Come into it with an open mind and think about that time where you've been stuck and had to throw in a hack to make it just bloody work this time.


Stop being a judgemental prick please.

...Sorry.... Been a long few weeks.

https://redd.it/kxr39t
@r_devops

QUESTION Testing PRs across multiple repositories in a single build?

Current setup:

SCM Manager: GitLab

Build Server: Jenkins

Branching Model: Trunk-Based Development (in each repository)

I have the challenge of figuring out how to set up a way to trigger builds on Jenkins when PRs are created for multiple repositories. While this in itself is not a challenge, I want to be able to test them in a single run on Jenkins, since some of the repositories are tightly-coupled in such a way that our developers often work in multiple repositories to fix a bug / develop a new feature.

This leads to the issue. How can I create a PR on each repository and then be able to fetch those in the Jenkins build? I can set up a quiet period in Jenkins so the build waits for a duration before building - This should be enough time to create the PRs in all the different repositories.

By the looks of it, GitLab assigns each PR an incrementing number as name. These PR names might not always match, so I was wondering if there is a way to do some pattern matching to get the branches created from the different branches (labels, created by developer)?

Furthermore, is this even the right approach to take? We are currently on the steps of setting up versioning for each repository, but versioning the repositories will also slow down development time, since we'll have to wait for the packages being available before even being able to test if they work properly in other repositories.

https://redd.it/kx1nw4
@r_devops

How to run AWS CIS compliance and other checks with SQL and CloudQuery

I saw some interest on cloud compliance from a post yesterday on commercial product so I wrote a short blog how to run various compliance checks (CIS) with open-source tools and SQL - https://cloudquery.io/blog/how-to-aws-cis-compliance

https://redd.it/kxwsaj
@r_devops

Sometimes it's notjust technical stuff that needs devops

Having a technical idea is grand but being able to express that idea in business talk is still necessary.

This clicked for me today when realizing that all my top notch ideas were getting swept under because I was not properly laying out my arguments in a way that had a paper trail or was not to the right stakeholders.

One must be able to articulate these ideas and have workflow in place that make collaboration easy.

I'm looking tomove forward with project management software that has auto triggers in slack and deadlines so accountability and automation work hand in hand.

https://redd.it/kxz3ce
@r_devops

[Architcture] Multiples databases to query (Mongo)

Hello guys, we have several **mongo** instances (3, 4), with product data related on each of the instance. We would like to query 1 ID, or severals IDs, and get all its data across different instances and databases.

I was thinking to a kind of "GraphQL" Schema, where the app requires a product and the attributes it needs with ID by HTTP request. And then a "middleware service" does the multiple mongo query stuff. *Does it exist already? If not, how would you implement it ? :)*


Example:
An app web needs data on the number 156789, and wants to collect data from different databases where it matches the number 156789.
For a fruit could be: "give me data related to chemical attributes, reviews from people of this fruit, and basic info (name, color, average size, etc)" to the fruit number 156789. And via HTTP request a service would handle everything to give back the data.


GET fruit/345678

{
basic_info, chemicals, reviews
}

I am curious about any ideas or ways to handle this :)

https://redd.it/kxxs2e
@r_devops

Azure Pipelines Restore packages along side solution

When using the Nuget Restore task, what do i write in the Destination directory to make it restore the packages in a packages folder along side the solution.

I am asking this because, for some reason for one of our project, the Nuget Pack fails to find the packages when they are one folder above the solution. using the destination directory as "..\\packages"

what do i write to get it to go along side the solution.

is there a predefined variable for this.

​

i cannot hard code the path as i have it restore multiple solutions in multiple folders.

preferable a solution like this if it exists "$(Build.SolutionDirectory)/packages" but targeting the solution that is currently being restored.

https://redd.it/kxwwx8
@r_devops

Looking for tech blogs in the video game industry

Hey there, tittle pretty much says all.

Any recommandations about tech blogs? Programing, coding, AI, anything tech related to video games! Thanks!

https://redd.it/ky1yln
@r_devops

How do you manage your Github/Gitlab, etc?

I have a couple of people who are asking for private repo's in our Github, while the base permissions are "read" on every repo we have in our Github (which every repo is private to everyone outside our org).

​

My question is, what are you all currently doing? Do you have it set to where everyone can see every repo for max collaboration? Do you have repos siloed off to individual teams?

https://redd.it/kxwema
@r_devops

Hardening Docker and Kubernetes with seccomp

Hi /r/devops,

Container security is often overlooked topic, so I wrote an article that shows how you can improve security of workloads running in Docker or Kubernetes by leveraging seccomp profiles, which I think might be useful to some you who are interested in security topics.

So, here's link to the article: https://itnext.io/hardening-docker-and-kubernetes-with-seccomp-a88b1b4e2111

https://redd.it/kxqrke
@r_devops

Pulsar vs RabbitMQ

Hey all,

I'm apart of a small dev shop and we're looking at different message/queue services. Primarily we're looking at Apache Pulsar and RabbitMQ. I was hoping to get some feedback on what you guys think!


My main concern is operational overhead, we have a 1-2 man ops team, so ease of use is the most important. It's hard for me to tell which would be better suited, Pulsar is CNCF so it is built to be run on the cloud, but RabbitMQ has an officially supported k8s operator so it feels like that's a moot point.


Thoughts?

https://redd.it/ky52vk
@r_devops

Alert response automation vs systemic root-cause deep-dives

Background: Ex-Amazon turned first time founder building in the DevOps/SRE space.

I posted here a couple months ago and had some really good chats with experts and learned a lot about where the industry is regarding alert response automation. There seems to be a lot of unsolved pain here because we generated quite a bit of interest.

We've expanded our value proposition to not only reduce toil and MTTR, but also help uncover the systemic root-causes and contributing factors behind alerts. To address both the immediate pain and tackle the toil at the root.

Would anyone here be willing to jump on a quick call to share thoughts, war stories, and brainstorm with me?

Thanks in advance

https://redd.it/ky5eat
@r_devops

Monitoring iot devices and servers

For our company, we're looking into adding monitoring to our fleet of IoT devices and backend servers. We would like this monitoring to be exposed to customers through our own web portal.

In a previous job, we used Zabbix to monitor all the equipment we had and I'm very familiar with it. I played around a bit and although I can get everything I need out of it (monitoring wise), I wouldn't know where to begin to expose this to our website. Maybe I could add my own webhook under media types?

On the other hand, all our devices are also using MQTT for communication so maybe it would be better to use that instead?

I would prefer using a tried and tested package for monitoring because it would save a lot on development time.

Suggestions?

https://redd.it/ky1y7c
@r_devops

Performance Benchmarks on Pull Request

We've become pretty big fans of Github Actions at our company and our latest blog post describes a pretty cool feature we've launched for our developers that gives them performance benchmarks on pull requests https://www.dolthub.com/blog/2021-01-15-performance-benchmarks-on-pull-request/

https://redd.it/ky1eym
@r_devops

php ci/cd flow

Hi guys, first post here.

So, I have previously setup a java cd/cd pipeline with this flow.

git->bitbucket->maven->sonarqube->Artifactory->ansible->dockerhub->kubernetes

For a php pipeline, would it be much different. I'm not a php programmer so not sure how to design the flow.

Also is php built with a tool like maven or does one just copy the php files into an apache document root thereby eliminating the maven, gradle build tool.

Thank you in advance

Brian

https://redd.it/ky0cf9
@r_devops

Open source cmdb

I'm looking a lightweight cmdb solution to store very basic data of servers, vm's and k8s clusters. Like, mac, ip, custom tags and so. I need api to access the data and possibility to add custom fields to it. Any recommendations?

https://redd.it/ky08ac
@r_devops

What’s the best way to create an ansible user account on a fleet of Linux servers?

If you’re getting started in using Ansible in managing 20-30 Linux servers, what’s the best way to create and manage the ansible account on each server for SSH access?

https://redd.it/kxmr4a
@r_devops

Looking for a conference talk video on monitoring/graphing

I saw a video in early 2020 some time, which was talking about how to handle monitoring / graphing of metrics. There was an example given of an API or web service which had an update pushed, the mean response time went down but a histogram showed the majority of requests were slower, with some evens being far quicker.

We were talking about histograms in the office, and I mentioned that video and that I'd find it and share the link, but I can't for the life of me find it now. Hoping someone knows what I'm talking about and has it bookmarked and can link me to it!

https://redd.it/kydepf
@r_devops

Should library modules use terraform.workspace directly?

First, let me get nomenclature out of the way:

A terraform "root" module is a directory containing .tf files where one can run various terraform action commands (such as plan, apply and destroy) and expect them to do something.
A terraform "library" module is a similar directory, but not all inputs have been provided, so terraform action commands aren't expected to do anything other than interactively prompt for missing values.
Root modules often reference library modules by using the "module" keyword.
I'm sure that hashicorp offers more precise definitions, but I think this is close enough.

Now the question: when developing a library module, which implies use by one or more root modules, is it bad practice to refer to ${terraform.workspace} directly? The alternative is to define a variable for each usage where your library namespaces resources, and leave it up to the root module developer to optionally include the workspace in the values passed to the library module.

View Poll

https://redd.it/kxlb2u
@r_devops

What is the best CI if I need cloud gpu runners and on prem self-hosted runners?

I'm looking for advice on a CI system that could support my use cases. 1) I have a number of build jobs that will need to run on nvidia-gpu instances to test a deep learning powered app. 2) I need to run some tests on specialized hardware that is on prem (arm64 Linux). 3) I also have regular Linux builds to support, but those can be on typical cpu instances.

So far we've been using a Jenkins solution, but it has been unreliable of late and I'm looking to replace it with something more modern, secure and easy to maintain.

From my search, none of the managed offerings seem to match my needs without some big caveat like self-hosted runners push you into a crazy pricing plan (circleci) or gpu nodes are non standard and require workarounds (azure pipelines). Gitlab seems like it could work but the stack is quite involved and I'm not super clear on whether I can have gpu nodes handled with scaling and on prem runners.

We use GitHub for vcs.

Has anyone had success using GitHub self-hosted runners for on prem builds? I feel like this could solve things for me if paired with a ci solution that provides the gpu nodes as a simple choice in the job definition yaml. I'm considering GH self-hosted runner + AWS CodeBuild (since it allows me to pick a gpu or beefy cpu machine at fair prices and without any workarounds), despite what I feel is a much worse user experience on the visibility front.


Thoughts?

https://redd.it/kyopgv
@r_devops

Reka: a cloud infra tool reaper

Hello Guys,
Checkout this tool I'm currently still working on. A tool to help you manage infrastructural resources on your cloud provider. You can easily stop and resume resources, clean up unused resources and destroy resources based on info about that resource. It's a common issue to have orphaned resources from say test environments. This tool helps to curb that. It could also prove useful for cost management. Say stop your instances and resume them when you want to. You could set a cronjob with your config to run it. It's still under active development, I'd appreciate any help.
https://github.com/mensaah/reka

https://redd.it/kyo8c4
@r_devops