CircleCI vs Jenkins

Just wanted to know if CircleCI was offering something new that Jenkins hasn't offered before??

https://redd.it/lklsrn
@r_devops

Estimate cost per user for cloud infrastructure

Lets say that I have a product which allows users to sign up and sell personal services (think craigslist..etc). How do I estimate cost per user, cost per month..etc.

Basically, I am trying to figure out, how much should I be charging user so that I am atleast breaking even, and I want this trend to scale with users.

Is there a formula or other tools that allow to do this end to end cost estimation and growth simulation ?

https://redd.it/lko5ym
@r_devops

K8s & Consul - Avoid cross-DC/AZ traffic

I have huge app in Kubernetes with about 90 containers on 5 k8s worker nodes. Nodes are spread between Availability Zones in AWS, like that:

node1 - AZ-A
node2 - AZ-A
node3 - AZ-B
node4 - AZ-B
node5 - AZ-C

Also, I got many small apps which use API from that huge app. All apps use external/public host of that app, so actually all traffic between apps goes through ELB.

I can use internal Kubernetes hostnames (now we got coreDNS), but I would like to avoid cross-zone traffic, so how can I implement consul to be AZ aware and prefer traffic between pods in the same AZ, e.g.

Diagram

Something similar but with HAProxy and Redis:

https://medium.com/helpshift-engineering/saving-inter-zone-transfer-costs-in-ec2-with-haproxy-c4217ac2ca04

https://redd.it/lkkeat
@r_devops

Where to get help?

Where to get help if you get stuck using or learning devops solutions? For example: Ansible, Jenkins...etc

The reason I am asking is for the past 2 days, Ansible wouldn't run playbooks for me. It took me 2 days to figure out the problem Which is that i installed the needed libraries for python3 while ansible was running on python2.

https://redd.it/lkkdtd
@r_devops

Azure DevOps Demo Generator

Just starting poking around here in this subreddit and I suspect I may be spending a lot of time digging around in here the next few months. :) I just stumbled into this Azure DevOps Demo Generator article from a few days ago and thought I'd share it in case anyone else finds it helpful.

The Azure DevOps Demo Generator | Azure DevOps Blog (microsoft.com)

Also just a side note, I'm aware Microsoft's unfortunate naming of "Azure DevOps" the tool is confusing along side general "DevOps" concepts of technologies LOL...but if you're someone in Microsoft's bubble, it's like a tool you'll likely end up running into.

https://redd.it/lky7hg
@r_devops

KubeHelper - simplify many daily Kubernetes tasks through a web interface

KubeHelper \- simplifies many daily cluster tasks through a web interface. You can easily do search, analysis, run commands, cron jobs, reports, filters, git synchronization and many more.

KubeHelper is not yet another attempt to display the Kubernetes API in a GUI. Not an attempt to replace Lens, the official Dashboard, or other similar products. This is my humble contribution to the Kubernetes Open Source community. KubeHelper does not have any highly specialized direction and contains quite a few different functions that will be useful in daily work with Kubernetes.

Read full article on Medium

GitHub

https://redd.it/lkk90p
@r_devops

Polling an API or MYSQL query to do alerting and monitoring?

Is anybody using polling of an API or MySQL query to make sure everything is normal?

Like alerting if the number of failed transactions in a time window is greater than the threshold, or if the API response has a field with a value beyond acceptable limits.

What tools do you use to do such alerting and monitoring?

https://redd.it/lkzkiv
@r_devops

Does anyone else have experience running spinnaker as their main CD strategy?

Does anyone else have experience running spinnaker as their main CD strategy?

https://redd.it/lkjje7
@r_devops

Create signed certificates

My CTO sent me a zip containing a .crt, .csr, .key and a .ca-bundle.

I was under the impression that I could use them to created signed certificates for use in nginx (or similar) for using https on our various subdomains.

I'm unsuccessfully trying something like the following

ROOTCAKEY=.mydomain.com2020.key
ROOTCA=.mydomain.com2020.crt
ROOTCSR=.mydomain.com2020.csr

openssl genrsa -out auth-proxy.api.mydomain.com.key 2048

openssl req -new -sha256 -key auth-proxy.api.mydomain.com.key -subj "/C=US/ST=CA/O=MyOrg, Inc./CN=auth-proxy.api.mydomain.com" -out auth-proxy.api.mydomain.com.key.csr

openssl x509 -signkey $ROOTCAKEY -in auth-proxy.api.mydomain.com.key.csr -req -days 365 -out auth-proxy.api.mydomain.com.crt

#openssl x509 -req -in auth-proxy.api.mydomain.com.key.csr -CA $ROOTCA -CAkey $ROOTCAKEY -CAcreateserial -out auth-proxy.api.mydomain.com.crt -days 500 -sha256
#echo crt=$(base64 -w 0 auth-proxy.api.mydomain.com.crt)
#echo
#echo key=$(base64 -w 0 auth-proxy.api.mydomain.com.key)

Can anyone here point me in the right direction?

https://redd.it/lkgvll
@r_devops

Kubernetes README

https://kubernetesreadme.com/

https://redd.it/ll7t5p
@r_devops

Multi tenant platform for running docker containers

Guys, i'm looking for open source platform for my users to sign up, request pool of resources and then run containers in each own pool. I have a cluster of virtual machines i want to split between my users (even multiple users on same VM).

I tried kubernetes+dashboard with RBAC/namespace resource allocation. While it perfectly limits cpu/memory usage, but couldn't limit disk space usage per namespace. Even with small persistent volume limits i could open container and create 10 Gb files inside without any restriction. Besides some malefactor can possible mess up cluster for every1 else.

My other idea was to run cloudstack to spin up small VMs for each user, but my CPUs don't have virtualization enabled.

I like how Loft is using virtual k3s clusters, but was hoping to find free alternative.

Lxd and kata containers looks interesting. I hope to find open source service to manage cluster preferably with web-ui. Or any other solution that comes to mind.

https://redd.it/ll7cz0
@r_devops

Need help understanding the concept of stateful serverless.

Hi all,


I'm trying to understand how tools, like cloudstate, from an architecture meaning.
I get it why FAAS are no good for working with data because you need to call the database every time that the function runs and it's gonna take some time.
But I'm struggling to understand how this tool is working that you can have both serverless architecture and stateful functions?


Thanks!

https://redd.it/llb6gg
@r_devops

Red Hat and IBM now have a Linux and Private Cloud Administration Specialization on Coursera.

If you're a Coursera subscriber and you're interested: https://www.coursera.org/specializations/linux-private-cloud-administration-power-systems

https://redd.it/llaop6
@r_devops

Scaling up with serverless computing

The rise of serverless computing is an interesting trend, but many teams still don't quite know why, how and when to use it.

My friend took part in quite a few projects based on the framework and wrote a case study that answers just these questions. In this you will learn why serverless is perfect for the service-based architecture, prototypes or MVPs, but not quite the best one for monoliths. Give it a try and let me know what you think, I can pass questions to the author.

https://redd.it/lla5ji
@r_devops

problems with spinnaker artifacts in gcp

Good morning everyone, someone with spinnaker experience, I am having trouble defining the artifacts to be able to deploy in a gke cluster, I did the installation provided by Google in gcp, and I am trying to get the artifacts (in this case manifest from k8s) from two different places, first from a gcp bucket and second from GitHub In both cases I get the same error, that the object cannot be resolved.

error for github artifact:

Failed on startup: Unmatched expected artifact ExpectedArtifact(matchArtifact=Artifact(type=github/file, customKind=false, name=k8s/deploy.yml, version=null, location=null, reference=null, metadata={id=8bed9f1f-0806-445d-9c14-f661c568922c}, artifactAccount=c-bordon, provenance=null, uuid=null), usePriorArtifact=false, useDefaultArtifact=false, defaultArtifact=Artifact(type=custom/object, customKind=true, name=null, version=null, location=null, reference=null, metadata={id=7dc222d9-1103-4564-95fc-0eebf0f4fcbd}, artifactAccount=custom-artifact, provenance=null, uuid=null), id=73cd74b1-4089-4a65-833c-1cb917414f51, boundArtifact=null) could not be resolved.

error for gcp bucket:

Failed on startup: Unmatched expected artifact ExpectedArtifact(matchArtifact=Artifact(type=gcs/object, customKind=false, name=gs://demo-pipeline-1/deploy.yml, version=null, location=null, reference=null, metadata={id=10025080-76c4-4926-96bb-75ca24f32262}, artifactAccount=gcs-account, provenance=null, uuid=null), usePriorArtifact=false, useDefaultArtifact=false, defaultArtifact=Artifact(type=custom/object, customKind=true, name=null, version=null, location=null, reference=null, metadata={id=7dc222d9-1103-4564-95fc-0eebf0f4fcbd}, artifactAccount=custom-artifact, provenance=null, uuid=null), id=73cd74b1-4089-4a65-833c-1cb917414f51, boundArtifact=null) could not be resolved.

​

I would appreciate any help

https://redd.it/ll6amj
@r_devops

How do you secure credentials when operating CI/CD?

I know a lot of people are happy to just pass your credentials to GitHub, Circle CI etc. This is fine depending on your risk profile but those credentials often have near admin access to your cloud account.

For those of you who are not happy with providing that level of trust to a third party what do you do? Does anyone know how someone like AWS manages their credentials with CI/CD?

https://redd.it/ll0yfd
@r_devops

Sensu

Is anyone using Sensu for monitoring? It has all the DevOps concepts that I like to see, but just don't hear that much about it. Everything is New Relic or Datadog..

https://redd.it/lll6m1
@r_devops

Serverless infrastructure

Hello all,

I’m looking into learning a bit more about the underlying technology under serverless architecture.

How would a provider approach the solution, load balancers, reverse proxies, and finally the application web server?

Thank you 🙏🏽

https://redd.it/lll3wj
@r_devops

Project Manager looking for Feedback

I am currently a waterfall based project manager who is learning more modern agile project management methodology & DevOps frameworks. The feedback I am seeking is – Is there value to you as a developer if the PM or the project leader knew or was familiar with programming?

I am under no delusion that I would be able to replace a developer nor do I have the desire to. I just want to know the best way to be part of the team.

*I am thinking about taking two Code Academy Career Paths (Code Foundations Computer Science) for those who are interested.

https://redd.it/lllgzn
@r_devops

Does AWS ECS add a price overhead if you don't use Fargate?

If you manage your own ECS cluster on EC2, then is there a price overhead from using ECS?

Also:

\- How does the pricing of Fargate compare to the pricing of EC2 these days?

\- How does the pricing of Fargate with spot instances compare to the pricing of EC2 (no spot instances) these days?

Thanks!

https://redd.it/llilj6
@r_devops

CI/CD Pipeline For Library + Backend Server

Hey all, I'm fairly new to DevOps and I am curious what my options are for the following scenario.

​

TLDR; How do you setup a CICD pipeline that builds two different repos, where one is dependent on the other, and it can handle the situation where you need to push out code to both at the same time?

​

To start off, I have a library A and then a backend server B that depends on A. The code for A and B are maintained in separate git repos. I want to setup a CI/CD pipeline so that whenever I push out changes to A and it successfully builds, it will trigger B to automatically rebuild using the new version of A. Likewise, if I push out new changes to B, it will automatically grab the latest version of A and rebuild itself. I think this is a fairly typical situation. I'm using CircleCI and have an idea of how to set this up in that environment.

​

Here's what I'm curious about though. What happens when I need to make changes to both A and B, and then push out changes to both repos back to back? I will end up with some weird synchronization issues if I followed the setup described above. For example, if I open a pull request for A and trigger the pipeline, it will attempt to build the previous version of B since the current version is probably still sitting in its own pull request that has yet to be merged. On top of this, I would really only want to build once, and the above setup I described would cause the server to be built twice assuming everything else worked out.

​

Is there a common way to deal with this issue? Hopefully I have explained it well enough. I appreciate any feedback or thoughts you can give me!

​

For some extra context, I'm working with Java using Gradle as my build tool. Both the library and server are built on Spring.

https://redd.it/llhf02
@r_devops