Tool to deploy static sites from Github repo into AWS account

Hey friends of /r/DevOps. I wanted to share a free tool that we built that connects up your Github repo and your AWS account and deploys static sites.

This came from something my team and I personally experienced - infrastructure is annoying to deal. You need to spend a lot of time and effort to set up the workflow and processes required to build, deploy, and monitor your infrastructure. At the end of the day, as a developer first, you just want to write code and not worry about the underlying infrastructure.

I think tools like Netlify and Heroku are great but at the end of the day, you never have the truly visibility and ownership. With what we built, you have full ownership of the infrastructure and if you decide to leave us, your stuff will still stay running.

This is our super early beta MVP. We're really looking for feedback and opinions.

Oh and here is a quick demo video in case you don't want to go through the hassle.

Thanks and looking forward to some comments!

https://redd.it/lmolv9
@r_devops

Curated GitHub repository on how organisations around the world use AWS

Curated GitHub repository on how organisations around the world use AWS

I’m curating engineering blog posts, videos and other cool resources on AWS

https://github.com/upgundecha/howtheyaws

https://redd.it/lmpu7g
@r_devops

Why not use Kube API and GitOps to manage infrastructure?

Kube API and GitOps are some of the major advances in the industry we got recently.
Yet, most Infrastructure as Code (IaC) types of tools do not embrace them, at least not fully.
Except, as far as I am aware of, Crossplane, combined with GitOps-type of tools like Argo CD and Flux.
Here's a quick overview.
\>>> https://youtu.be/n8KjVmuHm7A
What do you think? Can Crossplane be a blueprint for the next-gen IaC tools?

https://redd.it/lmqtua
@r_devops

Cutting Build Time in Half with Docker's Buildx Kubernetes Driver

Hey friends, I wanted to share with you all how we've recently undertook a project to revisit our build infrastructure. Long story short, we were able to cut our build times in half by migrating to Docker's Buildx Kubernetes driver. Here's a link to a blog article my peer wrote that fully details what we tried and what we found to be successful.

For the next few hours, we're hanging out in zoom to demo some of this or field any questions. (Check the bottom right of the page.) If you're free, stop by to hang out and talk Kubernetes, Docker, devops or whatever with us!

https://redd.it/lmt9rr
@r_devops

Cluster Infrastructure with Docker Swarm, Docker Compose, and Traefik

Hey, I thought I would share 2 articles I wrote about Docker Swarm, Docker Compose, and Traefik recently:

Container orchestration with Docker Swarm: https://juliensalinas.com/en/container-orchestration-docker-swarm-nlpcloud/

Routing requests to the right service with Traefik: https://juliensalinas.com/en/traefik-reverse-proxy-docker-compose-docker-swarm-nlpcloud/

I'm basically talking about how we're doing things behind the hood at NLP Cloud, where each spaCy NLP model is running inside its own container.

I hope some of you will find these posts useful.

https://redd.it/lmo6h0
@r_devops

Architecting with Google Kubernetes Engine

Colleagues, the Architecting with Google Kubernetes Engine specialization will teach you how to implement GKE solutions including building, scheduling, load balancing, and monitoring workloads, as well as providing for discovery of services, managing role-based access control and security, and providing persistent storage to these applications. It incorporates hands-on labs using our Qwiklabs platform.Gain high-demand skill in Google Compute Engine, Google App Engine (GAE), Google Cloud Platform and Cloud Computing.Training modules includes: 1) Google Cloud Platform Fundamentals: Core Infrastructure: Learn the computing and storage services available in Google Cloud Platform, including Google App Engine, Google Compute Engine, Google Kubernetes Engine, Google Cloud Storage, Google Cloud SQL, and BigQuery, 2) Architecting with Google Kubernetes Engine: Foundations: Review of the layout and principles of Google Cloud Platform, followed by an introduction to creating and managing software containers and an introduction to the architecture of Kubernetes, 3) Architecting with Google Kubernetes Engine: Workloads: learn to perform Kubernetes operations; creating and managing deployments; the tools of GKE networking; and how to give your Kubernetes workloads persistent storage, and 4) Architecting with Google Kubernetes Engine: Production: Learn about Kubernetes and Google Kubernetes Engine (GKE) security; logging and monitoring; and using GCP managed storage and database services from within GKE.

Enroll today (individuals & teams): https://tinyurl.com/y82vmyun

Much career success, Lawrence E. Wilson - Online Learning Central (https://tinyurl.com/gh8n9uq7)

https://redd.it/lmvbxv
@r_devops

Accessing large amount of files via distributed file system and kubernetes cluster

Hi,

I recently stumbled across this awesome subreddit and wanted to ask for some advice.

The main question for me is: Which distributed file systems would be suited to store large amount of small files in a kubernetes cluster? I saw that you can dump for example wikipedia articles and have around 2TB of data and around 5mio files Id like to get access to via a distributed filesystem and maybe do a small web-app in which I can search over all the files or upload new files.

I try to understand the topic of big data and dfs a bit more and am not a professional working in devops rn, I study cs and just got interested in that topic. I started to read more papers about infrastructures like hdfs on hadoop for example and that larger amount of files can cause performance issues since each file will be stored in individual blocks ranging from 128MB to 256MB in size and since namenode’s memory is limited it could be a problem, if I understood that right.

I found distributed filesystems like hdfs, ceph, lustre or filetypes like hdf5 but have to check these more out to write about that tbh.

Id love to hear of your experience with maybe a similar topic and like to learn from your guys experience.

have a great day

https://redd.it/lmsnhj
@r_devops

Service desk to dev ops

Hello guys I'm currently working in service desk as the title says but I want to get into devops and the cloud. I recently started studying for az-900 but just found it it's basically for sales people. I'd say I do understand the core fundements but I'd say I don't think I should take it because of money issues and I'd rather pay for a more helpful cert. Just wanted to know what kind of certs I can get and which types of roles I can move to step by step before I get to a devops role. I also only have a work laptop where I can't download nothing on or run vms or practice Linux so I feel like that might be something holding me back. Could someone shed some light plss

https://redd.it/lmxt2h
@r_devops

Jenkins - trying MSBuild plugin to build .net project in a linux system throws fatal error.

Hello everyone. I was hoping you guys could help.

I have Jenkins (version 2.263.4-1.1) installed on a CentOs 7 box. I need to build a project that is coded in visual studio (C#) and am trying to achieve this by using the MSBuild plugin. Every time I try to build I get the following error.

1. FATAL: Unable to use this plugin on this kind of operation system

I've pointed jenkins to the msbuild location `/var/lib/jenkins/plugins/msbuild/META-INF/maven/org.jenkins-ci.plugins/msbuild`

If I don't point jenkins to the msbuild location I get a "missing msBuild.exe" error.

I followed a few other tutorials and installed dotnet and nuget along with some SDK packages per the tutorial recommendations. but still, my project fails to build.

Is it because Jenkins is in a linux environement? I've been struggling with this for a month now. any help is greatly appreciated.

https://redd.it/lmx4ud
@r_devops

Using AWS CLI with Google apps Saml login

Hey guys.
In my organization users are logging in via Gsuite with Saml.

Therefore they don't have access key and so on.


How can they Access aws CLI easily?


Please assist.

https://redd.it/lmsixp
@r_devops

Integrating github Actions with gopaddle for a seamless CI/CD on Kubernetes

Continuous Integration (CI) and Rolling Update to a kubernetes service can be achieved seamlessly when using gopaddle Deck and Propeller together. However, in some scenarios, you may choose to use a different tool/platform in place of Deck to build the docker images and use Propeller deploying and managing the applications. For example, you may choose to use github Actions and then integrate with Propeller for rolling updates. Read more to know how to use github actions with gopaddle for a seamless CI/CD on kubernetes. #kubernetes #docker #github #devops #cloud #azure #microservices https://blog.gopaddle.io/2021/02/18/integrating-github-actions-with-gopaddle-for-a-seamless-ci-cd-on-kubernetes/

https://redd.it/lmsa7t
@r_devops

Discas de DEV

Boa tarde!

Qual e o básico que tenho que aprender para entrar no mercado de trabalho ? Atualmente estou estudando Python e um banco de dados. (Estou caminhando galera, todas as dicas e opiniões construtivas serão muito bem vindas)

https://redd.it/lmqh1h
@r_devops

Choose Life - Choose Terraform Enterprise

Was at a client site this week and caught this printed out on the wall.

* [Terraspotting](https://imgur.com/a/whjnoUi)

https://redd.it/lmepu3
@r_devops

Need ideas for end-of-studies project

I just started my end-of-studies internship a couple of days ago. Due to timing, I had to accept a project that I'm not very convinced of. The project consists of finding a solution to analyse and monitor logs (network equipment, mostly Cisco).

The steps that have been given to me are gathering the data (logs), creating a database to store the logs and creating/implementing a monitoring/analysis tool that includes dashboards and KPIs.

The project is not very well guided so I have to figure out everything on my own.

The first solution that I thought would work is using Elastic Stack, but I found that it would be way too simple or even too mundane as an end-of-studies project.

I've always wanted to work with DevOps tools or DevOps methods (CI/CD pipelines, containers, virtualization...) and on something cloud-based but that won't be possible due to the fact that the company's architecture doesn't include anything cloud-based.

What I'm looking for are ideas that would help me include any tools or methods used in DevOps (or any other idea) to help me make my project more interesting.

Any proposition is welcome. Thank you in advance.

https://redd.it/lmna4s
@r_devops

Less is one of my favourite "invisible" CLI tools, what are yours?

Tacit knowledge is sometimes hard to appreciate or even notice. A few days ago, I realized how often I use less for all kinds of things, and how many little tricks I've internalized over the years around it.

I'm curious - what's a very useful, but almost invisible tool in your CLI toolbelt. One you hardly notice, but without which your days would be a lot more toilsome?


Very curious to learn!

https://redd.it/lmhb4m
@r_devops

Nextjs10 & Serverless Framework

Working on a boiler plate project for serverless framework & NextJs10.

This seems to be the latest plugin: https://github.com/serverless-nextjs/serverless-next.js

I don't seem to be getting anywhere following this simple guide.

Does anyone have any advice or material I could use to get off the ground with this please?

https://redd.it/lmk7y6
@r_devops

Asking for a pay rise - new in devops

Sorry if this is the wrong place to post this but I'm looking to get help in asking for a promotion/payrise.

I've been a Windows engineer in a devops role since Jan 2020, since then I've learnt to use Linux, azure, aws, salt, terraform, git, powershell scripting (it's been a busy year). In that time I've gained my Azure fundamentals and completed courses is salt and terraform.

Job at the moment - extremely stressful, we work in a high pressure environment where we have to deploy alot of infrastructure in not alot of time and complete ad hoc tasks where needed, we have had 3 experienceed members leave meaning newbie like me has had to step up and now become a SME in one of our products.

I'm deffo on the low band on the pay scale for dev ops (26k) and with how busy we've gotton and the strides made over the last year, I want to approach my boss and ask for a (I think) well earned bump.

I have an idea in how to approach this, list my contributions and improvement over the year, accreditations ect, also I was thinking of showing a list of jobs available in the area as another good reason I should be bumped.

Any advice would be greatly appreciated!

https://redd.it/lmjvv5
@r_devops

Prevent user login before mount is complete

All our workstations are Ubuntu 18. The `/home/` is a network mount (nas). This is how it looks in `/etc/fstab`:

nas:/mnt/export/home /home nfs nfsvers=3,_netdev 0 0


The problem is: when users boot their computer, they login before `fstab` is able to mount `/home/` which messes up their desktops and settings.

If it matters, the `/etc/fstab` file is being managed by puppet, along with everything else on workstations.

My question is, how can I prevent users from logging in before the `/home/` mount is complete? In other words, how can I make it so users would be able to login ONLY after the `/home/` mount is complete?

Thanks ahead!

https://redd.it/lmiooq
@r_devops

VMware Tanzu Kubernetes

Can someone please explain in simple terms what "VMware Tanzu Kubernetes" actually is? Is it just a Kubernetes cluster running on VMware VMs? If so, why would anyone needed it? Can't you just build/buy a Kubernetes cluster which is not VMWare Tanzu and get exactly the same functionality?

https://redd.it/lmidme
@r_devops

What does those stickers mean?

Hello Reddit! About a mount ago I bought some "programming stickers" from Ali Express, and they finally arrived. While searching trough them I found those and I don't know what do they mean. Could you help me find out?
Here's the photo: Photos (gyazo.com)

https://redd.it/lncatd
@r_devops

Tools for experiment tracking, dataset management, and visualization

Hi!

I am looking for good solution set for the following tasks:

1. An experiment management tool to set experiment configs and save experiment logs. I came across sacred, and it seems to be suitable for me. Multiple different logs will be generated for one experiment, with different structures, and entries, on different machines.

2. All data generated in each experiment should then be collected centrally in a DB

3. A web-based tool to access those datasets in detail and experiment configurations.

4. Explore and visualize some metrics stored from the logs, mostly as time-series

I am not talking here about typical ML experiments and datasets. I have more networking and computing tests in minds, so not sure if dedicated tools from ML community like MLFlow, or similar can be suitable.

I started to work with ELK stack, seems to be great for logs indexing and visualizing, but not 100% sure how to deal best with dataset and experiment management. Any tips or alternatives?

https://redd.it/lmhhhi
@r_devops