How Can I Do Continous Deployment On My VPS With Docker?

I am trying to step up my devops chops by trying to implement a CI/CD (deployment) pipeline for my personal website.

On the CI side, I am working on implementing gitflow, automated testing, and pushing new Docker images on successful merge to master via Github Actions.

As far as the CD side, I am currently hosting my site on a Vultr VPS. I would like to set up a system that checks for a new Docker image for the site, and if one exists, it should pull it down and restart my website (probably via docker-compose) with the new Docker image. I could probably just write a cronjob that runs every few minutes and checks for a new image/redeploys but that seems inefficient and clunky.

I am aware that something like this could easily be implemented using AWS ECS/ECR but I think that would be overkill for my small site. Does there exist some tool that could do the "polling" for a new image and redeployment in a more streamlined way?

Any help/feedback would be appreciated.

https://redd.it/le9v8f
@r_devops

Building Python for multi-Linux distro deployment

We’re building some system-level applications that have to interface and configure Unix users/groups, other system tools like mail servers, nginx, etc.

The team is planning on writing the apps in Python, and distributing them via RPMs and Debs. Their reasoning for using system packages is primarily so that we can pin dependencies on the system. To me, this sounds fairly complicated, as we’d have to package the code up twice and test deployment across multiple versions of the multiple OSes.

I’ve mostly worked on ephemeral systems that were either configured with Ansible/Puppet that would configure the system + deploy applications, or baked Docker containers. Does the system package approach make more sense on non-ephemeral hosts? Is there a better alternative for Python apps? I’ve looked into something like Pyinstaller that can package the Python app into an executable, would that still be Linux OS-dependent?

Thanks!

https://redd.it/le8o13
@r_devops

Bare metal servers provisioning software

I am searching for a not very complicated system for provisioning bare metal servers (mostly Supermicro).

The steps for the new server deployment at the moment are:

1. Add server to a bootstrap network segment
2. Install base OS on the new server (Ubuntu)
3. Make some tweaks via ansible (add user/keys, some packages, iptables rules, etc)
4. Move server to the prod network segment and transfer to app admins.

Items 1,3,4 already automated with Ansible, now time to OS install process.

I'm looking to maas.io, but it looks some overkill for my purpose.

I would appreciate some advice.

Thanks!

https://redd.it/le76oz
@r_devops

Automate the initializing and unsealing of HashiCorp Vault without public cloud infrastructure

Hey everyone! Here's a small project I worked on recently that I thought people here might find interesting: https://github.com/jace-ys/vault-init.

It draws inspiration from https://github.com/kelseyhightower/vault-init, but doesn't rely on any public cloud infrastructure for the encrypting and storing of Vault's root tokens and unseal keys. You would typically use this if you do not have access to public cloud infrastructure, or if your Vault deployment must operate entirely on-prem.

Hope someone here finds it useful, and let me know if you have any issues/feedback!

https://redd.it/ldud7w
@r_devops

When using AWS route53 + Alb + EKS + ACM, is it possible to assign a different url/domain to each pod inside a cluster ? ex. foo.com = pod1, bar.com = pod2, dog.com = pod 3, cat.com = pod 4

I want to be able to access these urls using HTTPS. I want to be able to use the certs from ACM. Lets say all four as just hosting a static website.

If there is a better way of doing this, please do let me know!

ty

https://redd.it/leho36
@r_devops

SSH keeps disconnecting from my EC2

Hi, I'm new to devops and trying to figure out my way to creating docker images using ansible playbooks. I use t2-micro EC2 instances.. Everything is fine so far but the ec2 keeps disconnecting every 5 mins or so if i dont interact with the session..

Couldn't really get to the reason why this is happening..

Any suggestions in this regards are much appreciated. (Any logs that can be checked/time-out settings to be amended)

https://redd.it/lehd5t
@r_devops

How to use AWS S3 as a version control System using Terraform?

I created S3 bucket through terraform. And trying to using it as a version control system. By uploading terraform's state file into the bucket. However I am getting an error I'm posting below. I am using ubuntu 20.04 , terraform version v0.14.4. TIA

​

\# Define your AWS configuration
provider "aws" {
profile = "default"
region = "us-east-1"
}
\# Define your S3 AWS Resource
resource "aws_s3_bucket" "tf_course" {
bucket = "rahulwilliams11121985"
acl = "private"
}

\#after terraform apply
terraform remote config \\

backend=s3 \\

backend-config="bucket=(rahulwilliams11121985)" \\

backend-config="key=terraform.tfstate" \\

backend-config="region=us-east-1" \\

backend-config="encrypt=true"

After this i need to run these below commands .

remote configuration updated

Remote state configured and pulled.

But after the remote configuration updated. I am getting the below error. Could you please guide me where I am going wrong.

rahul@rahul-MacBookPro:\~/Documents/s3backend/s3back$ remote configuration updated

bash: /snap/bin/remote: No such file or directory

rahul@rahul-MacBookPro:\~/Documents/s3backend/s3back$ Remote configuration updated

Command 'Remote' not found, did you mean:

command 'remote' from snap remote (1.09)

command 'emote' from snap emote (1.4.0)

https://redd.it/lehty5
@r_devops

How can I come out of shell script execution?

I am running a shell script, using AWS Code Deploy on Linux EC2 machine. The script executes a binary which starts a web app on port 3000. Given it executes a binary which creates a process, the shell script never terminates which fails my AWS Code Deployment.

How can I come out of the shell script without terminating the binary process?

\#!/bin/bash

cd /home/ec2-user/go/bin/

./go_binary

https://redd.it/ldwgzh
@r_devops

A prototype framework for scaling cloud apps

Hi everyone! 😊

Over the last few months I've been working on a side project to automate the deployment of production cloud applications. It's really an automated way to deploy a local codebase into cloud services and automatically scaffold the pipelines and configuration.

My main driver was that configuring services and pipelines is a bit of a headache and cloud applications can be infinitely complex. So I wanted to simplify and drive design out of a scaffolding framework without losing flexibility to scale to more complex apps.

The way it works is "describe the thing you want to build" and it will be built end to end. The two use cases I have working so far are the end to end deployment of Dockerised app into Kubernetes with all routing set up. The second use case is deploying web application architecture. You can also replicate environments if you wanted to stand up dev through to prod.

There are a few guiding principles:

* Keep the language simple - I want to be able to describe the application I want to build (no complex new language to learn)
* BYOT (Bring Your Own Terraform) - flexibility in design
* Configuration / community driven - functionality scaled as modules (e.g. modules for Docker, ...)

Is anyone interested in taking a look and sharing some feedback?

https://redd.it/ldv70t
@r_devops

Need Advice for Getting Started with DevOps

I have a fun learning project in mind that I want to use to learn DevOps. Also, if it works, it may generate some side income that can fund my home lab dream.

I intend to use Python; I have spent the last few weeks learning Python and some Django.

I also want to containerize the application, with the sole purpose of having some hands-on experience in some of the DevOps concepts.

I will be building my test servers inside VMware Workstation. So far, this is what I have come up with:

(All servers are Ubuntu Linux)

1. Server A - Container 1: Nginx serving static files and JS
2. Server A - Container 2: Some business logic that is mildly CPU intensive
3. Server B - Container 1: Gunicorn
4. Server B - Container 2: Core business logic that is mostly CPU intensive

I have gained some experience in automated testing but not much in Jenkins. And I have no experience in containerization.

As I take the first steps, I need to know whether I am on the right track or not. I don't even know if I am asking the right questions.

Soliciting your valuable advice and inputs.

Thank you for reading the post.

https://redd.it/ldyave
@r_devops

Devops POC

So i writing a POC on implement a CI/CD pipeline using Jenkins, packer, AWX. Do I have to include a

Vulnerability scan in the POC?

​

​

Sorry if it is noob question, I am new to this

https://redd.it/ldrqts
@r_devops

Do network engineers need coding skills like Python? Microsoft Network Engineer speaks...

With automation demands increasing in all sections within IT sector, Networking engineers also need to gear up!

Do network engineers need coding skills like Python?

https://redd.it/leswni
@r_devops

Routing traffic from an API endpoint

Hey Guys,

We're working with some external vendor, And we only have one hardcoded endpoint in thier sandbox environment.



We want to use more development environments with this service,

So I wanted to ask if you have good suggestions (and how to do it...)

Lets say this enpoint https://blabla/alerts/api/v2/ received some request, so I want it to be distributed to many other development environments, but only the relevant environment will pick it up and run some logic. and all the others will ignore it (this is an internal process)

​

​

Let me know if you guys can help!

Thanks :)

https://redd.it/leqmd9
@r_devops

Test Run Dashboard - Looking for feedback

Hi All,

Some friends and I are working on a software startup called ObsidianQA. We are trying to validate the problem the app solves.

ObsidanQA is a test run dashboard that gives dev/qa teams better visibility into their applications quality. All of us work at software companies and realized the QA/Dev teams each had an internal built tool for viewing test runs. They were all very rough but the teams found them useful. This is why we decided to build a software tool that makes this quick and easy to setup for any team.

The app is free to use if you want to give it a try. Any feedback is highly appreciated.

https://www.obsidianqa.com/

https://redd.it/let1yy
@r_devops

Running Scripts or Commands in Spinnaker Pipelines

Hi everyone -

I'm trying to run scripts as part of some of my deployment pipelines in Spinnaker. I don't want to use Jenkins to run these scripts. I would use a Kubernetes job, but these scripts need to execute prior to the Kubernetes deployment.

I was debating creating ECS tasks in AWS which I'd like to run on demand during one of the stages in my pipeline. Does anyone know if it's possible to execute an ECS task directly from Spinnaker?

If not, are there any other ways to execute a command or script in a pipeline outside of using a Kubernetes job or Jenkins server?

https://redd.it/lesuu0
@r_devops

AWS CDK CodeBuild unable to upload Artifacts error

I am trying a simple example from the AWS documentation using the following link:

https://docs.aws.amazon.com/cdk/latest/guide/codepipeline\_example.html

For quick reference here is the buildspec.yml file for the CDK build. LambdaBuild is successful.

const cdkBuild = new codebuild.PipelineProject(this, 'CdkBuild', {
buildSpec: codebuild.BuildSpec.fromObject({
version: '0.2',
phases: {
install: {
commands: 'npm install',
},
build: {
commands:
'npm run build',
'npm run cdk synth -- -o dist'
,
},
},
artifacts: {
'base-directory': 'dist',
files:
'LambdaStack.template.json',
,
},
}),
environment: {
buildImage: codebuild.LinuxBuildImage.STANDARD20,
},
});

​

I am getting the following error :

​

Container 2021/02/07 03:44:53 Expanding lambdaStack.template.json 61
Container 2021/02/07 03:44:53 Skipping invalid file path lambdaStack.template.json 62
Container 2021/02/07 03:44:53 Phase complete: UPLOADARTIFACTS State: FAILED 63
[Container] 2021/02/07 03:44:53 Phase context status code: CLIENTERROR Message: no matching artifact paths found

THANK YOU !

https://redd.it/lemuu3
@r_devops

Skills required

Hi, I am currently pursuing post grad in cloud computing and I researched about devops and like to get into devops jobs after I complete my education. Can you guys suggest the skills I need to learn and how to get shortlisted for interview as a fresher. Mostly will be looking for jobs in Canada(Toronto). TIA

https://redd.it/lem6i7
@r_devops

Good ressources to learn devops

Hello guys, i'm a newbie in devops i've just started learning the concepts, i tried a first experience with Jenkins, GitHub webhooks docker .., but i feel like i'm missing a lot of things.
Is there any recommended resources u can suggest so i can build a strong base in the subject.
Thank you

https://redd.it/lek949
@r_devops

Continuous Integration (CI), Continuous Testing (CT), and Continuous Delivery (CD) are considered to be the key metrics to measure successful DevOps practice.

CT is considered to be the first step in the right direction when embarking on a DevOps journey. It is a continuous feedback mechanism that drives software delivery through SDLC. If the automated feedback at each checkpoint indicated no bugs or issues, then the process automatically moves to the next step in the delivery chain.

Read more on Need for Continuous Testing (CT) in DevOps CI/CD Pipeline

https://redd.it/lf45m7
@r_devops

Simple on-prem alternative to AWS for a solo-developer?

I am a developer and I am quite experienced working with AWS. I've been using a broad range of AWS services and I've been very pleased with being able to just use the AWS console and find my way around without sweating much.

Now I need to work on a project that needs to be working on premise bare metal servers. I'd I am decent software developer but my devops skills are pretty lacking. I've been looking at Kubernates, Terraform, Nomad, OpenShift, Docker swarm... Honestly, it's too complicated for me. I guess, it would make sense if you are working on projects where you need to manage tens/hundreds/thousdands? of apps/servers with a team of people. But's it simply feels like an overkill in my case.

From a solo developer perspective, what could be a simple alternative to AWS?
Basically, I need:

\- RDS (Mysql or Postgres with easy to use, automated backups etc.)
\- Elastic Beanstalk (deploying and scaling applications)
\- CloudWatch for dashboards, monitoring metrics, logs and events
\- S3 like object store (no need to be as fancy as S3)
\- And hopefully just a GUI to manage these all.

What direction can look for?

https://redd.it/lezq8h
@r_devops

Getting Better With Kubernetes

I've been working on devops for a number of years, but my experience with Kubernetes has been minimal.

Does any one have any suggestions on where I can learn more about Kubernetes? I really think I should just start at the basics and work up.

Just reading docs isn't super effective for me. I'm much better at learning by following asking with examples and reproducing those examples myself, so something with some hands on kind of learning would be great.

Thanks all!

https://redd.it/levunc
@r_devops