Docker and GastbyJS

Hi, I am currently building a devops pipline for my school club. The projects we are working on use many different back end technologies (NodeJS, Flask, Django, etc). To deploy the back ends I am currently using Docker Images + Github Actions + Google Cloud Run. This was the best way to handel the variety of frameworks.

A new project is planning to use GatsbyJS and there aren't any good resources / dockerfiles available online. I was wondering if anyone had experience with this and could point me in the right direction. I was going to treat this similar to the NodeJS deployment with an additional build step. Thanks for the help.

https://redd.it/f9zsg0
@r_devops

Jenkins replacement

Our Jenkins jobs execute Ansible playbooks. Soon we plan to migrate some Ansible "tasks" to Terraform. Jenkins gets the job done but I can't look at groovy anymore, I don't like how Jenkins logs, I don't like it's plug-in system. What modern solution would you recommend? Thanks.

https://redd.it/f9pjxh
@r_devops

Critique my attempt at devops for python microservice deployment and best practices

Hi everyone,

Currently a lead ML engineer that is very out of my element. I've inherited a code base that's a purely python microservice that serves ML predictions based on requests from a large internal monolithic app. The dev environment for the team is on multiple Windows 10 VMware virtual machines. I have access to a Windows 2012 VMware server. We don't have anything in place for CI/CD.

Instead of running a script to start the microservice, I was thinking of using Docker (with Docker Toolbox and enabling nested virtualization) so that a) I don't need to think about maintaining and running pip requirements.txt for upgrading packages and b) to streamline deployment in our users dev environment.

I can then use CircleCI to automate the "build" which essentially boils down to running unit tests for me upon code commit and emailing errors.

I'm only concerned with the deployment of the python microservice and nothing else.

Is this the best approach?

https://redd.it/f9pr2m
@r_devops

My personal devops project. https://www.linkedin.com/pulse/how-i-am-using-bitbucket-github-heroku-my-cicd-vasu-bhardwaj/

Hi all , this was my first devops project
I used github, bitbucket and heroku.
Let me know what you think.

https://www.linkedin.com/pulse/how-i-am-using-bitbucket-github-heroku-my-cicd-vasu-bhardwaj/

https://redd.it/f9pz7i
@r_devops

I want to setup these in my desktop: Gitea (and Postgres), CI/CD, Harbor, logging/monitoring; need advice

I'm learning IaS but I don't want to spend money for cloud instances yet, I only have i5 Skylake & 16GB of RAM, it will have to suffice for now. Which is why I'm choosing Gitea instead of Gitlab, and since it's also my main dev aside from laptop, I'm aiming for container centric GitOps workflow. I've installed minikube but I'm not sure I should do it in Kubernetes yet, and my line of thinking is as long as the architecture works, I can migrate it in the future since I do want to learn Kubernetes.

My plan is

- ✅ use vagrant to build Ubuntu 18.04 VM (my desktop is hackintosh), provision docker & other dependencies,
- use docker-compose to pull
- ✅ Gitea,
- Postgres,
- Harbor,
- CI/CD (maybe Drone),
- and Prometheus/Grafana or ELK.
- sets networking so I can commit git, trigger scripts etc without ssh-ing into VM

Roughly this is what the setup looks like https://i.imgur.com/14e7yu9.png

I had a problem that I haven't find a solution for a whole day yesterday, attaching local VM folder to `$PGDATA` in PG container made the folder owned by root & caused initdb to fail. I tried the attaching read-only /etc/passwd trick mentioned in the official README but it didn't work. I'm 50-60% sure i can work around it even if I have to build a custom image.

But anyway, what I'm asking is:

1. Is this a decent/sane plan?
2. Is there a better way to do it?
3. How far am I from being junior level DevOps/SRE/whichever title?

https://redd.it/f9o4yb
@r_devops

Are AWS t3a bad?

Comparatively speaking, value for money. I've done some of my own tests compared to DO and Vultr, which score quite a bit better in performance. Even compared to other AWS EC2 offerings, it's not that good.

What's your opinion?

https://redd.it/fa7666
@r_devops

Cloud infrastructure automation for a SAAS company

I am wondering what is the strategy to use in order to create cloud resources upon subscription of a paying client for a SAAS product. Eg: custom CNAME or a blob storage.

How do you keep the state? How to validate the completion? How do you queue the creation tasks? What other points I am missing?

Thank you

https://redd.it/fa7zsq
@r_devops

Looking for feedback for our automated version increment workflow for CI

My problem that led to this workflow was that in many cases we had to manually increment versions in git. This was tedious and error prone since would frequently lead to either unexpected extra commits or collisions when multiple developers are working or both.

Therefore, I built tooling around the problem which provides version synchronization capability outside of manual developer workflow. Such new automated workflow could be used in the CI process as described [here](https://worklifenotes.com/2020/02/27/automatic-version-increments-with-reliza-hub-2-strategies/).

I would be very interested to get some feedback from community whether this is useful or not. And if not - how are you currently solving version increments?

https://redd.it/fac5cu
@r_devops

GitHub is Down ... Again ...

[https://www.githubstatus.com/](https://www.githubstatus.com/)


I'm sure other folks here are considering alternative solutions to GitHub, given the reliability issues we've been seeing since Microsoft has taken over.

I was considering a self-hosted GH Enterprise cluster, but haven't researched too much on what's needed to make that happen. What are you folks considering?

https://redd.it/fadojf
@r_devops

Software Disenchantment

A colleague shared this link in our team's off-topic chat channel and it created a buzz. I thought our community may appreciate it as well.

[https://tonsky.me/blog/disenchantment/](https://tonsky.me/blog/disenchantment/)

https://redd.it/fadz5c
@r_devops

Jenkins Deleting ssh.exe!

Any job I run on one of my windows agents deletes my C:\\Program Files\\Git\\usr\\bin folder and thus my ssh.exe so I cannot clone a repo via ssh. I am baffled...anyone have any ideas?

https://redd.it/fafgtn
@r_devops

What should I look out for as a candidate in a devops job interview ?

I've got close to 2.5 yoe in technical support, I recently interviewed at a startup and the interview was way too easy. So I started having doubts if I'm doing the right thing?
The things that were asked very basic tcp/udp difference, one simple coding question, how to design a backup systems and some questions about my current role.

https://redd.it/fafu87
@r_devops

How to grow as an SRE/DevOps Engineer

This post is more so for exploring what I can do to grow as an SRE/DevOps Engineer.
A little about me: I have a B.S. in Computer Science and have moved into a DevOps/SRE role where I am writing a decent amount of code/scripts (python, bash), rolling out a decent bit of infrastructure (chef, cloud services) and troubleshooting application/network issues. I would consider myself on the above-average side of programming but the average to below-average side of Linux knowledge. I am very comfortable making my way around the Linux OS but struggle with some of the more in-depth troubleshooting/concepts from a knowledge standpoint.
The inspiration for this all is from interviewing for SRE positions at a couple of different companies and either making the on-site, pretty much acing the coding portion and then not passing through the architectural side, or rarely not making it to the on-site. The frustration for me is that I am given little to no feedback and struggle to find where to learn/gain experience that is needed to grow (large scale systems) and ultimately pass through all the interviews.
I generally have used Linux academy to learn but am constantly looking for more places to learn from.


Any insight/perspective/guidance/encouragement would be greatly appreciated.

https://redd.it/fakrm2
@r_devops

Lack of tooling that helps understand large codebases

**How does the code base work?**

There is a lack of good in-depth tolling for understanding big codebases for architects/team leads/developers. On one side we have simple tools for test coverage and on the other side, complex and expensive products like code climate focused on velocity.

But there is a problem left unsolved in the middle. As an engineer working in a bigger team, it’s hard to get an answer about how does part of codebase work. Or why & how & in what places is this piece of the system used.

Keep in mind that you can use static analysis and “find all” to find specific usage of a function but what I am interested in is connecting concepts/modules together and showing their relationships in easy to understand way.

**It’s a new decade and I still don’t have a way to have:**

* A clickable diagram explaining all subsystems of my app and their dependencies to one another
* A simple way to see the high-level flow for a particular feature. User registration code diagram for an example
* A simple way to see which parts of code are undertested / overtested. Code coverage is a bad metric in general. It does not tell you anything about the popularity/usage of tested code
* A way to have a good understanding of what side effects will I have to think about **before** starting the work on a feature. To measure scope and time needed. (Oh shit user registrations need to be exported to intercom...)

**What general questions should you be able to answer easily?**

* I want to change the functionality of this feature for new requirements. How can I know all other subsystems interacting with the feature?
* I want to quickly get a high-level overview of major subsystems. What are the dependencies, are there any hidden connections that people tend to forget? What should I watch out for?
* How can I easily know where does the feature starts and ends? I want to know how the card payments work. Where to start?
* How much effort will be needed to refactor this feature? Are there any connections/interloop that I should be careful about?

**Real-world example**

What parts of the codebase do Stripe payment gateway touch and how is it implemented at a high level? What do I need to watch out for when migrating to Braintree?

**Can it be better?**

It took me 4 months just to grasp the major subsystems and their interactions with each other in my company. There has to be a better way.

I understand that deep and comprehensive docs could be an answer, but really how many companies will do documentation at all levels of the tech stack? Usually what I see is that you have a few people at the company who have an understanding of the architecture and you kind of have to piece it all together from them.

Why can’t we automate at least part of this process? Something simple to give to the developers, so they can get a quick understanding of the features and their relations in the whole codebase.

https://redd.it/fah0jw
@r_devops

Looking for feedback from someone who is a user of Stackstorm / Rundeck

Hello, Just discovered stackstorm, but I want to know if someone can recomend which is better, rundeck/stackstorm or if them are pretty the same?

​

I know stackstorm is event-drivven but can also run scheduled simple tasks as rundeck? Also, can rundeck be event-triggered like stackstorm?

​

Thanks

https://redd.it/fakc5k
@r_devops

DevOps Roadmap & Me

I was given a really strange opportunity because of my powershell automation experience to be part of an amazing company, and I was wondering besides what’s on the monthly thread and the Roadmap:

What can I learn that will ensure I am successful meshing with my team?

I’ve wiped my PCs and servers and started running Linux only, learning bash.
I’ve started AWS and GCP free credits and am about to start learning K8s on AWS and go from there...

The employer knew I had no experience but they were confident in my ability to learn fast and enjoyed all the servers I have at home that I self learn with.

—

Any advice and encouragement is really appreciated.

https://redd.it/fanf2m
@r_devops

Azure Devops Agent installation (linux) – need help/advise

Hi we currently making use of Azure devops and have the agent deploy to a number of windows servers and everything works great. We make use of the install flags to configure it to run as a service and use a service account which has permissions to do what is required (ie local admin).

We have the need to install it on a linux machine, however the documentation is a little lacking. Firstly it seems to run as the user which configured the agent (ie my user account) and looking thought the documentation there seems to be no flag to change who the agent runs as.

I am looking for some advice on how best to install this agent. Should I install / run it as root or a dedicated account. If a dedicated account how do we deal with permissions (ie I do not see any options in pipeline to say sudo this task.

Any advice/recommendations would be great thanks

https://redd.it/fam2dq
@r_devops

What's your email QA process?

I run a small SaaS business which sends out report emails every day. And I haven't come up with a good process for email QA.

Last night we sent a blast of badly formatted emails to our subscribers, although after we did some testing on our server. The emails were badly formatted because we did not handle some corner cases. We checked some of the test sent emails, and they seemed to look good. But it turned out most did not.

Needless to say, this was embarrassing at best, churns at worst.

We need to set up some formal process for email testing to ensure this won't happen again (hopefully).

Here is what we think could work:

1. clone a testing VPS from the production server, with all the data and code;
2. update the code on the testing VPS with our latest changes;
3. redirect all the emails to some test email address which we will examine later;
4. run the code, check the test emails, make sure they are good;
5. push the code changes to the production server;

Care to share your process?

BTW: email QA is different than otherwise like dashboard, website, etc. Once you send a poor email message by accident, it stays in the user's inbox forever. There is nothing you can do about it. You can't recall it. Every bad email sent to our customer feels like a punch in the gut to me.

Updated: my main goal here is to implement a solid email QA process to get rid of manual testing as much as possible. Manual testing sucks most of the time.

Thanks good people.

https://redd.it/faof7d
@r_devops

Firewalling a docker container?

I have been given a system where DokuWiki lives in a docker container on an Unraid server. An nginx proxy handles directing what little internet traffic reaches the Unraid server. Notably, for the Wiki, you can only access it via port 80 (you can't ssh in or anything unless you RDP to a Windows machine from which you can SSH in).

The problem here is, the wiki holds sensitive data, and folks want it locked down completely from Internet traffic, with only a one or two external IPs which can get through. All employees are remote and there's no VPN for several reasons, so this is those employees dedicated static IPs.

Now, the question is: What would *you* do block port 80 traffic and only allow certain IPs? Should I just modify something within the nginx configuration?

Another option I have is simply to ditch docker and make a tiny VM, then write a few firewall rules in there. Anyways, thoughts?

https://redd.it/falyxx
@r_devops

Career in devops

Hello guys, I working in it field from last 3 years mostly in help desk positions. Right now due to some personal issues I cannot work for 5-6 months so I thought to study and complete some certification to get better job in cloud. I have basic knowledge of programming languages such as java, c#, html and css. My question is that what would be the path to get my career start in cloud and move to devops? Do I need to learn programming if yes than which languages? Should I give aws solutions architect ? Please guide me.


Thanks in advance

https://redd.it/falka0
@r_devops

Struggling trying to get my local Kafka messager project uploaded in the cloud using Aiven. Can anyone help?



https://redd.it/falhyl
@r_devops