How to learn about networking ?

Hello all,

I am a non-CS graduate devops engineer and I am seeing a lot of jobs out there require quite a bit of networking knowledge. At my current job I do a lot of development and don't really get to work with network layer. It is really a big gap in my knowledge and I'd like to start learning. Where do I start ? What kind of tasks I should be able to do ? Can we include networking and security related "getting into devops" links to the mega post ?

https://redd.it/faqiyo
@r_devops

Any tool chain to manage deployment/ roll out steps?

So in my organization we do roll out of tools (off the shelf).

There are like 40+ steps which need to be followed. The rollout itself us automated but the pre and post steps have to be done personally by a team member.

We use (unfortunately) excel for managing these steps.

I am wondering what kind of tool sets are available to kind of document these roll out steps and manage them (reuse for next releases, review etc).

How does your organization manages these?

https://redd.it/fb804g
@r_devops

Heroku to AWS Migration?

Hi, Imma final year at college doing some freelance development on the side.. I created the backend service for an app with Golang + Postgres stack that is hosted on my personal heroku account. How do i migrate it to AWS? Resarch tells me there are several options available like

a) Start an EC2 Instance with Postgres and Golang running

b) EC2 instance with golang and a Amazon RDS running postgres

c)Dockerize the whole thing and deploy

Which do you think is the best option or is there anything else altogether that i should know of

https://redd.it/faq31p
@r_devops

As a DevOps how do you deal with the management asking you to do Service Desk / Help Desk stuff?

I am seeking for advise. New user here and sorry if I am breaking any rules, feel free to delete this post if so.

Before reading further, I am aware that DevOps is a culture and not a job title, however I will be talking about a job title because that's what the job titles are named these days.

TL;DR: Those of you who work as a DevOps or Ops engineer, but at the company where you work for, apart from the usual DevOps tasks you are also expected to do the shitty Service Desk/Help Desk tasks (like fixing the laptops of the other incompetent coworkers, fix that video call, fix broken WiFis and broken LAN connectors, and other shitty stuff), how do you deal with it? Also have you ever tried to change this or you just leave?

​

The long story:

I always use to ask these questions before I start working somewhere during the job interview. *"What is expected of me for this position? What kind of tasks are given to the team? How is success measured?"* The last question give's me an oversight how do they measure success and if they have a KPI which is also another shitty trend.

Thing is, the DevOps is really a mixed concept among the companies. Usually (most often, actually) a company posts a job as a DevOps but what they really need is a one man army, where you are expected to do everything from Service Desk and fixing other people laptops, broken LAN wall jacks, IT Procurement and then pps stuff containers, Kubernetes, Jenkins, Sysadmin things like fixing broken Linux servers, administering mail servers, user account access, managing the physical equipment in the datacenter and what else. Now, please do not misinterpret me here. It is not that I complain because of these tasks, the problem arises when ALL of them are given to a single team, and when ALL of them are measured in the KPI equally for all team members.

With my 10 years of experience I have knowledge of many legacy and modern technologies so I rarely get rejected from a job and when I get and answer like that, usually I am the one who rejects the position.

In reality few companies offer a job position that really is DevOps (or Ops), and in most occurrences they ask for a combination of Service Desk/Help Desk and Ops. Myself for an IT guy of 10 years, to me it makes absolutely no difference if someone comes to my desk with his laptop asking me "please install the printer" or if he comes and asks me to mop the floor in his office.

To summarise IMO, it is humiliating and disrespectful from the company to ask these kinds of tasks from experienced engineers.

Thus I come to the last part of my story and my current job position. I was referred into this company by a friend of mine (who is also my coworker right now and in the same team). During the interview I knew almost everything they asked me and in the end they gave me a demo task that was totally DevOps oriented. It was really a joy for me to work on it and I completed it even ahead of schedule. The salary they offered was really competitive so I took the offer.

All this pleasant experience was so good at the beginning that I completely forgot to ask the main three questions that I mentioned above, so guess what? I landed in the same type of environment that I was trying for so hard to avoid.

What could I do and how to deal with this without leaving? (I have my reasons of why I do not wish to leave at least for another 6 months)

https://redd.it/faauvl
@r_devops

Rancher as a Kubernetes Dashboard

So at work we have Kubernetes clusters being managed by Rancher. A bad experience with Rancher and certificates has left everyone there very sour with it. So there's a plan to migrate our clusters to EKS.

However, there are less technical people in our company that like having a visualization of the clusters that isn't just via kubectl. Heck, even I really like some features that Rancher has, like the "press the + button to deploy more pods" one.

So I was thinking... Would it be possible to have clusters running on EKS, and just having Rancher be like a Kubernetes Dashboard for them? Rancher wouldn't manage any of those clusters, it would only be an interface, kinda like the actual Kubernetes Dashboard.

The thing that is important is that authentication via LDAP is a must, and I found it easier to setup in Rancher than in Kubernetes Dashboard.

Is something like this possible? If yes, how hard/flexible is it to configure?

https://redd.it/fa6unj
@r_devops

Should I modify a AWS DMS task?

I have AWS DMS task running in CDC mode for sometime and now we need to sync two more tables which were not included initially. Should I stop and modify current task or should I create a new task for the new tables?

https://redd.it/fa7db7
@r_devops

(Dead) Snakes on a… Debian System

> Python 3.6 / 3.7 / 3.8 Debian packages for both Stretch and Buster

The Deadsnakes PPA project originally built older Python releases for Ubuntu, so you could e.g. run unit tests on a new release using a Python version found on older releases (i.e. the ‘dead’ snakes).

Nowadays, the project also builds newer Python versions ahead of what a certain release offers as its default.

The packages contain the minor Python version in their name (e.g. python3.6) and can thus be installed concurrently to the default python3 ones. Originally based on the Debian source packages, they can also be used on Debian and not just on Ubuntu.

[more…](https://jhermann.github.io/blog/python/deployment/2020/02/26/deadsnakes_on_debian.html)

https://redd.it/fbatbc
@r_devops

What exactly is an “Infrastructure Engineer”, and can one become a Junior Infrastructure Engineer within 10 months of study?

I’ve found a lot of conflicting information as to what it is, so I’m a bit confused as to exactly what it is.

https://redd.it/fa5kx8
@r_devops

Kubernetes Namespaces Explained in 15 mins

This is a [**short but thorough video**](https://youtu.be/K3jNo4z5Jx8) about Kubernetes Namespaces. **What they are and how they can help you manage your Kubernetes resources.**

In detail I'll go through the following topics:

* What are Kubernetes Namespaces?
* Explain the 4 Default Namespaces
* Why to use Namespaces with 4 Use Cases
* Characteristics of Namespaces
* Demo of how to create components in Namespaces
* How to change the active namespace, which makes it easier to work with namespaces

​

Hope this tutorial is helpful for some of you 🙂 Also appreciate any feedback and suggestions.

https://redd.it/fbfpf4
@r_devops

Deploy to Kubernetes from CI/CD

Which is the best way for deploying to Kubernetes? Using Ansible, Helm or something else ? Also if I am not using a docker-compose file (and over that compose file Kompose for creating deployment and service files) is there a easy way for creating these deployment and services files from a java(spring) config yaml files(also know as application.yaml)

Also which is the best way for deploying to Kubernetes using Jenkins. Calling deployment and service files from Jenkinsfile or integrate Jenkins and Kubernetes with Helm or something else ?

https://redd.it/fbceyj
@r_devops

Intro to DevSecOps?

Any quality literature, website content or videos on implementing DevSecOps? I want to go beyond adding code scanning to a Pipeline. Any info would be appreciated.

https://redd.it/fbcpzl
@r_devops

Kubernetes isn't for me, what is?

I've been looking hard at kubernetes but my environment is basically two or more VMs that I get by ordering them from my hypervisor team, in a shared hypervisor environment.

So maybe there's a possibility to get a VMware API access but it would be complex since it's a shared environment.

In essence I have VMs to work with and I can't provision new ones easily. So let's say I have two to begin with.

I want to deploy 4-5 micro services on them to begin with. Maybe more in the future. I'm using Gitlab for CI/CD.

I probably want to route traffic between these deployed services, some talk to each other, some talk to the outside, some receive requests from the outside.

I have a docker registry too.

What are my options?

https://redd.it/fbag5z
@r_devops

Functional programming in DevOps

Hi everyone!

How prevalent are purely functional programming languages in the DevOps space?

I know a little Java, Ruby, Python, PowerShell, Shell scripting and am interested in learning Go or Rust.
I'm wondering if I should invest some time in learning a functional language too, maybe Scala or Haskell.

How relevant are functional concepts to build/deployment/etc.. in DevOps?
My gut feel is Python and Bash are more suited to this kind of thing; tying things together with scripting and Docker for consistent environments.

What's everyones thoughts about functional languages? Are there some concepts which are useful to know?

https://redd.it/fb9vao
@r_devops

How to i find contracts for designing new pipelines?

I'm a freelancer and generally the contracts that I can find through my external sales organization lands me maintenance positions within old legacy systems. What I want to do to test my skills and stroke my creative mind is to design new systems, selecting state of the art tools and putting it all together. That's what I enjoy and I hate myself every time I have to mess around with some old crap like jenkins, or bitbucket, or god forbid perforce.

I'm not really sure how to look for these types of contracts. I don't know if I maybe should select for a particular industry, if I should market myself as an expert on a particular tool that I like, or any other trick to attract the right recruiters on LinkedIn.

https://redd.it/fb9drd
@r_devops

Kubernetes and Spring Boot MVC

My company is deploying their app as one huge monolythuc mvc web app, deployed on kubernetes. Does it make sence to do this? No microservices are involved.

https://redd.it/fbn63o
@r_devops

I have a python script on AWS EC2 that connects to a website via websockets that is hosted on Digital Ocean. Every method I've tried to start it up on boot up has resulted in failure.

I'll try to be thorough in my description of the problem, what I've tried so far, and what I think the problem(s) is. The flow of information look like this: `transformer.py` starts up, loads autobahn for the websocket connection, tries to find another half of the websocket waiting on the other side of a URL, and attempts to connect to that. If the handshake goes through it it connections, if it doesn't connect then it either results in an error or just hangs. The websocket connection used to connect to AWS Lambda and it worked just fine; it was only once I had it connect to my own website that the autostart stopped working. When I start the script manually it connects just fine, but when try to start the script automatically on boot up I get some weird connection issues. This is a stripped down version of [`transformer.py`](https://transformer.py):

from autobahn.asyncio.websocket import WebSocketClientProtocol
from autobahn.asyncio.websocket import WebSocketClientFactory
import asyncio
import json

uri = "wss://<domain_name>.com/ws/ai/"
domain = "<domain_name>.com"
port = 443

class ClientProtocol(WebSocketClientProtocol):
def onOpen(self):
message = input("Enter Prompt: ")
message = {"action": "handlePrompt", "prompt": message}
payload = json.dumps(message, ensure_ascii=False).encode("utf8")
self.sendMessage(payload, isBinary=False)
print("Sent: " + str(json.loads(payload)))

def onMessage(self, payload, isBinary):
print("Text Message Received: " + str(json.loads(payload)))


if __name__ == "__main__":

factory = WebSocketClientFactory(uri)
factory.protocol = ClientProtocol

carousel = asyncio.get_event_loop()
socket = carousel.create_connection(factory, domain, port, ssl=True)
carousel.run_until_complete(socket) # This line is where the error below appears
carousel.run_forever()
carousel.close()

This is the error that I get with autostartup:

Traceback (most recent call last):
File "transformer.py", line 358, in <module>
carousel.run_until_complete(socket)
File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/usr/lib/python3.6/asyncio/base_events.py", line 820, in create_connection
sock, protocol_factory, ssl, server_hostname)
File "/usr/lib/python3.6/asyncio/base_events.py", line 846, in _create_connection_transport
yield from waiter
ConnectionResetError

The file structure looks like this:

/home/ubuntu/project/transformer.py
/home/ubuntu/project/start_script.sh
/home/ubuntu/project/venv/

`start_script.sh` is executable. The contents are:

#!/bin/sh
cd ~
cd home/ubuntu/project/
venv/bin/python3 transformer.py

While the autostartup worked with AWS Lambda, I was using a cron job to run a shell script on reboot:

@reboot /home/ubuntu/project/start.sh | bash &

I later found out that it's kind of pointless to pipe the script to bash, but neverminding that it worked. Once I switched the websocket info over to my own website the above error appeared. I switched the cron job to:

@reboot /home/ubuntu/project/start.sh > /home/ubuntu/startup.log 2>&1

This results in the error above. I then tried to use the package `supervisor`. We use this on our webserver and are very familiar with how it works. The thought process behind this was that maybe there were some user permission errors and [`transformer.py`](https://transformer.py) needed to be run as ubuntu. This resulted in the same error. So at this point I was thinking maybe it has something to do with the loading order of modules for linux. Like, maybe the networking part of linux isn't fully loaded by the time the connection is

tried? From here I went on to try to start it as a service, put the script inside the `/etc/init.d/` folder, changed paths in the shell script where necessary, and made a symlink in `/etc/rc5.d/`. This resulted in the same error. I may have done something wrong in this last part seeing as I'd not super familiar with creating services in Linux. I followed [this](https://github.com/OpenLabTools/OpenLabTools/wiki/Launching-bash-scripts-at-startup) tutorial.

tl;dr: When I run [`transformer.py`](https://transformer.py) manually after logging in as ubuntu everything works exactly as it should. It's only when I try to start it up automatically that I get what appear to be network errors. The fact that this only happens on startup leads me to believe that the problem lies outside of my program and in the boot process of linux.

My working theory is that there is some AWS module inside of my linux image that was put there which boots before my program, which is why I was able to connect to AWS Lambda. Once I switched away from their stack though, the boot order changed to my script loading first and then any necessary network connections. Does anybody know what is going on? How can I get my script to work correctly? Any help would be greatly appreciated.

https://redd.it/fb98iy
@r_devops

What differentiates Monolith from Microservices?

I get the difference between Monolith and Microservices architecture.

Monolith is where all functionalities of the app are in one huge component so that whenever we make a change in one service, we have to re-deploy the whole thing.

Microservice is something that we make all services independent so that even if one service goes down, it won't kill the app and each service can have its own tech stack and be independently tested/deployed.

However, I feel hard to differentiate what is Monolith and what is Microservice.

For example, I have a project that has frontend and backend deployed to different servers and separately deployed. They can be deployed and tested separately having its own CI/CD pipeline. Is this clearly considered microservice?

I also have a project where both frontend and backend component goes in a single repo and has a single CI/CD pipeline where I build, test, and deploy both services. Is this a monolith because they are in one single repo?

What is the factor that makes a project monolith or microservices?

I understand the pros of the microservice, but I am not even sure if what I am building is a microservice way. Thanks in advance.

https://redd.it/fb3ufg
@r_devops

Using Terraform with public CI/CD outputs

A few CI/CD tools offer unlimited free execution minutes for public projects (eg: GitLab CI/CD and [Travis-CI.org](https://Travis-CI.org)).

I have a project which deploys to AWS using Terraform and my CI/CD pipeline consists of pushing a Docker image and running the \`plan\` and \`apply\` stages to deploy to ECS.

My question is: Assuming I use masked/secure variables in my Git project, is it safe to use Terraform on a project where the logs are visible to the public?

https://redd.it/fb24ma
@r_devops

Would you accept lower salary than your current job for a company with better fundamentals?

&#x200B;

I've been interviewing lately because of some concerns I have about the future of my current company. I really like my job, and my salary is competitive, I think. I am a fully remote Sr. SRE with a base of 145k and some bonus, good benefits, and good work/life balance. I'm self taught and have a lot of job experience but no degree.

&#x200B;

Today I received my first offer after several interviews. I've done more interviews this time around than ever before and it's been fairly exhausting and stressful.

&#x200B;

The offer is for 125k starting. This is \~14% less base salary than I'm currently making. I'm trying to get some more info so I can calculate the total comp, but at first glance it looks like a lesser package across the board, ie my current company covers 50% of my wife/daughter insurance etc, which this company does not.

&#x200B;

I'm definitely considering countering the offer before outright declining it, but they sortof told me in the email that this was what they could offer and it was partially based on the cost-of-living of my area, which is fairly LCOL. However I work remotely currently, and would work remotely for them as well (most of the time), and think of myself as a citizen of the internet and not of my city really when it comes to compensation.

&#x200B;

This seems like a no-brainer, but there are two big factors I'm considering.

&#x200B;

1. My current company is not profitable after \~8 years. There has been significant employee churn since I started, key players in engineering have left. It is V.C. funded, it's completely owned by investors and I assume extremely diluted stock wise. I don't really see a path upwards for myself, and the company could be insolvent in 12 months for all I know.
2. The new company was self-bootstrapped by it's founder/CEO into immediate profitability and has remained so for \~6 years. They are experiencing high growth, have told me that they want me to grow into the SRE lead/manager role at which point my comp could be raised. They've also supposedly 2.5x the options grant to try to bridge the gap. But these are just numbers of options, which obviously gives me very little information on actual value.

&#x200B;

If I had to guess I would say its going to be more work and less money though.

&#x200B;

&#x200B;

Thanks for your time and thoughts.

https://redd.it/fbpp8d
@r_devops

Grafana, K8s install troubles

hi community
Does anyone know why I would be able to see clusters but i cant see nodes or pods for my Grafana deployment?

https://redd.it/fbpktr
@r_devops