HackerSploit Docker Security Essentials

The HackerSploit: Docker Security Series aims to provide developers, system administrators and DevOps engineers the necessary skills to be able to audit, secure and manage Docker in the context of an organization or in their own personal projects.
https://www.i-programmer.info/news/150-training-a-education/14785-hackersploit-docker-security-essentials.html

https://redd.it/p1unho
@r_devops

MongoDB scaling and speed

Hello there,

Context: I work in a small company where I am the only Ops/devops/make-some-magic-so-everything-is-running-smoothly. I use a bit ansible, a lot of docker & kubernetes (but always for jobs & low availability needs)-

We have one big mongo that starts to become huge and gets slower for some queries. We have more than 100 databases, as we have a multi-tenancy service.

The actual set-up for the mongo is:Docker-compose running a mongo container on a server, and we use a DO volume to write the data. As we approach the 500GB storage and it is a BIG single point of failure, it may be the best moment to use shards and/or replica sets.

**What is the best to manage and scale such a setup?** *Keeping in mind we are growing fast, availability and speed are our main focus (not confidentiality).*

1. Use a [shared cluster](https://github.com/mongodb/mongodb-kubernetes-operator) (via Helm chart probably) in Kubernetes;
2. Doing it manually with droplets, docker-compose, and command line;
3. Using Ansible to manage the different servers;
4. others?

*Managed mongo services are too expensive for the amount of data we use, that's why I don't include them.*

I have some points/concerns such as:

* I am not a Database administrator, I want to keep it as simple as possible.
* Is running a DB on Kubernetes a good idea? (I've read very different opinions online)
* If something goes wrong, the *meantime to recovery* is really important. 1h of downtime during the week is bad but OK. 2h is really bad. Half of the day we could lose clients.

I am curious to have your opinion on this one :)


Edit: With DigitalOcean, ReadWriteMany volumes is not available, only ReadWriteOnce.

https://redd.it/p1tzbo
@r_devops

Hiring Mgrs, do you even care about Certs?

There's a ton of talk about getting certifications here as a way to get into field. Almost a post everyday it seems. Is this really necessary? Would you rather see a list of certs or a resume full of projects showing you can do whats done on the job... maybe w link to github repo so scripts, yaml, code is reviewable. The cert obsession seems like a carryover from traditional IT industry.

I ask bc there are basically no certs in software engineering other than a degree or maybe a bootcamp. So if you don't have experience you fill resume w relevant projects and that is basically how u get in the door.

As entry swe who came into DevOps this is basically what I did and landed job where they were looking for some overlap. I have 0 certs.

I dont see a lot of mention about doing projects for resume. Just as a suggestion for learning in general.

so HM's, what's your perspective.

https://redd.it/p1umde
@r_devops

Issues with setting up networking in Podman for Prometheus Containers

I'm trying to run Prometheus containers with Podman on RHEL8.

My Prometheus container can't see the Prometheus Node Exporter Container and vice versa. My end goal is to get both on the same network, and also get a better understanding of how networking works with Podman.

I'm not able to list any networks from the CLI with the following command either with or no sudo:

sudo podman network ls

At this point, I'm not sure what is wrong. I'm coming from a Docker backgroup where we could create networks on the fly and I don't seem to have the capability to do that. I'm not root, unprivileged user and can do some sudo commands.

​

[user_a@host_a prometheus]$ podman version

Version: 1.4.2-stable2

RemoteAPI Version: 1

Go Version: go1.12.8

OS/Arch: linux/amd64

​

Let me know if we need more info.

thanks in advance

https://redd.it/p1wh3k
@r_devops

Introduction of Snowflake data warehouse for cloud | Many Data Workloads, One Platform

https://www.youtube.com/watch?v=PyAx0K1KLGE

https://redd.it/p1z5k7
@r_devops

How to get a job that uses Cloud if I don't have indurstry-level experience?

Hey guys,

I've been a DevOps engineer at a major bank for two years and am looking for my next move.

I am in a hard situation where almost every job (SRE/DevOps/Cloud Engineer) that I applied for requires Cloud/K8S experience.

However, in my current position, it doesn't involve these technologies. Maybe in two or three years later, some cloud technology would be adopted. But it's too late for me. Inside the organization, there are not that many opportunities for cloud technology.

​

Do you guys have any suggestions?

How should I plan for my next move?

​

Some ideas that I can think of:

* Get Cloud Certifications.
* The certificate could help on passing HR screening, but during tech interviews, the production-level experience is preferred.
* Personal projects deployed on Cloud.
* It can pass HR screening but seems not preferred to tech interviewers.
* Look for an entry-level cloud engineer position.
* Since I have almost 4 years of work experience (2 for current and 16 months for internship), I would prefer an intermediate/senior-level position.
* Even I ask for a reference, I guess I would be stuck with points 1 & 2 again.

​

Any suggestions would be appreciated!

https://redd.it/p1znrj
@r_devops

Understanding workflow of multi-stage Dockerfile

There are a few processes I'm struggling to wrap my brain around when it comes to multi-stage Dockerfile.

Using this as an example, I have a couple questions below it:

# Dockerfile
# Uses multi-stage builds requiring Docker 17.05 or higher
# See https://docs.docker.com/develop/develop-images/multistage-build/

# Creating a python base with shared environment variables
FROM python:3.8.1-slim as python-base
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIPNOCACHEDIR=off \
PIPDISABLEPIPVERSIONCHECK=on \
PIPDEFAULTTIMEOUT=100 \
POETRYHOME="/opt/poetry" \
POETRYVIRTUALENVSINPROJECT=true \
POETRYNOINTERACTION=1 \
PYSETUPPATH="/opt/pysetup" \
VENVPATH="/opt/pysetup/.venv"

ENV PATH="$POETRYHOME/bin:$VENVPATH/bin:$PATH"


# builder-base is used to build dependencies
FROM python-base as builder-base
RUN apt-get update \
&& apt-get install --no-install-recommends -y \
curl \
build-essential

# Install Poetry - respects $POETRYVERSION & $POETRYHOME
ENV POETRYVERSION=1.0.5
RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

# We copy our Python requirements here to cache them
# and install only runtime deps using poetry
WORKDIR $PYSETUPPATH
COPY ./poetry.lock ./pyproject.toml ./
RUN poetry install --no-dev # respects


# 'development' stage installs all dev deps and can be used to develop code.
# For example using docker-compose to mount local volume under /app
FROM python-base as development
ENV FASTAPIENV=development

# Copying poetry and venv into image
COPY --from=builder-base $POETRYHOME $POETRYHOME
COPY --from=builder-base $PYSETUPPATH $PYSETUPPATH

# Copying in our entrypoint
COPY ./docker/docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh

# venv already has runtime deps installed we get a quicker install
WORKDIR $PYSETUPPATH
RUN poetry install

WORKDIR /app
COPY . .

EXPOSE 8000
ENTRYPOINT /docker-entrypoint.sh $0 $@
CMD ["uvicorn", "--reload", "--host=0.0.0.0", "--port=8000", "main:app"]


# 'lint' stage runs black and isort
# running in check mode means build will fail if any linting errors occur
FROM development AS lint
RUN black --config ./pyproject.toml --check app tests
RUN isort --settings-path ./pyproject.toml --recursive --check-only
CMD ["tail", "-f", "/dev/null"]


# 'test' stage runs our unit tests with pytest and
# coverage. Build will fail if test coverage is under 95%
FROM development AS test
RUN coverage run --rcfile ./pyproject.toml -m pytest ./tests
RUN coverage report --fail-under 95


# 'production' stage uses the clean 'python-base' stage and copyies
# in only our runtime deps that were installed in the 'builder-base'
FROM python-base as production
ENV FASTAPIENV=production

COPY --from=builder-base $VENVPATH $VENVPATH
COPY ./docker/gunicornconf.py /gunicornconf.py

COPY ./docker/docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh

COPY ./app /app
WORKDIR /app

ENTRYPOINT /docker-entrypoint.sh $0 $@
CMD "gunicorn", "--worker-class uvicorn.workers.UvicornWorker", "--config /gunicorn_conf.py", "main:app"

The questions I have:

1. Are you docker build ... this entire image and then just docker run ... --target=<stage> to run a specific stage (development, test, lint, production, etc.) or are you only building and running the specific stages you need (e.g. docker build ... -t test --target=test && docker run test ...)?
2. When it comes to local Kubernetes development (minikube, skaffold, devspace, etc.) and running unit tests, are you supposed referring to these stages

in the Dockerfile (devspace Hooks or something) or using native test tools in the container (e.g. npm test, ./manage.py test, etc.)?

Thanks for clearing this questions up.

https://redd.it/p1sn27
@r_devops

How do you like to learn new tactics and tools?

I'm investigating ways to improve how systems engineers take on the breadth of their role.

One of the ways I'm looking at this is through learning design.

Looking at microlearning as one avenue for time-poor individuals.

So, my question to you is how do you prefer to learn new tactics and tools?

Pick as many types as you find useful.

View Poll

https://redd.it/p1nh49
@r_devops

How to choose the correct PATH to get into DevOps from Ops...

Hello, my friends

After a long journey and many interviews, I have got a job position as a Junior Application Operations (my first Ops role) and I will start it in September. I have only 1 year 8 months of experience as a Quality Assurance Technician (in the gaming industry/game tester). I love improving myself and I have a great passion to become a DevOps engineer in the future. I would like to read here Advice/Tips from DevOps engineers, especially those who come from an Ops background. I want to know what kind of courses should I follow in Udemy, Coursera, or in other platforms? Tbh, I'm a Business Management student with a master's degree who doesn't have an IT background. Right now I am following Shell/Bash scripting course for myself and I hope this is a good start. I wouldn't want to skip your advice here. Thanks everyone in advance!

https://redd.it/p17azd
@r_devops

MINIKUBE AND KUBECTL - HELP NEEDED


~ % minikube start
😄 minikube v1.22.0 on Darwin 11.2 (arm64)
✨ Using the docker driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
🔄 Restarting existing docker container for "minikube" ...
🐳 Preparing Kubernetes v1.21.2 on Docker 20.10.7 ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

~ % kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:60186
CoreDNS is running at https://127.0.0.1:60186/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

kubectl get pods --all-namespaces

NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-558bd4d5db-4c2qc 1/1 Running 1 46m
kube-system etcd-minikube 1/1 Running 1 46m
kube-system kube-apiserver-minikube 1/1 Running 1 46m
kube-system kube-controller-manager-minikube 1/1 Running 1 46m
kube-system kube-proxy-wtdcr 1/1 Running 1 46m
kube-system kube-scheduler-minikube 1/1 Running 1 46m
kube-system storage-provisioner 1/1 Running 3 46m


why does cluster-info or namespace not show the dashboard, and kubeDNS?

https://redd.it/p251r3
@r_devops

Git Switch and Restore commands came in version 2.23. In this article, we will go through all the new commands that are here to make our life a bit easier. To understand more about the new Switch and Restore, we will look at "Checkout" first. Let's Start!

https://www.p3r.one/git-switch-and-restore/

https://redd.it/p274k4
@r_devops

It has been such a wonderful week so far because another tool under the CNCF umbrella got its latest update. There are many changes, bug fixes, and new features and enhancements in this new update. We are going to talk about all of these in this article. But, let's get around with Keda at first!

https://www.p3r.one/keda-2-4-release/

https://redd.it/p274h5
@r_devops

Ansible - start at specific role

I have a playbook that only calls roles. This is what it looks like: (there are about 20 roles in it)

---
- hosts: prod1234
roles:
- role1
- role2
- role3

Sometimes, a role fails, and I don't want to start over as each role is huge and I would just like to start at that point or the next one.

With tasks, I know there's a flag for --start-at-task="task-name". Is there something similar I can do with roles?

My current solution is to comment out all the lines I don't need and run it again..

Thanks ahead!

https://redd.it/p28yv6
@r_devops

advice to handle work

hey guys, so i was selected as a trainee for the role of devops. This is like my first experience of doing any job and its been almost 2 and a half months since i joined. At first i was told to study git, gitlab , docker jenkins and all those basics related to devops.

after a month i was asked to work on puppet on which i had no clue about.. i did up to puppet installation and some configuration and establishing some connection... i took some time to do this stuff since i had no clue in the beginning and got a lot of errors while proceeding.

so, i was aked to stop it and do another task related to running a flask on docker. i did it and after that i was asked to run flask using tensorflow algoirithm on containers again.. I have know prior coding knowledge and tensorflow is totally different.. i took like 3-4 days and then he told me this is getting compliacated so do work on site realibility engineering. so i stopped tensorflow with docker and started reading about sre.

i had to read a lot of sre related stuff and tried to understand what and why we need it. took me 3 days( could have taken less time) . then i was asked to implement sre in my organization which again i have no clue... i dont even know how to approach and tried for few days and told him i need more time..

again, he told me to chuck it and start looking at prtg which is a network monitoring tool.

today in the daily meeting we had, he told me that i just raise my hands when asked if someone could do a particular task but do nothing about it. And i need to be committed and stuff. he said i dont do anything so why bother giving me a task.

​

i was like shocked and it aint going out of my mind no matter how i try. this is like my first time doing a job and it totally broke my confidence..

i just did what he told me to. i didnt even ask him why we are doing and stuff. i just said ok to him whatever he told. Now he says i didnt do anything.

​

could you guys tell me what went wrong and then how to actually talk, act and stuff realted to being in an office.. all suggestions would be helpful.

thank you

https://redd.it/p29v5n
@r_devops

What does the future of DevOps look like?

I'm beginning my transition of Sysadmin to DevOps. Still got a long way to go as I've not used some of the key tools before. What does the future of DevOps look like in the next 5 years or so? Do you think it will become more and more in-demand? Personally I don't think it's going anywhere anytime soon, but as I've got a long way to go to get to my goal it's something that has crossed my mind.

https://redd.it/p29rik
@r_devops

Creating Subdomains on AWS Route 53 with the original domain on GoDaddy

The thing started with wanting to route the traffic through AWS by creating a hosted zone on Route 53 and copy the name servers it generates to GoDaddy. When I checked the existing name servers on my GoDaddy domain before doing anything I found some name servers that belong to Nginx and I'm not sure if adding Route 53's NSs to the existing name servers in general would result in any consequences since the domain has traffic and it's operating!

Also, I want to know which is a better practice, whether to create the subdomains on AWS Route 53 (since I'm using many AWS services) with my domain itself being on GoDaddy or create the subdomains from GoDaddy and just route the traffic through AWS by creating a hosted zone on Route 53 and copy the name servers it generates to GoDaddy.

https://redd.it/p2c22k
@r_devops

How To Automate Your Mobile App Releases using Fastlane and SemVer for Hybrid Applications

Hey guys,

I wrote an article on how you could automate your mobile app releases using Fastlane and SemVer for Hybrid Applications.

I hope you find it useful for your release management process in your pet projects or companies.

How To Automate Your Mobile App Releases using Fastlane and Semver for Hybrid Applications

Do leave a comment or recommendations on how the tool can be improved.

https://redd.it/p2dls7
@r_devops

A serverless platform for running containers globally - feedback?

Hi r/devops

We are validating a new serverless product to deploy and manage containers globally (seaplane.io, the website needs updating). We are looking for feedback.

We found that many engineering teams spend hundreds of hours building and maintaining infrastructure where they could be working on their core applications instead. We aim to solve those problems.

Our platform lets users deploy containerized workloads on a global compute cluster that runs on top of multi-cloud (AWS, Azure, GCP) and bare metal (Equinix, Hivelocity, OVH, etc.) and custom edge. The platform automatically senses your traffic and adjusts the infrastructure accordingly (much like a CDN does for content), scaling horizontally and adjusting where the compute runs to minimize latency.

Besides the compute, we also run a data layer currently supporting Postgres. The DB supports multi-region multi-writer in 400+ locations and is strongly consistent.

The goal is to give engineering and DevOps teams superpowers to build on top of strong infrastructure without worrying about zones, regions, clouds, redundancy, and anything else. The system takes care of all of that while still giving you a granular level of control.

Would you use a system like this? Anyone interested in providing feedback, we would love to hear from you!

https://redd.it/p2g516
@r_devops

Are managed Kubernetes and managed databases worth it for a one-man show?

Hey folks,

Making a webapp by myself and have very little ops experience (mostly do data engineering at my real job but I do some generalist stuff in my spare time like webdev and some backend).

I don't know jack about Kubernetes and was was wondering if paying for managed Kubernetes on DigitalOcean, Linode, or Oracle Cloud (my use case is extremely egress-heavy so I can't afford to use AWS/GCP/Azure) is worth it for me. If not, I was considering using Hashicorp Nomad instead (edit: feedback on this idea welcome).

Will be using Hashicorp stack ops-wise unless I use Kubernetes over Nomad.


Also -- would $15/mo for DigitalOcean's managed databases (because fuck paying for Oracle databases) be worth it for me time-wise? Or could I reliably back up a DB I set up myself with just a cron job?

https://redd.it/p2ge8v
@r_devops

Looking for Suggestions on git training courses for senior employees that need retraining

I'm looking for git training course suggestions:


The Situation:

I have been tasked with providing a plan for my companies migration from perforce to git; eventually moving our current process into bitbucket. Obviously part of this involves training employees on git, some of which have never used it. We are a small company, so in-house training from other employees would be too much of a time sink to be a viable solution for us.

I've been scouring the internet but i'm having trouble finding unbiased reviews of various git training courses.

Another consideration is that, ideally, there'd be some form of testing/interaction involved. Unfortunately, I'm worried that pure reading/video type courses will result in employees just clicking through it to get it done as quickly as possible and causing a knowledge gap.

Paid or Free doesn't matter for us

https://redd.it/p2im5u
@r_devops