Pros/Cons: Containers vs immutable virtual machine images

Hi /r/DevOps,

I just started a new gig and find myself in a position where I need to pitch a pros/cons list of building a microservices application using containers or virtual machine images.

I'm familiar with both and have used them both to great success, but I have to convince management that containers are purpose built for our architecture. It's more or less a mix of python, Java, and node apps.

We're going to have about 70 total services, and if we follow the pattern of 1 service to one container/one image, it's a no brainer to me that containers (either on swarm or k8s) will win out in terms of operational overhead and cost.

How would you present pros/cons of each approach?

https://redd.it/fe4y75
@r_devops

Just got an offer as a Devops team lead, now what?

I have been escalating real quick In my career, I came out of college 4 years ago and now I got in a pretty good company by working 1 year in each of the following roles

Software developer > Scrum master > Profesional services consultant > Sysadmin > Devops team lead

This is my first time a as a team lead I might not be SR enough as many people here but you can count my experience x2. I'm really dedicated and work really hard from the beginning to the end of the day

I will be in charge of a team of 3 people working together with other devops teams in other countries but to be fair I'm a bit clueless where to start in this new role any advice is welcome especially in the "lead engineer" part

https://redd.it/fe99fw
@r_devops

Python For DevOps: Learn Ruthlessly Effective Automation

Has anyone had the pleasure of reading this book? I was wondering what people thought of it

​

[https://www.oreilly.com/library/view/python-for-devops/9781492057680/](https://www.oreilly.com/library/view/python-for-devops/9781492057680/)

https://redd.it/fdv7vv
@r_devops

Python script for bulk find and replace in multiple files in multiple git Repositories

This script is useful when you want to change multiple part of multiple files in multiple git repositories in bulk. Script is creating a new branch from the given base branch and committing changes into it. Maybe push to remote also can be added with small change.

BTW, any feedback would be much appreciated.

I will drop this here maybe it can be helpful for someone else:

[Medium Post](https://itnext.io/bulk-find-and-replace-in-multiple-files-in-multiple-git-repositories-487c7c433b75)

[Github Repo](https://github.com/omerkarabacak/bulk-find-and-replace-in-git-repositories)

https://redd.it/fdxt9a
@r_devops

Authfailure: AWS was not able to validate the provided access credentials

I am trying to create my Gitlab CI/CD pipeline with AWS. The goal is to Terminate the Existing EC2 Instance, Run a new instance from a template, then Associate an Elastic IP to the new EC2. The runner I'm using is a Docker runner using the python:latest image. When I run my CI/CD pipeline I get

An error occurred (AuthFailure) when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials My .gitlab-ci.yml is as follows:
```
stages:
- build
AWS_Install:
image: python:latest
stage: build
tags:
- Docker
script:
- pip install awscli
- export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
- export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
- export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
- echo "running script :)"
- OLDEC2=$(aws ec2 describe-instances --filters "Name=instance-state-name,Values=running" --query "Reservations[*].Instances[*].[InstanceId]" --output text)
- aws ec2 terminate-instances --instance-ids "$OLDEC2"
- sleep 200.0
- aws ec2 run-instances --launch-template LaunchTemplateId=[launch-template-id],Version=12
- sleep 120.0
- NEWEC2=$(aws ec2 describe-instances --filters "Name=instance-state-name,Values=running" --query "Reservations[*].Instances[*].[InstanceId]" --output text)
- aws ec2 associate-address --allocation-id [allocation-id] --instance-id "$NEWEC2" --allow-reassociation
```
```
What I've checked/tried:
- AWS credentials: They are correct and valid
- aws configure: Everything sets correctly (checked using aws configure get)
- Ensured UNIX line endings were being used
- Adding a variable section to the YAML file to include environment variables
- Hardcoding credential values
- New user on AWS with all necessary credentials
- Using export to get the environment variables
- Running everything in one script rather than having a before script
- Having multiple stages/Jobs
```

https://redd.it/fdwxo3
@r_devops

Company would like to have a call to make a formal offer to start working, and answer any questions you may have at this point of the process.

I don't really want to have a video call and have to answer straight in the moment what I think, how much I want, etc.

I'm a much stronger negotiator when it's all written and digital. Any tips on how to handle this?

Should I e-mail back:

>Is it possible to get the offer in writing before having the discussion tomorrow?

I feel it's a strategic move to talk me down financially. I'm definitely looking to maximize the offer and I'm quite introverted, so this would make me feel uncomfortable.

https://redd.it/fdwel5
@r_devops

What does a typical raise/bonus look like after 1 year at your company?

I’m pretty new to devops field and young so i was wondering what i should i expect and if what i got was too low or just right.

https://redd.it/fdvr7l
@r_devops

When to use devops? (noob question)

Hi All,

I' trying to learn about DevOps and have a very basic understanding about the concept around it. However, I cannot wrap my head around where exactly DevOps principles can be applied in a process flow. Hoping to pick all your brains.
Lets say I have a process flow that does the following for the organization I work for:
-inbound files are received in my organization that contain information that need to eventually go into our back end (lets pretend a purchase order from a customer).
-files then need to be 'massaged' and reformatted so our back end can read it. I assume at this point I can create a service using DevOps tools to accomplish this.
-Then send file across to back end application (like netsuite, SAP, etc.).

So in the end, I would really be applying DevOps to one portion of my project. Does that sound right? I know this is a vague example. I am just trying to understand where you would use DevOps in a project.

https://redd.it/fduwpz
@r_devops

Alerting - an overlooked support tool

Found this piece on alerting useful, thought I'd share it for others. Check [it out here](https://caylent.com/alerting-handling-on-call-management-and-notifications).

https://redd.it/fed354
@r_devops

Use KubeEdge to Collect the value of the Temperature Sensor connected to the RaspberryPi to Cloud



KubeEdge is an open source system extending native containerized application orchestration and device management to hosts at the Edge. It is built upon Kubernetes and provides core infrastructure support for networking, application deployment and metadata synchronization between cloud and edge. It also supports **MQTT** and allows developers to author custom logic and enable resource constrained device communication at the Edge.

Using the device management capabilities of KubeEdge, we can use the Raspberry Pi on the edge to transfer the value of the temperature sensor to the cloud.

A application called **Temperature Mapper** contains code to collect temperature from a temperature sensor which connected to a raspberry Pi through gpio. Depending on the expected state of the temperature sensor, the program collect the temperature.

The details for how to setup: [https://github.com/kubeedge/examples/tree/master/kubeedge-temperature-demo](https://github.com/kubeedge/examples/tree/master/kubeedge-temperature-demo)

[https://github.com/kubeedge/kubeedge](https://github.com/kubeedge/kubeedge)

https://redd.it/fdrd6x
@r_devops

Tell me about your artifact repository

Where is located? or what service do you use?
What tool do you use for ci?
How do you connect to it from ci tool, dev machines?

Small java shop with old and locked down on prem nexus repository and jenkins.
We dont have time to keep these updated, operational and they are running in old expensive setup.
We want to take advantage of the services available from different ci tools, but are in doubt what the best path forward is.

https://redd.it/fees6p
@r_devops

Most frecuent devops tasks

Hi all,

What's the most popular or frecuent task/configuration/deployment that you make as devops role? and components or languajes that it included?

https://redd.it/fe9tpl
@r_devops

SREs of Reddit who don't tag their cloud resources, why don't you tag your resources?

I have also gad a phase where I didn't care much about tagging my resources but I managing resources became a very real headache very quick. I am trying to get some more perspective around why people don't tag their resources.

Is it because you are too lazy and tagging is a lot of effort? Do you not know what tags to use?

Follow up - people who have standard tagging policies at their workplace, what do these policies look like and how are they enforced?

https://redd.it/fe9bes
@r_devops

AWS assume-role automation

Hey everyone!

I'm seeking some advice on automating assuming a role using AWS session manager, and running some commands.
I don't think this is a particularly difficult thing to do; I'm more interested in hearing from others on what tools to use to do this.

Here's what I'd like to achieve:

1. Export my AWS ACCESS_KEY_ID & SECRET_KEY to the shell environment.
2. Use `aws sts assume-role` to assume a role.
3. Automate the export of the temporary credentials into the environment.
4. Use `aws ssm start-session` to start a session and run some commands.
5. Terminate the session.

What I'm interested in is how I can parse the returned JSON and export to environment variables in step 3.

My current thinking is to use `jq` and some additional shell commands to achieve that. I should mention I'd like to stick to shell tooling if there's no strong reason to jump into using a programming language for this task.

How would you go about doing this? Is there an existing Docker container that wraps this kind of scenario? Is this the right way to achieve this?

I don't mind spending some time to increase my knowledge of the right tool (jq filters?), but thought I would see what everyone thinks, and how you would tackle something like this.

Thanks!

https://redd.it/fe985u
@r_devops

Vancouver devops salaries

Hi all,

Anyone here work in Vancouver and willing to share their salary? Am planning on moving in a few months.

Am seeing quite varied amounts on various websites (80-130k CAD). I have about 3 years experience as a DevOps engineer. AWS, Kubernetes, Jenkins, Terraform. Etc.

https://redd.it/fejs0l
@r_devops

What are the limitations of using AKS in prod?

Hi,

I'm in a battle with a PM who has not much idea about AKS. He'd like to use AKS for full-blown prod environment for biz-critical applications without much of POC or anything.

One issue that I can think of off top of my head is the client requires compliance-related software installed on nodes, which aks won't officially allow.

Does anyone have argued for the same issue? If so, could you please share some of the pain points you raised to avoid the disaster?

​

Thanks

https://redd.it/feni2f
@r_devops

How to Catalog many of webapps

Hi,

I work at an agency and we have dozens of web apps under active development and and low hundreds in maintenance or deprecated.

At the moment we catalog them all via a combination of bitbucket projects and confluence spaces describing things like
* client details
* prod server details
* links to monitoring (some times new relic, some data dog etc..)
* some times infrastructure as code
* etc..


I feel the wiki structure is to ad hoc and hard to audit. If some one new starts on an app or needs to maintain something every 6 months or more we can't quickly get up to speed

I don't know what it would be called or what keywords to use so my googling has fallen flat.

I want a service to catalog all our ad hoc apps, so i can find stakeholder details, run books, links to (or graphs) of monitoring.

If it could do audits for me that would be great, eg: show me all apps that don't have monitoring.


I am posting this here because it feels like knowing all the details of all the apps we manage in one easy to read place is a good place to start to move our company towards a devops type culture

https://redd.it/fenaez
@r_devops

What have you guys automated in your DevOps role ?



https://redd.it/felafv
@r_devops

Deploy headless PDF/PNG renderer on AWS Lambda at scale with a single command

Hey, guys!
There are many articles about how to get Chrome running on AWS lambda, but most of them are overcomplicated and require a lot of time to get them running, so I tried to create a really simple example of how to get headless chrome running with a single command with serverless.

[https://github.com/deepsyx/simple-headless-browser-serverless](https://github.com/deepsyx/simple-headless-browser-serverless)

Let me know what do you think :)

https://redd.it/fefwyv
@r_devops

advice needed for a ci/cd pipe line

So, i will need to provision 25 Wordpress sites in aws, i am trying to follow the ci/cd framework in deploying the project; what do you think the best way to approach this or the tools needed. I am new to the ci/cd pipeline.

who has done something similar?

https://redd.it/feqw82
@r_devops

CI and CD tool experiences for edge-optimized use cases

If you wanted every code review to have a dedicated environment (or multiple, parallel ones) spun up ephemerally for integration testing, what CI would you use?

If you wanted edge-usable CD tooling, what considerations would come into play? What tools would be interesting? "Edge-optomized" meaning: latency tolerant, disconnect resilient, network-agnostic deployments.

How do these two questions interact? E.g. are there distinct tools you'd use for one or the other, but aren't very compatible together?

[k3s](https://rancher.com/docs/k3s/latest/en/) is interesting as a potential edge-optimized orchestrator, especially since it uses a phone-home approach for node registration that helps for a lot of the edge use cases. It could also overlap with the desire for ephemeral environments for PR purposes, maybe in conjunction with something like [k3d](https://github.com/rancher/k3d).

[Spinnaker](https://www.spinnaker.io/concepts/) seems like a fantastic CD tool, but it delegates it's CI to other services and is very cloud-centric; I'm not sure how an edge/IoT use case would work with it. Lots of overlapping concepts and terms between it and k8s in general.

[Jenkins-X](https://jenkins-x.io/docs/overview/) is an opinionated CI and CD toolset that seems like it would be annoying to integrate with any of the above; seems like more of a standalone option.


What have I missed? What have your experiences been like with any of these tools? What non-k8s tools come to mind?

https://redd.it/feoosa
@r_devops