Recommended books for microservice architecture?

Hi all,

I'm looking for book/resource recommendations for architecting with microservices.

Given some set of functionality that my service needs to achieve, how can I best decompose it into microservices? What qualifies as a good design vs a bad design?

Design patterns for microservices are also of interest!

https://redd.it/eey3zg
@r_devops

I’m not a devops guy and I’m really struggling to understand the problem solved by secrets management from a devops perspective. What is the problem, with examples please?



https://redd.it/ehptbs
@r_devops

.NET project manager here. Having an impossible time hiring a devOps position. What am I doing wrong?

Hello all, as the post title states I am a .NET developer turned project manager for a cloud hosted IoT web application. The project is growing fast and I've been pushing to get a devOps person in house for a while and 6 months ago I was given the green light to hire one.

The problem is that we have received only 1 decent resume since the job posted. We are using all the usual job boards plus 2 recruiting firms. We have not posted a salary range but we are willing to go over 6 figures for the right guy. We aren't requiring an exam to interview and I think I'm asking reasonable whiteboard interview questions. (*for example*: Diagram a simple, three-tier web app, and then tell us how you'd write some custom monitors to track performance. *If they can do that I add on*: How would you handle 2x traffic growth over the period of a year? 10x growth? How would you help the dev team handle a page that tested well but has 30 second response time once deployed, and so on)

At a high level we are looking for someone to automate builds, automate some simple application testing, be responsible for our cloud environment, and provide direction when we are having cloud-related application issues (either through performance monitors or application telemetry). I'd be happy with a candidate that has 2 out of those 4. I am obviously not a devOps guy but I have worked on a team that had them in the past and they handled these responsibilities. Am I way off base with my expectations? Am I actually looking for a cloud engineer and a QA guy?

Any help is appreciated.

https://redd.it/ehoo2c
@r_devops

I'm beginning the Facebook interview process for the `Production Engineer`role. Graciously accepting any advice or study guides

So a Facebook recruiter reached out to me for a position on the PE Team. I haven't "properly" prepared for such an interview as I wasn't planning on even pursuing an interview at a FAANG for another year or so. I currently work as a DevOps Engineer at a startup with a lot of focus on AWS, Kubernetes/EKS, Terraform, Bash, and Gitlab CI, as well as a little Python.

I was provided with some study materials, but I was wondering if anyone here has interviewed for this position or currently works for Facebook that can also provide some insight and tips for acing this interview.

Also, if I do not do well in this interview, how does that persist in internal records and will I be able to interview at my "planned" time in the future after more studying, with a fresh slate?

https://redd.it/ehrjia
@r_devops

Terraform for loop issue

I have some terraform in which I would like to loop over an array and have it create a resource per item in said array.

​

Below is my current code, the first part of creating the repos works as expected, my issue is the second part of the life cycle policy the repo name line I get an issue, I know that I need to input the value of the repo name however I'm unsure how to.

​

resource "aws_ecr_repository" "repo" {
for_each = toset(var.repo_names)
name = "${each.value}"
image_tag_mutability = "IMMUTABLE"

image_scanning_configuration {
scan_on_push = true
}
}

resource "aws_ecr_lifecycle_policy" "repo_policy" {
for_each = toset(var.repo_names)
repository = "${aws_ecr_repository.repo.${each.value}}"

policy = <<EOF
{
"rules": [
{
"rulePriority": 1,
"description": "Keep last 10 images",
"selection": {
"tagStatus": "tagged",
"tagPrefixList": ["v"],
"countType": "imageCountMoreThan",
"countNumber": 10
},
"action": {
"type": "expire"
}
}
]
}
EOF
}

https://redd.it/ehps1e
@r_devops

Terraform & Jenkins

I'm on a mission to setup a pipeline in Jenkins for handling the Terraform/IAC for a specific application I have. I can't find anything real helpful on the internet outlining the steps for setting this up. I know there's a Terraform plugin for Jenkins, however it's outdated and doesn't support the "module" structure that the newer versions of Terraform support.

Can anyone point me in the right direction? I just need to setup a pipeline for init, plan, apply - however I'm also guessing I'll need to load the Terraform files to the agent since the plugin is outdated.

https://redd.it/ehge5c
@r_devops

Free DevOps Bootcamp on Twitch - T-Minus 7 days!!!

Happy Holidays all!

Just wanted to drop back in and remind you all that my free DevOps Bootcamp, called MASTERMND Academy, starts next Sunday on Twitch!

It's a 12 week course focused on learning the ethos and concepts of devops, so if you're someone who is looking to get into DevOps in 2020, check out the channel! The curriculum can be found at [https://academy.mastermnd.io/curriculum/devops/](https://academy.mastermnd.io/curriculum/devops/) and you can checkout the Twitch channel at [https://www.twitch.tv/mastermndio](https://www.twitch.tv/mastermndio)

There is also a Python bootcamp starting the day after this one if anyone might be interested in that.

Let me know if you have any questions!

P.S. I am running a stream on January 1st where we'll be setting goals and getting pumped for all the winning were all going to be doing this year.

https://redd.it/eherww
@r_devops

Cloud Engineer here....feeling lost on what I should do next?

Hi all,

I recently got hired as a Cloud Engineer (about 4 months ago) and have been super grateful as to how in the hell ended I up here. My role mainly has me programming automation and solutions regarding Cloud Security (governance and compliance).

However, being the goal-oriented person that I am, I have no idea where to focus my efforts and studies into now. I've just been feeling lost and frustrated, I hate not having something to strive towards.

So I thought it be best to ask those who have been in my position, or are currently working in the Cloud/DevOps realm as to what the possibilities in this field are. How do I find out what to aim for next? How do I find out what I should be doing?

Is there a certain position I need to be striving for? Are there certain skills I need to focus on acquiring? I find all these different disciplines (Security, networking, programming, etc) so I have no idea what to focus on. I guess I'm mainly motivated by what gives me the most money to better provide for my family.

I appreciate any advice and wisdom.

https://redd.it/ehfvhs
@r_devops

Wrong image being used?

I am using Helm to deploy to Kuberenetes which is running a Docker image from an ECR repo. This image is built uisng Jenkins in a simple pipeline. I've checked the Jenkins server and the code is as expected - ie right service name and port.

&#x200B;

When I check the describe on the pod for the effected service is show's it can't connect to port 8088 (the port the container should be on). The container that is actually running is a different micro service running on port 8086. I can see this by using kubectl describe pod <name> which also shows me the container uri, the ecr repo plus the tag both of which match with what it is should be pulling from.

&#x200B;

Service A and Service B use different ECR repos which should mean I can' have a cross unless I put Service A's repo name where Service B should be, which is not the case for me - I've checked and had others look too.

&#x200B;

I'm wondering in what cases could this issue come up where Kuberentes is running the wrong container yet manages to tell me it's running the right one?

&#x200B;

I've downloaded the container to my local and tested that it has the correct code as well.

https://redd.it/ehavey
@r_devops

Test automation: Why use a tool like Jenkins? Productivity Anti-pattern?

Disclaimer: I've not used Jenkins in production.

From my POV, using jenkins for test suites, is something of a productivity anti-pattern.

Scenario:

* pull + update local code
* pick bug to work out, begin coding
* complete bug fixes + commit
* push to dev repo
* run test suite via jenkins of the dev repo
* move onto new bug + begin coding
* realize tests fail
* need to stash local tree and switch-tasks
* lose 'local RAM' of what was going on within the initial bug pull in #1
* incur brain-space switching costs everyone is so fond of.


Am I missing something here?

Am I incorrectly opinionated in that tests should be sufficiently small + fast that something can be run on local machine?

Else risk the switching costs associated w waiting for a test-suite to run.

Is this not an anti-pattern for productivity?

https://redd.it/eh9kck
@r_devops

do any Windows terminals support tmux -CC like iTerm2 does?

iTerm2's tmux native integration via tmux -CC is perhaps the one thing that I would \*really\* hate to lose if I switched my work machine from a macbook to a windows machine. I'm just curious if anyone knows if this is supported by any windows (or linux/X for that matter) terminal emulators?

https://redd.it/eh13br
@r_devops

Makefile + Docker = CI Pipeline

I created this project [https://github.com/rosineygp/mkdkr](https://github.com/rosineygp/mkdkr). That helps to create agnostics CI pipelines. It can be shipped in any other big player like Github Actions, Giltab, Circle CI, Travis... and more, also you can run it locally as well.

So take a look ;)

https://redd.it/eh02lc
@r_devops

Help a Developer learn DevOps?

Guys, I am a developer. Worked mostly on .NET and Angular projects. Believe it or not, 6 months ago. I did not even know the meaning of word DevOps. My company had on premise applications that were deployed manually on IIS.
It was only recently that I was exposed to DevOps in my new organization.

I am in a desperate need of a course/courses that cover:

1. CI/CD pipelines for .NET and angular projects
2. Creating Tasks in Pipeline
3. Integration with Github
4. Deployment to Azure
5. Docker
6. Kubernetes (optional)

It is very hard for me to read long documentations. I am a very slow reader. It is very tiring and boring process for me to read long documentations. I lose focus while reading. Therefore it would be great if you could stick to video courses please.
Thank you.

https://redd.it/egvrjs
@r_devops

Monthly 'Getting into DevOps' thread - 2020/01

**What is DevOps?**

* [AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

**Books to Read**

* [The Phoenix Project](https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/1942788290) - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
* [The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
* [Google's Site Reliability Engineering](https://landing.google.com/sre/books/) - Google engineers explain how they build, deploy, monitor, and maintain their systems.
* [The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book
* [The Unicorn Project](https://www.amazon.com/Unicorn-Project-Developers-Disruption-Thriving-ebook/dp/B07QT9QR41) - the "sequel" to The Phoenix Project.
* [DevOps for Dummies](https://www.amazon.com/DevOps-Dummies-Computer-Tech-ebook/dp/B07VXMLK3J/) - don't let the name fool you.

**What Should I Learn?**

* [Emily Wood's essay](https://crate.io/a/infrastructure-as-code-part-one/) - why infrastructure as code is so important into today's world.
* [2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
* [This comment by /u/mdaffin](https://www.reddit.com/r/devops/comments/abcyl2/sorry_having_a_midlife_tech_crisis/eczhsu1/) - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
* [This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

**Previous Threads**

https://www.reddit.com/r/devops/comments/e4pt90/monthly_getting_into_devops_thread_201912/

https://www.reddit.com/r/devops/comments/dq6nrc/monthly_getting_into_devops_thread_201911/

https://www.reddit.com/r/devops/comments/dbusbr/monthly_getting_into_devops_thread_201910/

https://www.reddit.com/r/devops/comments/cydrpv/monthly_getting_into_devops_thread_201909/

https://www.reddit.com/r/devops/comments/ckqdpv/monthly_getting_into_devops_thread_201908/

https://www.reddit.com/r/devops/comments/c7ti5p/monthly_getting_into_devops_thread_201907/

https://www.reddit.com/r/devops/comments/bvqyrw/monthly_getting_into_devops_thread_201906/

https://www.reddit.com/r/devops/comments/blu4oh/monthly_getting_into_devops_thread_201905/

https://www.reddit.com/r/devops/comments/b7yj4m/monthly_getting_into_devops_thread_201904/

https://www.reddit.com/r/devops/comments/axcebk/monthly_getting_into_devops_thread/

**Please keep this on topic (as a reference for those new to devops).**

https://redd.it/ei8x06
@r_devops

Choosing the best future-proof platform

Hi, all. I'm the sole developer at a small brick-and-mortar business that has been spending the last couple years modernizing and building its IT infrastructure. We have a custom Laravel-based Point-of-Sale system, lumen-based API Gateway, and e-commerce site that are all running under AWS in a VPC environment. Our VCS is git via Bitbucket. Servers are currently separate EC2 instances, with our MariaDB on RDS. Currently, as I am the sole developer, I have been using Laravel's Homestead for my local development, and Laravel Forge to deploy to our staging environment for testing when we make a commit to our master branch. However, for our production deployments I am still performing the process manually at the end of each sprint.

Our basic layout is this: we have a public-facing web server that runs a Vue/Quasar SPA app. That app communicates with a Lumen-based API Gateway that funnels requests to our Point of Sale/API server (which sits behind a VPN) and back. So, that's three EC2 instances we have to maintain that talk to one another in various ways. I do plan to further segment to microservices in the future (i.e., completely split off the API calls so the Point of Sale is just a frontend to the API). But for now, we have it as-is.

I am planning to move towards a more cohesive CI/CD model in the new year, and with that, Docker containers at the very least. That said, I see so many new tools and platforms for orchestrating deployments and I am unsure what the most time-tested and forward-thinking ones are that won't be dead in a year or lose steam. My current plan is to migrate my local development environment to Docker containers, but use Minikube to run them locally, and then deploy to a Kubernetes Cluster on EKS, thereby bringing us down to one EC2 instance that can handle scaling our app as needed.

I know I am oversimplifying things to a large degree here (and I admit I am not a DevOps expert—I have so much to learn), but first, does this seem like a solid plan? Or should I be segmenting our three servers differently/more? Are there other tools or topologies I should consider? And then there is the question of how to handle the deployment to EKS. I am currently thinking of using Bitbucket Pipelines for that portion.

Basically, in addition to making our environment more team-friendly, CI/CD compliant, and future-proof, I also want to level up my own DevOps skills for career reasons.

Any advice or guidance would be most appreciated. Thanks in advance.

https://redd.it/ei6z7b
@r_devops

As a DevOps Engineer currently in a role, what does your typical day look like? If you were to express it as a percentage, how much coding vs ops do you actually do on a daily basis?

I come from an operational background and was curious how much OOP do you do in comparison to daily operational tasks? What task or group of tasks do you find yourself devoting most of your time?

https://redd.it/ei2b9p
@r_devops

Infrastructure advice for a batch cluster

I've recently been put in charge of a cluster that I had no hand in creating, which is primarily used as a batch system (HTCondor as the backend) and I have the opportunity to make some changes to the underlying infrastructure. I'm quite new to the world of devops, so I've spent some time going over the layout of the infrastructure and it is currently set up like so:

OpenStack is the host, with a terraform config to setup the different types of nodes. Ansible, to set up docker, condor, and Nvidia drivers (among other base tools). Then, docker images are built for each type of node to include the required tools for each (worker node may require python ML libs, login node will need very little, etc). And finally, kubernetes is used to orchestrate the images across the nodes.

At the moment, the system works well and it is easy to roll out changes as I simply need to modify a docker image and provision it with kubernetes.

I'm wondering if there are issues with this type of setup though, particularly with the docker/kubernetes layer. Would it be beneficial to gut that layer, and simply provision each type of node through ansible or terraform straight on the VM? What kind of benefits would that have (if any)?

One of the issues that has come up with this setup is if a job requires a container, we run into a container-in-container situation. I have read about how this is possible but advised against, and instead sibling containers should be used. One issue with this though is from my understanding of this setup, the docker containers are an additional layer of security, so if someone was to run an adjacent container, it would be possible to mount the host root directory.

As stated, I'm new to this world and I wanted to see what the wider DevOps community thinks. I don't plan on making any sweeping changes until I am confident in my knowledge (which I am currently not) so if there are any good resources I should read for this kind of setup, feel free to share them as well, I want to learn all that I can. Thank you!

https://redd.it/ei8li5
@r_devops

I have two vpc stack templates that produce two seperate vpc environments. How can I consolidate them into a single vpc stack? (code below)

So I have a root template that deploys two vpc stacks. Like this:

root.template
\-> vpcstack1.template
\-> vpcstack2.template

In each template vpc resources (vpc, security groups, subnets, ect.) are deployed. They both belong to an app stack on their own vpcstack1 goes to app1 and vpcstack2 belongs to app 2. I'm being required by my organization to deploy both apps into one unified vpc stack. Which makes perfect sense. Why do I need 2 vpcs for 2 apps when I can have 1 vpc hosting 2 apps and all their required network resources.

Below is a snippet of my vpcstack2.template

Vpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock:
Ref: VpcNetBlock
InstanceTenancy: default
EnableDnsSupport: 'true'
EnableDnsHostnames: 'true'
Tags:
- Key: Name
Value:
Fn::Join:
- '-'
- - vpc
- Ref: AWS::StackName
- Key: Platform
Value: nexus-repository-manager
- Key: Version
Value:
Ref: NexusRepositoryManagerVersion


This gives me my second VPC and all the resources in them. How can I get rid of the second vpc and take all the network resources and dump them into vpcstack1.template for a unified stack?

https://redd.it/ei6awx
@r_devops

VMware completes the acquisition of Pivotal. What will happen next?

Business Insider reported saying Pivotal is preparing for layoffs ahead of its $2.7B acquisition by VMware, and this sparked a protest by Pivotal employees internally and then went out on the public. As per the report, Employees started circulating an open letter asking VMware and Pivotal leadership to retain the workplace operations team.

When these acquisitions happen, it is usually so much confusing.
What to expect next?

https://redd.it/ehxtv2
@r_devops

Here's an off the wall question. Anyone in here dealing with or providing care for those with chronic or major health issues, what is your opinion of 'operations' in the medical industry?

After two years and about a million dollars spent on my wife's health, I have to say the entire thing is not inspiring confidence.

https://redd.it/ehws4a
@r_devops

IPAM / DCIM for the cloud

My organization maintains a spreadsheet with a list of all our subnet assignments across several dozen AWS accounts. This list is updated regularly by several people and is not always up to date, and overlaps do happen.
As our deployment continues to scale, the spreadsheet will not. So I'm looking for an IPAM solution, preferably something with auto-discover, that we can use to centralize management of limited networking resources.

I looked at Netbox which seems more geared towards physical DCs, and also doesn't have auto-discover capability (or at least I couldn't figure it out). Before I go and write a script to loop through all our AWS accounts and populate all the VPC and subnet data, does anyone have a better solution?

Thanks

https://redd.it/ehz9mg
@r_devops