Rundeck and Ansible inventory management

For the people using Rundeck and Ansbile how do you manage inventory which will need different user accounts?

My inventory looks like this:

Lets say I have an inventory file like this

[webserver]
wsp01
wsp02

[webserver:vars]
ansible_user=apache

[database]
dbp01
dbp02

[database:vars]
ansible_user=postgres

When I create a project, I don't know what to put to discover the nodes because each type of server(web server & database) have a different user.

https://redd.it/frpx5p
@r_devops

Unable to run AWX on port 8080

I'v installed AWX on a server host on the google cloud platform.

When I initially installed AWX it ran on port 80 which worked fine. I then wanted it to run on port 8080 so I could later set up a reverse proxy with nginx, so i changed the inventory file and ran the playbook again but I could not connect to the app on port 8080.

docker ps output:

c95d21f1c0f9 ansible/awx\_task:9.3.0 "/tini -- /bin/sh -c…" 22 hours ago Up 22 hours 8052/tcp awx\_task

88cb561af12f ansible/awx\_web:9.3.0 "/tini -- /bin/sh -c…" 22 hours ago Up 22 hours 0.0.0.0:8080->8052/tcp awx\_web

e42db0627be7 postgres:10 "docker-entrypoint.s…" 24 hours ago Up 22 hours 5432/tcp awx\_postgres

02ee5536be70 redis "docker-entrypoint.s…" 24 hours ago Up 22 hours 6379/tcp awx\_redis

f9d017d25160 memcached:alpine "docker-entrypoint.s…" 24 hours ago Up 22 hours 11211/tcp awx\_memcached

I am confused as to why it works on port 80 but not on port 8080 after re-configuring the inventory file. am I missing something?

Thanks for the help :)

https://redd.it/frp8o5
@r_devops

Need Help with zookeeper and nifi cluster set up

Hi,

I am a noob here and have been given a task for nifi with zookeeper set up on the same machine and I am unable to wrap my head around it.

Can someone please help with some resource

https://redd.it/frp7jr
@r_devops

CloudOps Monthly Newsletter: March

I write a monthly newsletter that was originally being published directly in the email, but I've recently decided to move it to my website instead. I believe this will make it easier to share.

[https://www.thecloud.coach/newsletters/march/](https://www.thecloud.coach/newsletters/march/)

I focus on specific core technologies like Terraform, GitLab, Ansible, AWS and public Cloud in generally. I also include interesting articles and important pieces of knowledge sharing that I find.

https://redd.it/fsa1hz
@r_devops

Working From Home [Self Isolated]

I always thought working from home would be the dream, maybe it is, but not when it's forced upon you!

I'm coping with it ok but I find it hard to stay focused when it's just me in a room, any tips from those that work remote full time?

I also decided to capture just how mundane my day was yesterday in a timelapse 😂

https://youtu.be/q6AZTa-VEZI

https://redd.it/fsb01o
@r_devops

Are there any tools you find yourself using more now that you're home?

I'm not sure why, but I'm curious if there are certain things you find yourself doing more while working from home than you were doing when you were in the office?

https://redd.it/frm1en
@r_devops

Can anyone help in API implementation?

I am facing a problem with the implementation of API, it's a bit complicated thing and I'm done searching it, can anyone help?

https://redd.it/frp5ng
@r_devops

Do you decouple build logic from build server?

I'd like to get a sanity check on my approach to build servers and scripts if you will. I've been writing and maintaining build scripts in CI servers for years - Jenkins, Travis, Bitbucket Pipelines, Gitlab, etc. Each of these servers has some kind of "trigger script" (my name)- Jenkins UI scripts, Jenkinsfile, travis.yml etc what sits in source control but works only on the server it was designed for. My approach to trigger scripts is to write as little as possible in them as possible - normally just some trigger conditions (build on tag for travis for example), set env variables, etc and then I call a single "build. sh \[some args\] ".

Build. sh also lives in source control - it's a monolith script, normally written in bash. It contains \_all\_ build logic, and is written to be run by a human on a developer PC, and also by Jenkins on a build system. It has sane/safe defaults, and these can be overwritten with shell or env arguments. I use no Jenkins plugins at all, except for VCS watching and credential management. I don't use pipelines, mainly because I work in gaming where a full pipeline run could take hours and span many different machines. Instead I wrap each pipeline stage in an independent HTTP microservices and chain those together outside of the CI server, but often the CI server is responsible for triggering the first microservice in a chain.

My approach is to treat the CI server as just another user on just another host machine, and all users and hosts machines are equal. A CI server is just a cronjob & credential manager, with a UI. Running a build should always be achieved by calling a single shell script, with optional arguments to modify that.

Based on your experience, am I missing out one some fabulous new advancement in CI tech (I use containers when possible)? Is this approach over or under-engineered? Is it more error prone, or difficult to maintain by others?

https://redd.it/fschiv
@r_devops

Fetching gitlab usage information

Do you guys have any idea how to get all the information about the projects running in gitlab in term of number of commits and builds, and if such info is exposed by the api so i can write an exporter that collects them ?

https://redd.it/fs9bw5
@r_devops

Why is DevOps so hard to get right?

Check out my article and let me know what you think! What challenges does your organization face? Share your experience!
“Why is DevOps so hard to get right?” https://link.medium.com/1LUEnzS9i5

https://redd.it/fsgi73
@r_devops

Automatically generate SSH config files for AWS

I usually have to deal with more than two dozen servers on AWS with different jumphosts. I wrote a script to simplify generation of SSH config files since servers are often deleted and replaced. It's been a useful utility in my toolbelt and I hope it helps other people than me.

[https://github.com/danihodovic/generate-ssh-configs](https://github.com/danihodovic/generate-ssh-configs)

https://redd.it/fsi4vq
@r_devops

Would you take it?

I just received an offer as a Senior DevOps Engineer for a small company. The role is following DevOps practices but is maybe 15% in cloud. This role for me would be a large pay increase (\~35%) and better benefits, less hours (currently working 70-80 hours a week).

The organization is using CI/CD, Terraform, Ansible, Containers, etc..

My main question is, do you think it would be okay to move from a company that is 100% in the cloud to a company that is mainly on-prem? Though my current company is 100% in the cloud, it isn't using DevOps tooling/practices as much as I'd like (I'm the only one using Terraform, Ansible, CI/CD within my cloud engineering team).

From an organizational perspective, I'm confident in the move. I really want my life back, the people seem good to work with, and the company is doing good in the world.

https://redd.it/fsgvyx
@r_devops

Simple mail server with REST API.

Hi,

Been searching a bit, I'm looking for a simple mail server with a REST API for user creation. The use case is embedding a dedicated mail server as part of a Docker stack I'm deploying, ideally it would come with a web interface.

I looked at mailu, doesn't have a REST API and Postal which has an incomplete API (no user creation).

Any suggestions are welcome!

Thanks :)

https://redd.it/fsi35p
@r_devops

Automatically deploy Jitsi Meet (free videoconferencing) to AWS with Terraform

Hello

I wrote a blog post about deploying Jitsi Meet to AWS with Terraform. Maybe some of you are challenged with the task of providing videoconferencing infrastructure and this will help.

I would really appreciate some feedback! :)

​

[Terraform: Deploy Jitsi Meet to AWS](https://avasdream.engineer/terraform-jitsi)

​

Stay at home and healthy!

https://redd.it/fsf7xz
@r_devops

How did you learn networking from level 0?

Projects? Books? For someone who wants to learn networking at a fundamental level, does anyone have any resources to get the practical knowledge down?

https://redd.it/fsh3r4
@r_devops

Help with Apache and Tomcat memory config!

Hello guys, here's the deal. I inherited a system that uses mostly webservers with Apache and Tomcat, on Java8, and they run on 4CPU, 7.5G RAM machines (specifically the c5.xlarge instances of AWS).

We are experiencing heavy traffic and adding more clones of those to the mix work, but its way inefficient. The fact is that I know squat about how to fine tune Apache and Tomcat, and while I'm adjusting daily, I'm sure there might be someone looking at these values that can tell me if they look OK, even OKish, or maybe super wrong. These values came from an old hardware infrastructure and, while I changed certain things, some stuck on these old values that I don't dare touch too much.

Here are the values:

<IfModule mpm_worker_module>
ServerLimit 22
StartServers 10
MinSpareThreads 75
MaxSpareThreads 250
ThreadLimit 64
ThreadsPerChild 25
MaxClients 550
MaxRequestsPerChild 1000
</IfModule>

and Tomcat config

-Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Dsun.java2d.fontpath=/usr/share/fonts/truetype -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dlog4j.configuration=file:///etc/tomcat7/log4j.properties -Xms2560M -Xmx2560M -XX:MaxPermSize=512m -XX:NewSize=20M -XX:MaxNewSize=40M

I am finding at the moment that of those 550 MaxClients, when it get above 450 or so it's tomcat that is failing to respond on port 8009. Do I need more heap?

Anyway thanks for your time if you read all this! Thoughts?

https://redd.it/fseaok
@r_devops

Python devops showcase without the cloud

Hi guys, I am trying to think of an idea of a personal project, not a huge one, but enough to show people at interview about my skills in Python, but relating to Devops. I know most people probably will think the Boto3 library in automating stuff on AWS, but I'd love to hear about ideas that doesn't involve the cloud but still would need python to automate stuff on Windows or Linux, or networks, anything relating to the CI/CD pipeline. Maybe Saltstack python codes would be helpful?

https://redd.it/fs7os9
@r_devops

nested terraform outputs

I have an output

output "route_tables" {
value = module.oci_network.route_tables
}

That results in outputs in the following following format

route_tables = {
"cool_systems" = [
{
"compartment_id" = {}
"defined_tags" = {}
"display_name" = "cool_systems"
"freeform_tags" = {}
"id" = "id-123"
"route_rules" = [
{
"cidr_block" = "0.0.0.0/0"
"description" = "sauce"
"destination" = "car"
"destination_type" = "CIDR_BLOCK"
"network_entity_id" = "foo"
},
]
"state" = "AVAILABLE"
"time_created" = ""
"vcn_id" = "vcn-123"
},
]
"tool_systems" = [
{
"compartment_id" = ""
"defined_tags" = {}
"display_name" = "tool_systems"
"freeform_tags" = {}
"id" = ""
"route_rules" = [
{
"cidr_block" = ""
"description" = ""
"destination" = "0.0.0.0/0"
"destination_type" = "CIDR_BLOCK"
"network_entity_id" = "foo"
},
]
"state" = "AVAILABLE"
"time_created" = ""
"vcn_id" = "vcn-123"
},
]
}


The remote state is read in via a data sources called vcn.

I want to read the "id" attribute but this is nested within the cool\_systems, from this data source how do I access this nested element?

If this was flat, it would simply be

route_table_id = data.terraform_remote_state.vcn.outputs.route_tables.cool_systems.id

but that results in the following error.

| data.terraform_remote_state.vcn.outputs.route_tables.cool_systems is tuple with 1 element

https://redd.it/fsa69t
@r_devops

Showcasing my latest project: a Hacker News Who is Hiring aggregator

Hey r/devops,

I checked, and I don't think showcasing is against the subs' rules–apologies if it is.

During the Christmas break, I launched this Who is Hiring aggregator at [https://www.whoishiring.work](https://www.whoishiring.work/) ([source code](https://github.com/jerroydmoore/whoishiring.work)). Check it out and tell me what you think. I'm also [accepting feature requests, bug reports, and PRs](https://github.com/jerroydmoore/whoishiring.work/issues/new). Now, there are a few Who is Hiring aggregators out there already, but I didn't very much like their look and feel, and it was a great opportunity to practice my full stack + devops skills outside of work on a brand new project!

Now let's talk about the infrastructure. The frontend is a gatsbyJS project that is hosted on an S3 bucket. HTTPs was enabled via CloudFront. CloudFront + S3 hosting has a weird quirk about requiring the trailing slash in the URL, which was solved via a Lambda@Edge function. The data storage is an RDS instance; I had considered Aurora serverless, but they didn't support the version of postgres I wanted, and they weren't offered in the us-west-1. The backend is a nodejs expressjs application hosted in Lambda hooked up to API gateway. Another nodejs application loads the Who is Hiring data into the database every 15 minutes. Finally, last week, I've started adding everything is terraform. It's currently about half way complete.

Some of the design decisions I made about the infrastructure focuses on the backend and database choices. I do want to go serverless with the database, since the database is currently my largest expense ($209/yr for a db.t2.micro). For the backend services, I decided to write them in nodejs using expressjs, which is 100% compatible with lambda out of the box and requires a shim. However, this will allow me to transition from lambdas to ECS with very little ease if I ever need to. I didn't start with ECS, since you're paying for compute time even when there is zero traffic. Finally, I wanted to launch everything in a VPC, but I didn't want to pay for an NAT (minimum $420/yr).

Let me know what your thoughts are and feedback is always welcome!

https://redd.it/fs3k4w
@r_devops

I've been bored, jotted my thoughts down on cloud vendor lock-in

This article is more for the guys running SaaS products, not for people running corporate offices or cots products with small "development" teams.

Here's the article, but if you don't want to read it, I put the main take away below.

https://blog.kwnetapps.com/vendor-lock-in-on-the-cloud/

The main gist is this: Don't worry about vendor lock-in.

The main thing you can take away from the article is this:

> You are a race car team owner and manager. AWS cars and Microsoft cars run the same as normal cars, but they also support a few extra features not in normal cars. These features can make them faster, more convenient, and more reliable.

> You have some drivers, mechanics, pit crew, etc. and they are all experts with normal cars. So everything on AWS cars or Microsoft cars that makes the car work and function like a normal car is the same, and these guys can work on them and drive them. The extras they’ll learn as they continue to work on them, they’ll gain more expertise as time goes on.

> Now you need to decide.

> You can get AWS or Microsoft cars and stick with one brand. Then make sure your drivers and crew understand how to drive them and get the extra features out of them. Keeping up with the metaphor, let’s say those extra features are like automatic tire changes while driving, unlimited fuel, etc. Starting out you’ll do fine, driving the cars like normal cars, but as you learn the features you’ll start to blow your competition away.

> OR

> You can worry about AWS or Microsoft raising prices on your cars and parts. You decide to go with both to balance this risk and miss out on all those early wins in the series because your drivers and crew weren’t specialized. Hell, some of them were still putting the cars together when the races were starting, unless you decide to hire more crew.

https://redd.it/fsobh6
@r_devops

Terraform enterprise tutorial

Does anybody know of a good terraform enterprise tutorial?

I learn best by doing. I would love to see a step by step that I can follow along.

Even if it’s not enterprise, I’d still like to read about it..

https://redd.it/fsqjw8
@r_devops