Building in a chroot for pxe booting nodes

So I've got a slightly different situation but I'd really like to use packer.

I've got a bunch of nodes that boot from PXE. They boot into an NFS root, grab a squashfs on that NFS root, load it into ram and then switch root into that and ditch the nfs.

The images, both NFS and squashfs are just built as flat folders, so you can chroot into them on the build/boot hosts. No VM's or other stuff. But I'd really love to use packer to make them.

Currently we have to by hand chroot into the image, install stuff/change settings and whatever, then squashfs the resulting folder up.

It would be far nicer to just edit the packer files and run a build.

I've looked at the available chroot builders and they don't quite do what I need.

I guess I'm interested in how hard it would be to make a new builder/whatever to do these from scratch. I mean its just a "rpm --initdb --root blah" "rpm --root blah -ihv centos-release-whatever.centos.x86_64.rpm" and a "yum install --installroot=blah bash yum rpm" to get to a usable chroot state.

I realise it's a bit of a departure from packers standard mode of operation but I don't think it's too far.

Any tips/advice or builders I've missed (no kvm use please) would be welcome.


EDIT: It looks like I might be able to modify this builder https://github.com/summerwind/packer-builder-qemu-chroot to do what I want. Now I just need to learn go.

https://redd.it/oapnc5
@r_devops

Vmware infra automation

Hello devops!

I've started an automation related role and looking for some advice,

I need to automate full deployment of esxi - or at least as much of it as possible.

This means everything from vswitch, datastores, users on the vcenter, roles, permissions, etc to VM deployments.

The latter part - VM deployments there are many examples online but not so much the first part of the more basic infra.

Anyone here with experiance of doing the above or part of it?
I know there are tools such as ansible, terraform, chef, puppet but each one of them has its drawbacks and I would like to hear which tools you guys are using.

​

Thanks greatly.

https://redd.it/oaqzis
@r_devops

Azure DevOps over Spinnaker+Jenkins combination

Why would you choose Azure over a combination of other open source CICD tools. ? i understand that the TCO will be less. But i want the view point of an developer/devops engineer.

OR

Why will you not choose Azure DevOps

https://redd.it/oasorm
@r_devops

Azure Pipeline running task in background?

Hey there,

I am in the need to start a node.js server before I am running the tests in the pipeline. But using plain tasks wont work, since Azure waits until a task completes.


We have a multi-stage pipeline set up and run unit tests in parallel as well.

I am really lost here, since I am not too used with Azure pipelines but other CI where you can define a task as a background task and it continues running, while the other tasks run.


The documentation on this one is really lacking so I hope someone here can help me out.

https://redd.it/oatbaw
@r_devops

Is there some way to make datadog monitors/alerts less full of useless info?

We are using datadog monitors/alerts in our company.

The developers complain that it is hard to see what the alert says, each alert doesn't contain just the info you need, it contains lots of extra junk.

I'm for example tried removing tags from the alert, but if your query makes use of some variable for example {container_id} then the tags in the alert will also contain this. Even if your alert contains no actual tags in the frontend.

https://redd.it/oau4wb
@r_devops

"Code Smells" not reflected in sonarLint (IntelliJ IDEA)

I've managed to connect my remote sonarQube to sonarLint in my IntelliJ project. The bugs + vulnerabilities are shown. However, Code Smells are not showing up.

Have anyone experienced this before? And do you know how to resolve this issue? Thanks!

https://redd.it/oasa04
@r_devops

Virtual RabbitMQ Meetup - RabbitMQ and PHP with PHPAmqpLib

In the lead up to RabbitMQ Summit 2021, our virtual RabbitMQ users conference, we're running an exciting free Meetup. As well as a fantastic talk, we'll introduce you to a few of the key speakers at the event and have a social networking session for all attendees on the Toucan Lounge (an awesome virtual space that is perfect for Meetups and socialising).

You can register to join free at https://www.meetup.com/London-RabbitMQ-Meetup/events/278994466/

https://redd.it/oavxxa
@r_devops

Jenkins / Terraform - Full automated deployment and clean up based on git branches/merge events

I realise the title is misleading. I am asking if it's possible and if someone has some pointers / links / etc to help me go through with it.

I've jenkins setup to manage QA/UAT and production pipelines but while the production is quite more "stable" (deploys the applications in kubernetes) the QA/UAT one is much more dynamic.

As a starting point I'm having that environment bound on a specific branch but as the dev team is growing, there will be multiple parallel features on a same product and I think it's better if each feature would have it's own environment deployed, connected to a branch.

What I am trying to achieve here is to have each branch starting from develop having a terraform config (a default template) deploying the resource, then the application deployed. Let's skip the details for the "app deployment" as that could be a docker image or barebone code, it really depends on the application and its current state.

I know I can do that using Jenkins multi-branch pipelines and even if I haven't done that yet, there are plenty of tutorials etc. I wonder, however, if it's possible to write that pipeline in groovy instead of using the UI as all tutorial I found seems to do. I suppose so.

However, the question I have is: can I make it so that when a branch is merged, closed or deleted, the entire deployed resource (via terraform) is removed as well? Can you point me to the right documentation/tutorials/etc for, I assume, jenkins?

I searched online in these past couple of days but I must be using the wrong terms because google is not returning anything useful or is getting confused by having jenkins, terraform, git in the same query.

https://redd.it/oavrin
@r_devops

Can I Please Get Suggestions on a Good Linux Networking Course - Not For a Certification

I am increasingly finding myself in situations where I need to make networking configuration changes. Each time I encounter one of these problems I have to spend time sifting through tutorials and documentation online to figure out how to accomplish my task. I would like to get some recommendations for some good Linux Networking courses for someone in a DevOps role.

I would rather not take a 25 hour in depth course, I was hoping for something more in the 4-10 hour range that will help me understand DNS, VPNs, GateWays, Routes, etc. I have a basic understanding of networking, and I have several AWS certs and have a decent understanding of networking in the cloud, but even something as simple as setting hostnames for local nodes required me to do some research last week. Instead of having to do this every time I run into a problem, I would like to just have a practical knowledge so that even if I don't remember how to do something off the top of my head, I will have a better idea of what resources to seek out to find a solution.

I am not really interested in getting any kind of networking certification, so I was really just hoping to find a course that will teach me the practical things that I need to know when setting up a local K8 cluster, or creating a local DNS server, etc.

Thank you in advance for your suggestions!

https://redd.it/oawmwe
@r_devops

Looking for an alternative to EC2Box (multiple SSH terminals for EC2 via web)

Basically looking for a tool I can run in a container on our dev server and access via browser to open multiple SSH terminals connected to EC2 instances. Simple, fast and modern. We've been using EC2Box for too long (and it still works I guess), and I'm sure there are better alternatives right now that are currently maintained.

I've looked at some candidates like Boundary but I'm not sure those are simple enough, maybe there are simpler tools out there?

https://redd.it/oazd1d
@r_devops

Integrate Rundeck with Github

Is there a way to integrate Github with Rundeck where when a job runs, the 1st step, it will goto github and download the latest version of a script and run the latest version?

https://redd.it/ob06jj
@r_devops

How to set how much time a kubernetes pod is replicated when a node goes down? Five minutes is too slow

I have a cluster and when a node goes down it require 5 minutes to replicate the pod on another node. I would like to have that this will happen in few seconds. Where can I set this? Thank you

https://redd.it/oaxykw
@r_devops

What are your thoughts on Portainer.io

The GUI looks interesting.

Still testing some of the features though.

What do you guys think?

https://redd.it/ob0wic
@r_devops

Rest api and apigee

Hi guys my new company wants me to learn rest api and apigee
My role there is a devops engineer
The new fancy term they call for build and release engineer / ops engineer
What should I know about rest api and apigee w.r.t to devops
I already know what is that just want to know where it will be used in our role

https://redd.it/ob4565
@r_devops

List AD users as a drop down menu option in Rundeck

Is there a way in a Rundeck job to list a dynamic and updated list of all Active Directory users (in a specific OU) so the user can select a valid user?

https://redd.it/ob40ns
@r_devops

Can someone pls explain like i'm 5 what the fuck hashicorp terraform is?

I'm about to graduate with a degree in information systems and feel fucking clueless.

Why is hashicorp so hot? I'm talking about terraform mostly. Like ... why don't the public cloud vendors have software to provision themselves? Why do you need something like this? And what the fuck actually IS terraform provisioning?

And I've seen this a number of times but still don't totally get it ... why do you need terraform and ansible?

Is terraform the reason that everyone is always raving about hashicorp, or is there some other thing? I understand vault is big and that makes sense to me from a tech perspective. So maybe it's just those two.

I don't want to derail this subreddit into idiocy for too long, but I have a hunch that a TON of people that are either newcomers or only dabble in DevOps will benefit from having this explained in 5yo terms.

Thanks so much!

https://redd.it/ob47kx
@r_devops

What is the best way to become an “expert” in systemd?

I’m sorry if this question is a bit sparse on details, but I started a new devops job recently and was told that it would be great if I could master systemd, as we only have one person who has expertise in it.

I’m currently going through the systemd docs right now but am a bit vexed on how to gauge grasp / mastery of systemd.

Thanks for your thoughts and comments!

https://redd.it/ob869j
@r_devops

Devops vs DevOpsSecs

What does DevOpsSecs discipline have over traditional(loosely termed) DevOps? Is it knowledge of DevOps culture + tools with a focus on security? I'm curious to know what the professionals in either organization have to say. Thank you all in advance

https://redd.it/ob7p01
@r_devops

Closing CFP DataOps Summit 2021 closing today (6/30)

If you are interested in showcasing your leadership in the DataOps space. See more info here. https://www.dataopssummit-sf.com/about

https://redd.it/ob7iid
@r_devops

Looking for API feedback on a TypeScript DSL for dynamic CI config

As I've spent more time on my team's CI pipeline and investigating various CI providers (we've had a lot of trouble with Circle and I'd like to get away from them if possible), I've noticed that most providers/platforms now support dynamically generating the CI pipeline config at the start of the job. It's a [recent addition to Circle](https://circleci.com/blog/introducing-dynamic-config-via-setup-workflows/), but others, like GitLab, have supported it for longer. In Buildkite, it's almost mandatory, since your initial CI config has to be set in the UI -- if you want a pipeline, even a static one, checked into your repo, you have to create a minimal bootstrap config in the UI that clones your repo and manually selects the pipeline file to run.

I see good reasons for this -- YAML + bash isn't the most elegant or expressive combo for big, complex pipelines. I've thought about writing some tooling before, but I didn't like the idea of having to install my preferred scripting language onto every CI node to run scripts. The dynamic config feature has given me a new idea, though: a TypeScript DSL for **generating** pipeline configurations.

The idea is that you'd run a TS script as your CI-generation job, the output of which would be a pipeline config for your platform of choice. Because there's a lot of overlap in the functionality of different CI systems, I think it should be possible to define jobs and steps in a mostly-platform-independent way, then "compile" them to the platform representation. I don't expect to be able to migrate CI providers just by swapping output targets -- I'm sure every project has platform-specific details to work out -- but I do think it could make migration *easier* and reduce lock-in.

[This is an example](https://gist.github.com/davidwallacejackson/a5e4f02115b23d06c7588e9d5ed7535a) of what pipeline definitions look like in the API as I've implemented it so far -- the working library name is `build-gen`. I'm not married to any of the design decisions, of course, or even to actually using this thing. It feels like a good direction to me, but this is why I'm looking for feedback -- I'm wondering if there are good reasons people haven't done things like this before -- or, maybe they have and I haven't found them, or maybe there are just better tools to solve these problems.

And a few stray thoughts on my reasoning:

* Most CI platforms have some abstractions built in for conditional execution, "defined steps" and matrix running -- to me, this seems like a signal that most people will eventually want traditional programming constructs like \`if\`, functions, and loops in their CI pipeline, as it grows.
* I chose TypeScript because it's my favorite language to write in, but this tool or one like it could of course be written in any language. I was actually working on this before the Circle blog post I linked above was released, but I note that they express interest in a TypeScript-based config tool as a possible future development, which feels encouraging.
* Type-checking and documentation comments (in TS or any other "full" programming language) offer some nice ergonomic benefits that apply just as well to CI as they do to anything else.

Anyway! Thank you for reading all of that. Would you use this thing? Does it look interesting to you? How are you handling templating and dynamic configuration in your CI systems right now? Any insight would be very welcome.

https://redd.it/oaqs9r
@r_devops

opensource development competition

https://www.facebook.com/100009526965600/posts/2936613869999483/?app=fbl

https://redd.it/obbwq4
@r_devops