Help me understand how to handle access and firewall rules for APIs on different environments

I need to handle an application on AWS that has several API endpoints exposed to the world.

In Dev, Test and Quality environments, endpoints are only available for certain public IP addresses only.

In Production, the endpoints are going to be open to the world.

The application also has several management admin APIs that are available in all environments only from certain public IP addresses.

I need to setup up firewall rules on AWS for the 4 environments, trying to figure out the best way to do it, any insights would be greatly appreciated.

https://redd.it/m2n60l
@r_devops

What do you think about new virtual recruitment process(codility and hirevue)?

I have a confession:Codility and Hirevue frustrates me a lot because now we have to score alteast 95-97% in codility test which kinda of seems unfair. Earlier i remember that for erisscon I went to their campus and gave all the test especially coding test. In that test I didn't get all the coding question but i explained how I would approach the problem using pen and paper they were impressed by that and in the feedback i was rated among one of the highest ranking coder on that day. There is one more piece of virtual recruitment i dislike the most which is hirevue: these automated video test are impossible to pass.

https://redd.it/m2n3qb
@r_devops

OneDev 4.2 released with ability to annotate source with Jest/ESLint information

OneDev is an open source self-hosted DevOps server, with ability to manage issues and git repositories, as well as built-in CI/CD support.

With 4.2 release, one can setup CI/CD job to publish Jest and ESLint reports for queries, statistics, and source annotations. Checkout this short article for details:

https://robinshen.medium.com/annotate-source-with-jest-eslint-information-in-onedev-c622641caa45

https://redd.it/m3cflj
@r_devops

Random bad HAProxy-Galera check

Hi guys!

I have a bothering problem. I set it up 2 haproxy vm and a mysql galera cluster with 3 nodes. I use a simple passwordless haproxy user for check, this is working fine base on the logs.

But, for safety sake i want to setup AWS R53 check. I have a simple php what log in to mysql with this haproxy passwordless user and if it can, it's print out Ok. If no, print the error message.

On haproxy-1 i have no problem, everything is working fine, but on the haproxy-2 i got random error message: Connection failed: MySQL server has gone away

The joke in the whole thing is that the system has ben set up with ansible, so the 2 haproxy and the 3 galera node is same config, everything is same.

Have you any idea?

https://redd.it/m2kvlz
@r_devops

Can someone tell me how to create a shell script like this?

I come to know this installation shell script from
https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh
and I hope to start from the basic.


Can someone point me to some resources where I can start building some shell scripts that I can automate things like that link?

Thanks.

https://redd.it/m2h4nm
@r_devops

Jenkins inside Kubernetes or not?

I'm planning on doing a home lab to learn devoos. However I'm not sure if Jenkins and other services like gitea should be in a pod inside Kubernetes or If It is better ti have It on a separate VM and why

https://redd.it/m3lc3r
@r_devops

How to create/update registry settings in Windows via Hashicorp Packer

Hi all,

I am building Windows 10 multi-session images in Azure. I install about a dozen apps, add some users, etc. Everything works great except that I cannot figure out how to set registry values. I get strange errors when they attempt to run...and they fail. I am assume this is a security issue (UAC). What confuses me is that I am performing other administrative tasks (like creating a local admin) that work fine.

Is there a trick to set registry values in Packer? My Powershell statements are correct, it’s just that I cannot get them to take when running the Packer build process. Is there a way to elevate and bypass UAC? Is it something else?

Thank you to anyone who can help. I have been working on this for several days.

A few details:

* Im using the Powershell provisioner and the Azure ARM builder
* The commands I am trying to set are:
* New-ItemProperty -Path "HKLM:\\Software\\FSLogix\\Profiles" -Name "Enabled" -Value "1"  -PropertyType "DWORD"
New-ItemProperty -Path "HKLM:\\Software\\FSLogix\\Profiles" -Name "VHDLocations" -Value "\\\\10.xx.xx.xx\\FSLogixShare"  -PropertyType "MultiString"
New-ItemProperty -Path "HKLM:\\Software\\FSLogix\\Profiles" -Name "ProfileType" -Value "3"  -PropertyType "DWORD"

In the doc I see ways to enter the elevated user/password but I assumed that the Packer user that runs and connects to the machine is already an admin.


Any thoughts are much appreciated.

https://redd.it/m3ku8z
@r_devops

WordPress workflow - newbie edition

Hi peeps,

First, I hope this question fits in this sub - I found a similar one anyways, just super older.


I've been hired to handle a company's website which is based on WordPress.


I'm trying to figure out what the best workflow would be, taking in consideration all the involved parties.

So far I'm thinking...
Local by Flywheel to develop locally, then Github would save the changes and Buddy would be used as the middle man, to update the server's version (AWS) whenever changes are added to the main branch. Thoughts?

I've heard of Docker as well, but I have to dive into that one yet.


Keep in mind - I'd be working on design / themes and plugins mainly, and those are the only files on the github repo and (hopefully) the only ones to be updated through each push, but I have no idea if it'll work like that. + that'd hopefully allow the Content Creation crew to go into the WordPress dashboard and put up posts as usual, without stuff getting overwritten - which again, I have no idea if it'll work.

Another step would possibly be to do all of that in a staging site, and then use a plugin to get a static version which will be used for the production site - to hopefully make it faster as well.

Any advice or tips will be highly appreciated. :)

https://redd.it/m3i7u7
@r_devops

How do you guys are getting most out of Prisma Cloud

Hello Guys,
We are using Prisma cloud mainly focusing on vulnerabilities and runtime alerts/events. Every time we receive an alert of container runtime from Prisma cloud we have to involve a dev team member and verify if it is a false positive or a genuine alert(I am part of sec team and I don't have access to the machines directly). I am just curious how you guys are actually using prisma cloud at your place and may be get some useful tips. Thanks

https://redd.it/m3e056
@r_devops

Fluentd_fluent-plugin-sanitizer

Hello All,

Need some suggestion on implementing encryption on the fluentd logs for the Kubernetes cluster. Does anyone have any idea how to integrate "fluent-plugin-sanitizer" to encrypt the logs data? Any other technique to achieve this.

Your help is highly appreciated.

Thanks

https://redd.it/m39uxq
@r_devops

Open Source Projects

Hello,
I'm looking to contribute to an open source project as a devops engineer. Any recommendations?

https://redd.it/m3sj8r
@r_devops

DevOps and SSIS scheduling (Operate)

I’m looking for an SSIS Package scheduling tool to monitor all SSIS packages with one central console. In the past I have used autosys and also looking into JAMS and ActiveBranch.

Before I move in above direction I’m wondering if there is another type of tool that better fits in a DevOps ecosystem and will accomplish the same (central execution and monitoring of sql jobs)

https://redd.it/m3tkxy
@r_devops

New to Azure DevOps. I hate it. It's a labyrinth.

Hi folks,

I'm a scientist, device engineer, and programmer. I've managed small teams of 3 or 4 programmers without a formal version control system. My preferred stack is Linux and Python, but I can work in other environments. Last year I started to participate in GitHub, and I was just starting to the hang of it. Then I got a new job where they use Azure DevOps.

I'm only two weeks in to the job, so maybe in time it will all become clear. But right now I'm hating Azure. I have spent hours trying to accomplish what should be simple tasks. Yes, I am RTFM. This reading will consume an unknown amount of my time. I can also see that Azure's documentation considers its customization features as a plus. I hope that customization doesn't involve too much flexibility in the menu options, because that will greatly decrease the value of RTFM if my company's sysadmins did something cute.

Right now, all I want to do is to find and download a software build. There used to be a place on the company network drive where the company was keeping its software builds. They decided to move the builds onto Azure today, and all new builds will go there.

I got a link to a Releases page. The builds are not actually accessible directly from that page. There was a bewildering array of choices on that page, all of which seemed to link to documentation of the branch history, what tests were run on this particular Release, etc. But where was the link to the installer binary? I couldn't find it. A colleague helped me with this task earlier today. I remarked that I seemed to need to click on Releases on multiple menus to locate the installer build. There were two or three menus to navigate from the starting point to actually get to the executable.

Alas, my note taking was less than thorough, and I can't seem to find my way back.

I feel like many of the changes that Microsoft made between Windows 7 and Windows 10 were "just because." Menus were rearranged. Settings were split from Control Panel. Popup windows from the OS cover critical information while you are working.

Azure feels the same way, like it was written to be the not-GitHub, different "just because."

Please feel free to change my mind. Thanks.

https://redd.it/m37zkr
@r_devops

Handling configuration for a single-tenant application with different startup modes

I am wondering what the best way of handling configuration for a single-tenant application would be. Bit more details on the project, it is a single-tenant application with a bootstrap mode and an active mode. On the initial deployment, it should go into the bootstrap mode, and then after that if it ever has to restart it will go into the active mode. This application will be deployed to AWS EKS in a namespace per tenant. The database will be MongoDB Atlas.

My question(s) come in how should I have the application pull the configuration from the DB both on its initial startup as well as when the pod is rescheduled to a new node due to a scaling event or instance failure. Is Mongo a good place to store this type of configuration?

Normally I would use a YAML or JSON file and have the application read that though I think that might not make much sense here because the file would be local and would need to have it reupload the config file to say S3 when a configuration change is made. How should I handle this configuration?

I don't expect configuration changes to happen often except during the initial setup by the tenant.

https://redd.it/m37eij
@r_devops

Best way to merge s3 files

I have a bunch of .ts files and an m3u8 on an s3 bucket (basically hls format). What's the best, most reliable and scalable way to convert them into mp4 and upload to onedrive? Zero data loss will be pretty important.

I am thinking of couple of solutions like lambda and elastic transcoder but some of the solutions feel like hacks. What would be the best way to do this?

https://redd.it/m364ez
@r_devops

Deploying same container with different config and then scaling them individually

So I have a web app (or 3 docker container, nginx to act as reverse proxy and web server, nodejs api container and the db container)

Everytime a new client wants to onboard, we spinup a ec2 (via terraform) and run the site_config.py (internal tool to config the site) and deploy it

It works, but updates are just painful and back-up are very basic.

So I was wondering is there anything like this which is built for it, I know bit of kubernetes and helm but we need each service to be deployed with its own config and then can scale-up or down that from there, I am very unsure if this is at all possible in k8s or helm or is there some better tooling that I could use.

PS: I hope this is the right forum for this question, if not please guide me to the right ones.

https://redd.it/m34lhu
@r_devops

Unattended Ubuntu 20.04.2 packer template on vsphere

Has anyone managed to get packer to create a Ubuntu template without being promoted for autoinstall confirmation during install? I have managed to get everything working, but I just cant work this one out..

https://redd.it/m34i6o
@r_devops

New to Cloud, new to AWS, new to Devops, new to the company - we're doing multi-region active-active

I've recently gotten my first 'real' DevOps job after being a traditional sysadmin for a while and only having to care about resources that were maybe used by tens of people at a time. My new company does custom development and has started to get into the tens-of-thousands region quite recently. I'm currently the only ops person there. (Like 99% of companies, we of course don't use "real" devops.)

I'm now tasked with two projects that will require high availability with zero experience in any of Terraform, cloud, AWS or providing services on this scale at all. (I started on 21-02-01 and on these projects some 1-2 weeks later.) One of the projects is even multi-region active-active which I've gathered is pretty much the pinnacle of achievements in cloud tech. Another will require insane throughput - large numbers of terabytes of video processing a day.

I've been making decent process on implementing these infrastructures but I'm quite terrified of what will happen when the products launch. So far, I've been able to write a few test services that show the basic principle does work (i. e. cross-region replication and such) but I have absolutely no idea how that relates to practical application development. (No background in dev work.) Yesterday, I didn't even know cloud-init existed, today we're absolutely relying on it for some core services. And I still have absolutely no idea how to implement a large number of things I'm supposed to finish next week, like KMS or CloudWatch - I haven't even started reading about them. It seems achievable to make them work as well if you only read enough Stackexchange threads but ... is that really all there is to building a good architecture?

I'm doing my best to try to learn all these services and how Terraform maps to them but it's just entirely overwhelming - especially when you factor in that I also have to predict the cost of the resulting infrastructure. One of these projects will launch next weekend because it's time-sensitive. I don't even know what my question in this post is, maybe I'm just venting, but... help?

To note, the company is not malicious at all, it's pretty much still a startup and the bosses' mentality is literally "Fuck yeah!". I think they just don't realize what they're getting into. If this goes well, it might be a career-defining moment for me because I know that they do reward performance. One of them also invited me over for drinks tomorrow so it'd be awesome if I had some appropriate way to express how I feel by then.

Edit: I've also made sure to very explicitly state that I have no idea what I'm doing on Slack so I don't think I have to worry about personal liability.

https://redd.it/m3qh1i
@r_devops

Part IV: Klustered - Live Kubernetes Debugging

Hi,

Another week, another episode. Thank you for the continued kind words, it really makes it all worth while to know that watching me suffer is helping other people with their Kubernetes journies; so thank you for getting in touch.

https://www.youtube.com/watch?v=Cp6zvBIo5KM

​

This episode was rather amusing, our first cluster breaks (a pair, Matt Moore [Knative\] and Dan Pop [Falco\]) decided to amuse themselves by the cluster suffered a death by a thousand paper cuts.

The second cluster was a tough one, I won't give anything away; but it was difficult due to the lack of errors; it's funny when you see what happened.

​

Enjoy!

https://redd.it/m3nlgg
@r_devops

When would you consider one knows enough to put "familiar with Terraform and Python" on their resume? And are my blog entries helping or hurting me in the job search? Hoping to get my resume more attention - interested in cloud/aws - but may have to start with a sysadmin role?

Here is my resume. https://imgur.com/a/eqAJGgG

I'm the type of person who never tries to B/S people with my resume. It may just be my lack of confidence in myself, but even though I might be able to talk about terraform and python, I'm worried I'll be expected to do something I am not capable of. Of course, I feel I could certainly learn it.

On my resume I list my personal website that hosts my resume, a link to my linked in page, and some blog entries where I discuss what I am learning. an example of a blog entry is...

​

Setting up an Ubuntu server using Terraform

For this project I wanted to learn the basics of Terraform. I first installed and set up Visual Studio Code because it's a nice interface and came highly recommend from other users online. Then I wrote out what I wanted my deployment to look like such as my subnets, internet gateway, and security groups.

​

Once I knew what I wanted to build, I felt the best way to get started with Terraform is to use HashiCorp's documentation because they provide examples of how to do things that you can easily modify to fit what you're trying to do. However, it can be a little difficult to navigate their website so I spent a lot of time googling "terraform aws [resource I was trying to deploy\]". For example, "terraform aws vpc" pointed me to the Hashicorp documentation for deploying a vpc into aws using Terraform which also gave me example code of a vpc deployment I used to launch my vpc.

​

After a lot of googling and converting my plans into Terraform code, I went through and made sure all my subnets, IP address assignments, security groups, etc. were accurate and built in a way that wouldn't cause any errors. Of course, once I entered the "terraform apply" command I was made aware of some typos which I quickly fixed and then finally once my code was successfully executed, I had a publicly accessible Ubuntu webserver!

​

What is great about Terraform is I can easily deploy resources in AWS without having to manually do dozens and dozens of clicks in the AWS console. Being able to quickly grab some code I've used in a previous deployment to use in a new deployment is also something I'd like to learn about in the future as well as the power of automation. But for now, I just wanted to learn the basics so I can start to see potential there is in infrastructure as code.

​

Please note the link below points to an elastic IP but I may terminate the instance as I do more testing so it may not be active 24/7.

​

[link redacted\]

​

​

If this is stupid, PLEASE let me know so I can fix or remove it. I am not looking for sugar coated responses. I am seeking feedback from those in the field.


I have already had my resume reviewed several times in the r/ITCareerQuestions sub and now feel I'm ready to apply for jobs but would LOVE to put "familiar with Terraform and Python" on my resume or some variation. At least so I could get past HR and have an opportunity to sell myself!

​

Thank you and I hope the sub will permit this post and it doesn't violates any rules! If this is inappropriate for the sub, please advise on a better sub as I'm not 100% familiar with all the subs. :)

https://redd.it/m3m6np
@r_devops

Is jfrog artifactory good for the long run, or nah?

My org is looking to standardize on one repo manager (we are currently using nexus and bitbucket depending on the team).

Lately we've been trying to be a forward looking org because we don't want to get fuckin stuck again (lol) on a tool that will become less relevant over time.

So my main question is to those that use jfrog, will it still be relevant to your org years down the road? Also any feedback you have about the trajectory of the product would be so help.

View Poll

https://redd.it/m3m6e3
@r_devops