Port forwarding on linux

Hi guys. The task may be simpler than I think, but I'm stuck. I "work" at the university as a student and I have no boss to help me with this task. So the problem: There are 2 servers, call them AI2 and AI3. I need to connect to the uni vpn in order to ssh into them.

On AI2 we have a Virtual Machine, on that runs a SpeechToText API to which I should connect with a websocket from AI3. I can ssh from AI2 to VM on port 5022, and on port 8181 runs the API. I tried to forward a port from AI2 to the VM's 8181 port, but it doesn't work for some reason.

ssh -L 3006:127.0.0.1:8181 -p 5022 <username>@AI2.com

I tried this command and works well on my own pc, I can connect to the API, and it works fine, however, when I do the same on AI2, and try to connect to it (wss://<AI2's IP>:3006) from AI3, it gives a connection timed out error.

How can I solve this issue? If you need more information feel free to ask. Thank you guys.

https://redd.it/qsyts5
@r_devops

Docker, Nginx, Gunicorn resources?

I've been self-teaching myself computer science for a while now. I've created stock portfolio web app in python that has scheduled tasks to update the db and send notification emails. It's rudimentary but I'm proud of it. Locally everything works great. I dockerized everything and upgraded the flask app to use gunicorn and nginx for production. I have a domain name and think I configured the DNS correctly. Locally everything works correctly but when I clone the repo to a Linode server I get a 502 error.

This has been a really long journey (300+ hours) to get this far and to be honest I am very close to just giving up despite being so close to being able to show it to employers. To make things worse it's much harder for me to debug things on a VPS inside a docker image. I feel like I bit off too much to chew.

Does anyone have good books or resources to truly understand how docker, gunicorn, flask, and nginx work? Stackoverflow has been great but I really think If I was able to sit down with someone and walk through my project I could figure it out.

Here is what I think happens.

1. client asks to send a url request
2. the DNS resolves the url to <MY_INSTANCE_IP>
3. the client sends the request to <MY_INSTANCE_IP>:80
4. nginx is listening on port 80 and accepts the request
5. nginx send the request to gunicorn
6. gunicrn serves the request to flask
7. flask does something
8. flask returns the response to gunicorn
9. gunicorn serves the reponse to nginx
10. nginx serves the response back to the client

https://redd.it/qtccdo
@r_devops

Simplifying AD/SAML/LDAP logins (Authentication Spaghetti)

I'm really not sure where else to post this, so forgive me if it's a bit out of the main scope for this sub.

Long story short (as best I can) due to companies buying companies, and all of the political issues that ensue, I'm left in a mess, trying to maintain a global platform with no one central piece of authentication.

* We have a legacy AD domain left over by one company, but the corporate direction is to move away from it. Several key systems still rely on this though.
* We have Azure AD tenant which works for like 80% of things, but it doesn't help the unix/windows hosts that use the legacy AD for authentication.
* We have a lot of things tied into Okta (which is federated against our Azure AD) but there's no ability to take user groups into consideration there (unlike LDAP for example).

This is complicated further by my complete lack of knowledge in this area, the fact that anyone who knew anything has long since quit, no one who remains knows how to set anything up, and we don't have enough resources to properly implement something like Azure AD DS and LDAP-S.

So, I am in desperate need of some solution that allows me to try and pull it all together, a sort of middle man. This middle man would organize users into groups (so LDAP can work) and let Azure AD do the actual authentication. I need something a Windows Server can authenticate against, a Unix Box, web apps like BeyondTrust or Skybox and that allows all of them to make use of user groups so various users get the correct permissions to everything as soon as they are placed in a group.

Maybe [this diagram](https://viewer.diagrams.net/?tags=%7B%7D&highlight=0000ff&edit=_blank&layers=1&nav=1&title=sktch.drawio#R7Vpdc6M2FP01nmkf1sOn7X30R5ztbNzJNNvu9skjQAE1AjGSSOz8%2Bl6BMGARZ7O1E7fpZBKjCxLSPedK59544M7TzSVHebJiEaYDx4o2A3cxcBzb9kbwoSzbyjK2tSHmJNIPNYYb8oi10dLWgkRYdB6UjFFJ8q4xZFmGQ9mxIc7ZQ%2FexW0a7b81RjA3DTYioaf1KIpnU67Ks5sYnTOJEv3ri6xspqh%2FWBpGgiD20TO7FwJ1zxmR1lW7mmCrn1X6p%2Bi2fuLubGMeZ%2FJ4O3uzzJ%2FtXF39bfrj5xdvgzSL5%2FEGPco9ooRc8fSw4BtN0oactt7UvkMgrF9%2BSDYZRZ4lMKTRtuMwZyWTpa3828BdgQZTEGRhCmB%2FmYCBp6evZLcukBtp2GvuCpDHMnpIA%2FiI1C7UYQD%2BTRG7hspzZerpYz5z5UNzH0NX0Qb0gzCXetEzaJ5eYpVhyGM7Sdx1P46MJ6rq6%2FdDAPfpYmZIW0CNPk0wTLN6N3GAAFxqGF0DiGn7HEVBSNxmXCYtZhuhFY51xVmSRgmRhQat55oqxXAP0F5Zyq92OCsm68OENkd90d3X9p7oe%2Brq12LRuLba6ISTicqpiTKFMkRAkrM1LQuuhn8RIsIKH%2BIAnHB3viMdYPk9i5aaDiHNMkST33cg%2BOn6TN8Vv6PhtCA8CCIDwbdVrVDdVL3toTca1oelatjp9rzEn4DQV3S9hBM6i%2BqGMZbiy6PvWqzDGOyvGfHxbxox%2FiDFelzL%2Fabr4Z0WXWhq9DV%2FsNlma3eY5vrTJog%2BWA3T51zBjdF7MsM%2BHGeN3zozxWTHDMXT%2BikQRVUJ%2FhbKBM6KwllnA4SqWpX%2B0pTZMC%2BAFJ48wV7Z7HqYSGD12Y%2Fz0u8BcKDdTFALBYIpZmeexIhc%2FG0yt04MWs%2Foyg5CSfA1gq0uW5oUs37FcIIkCJPDadiYb%2BB3mWXyE7MAb72UHjmNkBxPLzA5q29GB9Awgv5IsUmmuY91gDiszHCvusAwTTfxXzOFSESIg2PIPwmWB6AqFCcmwnu7xEjjP7kJkj80Ezu%2BBqK5MHB0i34DoimTF5owBWq3LGR4PE398ZpiMzLDBgXpRnvdsXBh8ZM3wlmXRF14IaW5WBkp5S9vCfEgucEvutkE0INsVhhT8ERLJ7jAWOQpJFn8pD2K3hW5vySRUx8g6IhxmxpTPlg84WIuSc8eDdrQH7Q7HFrReD7T%2BqXbEsQHtDUI3YJmzTDB1xO3jW8I7B3UQCcnJnToEGVcmxAN1uFkzOLDu%2Fsd8h%2FnY6mK%2B23LfCnPb1DNKoKhSYagVyh54sFL5DCbKH9CdTvWNtFJIoJCxAOETlENZ%2FdswiGKhd9tSjXJ2h%2BeMMt4I0FtQn%2FsmtUnrOfbUc1%2BMk%2Bv5XbXimTjtKtdtoJyTAeWa0YnDqsKs5CF8XCo9ePhMLB2O%2BcU9rvxuH4ygFsbPAjFwXN9SP2CnKMD0GnAsCdRDjKu9BwImJUt7mCNV8M5YISkInvnuPxPWEztBrtacbmL1H5Qhu70lIR4WSjsPiwzCmwtEIaTBa0Ru16V8PpJymtjdsO47pSc9YT05FVt8gwevWgnrFMKeqX6rxsGClmZbOzO1uplpk742yek%2Fq5jbOj14vmSu4%2FJM0lPbTGtmLDA3BR0rRUqn6rz78dhswd8bpkeILmdPKPmOGV1uz148OtlebIrga1qKDSV5lLurBB3O0TJX6d2m97XUlNIq%2BUxIqvoyNfMwxELZMHhrK5PqDQ8EVq1HH6FUoZAFIu8tHeSc5eVbf5tN56UvBKBX1hDKN1QvU%2Fm%2BMrFdWQE%2BU5wGsGUmxDxTzlIDHOPYd%2FeYNuphWh%2FVTqfPekT5dHX1bhAZOfuC2TdP1lfVYRMDkKvF9Pr9ALJXJHLGbwxIPXA7g1m8Gzg8v5uoOL79XWfj6eDo%2BSLM%2B4HD3%2FsOzAnhgGbzlafyXuuLY%2B7F3w%3D%3D) will help visualize things a bit.

I think maybe something Keycloak or Authelia (Identity Brokers??) could be the answer, but as I said, this is really over my head. I'm just a guy with very little experience or know-how in this, trying to find a way to make

the impossible work when politics and internal bickering is stacked against him.

&#x200B;

I'm pretty desperate at this point, so I'd be grateful for any assistance you guys can provide in pointing me in the right direction.

https://redd.it/qskn14
@r_devops

How to run only some from am Ansible role

I have a role with a lot of tasks. When I include that role in my playbook, how do I run only tasks from the role with specific tags?

https://redd.it/qtgw29
@r_devops

Overwhelmed with so much to learn :(

I’m a azure cloud engineer with a windows sys engineer background, I’m about 6 months into the job and have learned quite a lot about Azure DevOps but still feel I’m missing a lot of skills to become a full DevOps engineer focused role.

There’s just so many things to learn, I don’t know how it’s possible to learn them all: AWS, Python, Powershell, Jenkins, Ansible, Terraform, kubernetes, the list goes on…

How do you up skill when there is so much to learn? Do I just simply start with 1 focus point and move on to the next?

To summarise, I only have exposure with Azure, M365, Azure DevOps but I haven’t done much creating of my own pipelines or automation.. I also don’t have a strong coding background

Appreciate any advice!

https://redd.it/qtlhjo
@r_devops

Dockerless containers with Podman on MacOS

Docker the company has been throwing wrenches lately into what used to be a smooth user experience with their new Terms of Service for Docker Desktop and various limits imposed on image pulls from DockerHub. Understandably, many people have been looking for alternatives which are easier to implement on Linux but not so much on MacOS.

We are showing here a way to build and use containers on a Mac without Docker, using Podman.

https://gokloudtech.com/index.php/2021/11/13/dockerless-containers-with-podman-on-macos/

https://redd.it/qthao4
@r_devops

Python Selenium Debugging

I have the following [webpage](https://i.imgur.com/gZLwTgh.png) . I'm trying to double click the "Blocked_IPs" text.

This is the code that interacts with it:

blocked_ips = driver.find_elements_by_xpath('//td[contains(.,"Blocked_IPs")]')
print(len(blocked_ips), blocked_ips)
action = ActionChains(driver)
action.double_click(blocked_ips[0])

Problem is, it just doesn't seem to double click it. When I do it manually, it works. When I execute the code, it doesn't. There's only one occurance of the word "Blocked_IPs". This is the output in the terminal:

1 [<selenium.webdriver.remote.webelement.WebElement (session="82b277a5f85cbb202f5cd57c0b800f3b", element="530b1a15-a190-401c-8495-921777f8fa84")>]

Does anyone happen to know why it's not working? How can I test it? Thanks ahead!

https://redd.it/qtq1ok
@r_devops

If you could automate any troubleshooting workflow for k8s errors, what would it be?

You all have a series of checks and processes you go through to find the root cause every time there's a recurring error. Which of your workflows would you like to automate? Imagine every time you got XXXX error it resolved on its own.
For example, when I get an ImagePullBackOfferror I check for changes in the image tag and repo, then I verify that the repo is specified, then if it's a secret I check to see if it's expired or misconfigured, etc. And each one of those checks can branch out and spiral into endless steps and iterations until I find the root cause. And that's one of the (relatively) simple errors.


If I had to choose I'd go with OOMKilled, since it's so recurring and annoying to debug. What would you choose?

https://redd.it/qtpu8f
@r_devops

CI CD for low level Linux and kernel modules

Hi all,

How do you guys do CI CD with kernel modules?

Do you just setup VMs with proper kernel configurations and run your jobs on them through jenkins or gitlab ci?

Did you manage to do them through docker containers and somehow made it work? if so how did you do it?

I would love to hear from all of you devops people who work in embedded linux and kernel development, how do you guys work on such projects?

https://redd.it/qto0ly
@r_devops

One platform, one tool.

What if you could do everything from one platform. From CI/CD to cluster deployments, and perhaps even deploy your regular infra tools.

View Poll

https://redd.it/qt995v
@r_devops

My Containers Learning Path

Following up on the feedback I got here last time, I decided to share my containers learning path as a blog post. The following learning order turned out to be particularly helpful for me to understand Docker (and not only) containers:

1. Linux Containers \- learn low-level implementation details.
2. Container Images \- learn what images are and why do you need them.
3. Container Managers \- learn how Docker helps containers coexist on a single host.
4. Container Orchestrators \- learn how Kubernetes coordinates containers in clusters.
5. Non-Linux Containers \- learn about alternative implementations to complete the circle.

I've been digging into the internals of the containerization tech for the past few years, meticulously documenting my findings. However, I've come up with the above order only recently. So, this blog post is an attempt to organize my in-depth but ad-hoc write-ups into a structured way to learn containers' fundamentals.

Hope someone will find it useful.

https://redd.it/qtw0qo
@r_devops

Homelab CI/CD Pipeline

I am looking for a tutorial to build a simple CI/CD pipeline using open-source tools on my homelab. Everything I have found to this point has been overkill for what I want to accomplish. I appreciate the help!

https://redd.it/qtw75h
@r_devops

Looking for hosting recommendations for a static site with some form of access control (ideally free or cheap)

Hi I looking for some recommendation on the best and cheapest option to host a small website with some form of access control.

Essential I have collated all my personal notes from differ formats and ported them into markdown files and saved them to a private repo. What I am now looking to do is make these into a knowledge base / wiki website for personal use that I can easily refer to when not on my personal computer, however I do not want to make them public accessible without some form of authentication (basic auth would do).

Currently I looking at using something like Hugo or Jekyll or mkdocs to generate the site every time there is a commit to the git repo (using github free actions) and then copy them to the website server.

I been looking at a few options (ideally free as money is a bit tight atm, but I am expecting that I might have to pay for something), however all free options I seen so far do not offer any form of access control on the website (ie github pages, cloudfront pages etc). Does anyone have any recommendations on what hosting to use? I would be fine with a cheap VM that I can run nginx/Apache on and make use of .htaccess files to control access.

&#x200B;

Thanks

https://redd.it/qu0ntm
@r_devops

Can Therapy lower chances of getting hired for Gov positions?

Not really a dev ops question, just related but I’m not sure where to ask. So sorry if this isn’t the right place. But asking in case anyone can help. I have a friend that needs therapy, he’s a Devs Ops Engineer. He is avoiding therapy saying he doesn’t want the fact that he went to Therapy to show on his background check. He says he might get rejected for US government positions if they see that on his background check. Can that happen? Anyone have any experience or knowledge about this? Any answers would be appreciated, thank you!

https://redd.it/qtxmqz
@r_devops

Is Nana's devops bootcamp worth it?

I am planning to take DevOps bootcamp by Nana. Can anyone who have experience or have done let me know whether it's worth it to invest in it.
thanks

https://redd.it/qu8dw3
@r_devops

DevOps Bulletin Newsletter - Issue 26

DevOps Bulletin - Digest #26 is out, the following topics are covered:

Keeping K8s clusters clean and tidy
Git techniques to get out of hairy situations
Efficient communication during an incident
Mini-projects in Python for DevOps

Complete issue: https://issues.devopsbulletin.com/issues/git-techniques.html

Feedback is welcome :)

https://redd.it/que8hh
@r_devops

Managing a Docker application with systemd

Linux service manager systemd can be used to manage long-running Docker applications on your server by automatically starting and restarting applications.

https://abhchand.me/blog/managing-a-docker-application-with-systemd/

https://redd.it/quh6a8
@r_devops

Unified or automated timeouts and contracts between microservices

Hello,

I wanted to ask about best practices and maybe tools available on the market for my purpose.

So, let's say that I have a product consist of multiple services. Different services defines different SLAs and timeouts in especially. I'd like to somehow automate the process of reading those SLAs, adjust timeouts dynamically if one of the downstream services agreements changes.

&#x200B;

Consider these services:


A -> B -> C & D

and a situation where e.g. service B defines timeout of 5s, but a request that's processed in 7s in service C can be a proper response from service C point of view.

I'd like to have some kind of automation, or maybe semi-autiomation, where service B can ask service C what are his SLAs or timeouts, and set its own timeouts properly to that.

https://redd.it/qup48x
@r_devops

Interactive Architecture Diagrams

Does anyone have experience or recommendations for a tool that would allow an engineer to create a multi-layered, interactive/explorable infrastructure diagram? I'm looking to create a diagram that encompasses everything from VPC, subnets, security groups, EKS, statefulsets, deployments, etc. etc.

I did some Google searching, and the only product I found that seemed to fit the bill was Terrastruct. Are there other alternatives? Is Terrastruct a good fit for this use case? Does anyone have experience with Terrastruct, a similar tool, or creating this kind of infrastructure diagram?

https://redd.it/quv52y
@r_devops

Webhooks in Kraken CI for GitHub, GitLab and Gitea

Hello, I have extended webhooks in Kraken CI, in the latest release 0.753. Besides GitHub, there is now support for GitLab and Gitea.
A guide about webhooks in Kraken can be found here:
https://kraken.ci/docs/guide-webhooks/

https://redd.it/qv16g5
@r_devops