mTLS using a forward proxy?

I have an application that needs to connect to postgresql, using a binary protocol over TCP.

If the postgresql server requires mTLS from the client, is it possible to use HAProxy as a forward proxy to handle the secure TCP TLS connection to postgresql? Thus freeing the application from the obligation to learn how to use mTLS? Is it possible to use Nginx as a forward proxy? For Nginx I believe the TCP forwarding can be done with the stream {} block, but I haven't seen an example where mTLS is used -- I have done this for HTTP traffic. Similarly for HAProxy, I have seen configurations with TCP forwarding only.

APP --- (TCP plain) ---> HAProxy OR Nginx --- (TCP with mTLS) ---> PostgreSQL

https://redd.it/1dxu42m
@r_devops

"no such file or directory" error on container on Windows

FROM python:3.10-slim-bullseye
RUN apt-get update && apt-get install -y dos2unixCOPY yourscript.sh /usr/local/bin/yourscript.sh
RUN dos2unix /usr/local/bin/yourscript.sh
RUN chmod +x /usr/local/bin/yourscript.sh
ENTRYPOINT "/usr/local/bin/your_script.sh"

I am getting: no such file or directory when trying to run the container on Windows, but on Linux it works fine. I thought it was a permission error due to the bash file coming from WIndows, but the error is due to the fact it cannot find any file.

https://redd.it/1dxvwj0
@r_devops

Any good price/quality security scanning tool for small team?

Hello!

We would like to add some SAST and SCA to our products, however we are a small team (\~ 3 to 5 people) and tools out there are quite expensive:

- Github Security: 49$/developer/month
- Snyk: 25$/product/developer/month, min of 5 developers per product (min 1,500$ per year)
- SonarQube: Open-source not enough for security; Developer plan exponential price in terms of lines of code

Do you know any interesting tool?

https://redd.it/1dy2q7v
@r_devops

Cert recs for an incoming masters (MSCS)student in Germany. (Related to cloud/devops/sysadmin/linux)

I'm an in coming masters (MSCS) student for the october 2024 intake in Germany , Ive got some time to work on my skills to land a student job preferably related to cloud/devops/sysadmin/linux. Now ik getting a cloud or a devops role right off the bat is impossible without some practical experience. So assuming a helpdesk job is a good start what are some cert yall would recommend to upskill myself to land one?

I know certs dont guarantee a job but i think its a good thing to have for somebody with no IT experience

I have done some googling and surfing on the internet and found that a linux cert would be a good starting point like the RHCSA or the Linux foundation though most tend to prefer a vendor certificate.

Also i did read that the AWS foundation cert is another beginner cert but it feels more like a sales cert with not much technicality to it so im not sure what to think of it.

So what would yall recommend I start with? (Even if not for a student job, I'd like to start upskilling myself right from my first year of masters so advice unrelated to the posts are also welcome!)

PS: I'm learning German and hope to get better by interacting with native German speakers once I move there cause i feel like i need to keep listening to the language to get the hang of it.

https://redd.it/1dy3lyf
@r_devops

What platform do you use for your Mobile DevOps?

We used App Center and Xamarin UI tests but with both being shut down we need a new platform for build, distribution and testing of our Mobile Apps. What platforms do you use for your Mobile Apps?

https://redd.it/1dy4v9w
@r_devops

Update to HR ghosting me after 2 technical interviews

See previous post for details

So I decided to send HR an email after being ghosted. Got a reply in less than 5 minutes from an entirely different person saying "sorry for the delay" and that they would like to schedule a 15-minute HR interview as a "final step." Note that the previous HR interview was supposed to be 40 minutes "screening" where I assumed I'd get behavioral questions and provide a rundown of my experience. Now it's a short 15 minute interview with a completely different person.

Okay, so what is going on? Is 15 minutes even enough to discuss HR related matters like salary? What should I expect here?

https://redd.it/1dy6a6x
@r_devops

CI/CD process for Azure Data Factory on GitLab

Hi,
I encountered a little problem. I'd like to create a CI/CD process for ADF(pipelines, datasets,linked services etc., ADF instance is created by dedicated team) but all stuff on the Internet is about ADF and Azure DevOps. I have a requirement that it must be Gitlab(I had discussed it already and it cannot be changed).
I am wondering:
- how to structure repository for my ADF and associated resources like Azure Functions or Synapse Spark Jobs which are triggered from ADF
- what steps CI should contain and what CD process
- how development process should look like, I imagine that somebody creates ADF pipeline on UI and then what...copy and paste a json definition to repository?
- how to move DEV to QA and PROD envs
ends
- how to store state of resources?

Thank you in advance ☺️


https://redd.it/1dy6ks6
@r_devops

DevOps on demand is Exhausting

I am DevOps/cloud engineer on demand. My company has multiple clients and work with 2 of these clients, 4 hours each.

I feel drained and unmotivated, and I feel when am split up I can't focus.

The worst part is that one of these client is a software on demand firm and has 2 clients.

So I work for 3 different projects.

Is this normal? Or it's me that is not adapting?

I am even thinking of changing careers for something with more programming.

Sorry for the rant.

https://redd.it/1dy8r2o
@r_devops

Nginx sticky sessions

hi everyone,

Im learning devops and i have this assignment, it requires me to run

3 python app servers with Flask that route to my localhost

DB server using MySQL that logs a counter for each time a connection is made to localhost and the IPs of server/person trying to connect (the apps are configured to store these logs

NGINX serving as a load balancer

With docker compose im running all those containers.

Now the issue is that i need to implement sticky sessions to the nginx load balancer to stick to an IP for 5 minutes.. for this i need some modules. It seems like I can’t configure the modules to download with the dockerfile as the nginx:latest image does not include them. I cant use ip_hash as well..

Can anyone help me please?


https://redd.it/1dy9c24
@r_devops

Which internal developer portal should we use?

We're gonna set up an internal developer portal for a team of 28 devs, and I'm on the case to figure out which one we should go with. I want to know your thoughts - what have your experiences been like with different portals? I've only a little experience with Backstage but have experienced (and heard) some mixed things about it, and luckily we've got some budget. Let me know!

https://redd.it/1dyb1f1
@r_devops

Handling Docker File/Compose Creation

How are you handling your dockerfile and docker-compose.yaml file creation?

I've been letting two of my Sr Devs handle the creation of them and then hand them to me, largely they have been fine. I do end up having to do some work in them though.

I am considering creating a template for both compose and file that they can pull down and add what's needed to them, of course with the deploy compose files being mostly variables to pull from GitHub.

https://redd.it/1dyb3ah
@r_devops

3 YOE A bit quiet 4 Months into my new role . What to focus on?

For reasons out of my control, and my manager knows, I don't have much work going on. My manager has outlined my eventual responsibilities, but this is some time away. You might think this is the best thing ever, but it is driving me insane. I've always been used to putting out fires constantly, albeit in interesting lines work but I was always severely underpaid.


Fast forward to now, I am paid double compared to previous role, but the caveat is I don't have much to do. The standard suggestion would be to get a cert of some sort, but I have 4 AWS certs and recently did the CKA so I am bored out of my mind and I cannot go through doing another multiple choice exam/Preperation.

I am strong in AWS, Kubernetes, Networking, Docker, did a lot of Python some time ago but It dried up. What should I focus on? Should I try and learn Azure etc? did a lot of CICD previously, including Jenkins, GitLab, AWS pipelines etc etc.



TLDR - Happy with salary, but not much work at the moment. Worried about skill atrophy and also sick of studying for certs.

https://redd.it/1dyfy9j
@r_devops

Passed KCNA. But have a few questions!

So, I passed my KCNA exam yesterday, I just received an email saying that I achieved the passing score and I can download my cert from the portal, however can’t see my score on the portal even after 24 hours of finishing the exam. Also, now that I have passed this exam could ya’ll guide me on which cert should I prepare for now? I have 3 weeks before my college starts.

https://redd.it/1dyi8u8
@r_devops

How much do you care about the cloud infra costs that your company incurs?

As an SRE, architect, or software developer, how much do you care about the cloud costs that the products you build and support incur? Do you think much about cloud cost when designing systems? Do you ever look at the cost once a product/feature is in production?

https://redd.it/1dyj1zf
@r_devops

Improve website performance using cache

Hello everyone

I have this ecommerce website developed with Nuxt2 and the API using Symfony framework (PHP), all hosted in AWS using beanstalk, and also using Cloudflare for the protection.

The performance of Nuxt SSR is not so great, so my idea was to add cache on top of it. In another project (self-hosted) I have done cache invalidation using cache tags, and for that I used Varnish. I was trying to avoid Varnish and just use Cloudflare, but just found out that cache invalidation just works with the enterprise plan, and that's out of our budget. I also though instead of using Cloudflare just make use of Cloudfront using it's WAF (for what I read not as good as cloudflare's), but also Cloudfront don't support cache tag invalidation out of the box

So now a little bit stuck with ideas, so my thoughts are:
1) Create some sort of middleware that stores the tags and the urls where they were used maybe on Redis? and that do the cache invalidation of those urls? (I believe cloudflare only allow 30 per request so could need many API request to accomplish it
2) Place Varnish in AWS infrastructure? (Cloudflare -> Varnish -> ELB)
3) Ditch cloudflare for some other CDN solution?

Any feedback would be very welcome

https://redd.it/1dyj7f3
@r_devops

Need advice for reducing the size a monolith repository

I have a big monolith (\~75GB) git repository for a particular desktop application. The repo has several gradle modules tightly coupled to each other in terms of their dependencies and exports. All the modules have Unit Tests which use test data (XML, properties, etc.) for their validation. The Unit Test data for all tests is around 30GB.

Apart from this, there are several automation tests in a separate folder within the repo. The automation tests also use certain test data (XML, ZIP etc) for their import/validation processes. The automation test data occupies another 35GB of disk space.

I need advice on how to go about reducing the size of the repository. Should I go about splitting the repository as microservices? Microservices for this seems to be challenging due to the tight coupling of code across modules. I still don't want to eliminate this option.

Extracting the automation test data and putting it into an S3 bucket/Artifactory seems to be a viable approach to reduce automation test data. And extracting the Unit Test data seems to be very challenging as thousands of tests need to be refactored in order to fetch the test data from another source.

Has anyone else faced this situation where their mono repos are bloated? Our entire infrastructure is hosted on-prem with in-house CI/CD pipelines. The tight coupling of modules within the repo, also the tight coupling of the CI/CD infrastructure with the repository layout seems to be challenging to address this issue.

Appreciate any inputs/advice on how to go about this. Thanks!

https://redd.it/1dyn8iu
@r_devops

How to upskill myself?v

Hi all. I am with a big 4 company working as a QA engineer. My project got over where I was doing automation using Java, testng, selenium webdriver. Before that, Python & robot framework. I have 10 years of experience doing the above. Now they are saying QA alone not gonna fly. I have to upskill myself. I am looking at devops as one of the options. So how do I go about upskilling myself in devops domai? I did cka certification..

https://redd.it/1dynzm0
@r_devops

Github Actions docker tag strategy

I am currently setting up GitHub Actions and got some interesting issues.

I have two workflows:
1. pr-open.yml: Triggered during a pull request. Builds a Docker image and pushes it to the registry.
2. merge.yml: Triggered during a merge. Pulls the image and deploys it.

At first, I tagged the image with the Git commit SHA. But, during a 'merge and squash,' a new commit SHA is created and merge.yml doesn't know which image to pull.

Then I thought of using the pull request number as the Docker tag. But during the merge, I can't retrieve the pull request number. What is the best practice for Docker tagging in this case?

https://redd.it/1dyp77j
@r_devops

Python Has Too Many Package Managers

#Python Has Too Many Package Managers

Note: I wrote this article

https://redd.it/1dytcq7
@r_devops

what should i learn first

im a spring boot dev, i dont know much about linux only the basics of the basics

i work with docker ( docker files - docker compose ...) and i want to start learning about devops
and the process of deploying the apps that i create
what should i learn fast because it is very overwhelming cuz of the amount of tools that exists
i dont want to master one thing i just wanna grasp a little bit of everything cuz i believe mastering one thing isnt the optimal way of learning ...
so please if u can guide me with a road map of what to learn i would appreciate it


https://redd.it/1dyu4tx
@r_devops

Can I use M1 MacBook Air for DEVOPS

Like to run 1 or 2 VMs
And all basic DevOps tasks and all

I have a budget of 60k to 75k, Please let me know your suggestions.

https://redd.it/1dyvayg
@r_devops