AWS NAT Solution for inbound and outbound traffic?

Hi guys! I hope everyone is doing well. I've run into a problem I can't seem to figure out and am looking online for suggestions, help, etc. So any help is well appreciated

**What we need:**

* We have a customer connect to us through a VPN. In our case currently a site to site VPN setup on AWS.
* We need the customer to send traffic/data to one of our resources, but customer has to send this to an IP outside of the VPC CIDR.
* We need a device that NATs this IP into our VPC and routes traffic to a specific resource. We also need the outbound traffic to go through the NAT back to the customer.

**What we've checked:**

* We've looked at the Transit Gateway, NAT Gateway, Client VPN... But we can't find a valid way of doing this.
* The Transit Gateway doesn't seem to do NAT, and we can't figure out a way of using the Transit Gateway together with the NAT Gateway to accomplish what we need.
* It also doesn't seem to be possible to configure the NAT Gateway to NAT specific IPs to specific resources for both in and outbound traffic.
* We've seen the option of using a NAT Instance (which AWS seems to have moved to the NAT Gateway...), and think that maybe this is the least complicated method?

Simple diagram to depict what we're trying to achieve:

[https://forums.aws.amazon.com/servlet/JiveServlet/download/8-343034-989711-34061/aws-nat.jpg](https://forums.aws.amazon.com/servlet/JiveServlet/download/8-343034-989711-34061/aws-nat.jpg)

https://redd.it/oiuhoo
@r_devops

Any team leaders on this sub?

I am curious to know what team leaders (whose teams participate in DevOps) think of a project I'm working on. Please, please, please find holes and critique as if you were aiming to start a flame war.

**Here's a concept summary:**

* It's a continuous feedback sharing and learning tool
* DevOps is the first space I want to address because of its sheer complexity
* You map the Ops activities your team does\*\* then write/link notes to them
* Your engineers spend about 5-minutes per day reviewing notes you and their peers share

\*\* Mapping is done by selecting from a DevOps capability map

Now, you might be thinking, "Why don't we just do this on Slack?". Slack channels better serve ephemeral content, so why not a clean, dedicated space for sharpening your abilities?

**Expected benefits include:**

* Supplements your 1-on-1 coaching and engineer's ongoing certification studies
* Boosts efficacy of work by linking feedback and learning direct to relevant areas
* Help neurodivergent tech workers grasp feedback and learning better due to visual context

So... let me know what you think :)

https://redd.it/oklwda
@r_devops

Been out of the job market for a few years and now everyone requires k8s experience, am I screwed?

Background: I've spent the last several years working in the public sector, which sometimes lags behind the private sector. While we use docker for local development and CI/CD pipelines, we're still deploying to bare EC2/ASGs. We do have plans to deploy to ECS/Fargate but god knows when that will get prioritized by stakeholders.

While I'm not exactly unhappy, I do feel stagnant and the job market is extremely hot right now. I started looking at listings for the first time in 2+ years and it appears everyone and their mom now expects "deep kubernetes experience".

So I ask you fine folk in other industries: is this just HR speak, or have I effectively been locked out from anything but public sector?

https://redd.it/oklqul
@r_devops

Is there a self-paced lab Kubernetes deployment practice like google cloud in AWS?

Hi Devop experts,

I will be going for AWS to try deploying my app which I have done like 2 years ago but have not the time to deploy it for one reason or another.

But, really, I am not sure if Kubernetes is the way I should go for..perhaps Kubernetes is for big corporate and not for my small app?

And after reading on the latest about FASS (Functions as a service) which is like a one time thing...I am not sure what part of the app or normally what people do to make that part of the app to deploy on FASS? Would it be authentication or what ?

Hope I can get some insights here. Thank you guys.

https://redd.it/okm20g
@r_devops

Debugging/Testing CI pipelines

At my company we are using jenkins as our solution for CI. We are using a self-host environment with a large number of pipelines. In many cases our pipelines get large with a lot of content.

A common thing we have are optimizations. For example if a job is rebuilt, e.g some flaky test failed, we want to avoid building the artifacts again. While this lead to a great improvement in the overall performance of the pipelines it added a lot of complexity to the development flow.

We tried to use tools like https://github.com/jenkinsci/JenkinsPipelineUnit. That ended up being even worse. It took a lot of effort to maintain these tests, mocking every single plugin that we use. Add to that the fact that you need to write it all in java. Which isn't the "home environment" for most DevOps at my company.

I saw that other tools, e.g CircleCI, also offer such abilities and potentially complicated plugins.
As can be seen in this guide https://support.circleci.com/hc/en-us/articles/360043638052-Conditional-steps-in-jobs-and-conditional-workflows. While they do offer some support for development, like https://circleci.com/blog/local-pipeline-development/, it still feels like something that will be really hard to use on a real project.

So it got me wondering - how do you handle this issue on your environments?

https://redd.it/oiue44
@r_devops

Permissions are driving me crazy - DevSecOps

I wanna quit work for that and only reason.

I am a DevSecOps engineer. More specifically I secure AWS, deploying native solutions in a multi-account setup. We manage everything through terraform.

I am at this position for 4 months but the work I have managed to complete corresponds to one month of work. The other 3 months have been wasted in waiting for permissions grants, from more senior engineers. The permission grants completely block my tasks everytime.

I cannot connect properly with my team either. Although they are willing to help, they respond to me with delays of 5-6 hours every time. And it may take days to resolve a 10 minute issue. In the end, its me who cant produce work and seems like the unproductive guy.

Have you been in a similar situation? How did you tackle this?

https://redd.it/okoww3
@r_devops

Trying to access redis container remotely, not able to get it to work

Sorry if this isn't the right place to post..

I'm building a redis container with Dockerfile:

FROM redis
COPY redis.conf /usr/local/etc/redis/redis.conf
CMD  "redis-server", "/usr/local/etc/redis/redis.conf" 

The redis.conf says:

bind 0.0.0.0

Under YML file services:

redis:
build: ./services/redis
ports:        
- "6379:6379" restart: always

I can access redis from other containers in the same machine, but I'm not able to access it remotely using redis-cli. It says "Could not connect to Redis at <ip>:6379: Connection timed out".

I have another container with nginx on the same VM, listening to ports 80 and 443. With my limited knowledge I feel like I don't need to make any changes to nginx configs to get redis to work remotely, but I'm not sure.

What should I do?

https://redd.it/okpogw
@r_devops

That's it I am never gonna get a job in this industry

I have a low IQ It takes longer to understand the question I don't do good with ticking time in the interview. I am good programmer I know it. I just can't do these interview coding test, I hate how they amount all the hard work to these couple of these question and decide whether you will get a job or not. I wanna know how can I get better at it

https://redd.it/okqcoy
@r_devops

How to update container when new docker image version is published to docker registry?

I new to CI/CD, I was successfully able to create a pipeline that build a new docker image of my repo and push it to docker registry, but I'm not sure excactly how to update the running container to use the new image? I'm using Drone CI and docker-compose.

https://redd.it/okqefn
@r_devops

Hikaru 0.6b released with support for the 1.16 and 1.17 K8s Python client, newest black formatter

Hikaru is a tool that provides you the ability to easily shift between YAML, Python objects/source, and JSON representations of your Kubernetes config files. It provides assistance in authoring these files in Python, opens up options in how you can assemble and customise the files, and provides some programmatic tools for inspecting large, complex files to enable automation of policy and security compliance.

Additionally, Hikaru allows you to use its K8s model objects to interact with Kubernetes, directing it to create, modify, and delete resources.

https://github.com/haxsaw/hikaru

https://redd.it/oku2qx
@r_devops

Can anyone help with adding --net:container:CONTAINER_NAME option in kubernetes for a container.

Hi all, I have 2 containers, in one I have a java application and in another one I have open-vpn. I need to make the first container use the network of the second one. I managed with --net option locally on my computer and it works. Container 1 shares the network of container 2.

My question is how to do this in kubernetes? I am new to this stuff and also I am not a devops but a java developer so bare with me if explanations are not so good. :)

Thanks!

https://redd.it/okvhck
@r_devops

How does Azure's Deployment groups work?

Recently, I've been working with Azure pipelines to setup releasing our project to our On-Prem Dev environment. I was able to set up the a Deployment group in order to get my release working pretty easily, but I do have questions regarding how that PowerShell script works and how secure it is.


I would like to get a better understanding how this works before I start implementing it in our Prod environment.

This is the article I followed while setting up the Deployment group (https://medium.com/software-development-turkey/deploying-to-on-prem-server-with-using-azure-devops-pipelines-842f4aa226a7)

https://redd.it/okwomj
@r_devops

Service Sizing Calculation

I'm trying to build a sizing calculator for a service which processes work items, so that I can give my customers an idea of what their compute costs will be. For example:

* I have 500 work items arriving per hour. We can assume these arrive at a constant pace.
* Each work item will take \~110 seconds to complete, on average.
* There is a tolerance of no more than 15 minutes of delay between work item submission and result.

Is this enough data to determine the number of workers I need to stay current? I can't quite get my head around it, and can't seem to find the right terms to google.

https://redd.it/okwij8
@r_devops

A live coding exercise

Hey All.

So I will be appearing for the first round of technical interview with a cloud provider company.

As per the process, they are going to do the a live coding exercise.

I have been out of the interview loop for a while and wondering if any one has any suggestions about coding exercise that I can take/do before this interview.

Any other general suggestions will be welcomed as well.

Thanks.

iking

https://redd.it/ol0qqu
@r_devops

Would learning Python be useful for DEVOPS?

My company is pushing me towards the Devops team and I'm trying get a jump start on being successful. Do you think it's a good idea to learn python?

Also, I got the green light to take a class. Im not trying to do a 3 month coder camp but where can I take a 2 week (in-person) course on specifically python?

https://redd.it/ol1olh
@r_devops

Pagerduty is down!

Pagerduty experiencing yet another major downtime. All hands on deck folks.

https://status.pagerduty.com/incidents/8q8whhkm4t6t

https://redd.it/ol2nml
@r_devops

API catalogue....

Hi guys

Current gig has API's documented and built all over the place. We have multiple vendors each doing their own thing. Some of it is in Confluence, other's just have a Swagger link...We prob do have YAML files for all the API's across the vendors though - in total, prob close to 100 API's

I could build a little database of API's by cobbling together all the relevant documentation. Maybe it's as simple as an Excel sheet to start with but I wanted to understand how others are dealing with this?

In my case, it's a growing org with lots of new joiners - still all working remotely and an increased need for 'self service' (to test at very least).

I'm learning bits about devops model as a PM but don't know if this is an issue others face?

Thx

https://redd.it/ol13cy
@r_devops

AWS Cost Analysis Comparing Lambda, EC2, Fargate

I've found a blog article while doing some digging into cost comparison between different services. I posted it to /r/webdev as well, but figured it's just as relevant for anyone who's poking around in here, too. Here's the link:

https://blogs.perficient.com/2021/06/17/aws-cost-analysis-comparing-lambda-ec2-fargate/

Here's the /r/webdev post:

https://old.reddit.com/r/webdev/comments/ol4xlv/awscostanalysiscomparinglambdaec2fargate/

This is my submission statement:

> I think this was an interesting breakdown of some possible costs when considering the different services that AWS provides for infrastructure. I was discussing with a peer about the price of engineering hours and possibly opportunity cost to migrate to or from an overall serverless system.

> The article doesn't go into detail about that topic per se, but does an interesting job considering other factors. I thought it'd be cool to generate some discussion and see what others thought.

> If anyone knows of any analysis of the cost of switching between Lambdas, EC2, containers, etc. in terms of engineering hours, I'd really like to see it.

https://redd.it/ol5328
@r_devops

Clarification on integration and E2E stage in a pipeline

The context of this is I am sorting out CI pipelines and their various stages using Azure DevOps Pipelines.

I have these stages pretty well laid out:

1. Check what Microservices have changed (ignore those that haven't).
2. Build the Microservices with changes.
3. Run their Unit Tests.
4. If their Unit Tests pass, COPY the passing Build into a Docker image for that Microservices.
5. Push the built Docker image to an image repository (ACR in my case) for more testing.

Now I'm sorting out integration and E2E testing. After spending more hours than I care to admit on sorting this out, I think I have a strategy and game plan to actually start implementing it (a lot of this is based on what is in this repo):

1. Deploy these images to a Kubernetes cluster in the Hosted Agent using something like KIND or Minikube in the pipeline.
2. Run my integration tests.
3. Run my E2E tests.
4. If they are passing, either merge to production (in the case of my PR pipeline) or deploy to AKS (in the case of my Release pipeline).

My questions are these:

1. Just looking at two basic examples of integration and E2E/Selenium tests, they look awfully similar. It looks like the integration tests are doing what the E2E tests should be doing.
My understanding of integration tests (which is probably wrong) is that they are kind of a gray area between white box unit tests and black box E2E tests. Whereas unit tests are meant to test a specific function() in the code, integration tests are meant to test multiple related function() in the code for the same microservice and will typically not involve the whole stack, mocking network requests when they are needed, and will not typically involve the other Microservices. You are looking at integration between units in the microservice and not between other microserivces.
E2E is what would be testing the whole stack and integration between Microservices.
If my assessment of integration testing is correct, then I don't see why they need to be run after the cluster has been deployed since the tests would be isolated to the microservice and could just be run after unit testing.
Can someone clarify this? Should the integration tests be testing integration between Microservices and not just units within the same microservice? Is there a need for integration tests if doing E2E testing?
2. Should the integration and E2E testing stages be run in parallel or should E2E depend on the integration tests passing?

Thanks for the advice.

https://redd.it/ol3cam
@r_devops

Deploy angular application using aws Codepipeline

Hi folks,

I need quick help
I have an angular applications. I need to deploy it on cloud-front using aws code pipeline and store Repo files on S3

Can anyone give me any hint or link or resource from where I can take help to complete this deployment

Thanks in advance

https://redd.it/ol090c
@r_devops

From NLP to software architecture, how?

Hello all,
Iama n NLP Engineer and life is good, except that recently I got interested in software architecture (backend +devops). I would love to ask how can I gain such a skill? I really get amazed by designing software like when to use this kind of database, how can we be easily scalable, what is a message a queue and when to use it?

I am now trying to find a part time job or an internship beside my main job to get the hang of it. If you can help me in any way I will deeply appreciate it (either advice, resources, a part time job!).

Also if you are a software architect, a backend engineer, or a devops guy, how do you feel about it and what do you find interesting in it?

Looking forward to hearing from you.

https://redd.it/ol8ew6
@r_devops