AWS NLB stuck on pending on new KOPS cluster

I have a new KOPS cluster I created today and am trying to get the cluster to apply a NLB so I can have my ingress work. I am using the YAML provided here: https://kubernetes.github.io/ingress-nginx/deploy/#aws \- I have taken the file and split it up into it's own sections and all depoys fine. Nothing wrong except the service for the load balancer is stuck in the pending stage and describing the service does nothing useful other than tell me how long it has been in that state.

Bottom of describe

Normal EnsuringLoadBalancer 103s (x47 over 3h27m) service-controller Ensuring load balancer

My ingress.yaml file

apiVersion: networking.k8s.io/v1beta1
# apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
# add an annotation indicating the issuer to use.
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-stage"
# needed to allow the front end to talk to the back end
nginx.ingress.kubernetes.io/cors-allow-origin: "https://api.dev.mydomain.ca"
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, PUT, POST, DELETE, PATCH, OPTIONS"
# needed for monitoring - maybe
prometheus.io/scrape: "true"
prometheus.io/port: "10254"
#for nginx ingress controller
ad.datadoghq.com/nginx-ingress-controller.checknames: '["nginx","nginxingresscontroller"]'
ad.datadoghq.com/nginx-ingress-controller.initconfigs: '{},{}'
ad.datadoghq.com/nginx-ingress-controller.instances: '{"nginx_status_url": "https://%%host%%:18080/nginx_status"},{"prometheus_url": "https://%%host%%:10254/metrics"}'
ad.datadoghq.com/nginx-ingress-controller.logs: '{"service": "controller", "source":"nginx-ingress-controller"}'
name: nginx-ingress
namespace: custom-namespace
spec:
rules:
- host: api.dev.mydomain.ca
http:
paths:
- backend:
serviceName: express-api
servicePort: 8090
path: /
- host: socket.dev.mydomain.ca
http:
paths:
- backend:
serviceName: socketio
servicePort: 9000
path: /
tls:
- hosts:
- api.dev.mydomain.ca
secretName: express-ingress-cert
- hosts:
- socket.dev.mydomain.ca
secretName: socket-ingress-cert

I am wondering how I can get an NLB to provision and allow me to point DNS at it and have the above ingress resource direct traffic where it needs to go.

https://redd.it/nq7yie
@r_devops

Manychat

When using manychat ( fb messenger chat bot ) how to publish the bot and make it work ? Ps : I published the flows but still didn't work when someone text the page .

https://redd.it/nq6orn
@r_devops

How to use strace on threads managed by supervisor? i.e. i want to ´supervisorctl restart someService´ and strace someService.

This would be great for debugging and for understanding the system i am working on better. A core mechanism of our system is a collection of services started and restarted using supervisor and I would like to use strace to see that system calls these processes make.

I think it is possible to look up the PID of a already running process and then hook strace onto it, but this will miss the first system calls of the process and I would like to get them all.

Any suggestions?

https://redd.it/nshax6
@r_devops

What are some good tutorials that would allow you to add a little CI/CD scripts or pipeline to any project?

What are some good tutorials that would allow you to add a little CI/CD scripts or pipeline to any project? Looking for some recipes that can be used in almost any project be it backend or frontend.

https://redd.it/nsiylk
@r_devops

Managing Windows VM Cloud Node Images question.

Hello I am a devops engineer. My company uses Jenkins for CI and we use Openstack cloud nodes as our build nodes.

As a result of this my department handles building our Linux and Windows VM images to be used on these cloud nodes.

The problem I am running into is the sheer amount of installations on the Windows nodes is crazy. We use a SCCM Task List that IT manages and that task list has over 350 steps for our image alone.

How do you manage to keep images a reasonable size and deliver in a timely manner for your developers. For reference the company I work at is heavy on the embedded device side.

Any advice would help. Although I would prefer more in-depth advice than (SCCM is old use Chocolaty).

Thank you for your time.

https://redd.it/nsigfj
@r_devops

Looking for work experience

Hi all

I am between a rock and a hard place and can't seem to get myself out of a situation.

I am transitioning into tech from the oil and gas industry as a metallurgist.

I have successfully completed the AWS solutions architect exam. I have also undertook training on docker, python, cloudformation but can't seem to land my first role even as a junior.

I wanted to ask the community if anyone is willing to give me some work experience. I have already lost 1.5 years because of covid and I am desperately trying to secure some work. I do not need paid work (although that would be nice) but a 3 month project would be really beneficial.

https://redd.it/nsdkz9
@r_devops

Dev site being attacked, cant access certain parts of CMS

I have 2 sites hosted on Azure setup as an App Service with a Front door. One site is the Dev site, the other is the prod site. They are getting hit within milliseconds and anon contacts are being created in the database and the DB is growing to be huge. Its slowing everything down and the CMS is difficult to access.

What should I do to protect the dev and the prod sites from these attacks that wont break the bank?

The people that work on the sites work remotely and the IPs they have can change depending on where they are.

Is there a way to get the IPs of the bot and block it via Azure?

https://redd.it/ns920z
@r_devops

Python use cases for devops

Hi guys I have been learning python lately
Is there a practical use case of python which you are doing in your work am not asking about development
I need to know the use cases in the ops side of things
Whenever I automate something I always shell it been shell scripting for several years so it comes automatically
Anyone tried replacing shell with python ?
Any good examples of that sort please

https://redd.it/ns8yp3
@r_devops

Puppet and Openshift?

Good afternoon /r/devops,

I have a question about puppet in regarding how it’s configured ideally in a openshift environment. From my general understanding isn’t Ansible better within openshift? I ask because for this job description within applying lately it states typical Linux system admin requirements but additionally states; preferable knowledge of “puppet and openshift”

So I am assuming their environments are most likely in the cloud going on a whim but not understanding the caveat of why use puppet with openshift instead of ansible ( I thought ansible was more used with openshift ). Additionally architecture wise, in a clustered environment within the cloud; is it common practice to have your masters separate from your workers to talk?

Example; would master 1, 2, 3, talk to workers a, b, c using the puppet files and have kubernetes configs within those same files?

Sorry if this is some noob question but I’m trying to really grasp the higher level of this so it clicks better and makes sense on why you would use X over Y thing. Analogies, differences, are welcomed. Thanks a lot!

https://redd.it/nt5dub
@r_devops

Automating database migration with CI/CD

Hi there. I like to automate deployment steps in our repos. Our current pipeline supports fully automated deployment of the software to our k8s cluster. But there's a catch. We have to manually migrate database before merging codebase to main tree. I'm currently using GitHub Actions for executing pipeline jobs. Previously used GitLab CI and also tried Azure DevOps but GitHub feels a bit more friendly to me (also has lots of community provided jobs on market).

So, I wonder if there's a way (I'm sure there's) to automate database migration steps in CI? How is it done usually? Any tips or links would be appreciated.


A bit more details:

We are using "code first migration" to migrate database manually. So It should be easy to execute the actual migration part. I just want to learn more about security and best practices about that before actually applying this. I'll probably create a workflow with dispatch_job trigger to manually trigger migration steps. But I haven't yet figured out how should I securely connect to database to do migration or should I do migration from our k8s cluster by somehow creating one time job to do the migration. I'm currently just exploring the possibilities...

Thanks

https://redd.it/nt0n6j
@r_devops

Troubleshooting a question

**How would you go about troubleshooting a server that is down?**


* Check the server logs to see if you can find anything event that might have shut down the server it.

* Reboot the server and see if that fixes the issue

* Check network connectivity by pinging another server on the network.

https://redd.it/nt8a7s
@r_devops

Automate Setting Up WordPress Server

Hey folks, my friend who I'm helping set up his Wordpress site had me do these steps three times already and I want to automate it:

1. Spin up an EC2 instance with Bitnami Wordpress as its AMI
2. Download its keypair
3. Create elastic IP and associate with EC2
4. Assign elastic IP to DNS record on Route 53
5. Get Wordpress password from EC2 system log
6. Use bncert tool on EC2 CLI to enable https on Wordpress site

Could someone please help me determine what DevOps tool should I use for each step?

https://redd.it/nsy9e6
@r_devops

Transition from Devops to Security Role

I came across a role for an Information security engineer but with Devops experience as they would be applying/evaluating security for AWS/GKE etc.

I wanted to see if anyone else made the transition and how interesting is the security role as compared to working on challenging projects in devops?

Security role entails applying security practices to teams building on AWS/GKE and wanted to get a better perspective on the role and what this means

https://redd.it/nsy6t1
@r_devops

What are some good devops course for React and Node apps?

What are some good devops course for React apps? Need some recipes I can apply to React and Node projects.

https://redd.it/nsvfqq
@r_devops

What it means to run a monolith on AWS Lambda

Over the course of 4 years I steered the DevOps culture in the company I work from EC2 to ECS + Fargate and finally to AWS Lambda, leading to better code that survives ephemeral environments, improved release practices, continuous delivery and cost reduction. I recently wrote a great piece on my experience running a monolith on AWS Lambda for the past 2 years.

https://blog.deleu.dev/what-it-means-to-run-a-monolith-on-aws-lambda/

https://redd.it/nst4iu
@r_devops

Managing Action Approvals with GitHub Actions

I'm a huge fan of GitHub Actions, after years of using every CI/CD tool under the sun, I see Actions as truly innovative and for the most part that's because of the Actions interface and ecosystem.

However, it is a whole new threat vector, I now have teams pulling in God knows what Actions that they've found online. And that is usually fine because there are tons of useful and free projects to choose from but it's a mirror of the import/dependency problem in repo source code. Everyone pulls in dependencies which in turn pull in their dependencies and it's a security nightmare.

GitHub has some security knobs in place for this but I'm just wondering if anyone here has devised a flow for vetting and allowing Actions in their teams' pipelines.

https://redd.it/nst35l
@r_devops

Bitnami Tomcat issue with the base64 enc database password

Hello All,

I am using bitnami tomcat 9.0.45 to deploy our containerized application onto kubernetes. This web app connects to a remote database, and the credentials for the same are defined as variables in the config map and the secrets, the values of these objects set in the helm values.yaml. I am deploying the containerized app through the Gitlab CI, wherein after the successful deployment I can see the DB password in the plaintext in the console logs of tomcat, whereas we have set DB password to be base 64 encoded on the GitLab CI. Not sure, upon deployment, it just shows the plain text password but not the base 64enc, which is a security issue for the organization. Please advise how to solve this?

​

Thanks

https://redd.it/nsr269
@r_devops

I'm an IRL DevOps Eng and was curious if my problem would interest you?

I thought hey- since I know there are readers of all sorts on this subreddit, from starting out to experienced vet, I thought this might be a nice idea to illustrate a fun problem I'm having to see if anyone wanted to give it some thought?

I've got a Jenkins Server running in Kubernetes. I have a Bitbucket server (internally). I am not able to connect to Stash via the"Bitbucket Server Integration" plugin... and the issue seems to be with certs as per the error raised in the plugin, however I have verified my certs are correct in the truststore for the Jenkins container inside my pod (remember, it's on kubernetes)... so, what do you think could be the issue? ;)

https://redd.it/nsmkx6
@r_devops

Goal Setting is Bullshit: Try this Instead

“Set goals and you’re set for success.”

These might be the exact words that you would hear across the internet from the millionaire gurus while you procrastinate thinking of your goals. You might even have a diary that has your goals listed and defined. But what’s lacking?

https://www.p3r.one/goal-setting-is-bullshit/

https://redd.it/ntlggy
@r_devops

Object and Block Storage: How They Differ?

The difference between block and file storage makes heads spin due to the complexity of definitions and technical jargon across the internet. Even a technical person sometimes forgets the business value and makes decision fatigue their best friend when trying to figure out the value proposition of

https://www.p3r.one/object-and-block-storage/

https://redd.it/ntld87
@r_devops

basic questions about getting into devops career...

Hi. Thanks for reading. First a little background about myself.

- Mid 30's
- 3 years exp as Software Developer mainly using JS.
- Some experience with other programming languages including Python and Java.
- Experience creating basic Docker containers, CI/CD (CircleCI), deployments on Heroku and DO.
- Just above basic linux knowledge (Fedora is my main OS)
- Previously passed CCNA (I still remember some basic networking concepts)
- Read the Phoenix Project

I was contemplating switching to a more DevOps role but had a few questions first. I've already done an extensive search for answers but would like some further clarification:

- How common are Jr/Entry level DevOps roles? Would I be considered for a junior role given my skills and experience?
- How difficult is it to learn some of the DevOps related tools such as Ansible, Kubernetes, Terraform?
- True or False: DevOps usually requires on call.
- What's the best way to practice and learn all of these DevOps tools? What should I be building and demonstrating in my projects?
- Which cloud service is best for beginners to practice out of the Big3 - Azure, GC, AWS?
- How much programming do you actually do on a day to day?
- True or False: DevOps is typically a thankless job.
- How accurate is this roadmap?
- What programming language is most useful for a DevOps role? The roadmap says Go.
- True or False: DevOps is typically more stressful/high pressure compared to Software Dev and other tech related roles.
- How much do certs matter?
- True or False: DevOps is basically Softare Dev, Systems/Network Admin, SRE all rolled into one.
- Do you feel any job satisfaction?
- Anybody want to partner up and learn all this stuff together?
- Are any of your companies in Canada looking for a jr/entry level Devops? If yes, feel free to send me a PM.

Thanks ✌️

https://redd.it/nte7d4
@r_devops