Reddit DevOps
278 subscribers
69 photos
32.2K links
Reddit DevOps. #devops
Thanks @reddit2telegram and @r_channels
Download Telegram
Nginx unable to get SSL cert to work

I'm trying to get a kubernetes cluster using the nginx ingress controller and cert manager to get an SSL cert from lets encrypt. Currently is appears the cert has been issued however I am unable to get it to load properly inside Chrome or via curl.

​

Below are some config files that show the current config and following that some describes.

​

apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
name: letsencrypt-prod
namespace: cert-manager-prod
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: letsencrypt-prod
namespace: cert-manager-prod
spec:
commonName: [email protected] <- This is not actually what I have just for example reasons
secretName: letsencrypt-prod-tls
issuerRef:
name: letsencrypt-prod

&#x200B;

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
# add an annotation indicating the issuer to use.
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
# needed to allow the front end to talk to the back end
nginx.ingress.kubernetes.io/cors-allow-origin: "https://portal.<domain>.com"
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, PUT, POST, DELETE, PATCH, OPTIONS"
# needed for monitoring - maybe
prometheus.io/scrape: "true"
prometheus.io/port: "10254"
#for nginx ingress controller
ad.datadoghq.com/nginx-ingress-controller.check_names: '["nginx","nginx_ingress_controller"]'
ad.datadoghq.com/nginx-ingress-controller.init_configs: '[{},{}]'
ad.datadoghq.com/nginx-ingress-controller.instances: '[{"nginx_status_url": "https://%%host%%:18080/nginx_status"},{"prometheus_url": "https://%%host%%:10254/metrics"}]'
ad.datadoghq.com/nginx-ingress-controller.logs: '[{"service": "controller", "source":"nginx-ingress-controller"}]'
name: prod-ingress
namespace: production
spec:
rules:
- host: api.<domain>.com
http:
paths:
- backend:
serviceName: api
servicePort: 8090
path: /
tls: # < placing a host in the TLS config will indicate a certificate should be created
- hosts:
- api.<domain>.com
secretName: prod-ingress-cert # < cert-manager will store the created certificate in this secret

&#x200B;

me@LAPTOP-LINUX:~/Projects/k8/tmp$ kubectl get certificates
NAME READY SECRET AGE
prod-ingress-cert False prod-ingress-cert 30m

&#x200B;

Name: prod-ingress-cert
Namespace: production
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1alpha2
Kind: Certificate
Metadata:
Creation Timestamp: 2020-01-26T21:29:07Z
Generation: 1
Owner References:
API Version: extensions/v1beta1
Block Owner Deletion: true
Controller: true
Kind: Ingress
Name: prod-ingress
UID: adec15ea-4cde-4376-8e6a-306586e57119
Resource Version: 2780
Self Link: /apis/cert-manager.io/v1alpha2/namespaces/production/certificates/prod-ingress-cert
UID: c86bf4aa-8435-4da0-a822-1a0e9d1df38e
Spec:
Dns Names:
api.<domain>.com
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt-prod
Secret Name: prod-ingress-cert
Status:
Conditions:
Last Transition Time: 2020-01-26T21:29:07Z
Message: Waiting for CertificateRequest "prod-ingress-cert-1024021523" to complete
Reason: InProgress
Status
: False
Type: Ready
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal GeneratedKey 30m cert-manager Generated a new private key
Normal Requested 30m cert-manager Created new CertificateRequest resource "prod-ingress-cert-1024021523"

\`\`\`

&#x200B;

That's everything I can think of that might be of use in resolving this issue. I should note I have a similar setup running on development (well did till I took the cluster down today) that is/was working.

https://redd.it/eueoxc
@r_devops
(A few) Ops Lessons We All Learn The Hard

https://www.netmeister.org/blog/ops-lessons.html

To be frank, It is not «a few», it is 88! I was thinking about choosing the points I agree with the most and write these down, but I realize it would be hard to choose a top three. Even a top ten. I might as well make cron send me the link once month as a quick and steady reminder. This just hit too close to home.

https://redd.it/eu7wv2
@r_devops
Code challenge sites for Devops/sysadmin scripting (Python)

Hey guys,

Wondering if anyone knows some good sites to practice and expand Python scripting skills with exercises related to linux admin/ devopsy tasks. Basically all things parsing files and linux related.

[PyBites](https://codechalleng.es/bites/75/) is pretty good, with some relevant questions. Example questions is "*Parse Unix cal to a weekday mapping*" or implementing grep.

Another one is [CodeSignal](https://app.codesignal.com/challenges/page/1) which has good questions in a 'Devops' category.

Alot of the other sites such as Codewars and edabit only have algorithmic type questions.

https://redd.it/euaua7
@r_devops
I'm a (36m) Sr. SRE / DevOps Engineer . Where do I go from here ?

Greetings and thanks for reading.

I am currently employed as a 100% remote "Sr. SRE" for a company that is primarily V.C. funded.

My base salary is 145k, I sometimes receive a bonus, and all my remote expenses (phone, internet, cowork space) are covered. I live in a lower cost of living area. All in all, it's not a bad setup. But as someone who is approaching middle-age, I am beginning to wonder how I can continue to increase my earnings, stay relevant, and not burn out.

&#x200B;

I have ambitions for technical leadership / management but I'm not sure how to get there. In my current role, I am an important contributor and regularly consulted about implementation details etc but the reality is that I am at the bottom of the hierarchy for my department. I believe my salary is in the upper tier for individual contributors within the company, but there is nobody below me in the org chart.

&#x200B;

Trying to start my own company sounds like a big risk now that I have a wife, child and mortgage, but I certainly don't hate the idea.

I fear the potential for age discrimination if I don't advance one way or another.

&#x200B;

Some stats abut me:

\- Almost 36 years old

\- Didn't go to University, have worked since I was a teenager.

\- Have been a linux hobbyist since 1998

\- I have something like 15 years of job experience that basically goes tech support -> sysadmin/I.T. -> devops -> SRE over the span of my career, with a little bit of training/mentoring and technical leadership sprinkled in.

\- I'm decent at python, for systems and webdev stuff

\- AWS DevOps Pro certified

\- Have some experience training/mentoring jr. engineers with sysadmin, config management, and basic programming

&#x200B;

Technologies I regularly utilize:

AWS, GCP, Ansible, Terraform, Salstack, Packer, Jenkins, Kubernetes/Docker, Nginx, Apache, postgres, mysql, monitoring tools like nagios, icigina2, checkmk, ELK stack ...

&#x200B;

I feel very comfortable working on large scale, production infrastructure. I consider myself to be a master of troubleshooting and very resourceful at learning new technologies on my own.

&#x200B;

If you've made it this far in my post, thanks for reading. Am I underpaid, overpaid? What can I do to move onward and upward in my career?

&#x200B;

Thanks so much.

https://redd.it/eua83x
@r_devops
Best way to create automations on the cloud?

I want to create an automation on a website without web hooks (pixieset). How can I create an automation on the cloud for this?

https://redd.it/euhe10
@r_devops
Secure website

Hello I hope someone can help me I get a task for a work I try to land.




I need to secure a website ruining in dockers, I'm using this image




[https://github.com/TrafeX/docker-php-nginx](https://github.com/TrafeX/docker-php-nginx)




I get some troubles making the SSL, I wonder if I can make the SSL in the same docker file,




Also if you have any other best security practices that wanna share I could appreciate any help.




Thanks

https://redd.it/eu6z6y
@r_devops
DevOps/SRE Python online learning platform

Hi all,

I am a DevOps/SRE with 10y+ exp. I want to challenge myself in order to progress in Python.

But i struggle to find some good online formation tool in order to get exercises (devops oriented or not) in Pyhon.

I have found things like "PyBites". There is other or better site like that ?

&#x200B;

Thanks for your feedback.

https://redd.it/euovp2
@r_devops
Best way to learn Containers / Kubernetes?

I've been searching for a new job as a DevOps Engineer and one of the things I'm lacking that everyone wants is experience with containers / Kubernetes. What are some good resources for learning these?

Thanks!

https://redd.it/euqbac
@r_devops
What’s your testing practices for Infra,Jenkins Pipelines etc?

We have a combination of Terraform/ansible modules for infrastructure provisioning &management.As it gets more and more complex,we are looking to invest in more unit and integration tests.Our eco system consists of Jenkins pipelines for CI/CD,a customer vue app we wrote to be the front end of the orchestration layer.We are looking to add more testing across our eco system.Planning to leverage local stack,Inspec and puppeteer for the front end app.

What are your testing strategies?What are the testing tools you use?What are your learnings?Was there any tools any one used for Jenkins pipelines?

https://redd.it/euproz
@r_devops
AWS Fargate loadbalancing need advice

Hey,

I have a website, frontend on S3 and 4 microservices for it which I want to put in Fargate. Really only one of them will most likely need to ever be scaled which is where my question arises:

How do I loadbalance the services? So far I can see 3 solutions:

\- put each container inside its own service, put loadbalancer in front of each one and don't worry about it (but each loadbalancer costs money)

\- put all containers in one service with a single loadbalancer in front and scale the whole thing (but I need an expensive machine)

\- use route 53 for service discovery and nginx container to dynamically resolve DNS records and hope it works because there is no algorithm behind this?

How do you guys set up your projects?

https://redd.it/euotug
@r_devops
Issue tracking for open-source projects

I have an open source project and we manage release planning in jira, but managing 2 repositories for issue tracking is cumbersome. Is there a way I can overcome this. How do open-source projects generally manage their issues for every release cycles ??

https://redd.it/euonmc
@r_devops
Guide to cracking interviews from insiders?

I have been in the process of doing interviews for Devops roles and been having mixed success. There are many rounds to even get to the onsite but typically hit a brick wall.

I seem to get picked by HR managers as my resume has the key words (buzz words use to screen) and then do countless calls set up by recruiters.

For the assessment, I do technical tests and for the most part my answers are right, but the managers are so picky that even if you answer is correct you don’t get selected. The questions they ask are not super hard. Most frequent URL from a file, ping IP addresses in a loop, replace a word using sed command. I ask is this correct, and you here the usual (we’ll get back to you).
There doesn’t seem to be any objective standard and it seems very arbitrary. Do you want me to write the code first time right without any errors? Did I take too long?
(You get a typical thank you applying while we were impressed , by your evaluation we have decided to move on.. yadda yadda .. very impersonal and no sort of helpful feedback.


When I go onsite, they start asking me super curve ball questions and sometimes obscure out of left fields. I’m not a networking guy.. yet they ask me What port is SNMP, what is difference between router and a switcher, what level are they on? And nuances between Entrypoint and CMD? And what are inodes?
The worst is when you are expected to memorise command line functions such as using stat or how to sort logfile and knowing the right arguments.

Things you usually look up on Wikipedia as I don’t memorise the documentation. Then I do white boarding and draw my setup - it’s a Kubernetes cluster with pods running micro services. Then would you make this multi regional or zonal, are you using VPC peering?
Basically I have to explain the architecture and theory behind this as If a lecturer, I’m a practitioner of tools

I am bewildered as how to play this game. I miss some super easy questions (such as what’s the docker process running in background- dockerd ..how do assign Kubernetes pods to nodes ..using labels). Obviously I try to go prepared but you can never know what they are going to throw and it you.

The recruiters are horrible and pitch you to get their quota, sometimes they even send the wrong job description and make appointments without your permission. Obviously they don’t want to gamble and make the wrong decision on a hire, the managers though are the real ones in charge and decide if they hire you. And it’s not easy to get past them.

Obviously I know this whole process is a vetting out competition trying to weed out weak candidates, but at least make it skills based and something you will do on the job. I can make a CI /CD pipeline, set up a cluster , do automated deployments all day, set up monitoring tools.

But asking me what is a replica set and deployment difference ( trick question ) .. they are the same.. or what is Kubernetes operator? It starts becoming more a sadist ritual on trivial jargon that an evaluation of your competence.

Sometimes it seems like companies don’t know what they want - or they send the wrong message about their company. Are they trying to measure ability or the persons ability for cultural fit? I’m even afraid to answer the right answer, as they will follow up with a bunch of even harder questions until I choke. I start thinking the Gods of tech are cursing me and delighted in making me look very foolish ...

I’m starting to think who these guys hire because it’s the unicorn who uses their exact tech stack, needs zero training, and subject matter expert in several domains with exceptional knowledge of system architecture.

I’m not trying to make light of a job selection process as it’s challenging, personally believe it’s about compromise .. if person suffices and satisfices rather than creating a mythical tech worker based on a checklist.

https://redd.it/eum46s
@r_devops
How to pass a variable from a Root Module down to a child module? Terraform

I'm not understanding the usage of Output Variables correctly or something ... the documentation is confusing and I can't find examples online.

Structure:

|- /api/datasources

|--------------------- dev.tf

main.tf

I understand how to broadcast the output, but how do I consume it from the child element?

namely ... SSM parameter value..

-----------------------------

module "transfer" {

source = "./api/datasources"

api_id = "${var.api_id}"

}

--------------------------

output "api_id" {

value = data.aws_ssm_parameter.myapi.value

}

------------------

The child module (datasources) needs to consume the output of this ... but HOW DO I pass it thru the module?

How do I consume the api_id inside of /api/datasources/dev.tf?? Here is dev.tf:


---------------------
resource "aws_instance" "name"{

api_id = var.api_id

}
---------------------

Help... please

https://redd.it/euv4n3
@r_devops
Next Generation Machine Learning IoT/Embedded Security

Hi guys!

We are working on an embedded security project. We actually released our first version targeting some webserver process in an OpenWRT on ARM32, in the following weeks we will release MIPS and ARM64 version too.

Hope you like and please give us suggestions!

Github: [https://github.com/Exein-io](https://github.com/Exein-io)

https://redd.it/euliln
@r_devops
Falco vs OpenShift Security Constraints

Hi!

Does anybody knows the main differences between these two alternatives regarding security on Kubernetes? I've found no paper/video explaining it. What do you use on your day to day and how is your experience with it?

Thanks!

https://redd.it/eukuor
@r_devops
DevOps or Agile?

Is DevOps a part of Agile, or is Agile a part of DevOps? Are they different things all together? SAFe believes DevOps (CI/CD) is part of overall Agile. I’ve heard and used both terms as parts of each other and their own items. I find it depends on the audience. This sub seems to be mostly focused on the technical pipeline piece of DevOps. Is that the “right” definition?

Maybe they’re just buzzwords with no clear definition...

https://redd.it/euys5d
@r_devops
How to move to using infrastructure as code / configuration management / devops tools in a monolithic environment?

Old school sysadmin here, and I am struggling to find a value add that I can use as a justification to start such a project in a windows environment. The idea of ansible/chef/terraform sound amazing, but we already have tools to do most of the things I read about and none of them require messing with YAML or learning python. We are still buying software from vendors that are struggling with the concept of virtualization & server core let alone stateless containers. We don't do much in-house dev work and most of apps are typical 3 tier (SQL/APP/WEB). So, where could I find value with these tools?

https://redd.it/euyzk6
@r_devops
How do you folks implement load shedding?

A common recommendation for running high availability systems is [load shedding](https://aws.amazon.com/builders-library/using-load-shedding-to-avoid-overload/). Last year I personally experienced an outage that was exacerbated by a lack of load shedding. I can find a handful of resources talking about this strategy but none touch on the practical aspects: Okay, I want to implement load shedding, how do I do that? What software can I use?

I've been using HAProxy as a reverse proxy in front of my application, which [has some queuing features](https://cbonte.github.io/haproxy-dconv/2.1/configuration.html#timeout%20queue) that I was able to work into a primitive load shedding system. Unfortunately, I'm going to have to move away from it (or pair it with nginx or similar) because HAProxy [doesn't support request buffering](https://www.mail-archive.com/[email protected]/msg23074.html).

Wondering what everybody else's experience is with this.

https://redd.it/euu7rw
@r_devops
I wrote a self-hosted slack bot that will notify you when you are mentioned in on GitHub projects or Organizations in Lambda.

GitHub project: https://github.com/bagel-dawg/github-slack-mentions

I was tired of having to check my email to find out if my username was mentioned, or if my PRs were approved/commented on. It also has the ability to remind you daily of any notifications that are still on-going.

The reminders functionality assumes it will be used with an organization, but the mentions functionality will work with standalone repositories OR organizations.

It is based on AWS API Gateway+Lambda integration proxy and DynamoDB with 2 indexes for key+value storage. It can be deployed manually, but there is also a serverless.yaml for convenience.

https://redd.it/euxzrx
@r_devops