Reddit DevOps
268 subscribers
1 photo
31K links
Reddit DevOps. #devops
Thanks @reddit2telegram and @r_channels
Download Telegram
Question on NGINX reverse proxy when there are services on the host and on containers

So, I have this architecture depicted in the attached picture, which consists
of a nginx server on the host machine, and 3 URLs, app1.com, app2.com, app3.com.

app1 and app2 are both in the host machine, lets say they are an HTML file on the host, on the container, there is app3 and app2.com/example url.

My question is, how can I reference the container in the nginx of the host?
Since I cannot expose port 80 and 443 of the container, because the host
is already using these ports, I am not sure what is the best approach to this.

I though about putting the container on a not used port, like 8999 for example,
and proxying the request to this port, and blocking these ports on the firewall,
so no one outside can access them. If I go with this option, does the nginx
configuration inside of the containers will need the server_name still? In the
case of the app3.com/example

The other option I thougth about is using the docker socket, or even the docker0
network, but I don't know if this is viable.

An example of the nginx configuration file of the host:
```conf
server {
server_name app1.com;
location / {
root /var/www/app1/index.html;
}
}

server {
server_name app2.com;
location / {
root /var/www/app2/index.html
}

location /example {
proxy_pass https://localhost:8999;
}
}

server {
server_name app3.com;
location / {
proxy_pass https://localhost:8999;
}
}
```

Example of the nginx configuration file of the container:
```conf
server {
server_name app2.com
location /example {
root /var/www/app2/index.html;
}
}

server {
server_name app3.com;
location / {
root /var/www/app3/index.html;
}
}
```

Rough picture of the architecture (sorry):
https://imgur.com/Hx3enGi.png

https://redd.it/g6jigv
@r_devops
How to prepare for a 50 min devops assigment?

I got an email to complete a timed assessment it has some Jenkins in it which I’m not too familiar with. I never done a Devops time assignment what should I expect?

https://redd.it/g593os
@r_devops
DB Migrations for build pipeline in Azure DevOps

So I'm kind of stuck on how best to accomplish this. I have a docker image that's being run as a Web App for Container in Azure. Build and release pipelines are already setup with the build pipeline building and pushing the image to the container registry and then creating an artifact to be used by the release pipeline. All of that works great but occasionally we need to be able to run 'rake db:migrate' against the image before it boots in the container.

Is the best way to do this by adding a step after build which just runs a one line script such as docker exec $container rake db:migrate ?

https://redd.it/g6r44k
@r_devops
How to hide nginx version in elastic beanstalk + Docker

Hello!

My question is similar to

[https://stackoverflow.com/questions/31603784/how-to-hide-nginx-version-in-elastic-beanstalk](https://stackoverflow.com/questions/31603784/how-to-hide-nginx-version-in-elastic-beanstalk)

I am running app on the platform docker + elasticbeanstalk and wish to hide the nginx version from the HTTP headers.

Should I just add a http block ? I am asking because the platform is different and also I don't see the http block in the current nginx conf file.

​

Thanks.

https://redd.it/g6q2aw
@r_devops
Wired dot1x, Linux machines out of domain

Hello!

First of all - nice to meet you all, as this is my first post here.

I have a quite an issue. Or more like a solution that is not entirely perfect and I'm wondering if there is some way to make it better.

A little background - we're currently implementing dot1x solution in our company that is mostly MS oriented so Active directory and so on. Unfortunately (for me) there are some specific projects that do not use Windows - so Linuxes and MacOS. Moreover - machines that are not windows are generally NOT in ad with users having access to root. As for if this is good idea. Obviously not, but what do I know, not my idea. Fortunately Ubuntu machines are mostly Ubuntu so when I write Linux I generally mean Ubuntu (18.04 is standard if that matters)

Radius server for dot1x is on Cisco use - which is good - we're mostly on Cisco and ise seems to work quite well with ad.

For the plan:
Windows machine is authorized via ad computer account to some profile and assigned some vlan id. Whenever some users logs into it then he's authorized via radius and if needed - assigned another profile.

Problem is with Linux machines. As I know there are two options right now:
1. Use certificate for machine to authorize with and bind certificate to profile
2. Use users credentials on wired configuration

Now for the problems with both of these:
1. We obviously would need to distribute certs manually and change them whenever people move from one project to another. This is just not scalable, but worst case scenario.
2. As users have root they can pretty much check password stored for dot1x, as user/password used would be from ad - this is not secure and will be axed the moment we mention it

Is there some smart way to make it work?
Maybe create same computer in ad, set hostname on Linux and make authorization work by forcing radius to query computer account based on hostname? Or maybe some way to encrypt all dot1x configuration so NOONE can check what credentials are stored for dot1x? This would require some synchronization with checks for password between as and linux, but better then nothing.

Or maybe there is some other, more 'gentle' way to make such solution work in my case?

https://redd.it/g6uqa8
@r_devops
Prometheus Exporter Java Agent not working

I am attempting to setup confluent kafka v5.4 and running the prometheus JMX exporter. I have found this blog for how to get this setup https://alex.dzyoba.com/blog/jmx-exporter/ . Kafka is setup and runs just fine but the endpoint for the exporter returns nothing. I've tried just about everything for how I call the javaagent in the systemd script but nothing seems to work.

[Unit]
Description=Confluent Kafka Broker
After=network.target network-online.target remote-fs.target zookeeper.service

[Service]
Type=forking
User=confluent
Group=confluent
Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.javaagent=/opt/prometheus/jmx_prometheus_javaagent.jar=8080:/opt/prometheus/config.yaml"
Environment=LOG_DIR=/var/log/confluent
ExecStart=/opt/confluent/confluent-5.4.0/bin/kafka-server-start -daemon /opt/confluent/confluent-5.4.0/etc/kafka/server.properties
ExecStop=/opt/confluent/confluent-5.4.0/bin/kafka-server-stop
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

Any ideas on how to call that java agent in the systemd script to get it to work correctly?

OS Centos 7
JMX exporter 0.12.0
Java openJDK 11

Logs that I have found are not telling me anything as to why it's not running. Maybe I'm looking at the wrong logs.

https://redd.it/g6teui
@r_devops
Frustrated with trying to do configuration management on Jira

Is anyone else really frustrated with trying to do configuration management on Jira Server, especially while trying to do configuration as code? It just seems like trying to automate some things seems all over the place, sometimes you have configuration files, sometimes you need to do it via the REST API, sometimes you need to fiddle with some database tables it just seems that everything is so inconsistent.

The worst offender of this seems to be plugins, we tried to implement SAML SSO via a plugin but automating that setup seems outright impossible.

Is there anything obvious I'm overlooking here or do other people have troubles like this too?

https://redd.it/g6ob66
@r_devops
Difficuly of upgrading hashicorp vault oss to enterprise

I am looking at setting up hashicorp vault. We are thinking of starting with the oss version and then later moving to enterprise. Is the upgrade procedure straight forward?

https://redd.it/g6mvlx
@r_devops
Tools for Terraform and Azure testing

Hey All,

What tools are you using to test your Terraform meets expectations in an Azure environment? I see tflint has a plugin but its just started development - anything else you guys use?

Cheers

https://redd.it/g6jqj6
@r_devops
Jenkins integration with Jira

Hi team,

Did anyone tried integrating jenkins with Jira. I saw couple of old video on youtube. However its not working for me. Can some please point to right link or how to video.

https://redd.it/g6o6y9
@r_devops
Communication between private AWS resources: Route53 and Security groups

My network-fu is not the hottest and I'm running into some problems with my AWS stack, hoping the community here can point me in the right direction. My problem is my security groups appear to be blocking traffic that uses domain names for addressing instead of raw IPs. For context, I am using ansible to provision and manage my AWS resources.

Basic picture is that I have a publicly available load balancer which proxies traffic to private app servers, which talk to a private DB instance. I have security groups roughly set up so that the load balancer is in a SG called `sg::load_balancer`, the app servers are in `sg::app_server` and the database in `sg::db`, so that `sg::db` instances only accept incoming traffic from `sg::app_server` instance which only accept traffic from `sg::load_balancer` instances. Now this works fine if all the network calls actually use the IPs of the relevant instances, i.e. if the app servers make networks calls directly to the DB IP, and the load balancer's nginx config has the app server IPs listed as upstream servers. However, I would much rather have semantically meaningful domain names in my configs (`app-server1.mydomain.com`) instead of templates with IPs injected into them at provision time. It kicks the can to having to update Route53 with the new IP for a given domain when provisioning new resources but it feels cleaner to me.

The problem is that when addressing between instances using domain names instead of IPs, the security groups block the traffic. I don't fully understand the situation, but I guess when using domain names, the traffic is proxying through the Route53 servers and so the incoming traffic to private instances is no longer coming from the machines in the specified security groups. So, is there a way for Route53 to play nice with my security groups or is there a better strategy for this altogether?

https://redd.it/g6o531
@r_devops
any recommendations for managing storage in a private cloud environment?

since adopting a more "devops-y" culture with engineering & sysadmin, our infrastructure team is struggling to keep up with storage demands now that we've adopted a higher velocity through devops collaboration, particularly around managing the storage to be used for clustered databases. unfortunately the collaboration is being done between engineering & sysadmins, but the infrastructure team still wishes to operate autonomously, so i'm trying to help enable them to keep up by solving some of their pain points and routine tasks.

we primarily use nimble & netapp storage platforms. i'm looking at their native tools and APIs, but i'm wondering if there's a better way to approach this.

initial search for terraform seems to be lacking providers for the on-prem offerings from what i can see. ansible has some good stuff for netapp, nimble seems to all be community created from what i can see.

https://redd.it/g6ng6e
@r_devops
Overriding Prometheus alerts

Hey, we've been struggling with overriding alerting rules in our Prometheus monitoring setup.

We have a set of generic rules that applies to all cluster resources across multiple environments and wanted to alter or disable alerts for a few subsets of resources.

Take a look at how we've solved it [https://medium.com/p/how-we-solved-our-need-to-override-prometheus-alerts-b9faf9a4558c?source=email-fe7d0a36f5f9--writer.postDistributed&sk=7eb8d3b8d2c0c6fd710a5f19cc649399](https://medium.com/p/how-we-solved-our-need-to-override-prometheus-alerts-b9faf9a4558c?source=email-fe7d0a36f5f9--writer.postDistributed&sk=7eb8d3b8d2c0c6fd710a5f19cc649399)


Feel free to post other ideas and suggestions.

https://redd.it/g6jz30
@r_devops
Tools for writing CI tests

Hi everyone.

I know this is a very broad topic but. Can people please like list tools or frameworks that they use or know of to make tests for CI pipelines. And maybe list what languages they are commonly used to test for. I know for some like selenium and behave. But would like to see what others there are. So I can make a better decision.

Thanks, Jpac14

https://redd.it/g6m5jk
@r_devops
[Article] I just finished writing my second medium blog post on Writing a Custom Concourse Resource - The check.

I just finished writing my second medium blog post on Writing a Custom Concourse Resource - The check. Please check it out! The more people read it the more motivated I will be to write the next posts ("The in", and "The out")!

https://redd.it/g6lr99
@r_devops
How We Build a Release in One Minute

Any team working on a growing software project eventually has to adopt more internal infrastructure to manage the code’s complexity and the development team’s size. I'd like to show you our article about how our team arrived at our current production build toolchain that helps us to build a release in a mere minute - [https://flussonic.com/blog/news/how\_we\_build\_a\_release\_in\_one\_minute/](https://flussonic.com/blog/news/how_we_build_a_release_in_one_minute/)

https://redd.it/g6kqb4
@r_devops
Any up-to-date books about Jenkins and Docker?

Hello,

I've been studying Docker and Jenkins for a week and a half now and I'm pretty comfortable with Docker in general. However, the book by which I am studying is about 3 years old and is made for Jenskins 2.36 (I think?) and some features are either missing in the latest Jenkins version or work in a completely different way, forcing me to troubleshoot or blindly guess which options I should use or what I should do.


Any book recommendations would be very appreciated!

https://redd.it/g6jm7b
@r_devops
What’s the cheapest way to get dashboard metrics on lambda functions?

I built a grafan instance to monitor lambda functions across accounts then attached it to cloud watch metrics and then realised it would be expensive.
Is there anything I could try that might be a cheeper solution?

https://redd.it/g6ivqn
@r_devops
On-prem: How would you clone a k8s cluster

Situation: I want my prod cluster to exist on another site, the hypervisor might change, but the cluster services must still be the same.

The cloned cluster will process the same data schema, its just different sites should have different clusters (so that means absolutely not extending the prod cluster to that site)

What I already did:
1. Packer to generate machine images - can export to vmdk/vdi/others
2. **Ansible** - specifically, **Kubespray**, when I need to deploy to a site I just need to get 3 IP addresses (plus another 3 for metallb)
3. use the clusterinfodump from the main cluster to deploy services to replicate the state of the prod machine


I know there are better ways to do step 3 like templating using helm charts, am in the process of cleaning it up so that I wont use the clusterinfodump anymore.

have you solved this in your environment?

**How would you assure the fidelity of site clusters to your core cluster state?**

^this is not an interview question, I have actually done steps 1 and 2 (in a way 3, when I borked the cluster), and after passing tests I'll need to have some idea on 3, I'm basically priming myself for information now on how to do it properly.

https://redd.it/g6fm9n
@r_devops