replacing certain lines in files depending on dev/prod environment?

I'm working on setting up multiple tools automatically (with containers) and there are quite a few configuration files to manage.

We have a testing/development environment and a production environment, so I'm wondering if there is a way to replace certain lines in files e.g. a url for dev/prod, maybe some permissions, etc.

I'm thinking of someting like Angular's environment.ts since that's the background i come from. Is there a tool for something like this just with normal files?

EDIT: If anyone ends up stumbling upon this, ansible's template module/jinja2 is a really good solution to do this

https://redd.it/g5erc2
@r_devops

How to run Python Selenium with Chrome GUI on Ubuntu Container with Browserfull (NOT Browserless) on Docker on a WIndows 10 host PC?

How to run Python Selenium with Chrome GUI on Ubuntu Container with Browserfull (NOT Browserless) on Docker on a WIndows 10 host PC?

I want to run **Chrome Browser** with **Selenium Python** from my **Ubuntu container** on my **Windows 10 Host** PC. I **DO NOT** want to run *headless browser*.

I want to change my **Dockerfile** to use Chrome browser with Selenium Python.

I have tried *downloading Chromedriver*, but it is failing.

I have tried out all the other suggestions from Google and Stackoverflow, but nothing is working. Any suggestions?

The following is my Dockerfile:

FROM ubuntu:latest RUN apt-get update --fix-missing; \ apt-get install -y software-properties-common; \ rm -rf /var/lib/apt/lists/* ; \ add-apt-repository -y ppa:deadsnakes/ppa; \ apt update -y; apt upgrade -y; \ apt install -y python3.6; \ apt install -y python3-pip; \ yes | python3 -m pip install --upgrade pip; \ yes | python3 -m pip install virtualenv pip; \ apt-get install -y python3-venv; \ yes | python3 -m venv my_py_venv; \ add-apt-repository -y ppa:git-core/ppa; \ apt update -y ; \ apt install -y git; \ yes | python3 -m pip install --upgrade pip; \ yes | python3 -m pip install virtualenv; \ source my_py_venv/bin/activate; \ yes | pip3 install wheel; \ yes | pip3 install selenium; \ yes | pip3 install git+https://github.com/behave/behave; \ yes | pip3 install allure-behave; rm -rf /var/lib/apt/lists/*;

I have downloaded the ChromeDriver ([https://chromedriver.storage.googleapis.com/index.html?path=83.0.4103.14/](https://chromedriver.storage.googleapis.com/index.html?path=83.0.4103.14/)) but I am not sure which version to use (*chromedriver\_win32.zip* OR *chromedriver\_linux64.zip*).

And while instantiating the Chrome Browser instance, should I use Windows Directory Syntax ('\\') or Linux Directory Syntax ('/') in the code below:

driver = webdriver.Chrome('/steps/chromedriver')

Thank you for your time and suggestions.

​

[https://stackoverflow.com/questions/61341084/how-to-run-python-selenium-with-chrome-gui-on-ubuntu-container-with-browserfull](https://stackoverflow.com/questions/61341084/how-to-run-python-selenium-with-chrome-gui-on-ubuntu-container-with-browserfull)

https://redd.it/g5dfn2
@r_devops

Hi, I want to clone specific folder using Jenkins.

Hello fellow redditors, I'm working on one pipeline and because of space issue on server I have to clone one specific folder using Jenkins. I'm using Git SCM plugin to generate pipeline syntax. Please help!

Thank you!

https://redd.it/g5bu0b
@r_devops

What tools do you use along with Ansible Tower to run initial unit testing?

I'm having some trouble coming up with the "right" tool to use. I'd like to test code that is submitted to my development branch for syntax, errors, as well as run with --check by using either a setting or variable in Ansible Tower. The goal here is just on initial commits to the staging branch, as I'd like each person submitting changes to verify their own code against Ansible Tower before initiating a merge request to the staging / dev environment.

https://redd.it/g604n2
@r_devops

Single click putty session

Hey Guys,

Hope everyone is doing good. I recently started learning about DevOps and following an online class. The videos are recorded and I can't contact the instructor.Now, I've created an instance in my AWS console, created a .pem key, converted it into .ppk using putty gen and launching a SSH shell using putty.

I want to know is there any way I can launch this session with a single click than copying the ipaddress of my instance and loading the .ppk everytime? I don't even know if it's possible, so excuse me if this is just a silly doubt.

Thanks in advance. Namaste!

https://redd.it/g5zrxm
@r_devops

Doing CI/CD for ASP.Net using containers?

Does anyone have experience with a CI/CD pipeline for [ASP.Net](https://ASP.Net) projects using Windows containers? We are mainly a Linux/PHP joint, but have lately also aquired some companies that also do .Net development.

We currently have setup a build agent (we run Bamboo) that has nuget and Visual Studio 2017 installed. Now they also want to build with newer .Net frameworks and that's giving us version conflicts (wrong version of msbuild etc.). We managed to sort that out, but it's not very flexible and I'm pretty sure we'll run into problems when the next version is there. On our PHP projects, we have a set of containers that each have their own specific PHP or NodeJS version, which makes switching between versions very easy (just switch containers and done). Is something like that possible for Windows as well?

Most deployment workflows are quite simple. They do nuget restore, build the solution and sometimes a nuget pack but that's basically it. I think the main problem right now is that they are relying on devenv.exe to actually build the solutions, which requires Visual Studio to be installed on the machine. A lot of the projects they are working with are not yet compatible or sufficiently tested on DotNet Core either (like older versions of Umbraco), so we can't just use that on Linux containers.

Any leads on where to start are more than welcome. Most resources I find online are focussed on running apps in containers (not using them for building) or tied to Azure. We basically just need to feed it a folder with the project, run some commands and hand the result over to the deployment part of the pipeline.

https://redd.it/g5wywf
@r_devops

How to pass values to serverless template without displaying them in the resulting cloudformation stack?

At my work we are using serverless to create all AWS resources. I want to create some parameter store parameters, is there a way to pass these values without displaying them in the template? I want to do something similar to cloudformation parameters with “NoEcho” option.

https://redd.it/g5yr02
@r_devops

Shared hosting microservices and reverse proxy

Hi there, I am trying to set up some microservices on a new EC2 server. Essentially we are setting up a SAAS product, but we want to use shared hosting for the software for different clients. Because of this, I need to route network traffic through a reverse proxy to make sure network requests reach the correct containers.

It is worth mentioning that at the moment, the infastructure for the different clients are not seperated by any VM or container, each service just operates in its own docker container. Each service atm also has it's own nginx instance at the moment, because that makes development much easier. I am considering moving to having only one nginx container per client, that each service uses. I assume this will be better long term.

Keeping docker containers in networks should allow me to dynamically route requests using the container name instead of the static IP, which might change. View image 1 for a picture of what I mean.

The reverse proxy works when I set an upstream like so:

http {
upstream storage {
server clienta_nginx_1:8000;
server clientb_nginx_1:8001;
}
}

However, nginx cannot boot if any of the declared upstreams are down for some reason, which prevents ALL services from being accessed when only one is down. This is clearly not allowable. To attempt to circumnavigate the issue, I instead use a variable syntax, using the nginx resolver parameter and the docker local DNS resolver like this:

server {
resolver 127.0.0.11 valid=5s ipv6=off;
set $client clienta_nginx_1:8000;
proxy_pass https://$client;
}

This resolves the name of the container into an IP address and attempts to pass the request through to the container, but gets this error:

6#6: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.24.0.1, server: clienta.example.com, request: "POST /files HTTP/1.1", upstream:
"https://172.24.0.3:8000/files", host: "clienta.example.com:80

I should also add, I can connect to the service just fine if I try and access it through postman on port 8000, which means that the port is correctly exposed to the container. It just gets refused when I attempt to access it through the reverse proxy. And the two nginx's are both hosted in docker containers, that are both connected to an external docker network.

I suppose I have two questions, do you have any recommendations or suggestions for my infrastructure design? And also, can anyone explain to me why this doesn't work and how to fix this error? Please find supporting documents and images below as well.

[Image 1](https://imgur.com/a/GjgqFxQ)

[nginx.conf](https://pastebin.com/yfeAQzFV)

https://redd.it/g5x5xm
@r_devops

jsonnet IDE?

So I wanted to mange my Grafana dashboards as code and found [grafonnet-lib](https://github.com/grafana/grafonnet-lib) which looked promising.

After reading about jsonnet and getting all excited I quickly realized there is no full fledged IDE for jsonnet, sure some IDEs have plugins (intellij, atom, vscode), but not one provides a complete set of feature like autocomplete, goto definition, code folding, vscode comes close but the plugin hasn't been maintained since March 2018, and I refuse to work with vscode/atom since they feel like toys compared to Intellij products.

How do people mange their jsonnet code? I want amazed to find that a tool release in 2014 still doesn't have decent integration with popular IDEs.

https://redd.it/g5yeeu
@r_devops

J COle Morrison awsdevops.io AWS Devops Master workshop

Hi

Has anyone went through/completed training -AWS devops master workshop of Cole morrison?Does it provide a thorough understanding and real time experience and is it good for who is new to AWS and devops?

https://redd.it/g5w7t2
@r_devops

POSTMAN request doesn't work for me but works for others

I have been trying to run a PUT in POSTMAN, it gives me a 400 error.

However I provide my CURL to somebody else and it works, any ideas what could be causing this?

Thanks!

https://redd.it/g5wzpy
@r_devops

What made you to hire another devops?

Do you remember a moment that made you to take this decision and hire another DevOps?
What was the cause?
Production Problems? Help to R&D? On Call?

https://redd.it/g5w74d
@r_devops

Custom error pages based on response header

I am trying to solve an obscure task here and I need some help guys.
We have an application running on kubernetes and an nginx is set up in front of it to proxy pass the traffic to kube services. I need to setup custom error pages for 5xx and 4xx on nginx. There are, however, cases when developers specifically need to show the error response body from upstream in the browsers with some 4xx code. We came up to an agreement that they will send a specific custom header in the error response which they handled and I need to check if that header is present in the response then let the error appear as is, otherwise I need to show my custom error page. Can anyone suggest a configuration example on how to achieve this?

https://redd.it/g5btv7
@r_devops

Continuously evolving full stack Web App dev ops flow

Hello!

I have a full stack project that I have been working on with the following stacks:

\- Express Backend (Exposes GraphQL + WebSocket endpoints) hosted on Heroku with Redis for cache/pubsub.

\- React SPA Frontend hosted on Zeit Now.

\- Static assets like images on S3 & CloudFront CDN.

\- Postgres hosted on RDS

\- GitLab for source control + CI/CD

\- Have 3 environment deployment set up (review => staging => production)

\- Sentry for error reporting.

​

As I scale up the app and as I add more features, I am running into different road bumps and would like everyone's thoughts. (Apologies if not everything is "devops" related.)

1. The web app crashes every time I deploy a new app until the user refreshes the screen (because the old javascript assets no longer exist). There seems to be a LOTs of solutions here; what do you guys recommend for updating users to a new version of the app? Couple solutions are... Continuing to serve up old javascript assets (but people will be on an outdated app until they refresh. Could be a problem if the server introduced some new changes.) Would Server Side Rendering help here?
2. Currently, the frontend/backend is in a monorepo, and every time i make a change to the app, I have the CI/CD set up so that both front/backends will get deployed in parallel. This \_can\_ be problematic if IF am introducing breaking changes in the backend. How can I
3. Currently using too many platforms ? (Heroku, Zeit Now, AWS, Sentry, etc.). The app is getting more and more traction, and would like to support growing needs and features as fast as possible. Should I consolidate to AWS? I want to minimize dev-ops maintenance. Thoughts?
4. I have to add queues (for sending emails, analytics, etc.). I was thinking Thoughts here?
5. I want to add smaller services for different tasks (external webhooks, invoice generation, queue workers, websocket connection nodes, etc.) but currently I have been piling stuff on top of the existing express backend. I can see this is going to become monolithic if I continue this way. Am I pre-optimizing for microservice? Should I spend some time to create something that helps spin a new service?
6. Want to set up terraform scripts for the overall infrastructure. Too ambitious as a one man team?
7. Feel like I am not getting a good exposure to the health of the systems beyond Sentry error reporting. What's the easiest way to get started with monitoring these?

Apologies for lots of questions. I think I am mostly torn on all these questions because there are just lots of different solutions and finding myself in analysis paralysis. I have a tendency to overcomplicate and overthink things so feel free to tell me to KISS if you feel this way.

As I start a company, there are simply TOO many things I have to set up. You can also point me to different resources for someone like me starting off a software startup!

Really appreciate your help in advance!

https://redd.it/g641fm
@r_devops

[Help] Trying to setup my first CI/CD pipeline.

Hello /r/devops,

I have a general devops question.

I'm trying to build my first CI/CD pipeline.

I have a simple node/react web app

the backend is in one repo, the front in another.

I was hoping that with circleci I would be able to push to either master branch and start a pipeline that would result in a single doker image that can be deployed on my server.

So far I haven't been able to figure out a way to connect the two circleci 'projects' to create that single docker image.

I think that I may have the wrong idea about something.

Should I be looking to integrate another service like chef or puppet?

Should I just be doing this all on my own server with a bash script?

Any tips and pointers would be greatly appreciated.

For the record, I'm very comfortable with docker, bash, git and linux. But I'm very new to tools like circleci and ansible.

Thanks very much

https://redd.it/g66yos
@r_devops

What tools are you using for loadtesting?

I have no favourite so far. Currently looking at [https://locust.io/](https://locust.io/)

If I would need to go for paid solutions, I guess it would be [https://loadimpact.com/](https://loadimpact.com/)

https://redd.it/g69ue8
@r_devops

[Article] How to set up a CI/CD pipeline with Github Actions

Hey, here's an article I wrote on "[How to set up a CI/CD pipeline with Github Actions](https://ryanprater.com/blog/how-to-set-up-a-cicd-pipeline-with-github-actions)". I hope it helps someone!

https://redd.it/g64pfr
@r_devops

How to expose a service on Nodeport under EKS cluster with private subnets ?

I have an EKS cluster set on private subnets and worker nodes don't have a public ip, i want to expose a service on nodePort so i can reach it on <nodeIP>:<nodePort> but on the browser it shows me **site can not be reached**

https://redd.it/g67dmn
@r_devops

Interview preparation and interview questions for DevOps and SRE

I have compiled a small ebook for DevOps and SRE preparation. This book is based on rounds that can happen in a DevOps or SRE interview at companies like LinkedIn, Atlassian, Visa, etc. It has below chapters that will help you understand each round and how to approach it.

1. Linux Questions

2. Cloud Specific questions

3. Python programming

4. System design round and how to approach it.

5. Incident management.

6. Basic system Troubleshooting.

7. How to do code reviews.

These are motivated by interview round that happens in big companies that are hiring SRE and DevOps.

I hope this will help you. Feedbacks are welcome.

Thanks to Rahul Sinha for his contribution. hashtag

\#linux #sre #devops #systemdesign #interviewpreparation

\#interviewprocess #book #python

&#x200B;

[https://www.amazon.in/Interview-preparation-interview-questions-DevOps-ebook/dp/B086ZVY7KM/ref=sr\_1\_1?dchild=1&keywords=interview%20devops&qid=1586755852&s=computers&sr=8-&fbclid=IwAR0iYwFaq4zS6mOuZAoY\_lIpgetHw2NbtyJf73KQZHP-Al-RI3GHQhuUZm8](https://www.amazon.in/Interview-preparation-interview-questions-DevOps-ebook/dp/B086ZVY7KM/ref=sr_1_1?dchild=1&keywords=interview%20devops&qid=1586755852&s=computers&sr=8-&fbclid=IwAR0iYwFaq4zS6mOuZAoY_lIpgetHw2NbtyJf73KQZHP-Al-RI3GHQhuUZm8)

https://redd.it/g67a4v
@r_devops

Secure EC2 SSH Access for Dynamic IP with Ephemeral Security Groups

Hi, it's my first submission here. I'm more into Ruby than devops but recently I started using this approach to closing SSH ports on all my EC2 instances and wanted to share it.

[Secure EC2 SSH Access for Dynamic IP with Ephemeral Security Groups](https://pawelurbanek.com/ec2-ssh-dynamic-access)

https://redd.it/g5agcr
@r_devops

Question on NGINX reverse proxy when there are services on the host and on containers

So, I have this architecture depicted in the attached picture, which consists
of a nginx server on the host machine, and 3 URLs, app1.com, app2.com, app3.com.

app1 and app2 are both in the host machine, lets say they are an HTML file on the host, on the container, there is app3 and app2.com/example url.

My question is, how can I reference the container in the nginx of the host?
Since I cannot expose port 80 and 443 of the container, because the host
is already using these ports, I am not sure what is the best approach to this.

I though about putting the container on a not used port, like 8999 for example,
and proxying the request to this port, and blocking these ports on the firewall,
so no one outside can access them. If I go with this option, does the nginx
configuration inside of the containers will need the server_name still? In the
case of the app3.com/example

The other option I thougth about is using the docker socket, or even the docker0
network, but I don't know if this is viable.

An example of the nginx configuration file of the host:
```conf
server {
server_name app1.com;
location / {
root /var/www/app1/index.html;
}
}

server {
server_name app2.com;
location / {
root /var/www/app2/index.html
}

location /example {
proxy_pass https://localhost:8999;
}
}

server {
server_name app3.com;
location / {
proxy_pass https://localhost:8999;
}
}
```

Example of the nginx configuration file of the container:
```conf
server {
server_name app2.com
location /example {
root /var/www/app2/index.html;
}
}

server {
server_name app3.com;
location / {
root /var/www/app3/index.html;
}
}
```

Rough picture of the architecture (sorry):
https://imgur.com/Hx3enGi.png

https://redd.it/g6jigv
@r_devops