Service in AWS/Azure for getting randomized Domain Names (via terraform or other IAC Tools?)

Hi,
sorry for the headline, it seems i cannot wrote without upper case letters there. for my question:
for starting up tests or shortlived infrastructure it would be EXTREMELY helpful to have an component in azure/aws to request random domain names which can be mapped to certain public IPs which i want to set.. Do you know if there's such a service within Azure or AWS? i do not want to use another dns provider i would like to have it WITHIN azure/aws.

https://redd.it/g2gu5d
@r_devops

What to do after Docker

Hi, so I am currently making a website as a side project and I want to make a CI/CD pipeline to deploy the app. My current set up hosts my code on github. I have 3 branches :dev, staging, master. I run my tests on pull request and build the Docker containers on push request on staging and master branches. This is all done with github actions and my containers are uploaded to docker-hub. I then have my docker-compose file which uses the previously built containers and set environment variables and various things. I host my website on the smallest DigitalOcean VPS. The current set up is a nginx server reverse proxying to web server in a docker container. I figured I would do this so I could maybe do "load balancing" to update my site, so there is no downtime I can spin up the new container and then shutdown the old. I am using the Laravel framework which is in PHP, with a db, nginx as the web server and redis(not sure if really useful as everything is on the same machine?).

Now I'd like to achieve deploying new versions without having to log in my VPS hosting the containers and if possible do it without any disruption to the service. I also sometime have migrations to run on a database and I'd like that to be automated as well, but I don't see how to do it. They are run via a command.

I know there is Kubernetes that can do it, but it looks a bit overkill and complex to set up. Devops looks pretty intimidating to be honest so much stuff and I don't know what I am doing. Would there be an easier alternative? I am also running into the issue that I will have to maintain 2 docker-compose set up for production and dev? Is that something normal?

https://redd.it/g2fcg5
@r_devops

When should I use Configuration Management Tools (e.g. Chef and Ansible)

Based on my research, I wonder where these tools fit today and how they are used. Since a lot of applications are moving to containers and Kubernetes I'm struggling to understand what's the current state these tools.

https://redd.it/g2f7ad
@r_devops

What is CORS for?

CORS (Cross-Origin Resource Sharing) can be complex to understand and to set up. [Here is an explanation of what is for](https://www.padok.fr/en/blog/cors), and how to configure it on Symfony and Django applications. If you have a CORS tutorial on other framework technologies it would be great for me, please share it with me.

https://redd.it/g2e691
@r_devops

Monitoring advice

Hello, I'm looking for advice for a monitoring solution.

My experience as a devops is reasonable as I'm a backend engineer that has take on devops tasks by necessity.

Our current stack is deployed using AWS ECS and we have mostly node.js app and some python.

We also would like to monitor tasks that are done asynchronously, currently using celery.

We also have a media server where connectivity is very important.

We currently have no alerts, and we have to go to cloudwatch and scroll forever when something goes wrong.. in prod it's almost useless as it's lost amongst http logs.

Basically we would like a unique endpoint to monitor, logs, apps, networking.

I'm a big fan of open source soultion but since I'm alone to do devops tasks, I don't want to spend too much time keeping it operational. So I'm ok with commercial solutions.

I've been looking at datadog and promotheus but I'm not sure which way to go. Also I've read reviews about datadog apm integration that can make your apps crash...

Thank you in advance for any help you would give me !

https://redd.it/g2aduq
@r_devops

BUILD Virtual machine image for multiple cloud providers

Basically, I'm looking for a way to build images or ideally just a single one of Linux and maybe Windows which will work for most cloud providers.

The image should work the same on AWS, GCE, Azure but also on some smaller providers like DO, Scaleway, OVH ...
I guess I could tinker such a thing together but maybe some kind of recipe already exists.
I'm also not entirely sure how feasible this would be or if the underlying architectures etc. would be too different

https://redd.it/g2c115
@r_devops

Code that shares docker images

Title maybe wrong as I’m not sure if this approach has a name.

Just wondering if I can get some opinions on a comparison between the two setups.

For context I started a new job and we are using a micro services architecture with containers with kubernetes for orchestration.

In my past work places we’re we used micro services we decoupled all the services and they had their own repositories and container images.

At my new job they use the same images for many services and adjust the entry point according to which service they would like to run.

These are the advantages/disadvantages I could see with this shared images approach.

Benefits:
- less repeat shared resources for code
- reduced cost, less repos

Disadvantages:
- harder to manage (all code in repo may not be related to particular service your troubleshooting)
- bigger docker images
- increases redeploying of services that haven’t changed (ci/cd will redeploy a service if image has changed but code may not have for that particular service inside the image)

I was just wondering if people had come across this approach before and what people’s opinions are on it?

I personally am not a fan of it but maybe someone could enlighten me.

https://redd.it/g1pfb3
@r_devops

Infrastructure as code vs. platform as code

With infrastructure as code (IaC), you write declarative instructions about compute, storage and network requirements for the infra and execute it. How does this compare to platform as code (PaC) and what did these two concepts develop in response to?

[https://jaxenter.com/infrastructure-as-code-platform-170465.html](https://jaxenter.com/infrastructure-as-code-platform-170465.html)

https://redd.it/g1owaw
@r_devops

What method/tool do you guys use to store/use your secure access credentials to a Cloud provider.

How do you guys are managing to get access to your Cloud provider in a secure way?. Let's say, to configure EC2's with Ansible or provision VPC's with Terraform or any other tool .

For personal stuff I have been using environment variables which can be tedious and sometimes I store the aws credentials in .aws/ which obviously is not secure in any way so when not lazy I add to it the password manager tool "Pass" .

I've been thinking to step into Hashicorp Vault + Raft + [Keybase.io](https://Keybase.io) but I am curious of what are you guys using for a simpler or better approach.

I also considered the own vendor Cloud provider Key management service but not sure if the price and lock-down to an specific provider will work long term for me.

Anyway, what about you? .

https://redd.it/g1o2xj
@r_devops

PagerDuty vs. VictorOps vs. OpsGenie

We are looking at implementing a tool to help manage alerting and incident response. How do these compare? Are there other competitors which I should look into?

https://redd.it/g34drp
@r_devops

Did an In-Place Upgrade on a VM running Windows Server 2016 to Server 2019... user is getting access denied errors starting up Docker?

Hi everyone,

As the title says, I recently did an in-place upgrade on a VM for a client from 2016 > 2019 on their request. Now they are getting this when logging into the VM with their domain account and trying to launch Docker:

Docker.Core.DockerException:
Could not save daemon configuration to C:\ProgramData\Docker\config\daemon.json: Access to the path 'C:\ProgramData\Docker\config' is denied.
   at Docker.ApiServices.DaemonOpts.DaemonOptionsLoader.Get(ContainerEngineMode mode)
   at Docker.WPF.BackendClient.Start(ContainerEngineMode mode)
   at Docker.Actions.DoStart(SynchronizationContext syncCtx, Boolean showWelcomeWindow, Boolean withNotifications)
   at Docker.Actions.<>c__DisplayClass26_0.<StartAsync>b__0()
   at Docker.ApiServices.TaskQueuing.TaskQueue.<>c__DisplayClass18_0.<.ctor>b__1()  System.UnauthorizedAccessException:
  Access to the path 'C:\ProgramData\Docker\config' is denied.
     at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
     at System.IO.Directory.InternalCreateDirectory(String fullPath, String path, Object dirSecurityObj, Boolean checkHost)
     at Docker.Core.Paths.EnsureParentDirectoryIsCreated(String path)
     at Docker.ApiServices.DaemonOpts.DaemonOptionsLoader.Save(ContainerEngineMode mode, DaemonOptions options)
     at Docker.ApiServices.DaemonOpts.DaemonOptionsLoader.Get(ContainerEngineMode mode)

He also said the folder/file path doesnt exist. So, he asked that we revert the VM to a Snap of when it was on Server 2016. I did that and he said he is still having the same issue?

https://redd.it/g34zfp
@r_devops

Need of multiple docker instances while on cloud

While we use cloud for deployment , it is autoscalable , which means if there are more incoming requests/load on the server then instances will be dynamically added and while the load reduces automatically . Docker does help us to do efficient use of the hardware resources. but what is the need to spin up more than one container for an API as docker over an compute instance (may be an ec2) will do the job as its auto scalable. Kindly share your thoughts.

https://redd.it/g33eci
@r_devops

Do you know/use any tool for defining and vizualizing dependencies beetween services?

I lack such a tool a lot. I have a system of about 150 microservices and it would be very helpful if I could define dependencies (service A talks with service B, uses mongo, kafka) and visualize it. Now if I want to see service dependencies I have to look into service code (which is not a big pain in the ass using monorepo). Do you know such a tool?

https://redd.it/g2z7zx
@r_devops

Distributing Secrets for PowerShell Scripts with Ansible

Hi guys,

I did a little project over the weekend:

I wanted to distribute secrets for Windows users to use in scripts, using the Windows builtin local vault.

link: [https://dev.to/mieel/how-to-distribute-secrets-for-powershell-scripts-using-ansible-3mne](https://dev.to/mieel/how-to-distribute-secrets-for-powershell-scripts-using-ansible-3mne)

**tl'dr**: I'm using the [Microsoft SecrectsManagement](https://devblogs.microsoft.com/powershell/secrets-management-development-release/) Module to populate the users local vault, and using Ansible to automate this for multiple machines/users.

I declare the user/secrets mappings in an ansible-encrypted file, and run a playbook to deploy them in my environment. When the play has finished, users can use `Get-Secret` to retrieve the secret.

Any feedback would be much appreciated! Thanks.

https://redd.it/g2wmuc
@r_devops

Jenkins-X vs Drone.io for a small team of developers

I'm a developer. My team is looking for a new CI/CD tool (we were using a commercial tool but we got the go from our IT department to try an open source alternative).

We are not very mature in DevOps and GitOps practices (except basic knowledge), so we are afraid to be overwhelmed with an opinionated solution like Jenkins-X. On the other hand, its capabilities are very appealing (like automatic previews on pull-requests).

Or we can choice [Drone.io](https://Drone.io) which looks similar (cloud native) but offers less features (but I might be wrong).

I don't want to launch a solution war in this post. I just want to know both pro and cons solely based on your experiences. We know nothing can be better than trying and finding our match by ourselves, but having real world feedbacks will surely help getting a mature decision and adjusts our expectations.

Thanks.

https://redd.it/g3bdc5
@r_devops

Pain point in implementing git flow

I'm trying to implement a git flow-like process for our company. Currently our process is a simple two branch model with a develop branch and a master branch. Weekly, we merge develop into master and stage master, test it and patch/re-stage as needed, then merge master back into develop with the patches and deploy master to prod.

This winds up polluting the master branch with all of the unnecessary commits and patches in develop. Furthermore, it makes rollbacks hard because a "stable release" is only defined as whatever commit the HEAD of the master branch is at when it's deemed ready. We tag it, but that's all.

I'd prefer to stage and test a release/qa branch, which is then tested and only merged into master for releases. I also like the git flow concept of every commit on master being a release or hotfix and guaranteed clean working code, but I'm having trouble implementing this in practice.

In my test framework I have three branches: `develop`, `qa`, and `master`.

Each week we cutoff by making the develop and qa branch the same:

git checkout qa
git reset --hard develop

The qa branch is tested and patched, as needed. When it's ready, I want to then merge qa into master a a *single commit* representing that stable release, then merge master back into develop. However I haven't figured out how to do this elegantly.

git-flow recommends using a --no-ff merge to force creation of a merge commit, and then tagging HEAD:

git checkout master
git merge --no-ff qa
git tag v1.2

However, this has the undesirable effect of migrating all dirty commits from develop into master. I have experimented with a way to get a single commit on master representing the whole release:

git checkout master
git merge develop --no-ff -m "Release 1.2"
git reset --hard HEAD~1
git cherry-pick -m 1 HEAD@{1}
git tag v1.2

This gets me pretty close to what I want, but it makes it so that when I then merge down master back into develop, I get both my release commit AND a merge commit. I can't merge `--ff-only` because I have a different set of commits on master and develop. And it makes diffs in github hard to read because they use three dot diffs instead of two.

It's possible I'm thinking about this all wrong, and would love some feedback. My ideal solution would let me have the master branch composed entirely of clean composite commits, each representing a release or hotfix. In addition, I'd also like to avoid adding unnecessary commits to develop, and wind up having a `Release 1.2` and `Merge branch 'master' into development` commits side by side for each release, summarized in the title: I want to merge dev -> master as a single commit, then merge --ff-only when I merge qa back into dev.

https://redd.it/g3ayi1
@r_devops

Run Command doesn't recognize AWS EC2 instance - help

*Crossposted in* r/aws *but autoremoved...*

Hey guys, got a little problem I can't figure out. Following a course yesterday on AWS, I managed to create a VPC, subnets, NAT gateways and a load balancer. I also created 2 EC2 instances in two availability zones and pointed them as a target for the Load Balancer. Then I did "Run Command" from Systems Manager to install a basic Apache webserver and it all worked well, I was able to hit both instances through the load balancer. Problem is:

Today I deleted everything and recreated it to just learn it really well. Everything seems fine but the instance(s) don't show up in the Run Command. I created the proper SSM role (with AmazonEC2RoleforSSM & AmazonSSMFullAccess) and attached it to the instance. But when I go into Run Command, the instance just doesn't show up! Why? I've been trying to figure this out for an hour...

I am in the same region (N. Virginia) on both the EC2 page and the Systems Manager page. I tried deattaching security groups or just having ingress and egress wide open for the instances and load balancer + NAT gateways but still nothing. What am I missing? Why is it not showing up?

Thank you for the help

p.s. I'm thinking it could be my subnets and my routing tables are not set up correctly, but I actually didn't delete them from yesterday so it should be the same. maybe I didn't configure the new NAT gateways properly?

edit: the instance image is the default AWS Linux one that has the AWS tools already installed by default. t2.micro, same as yesterday. I click on "Choose instances manually" in the Run Command console, and I get the "You do not have any registered managed instances in this region" error message. Tried both root account and IAM admin user.

https://redd.it/g39neh
@r_devops

Seeking Recommendations for a Knowledge/Training/Documentation Platform

Hey guys, I'm looking for recommendations for a solid platform that can deliver the following capabilities. I'm offering to paypal beer money and buy a 6-pack for whoever suggests the platform that seems like the best lead. Hit me with what you got!

**Capabilities Needed:**

* Store user-facing documentation for each department in the org as a sort of high-level organization directory for each team, tool, IT process. The goal is to build a an org directory and landing page with introductory materials for each team's tools, processes, interfaces, and common customer-facing informtion and links.

* Provide some light e-learning style training materials for each IT platform and tool in the org - stuff like videos, KB's, and tutorials. I would ideally like to organize these materials into courses/categories.

* It must be easy for each team to administer their own content

* Bonus Points if the platform can support doing qual-cards/skill matrixes/track internal certification.

Explanation: Our IT organization is huge and there is virtually no onboading orientation for new IT employees/contractors. It takes about 2 years for a new hire to get their bearings enough to navigate the system effectively, and each person must individually build their own list of processes, documentation, links, and resources.

Our goal is to centralize all of this information in a way that is easy to consume. The platform would support a new "new hire orientation" process designed to expose them to the key teams, systems, and processes they will need as IT staff. This system will also need to support light training/qualification courses where teams can build out week-long training sessions on specific tooling, such that people who need access can get their access provisioned, understand the major components and processes, gain exposure to best practices and common tasks, and generally gain a high-level understanding of the sprawling IT organizations and teams at play.

Willing to look at SaaS, open source, or commercial products (with a preference for Open Source) - from SharePoint to Confluence. What can you recommend for this purpose?

https://redd.it/g3duqa
@r_devops

Post deployment ECS API validation testing

Been working on expanding our post deployment tests for an ECS hosted REST API. Currently the suite has some basic smoke test calls the all the endpoints. I wanted to expand into more validating the, I guess you could say "ops" perspective, as the endpoints already have an extensive functional test suite which is ran prior to deploying. I'm thinking about doing some boto3 calls to check that the task/service is running etc. What type of stuff can/should I test for here to validate it's good?

https://redd.it/g3dnzr
@r_devops

Does Solr or ElasticSearch work better with search terms from a HSQLDB or MySQL database?

I am new to Solr. I just know it takes json queries. I wish to know as much as possible.

&#x200B;

Also if my hypothetical task is to search multiple queries into one unstructured lengthy text to see which queries get a hit, which search would be best?

https://redd.it/g3hd5j
@r_devops