ServiceNow DevOps, CMDB and Self Service Portals

1. Has anyone seen ServiceNow’s DevOps product? Looking for some reviews. Change Automation sounds like a compelling story. Not sure if it can also help with dependency identification.

2. Are CMDBs still a thing? How do you do dependency identification in change management without a CMDB?

3. How do you all utilize self service portals? Check-ins kick off builds is something I understand, but for a new app, where do you specify which build to kick-off (Gold/Silver/Bronze servers) etc.?

4. How do you like ServiceNow’s story around the need for a single dashboard showing the flow of a ticket from conception (idea) to release?

https://redd.it/g1pyxv
@r_devops

Ask r/Devops - Basic help with Jenkins&Kubectl

I am trying to get a very basic Jenkins-GKE CI/CD pipeline going with GitHub integration. Push change to GitHub, webhook to Jenkins (ubuntu VM) server running in GCloud, build Docker image and push to docker hub, update deployment in GKE with kubectl with "set image".

My problem is that Jenkins is running with the user "jenkins" and it always fails at the kubectl part. It gets the notification from GitHub, builds the new docker image, pushes it to Docker Hub but can't access my kubernetes cluster with kubectl.

The error:

`+ kubectl set image deployment/hellowhale hellowhale=timorwell/hellowhale:11 error: Missing or incomplete configuration info. Please point to an existing, complete config file:`
`1. Via the command-line flag --kubeconfig`
`2. Via the KUBECONFIG environment variable`
`3. In your home directory as ~/.kube/config`

I spun up the cluster on GKE with the gcloud-cli with my regular user on the VM, so that user can access the cluster no problem. I copied the .kube/config gile to /home/jenkins/.kube/config but it still doesn't see it. I chown-ed it to the jenkins user too.

Why can't it use my config file? Am I approaching this problem from the wrong place? Is there a better way to integrate kubectl into my Jenkins pipeline?

I just tried adding the `--kubeconfig` flag pointing to my config file but that results in "must be logged in to the server" error...

I'm trying to learn all this rn, thanks for the help. And yeah I know this isn't stackoverflow but I just like reddit more due to it being more "human" lol.

https://redd.it/g1vioz
@r_devops

DevOps test



Programming Pre-Screen

Conference room scheduling. Find the nearest open conference room for a team in which a team can hold its meeting. Given n team members with the floor on which they work and the time they want to meet, and a list of conference rooms identified by their floor and room number as a decimal number, maximum number of people it fits and pairs of times they are open - find the best place for the team to have their meeting. If there is more than one room available that fits the team at the chosen time then the best place is on the floor the closest to where the team works.

E.g.

rooms.txt

7.11,8,9:00,9:15,14:30,15:00

8.23,6,10:00,11:00,14:00,15:00

8.43,7,11:30,12:30,17:00,17:30

9.511,9,9:30,10:30,12:00,12:15,15:15,16:15

9.527,4,9:00,11:00,14:00,16:00

9.547,8,10;30,11:30,13:30,15:30,16:30,17:30

Input:

5,8,10:30,11:30 # 5 team members, located on the 8th floor, meeting time 10:30 - 11:30

Output:

9.547

Please explain: how you solved the problem and how it would behave based on the different parameters (number of team members, longer meeting times, many rooms with random booking times). How would you test the program to ensure it always produced the correct results?

For extra credit, can you improve the solution to split the meeting across more than one room if say only one room is available for a fraction of the meeting and another room is free later to hold the remainder of the meeting during the set time. If you want to make this more powerful - assume that the number of room splits can happen in proportion to the length of the meeting so that say if a meeting is 8 hrs long then the algorithm could schedule it across say up to 4 rooms if a single room was not available for the whole time



You may code the response in any DevOps programming language you like

• Bash

• Perl

• Python

https://redd.it/g24epo
@r_devops

Jenkins freestyle job ansible pluging sudo issues

So I'm running a playbook in linux based jenkins using the ansible plugin. For linux to linux, it works just fine. However, with windows I'm running into an issue where it keeps trying to do ssh instead of winrm I've narrowed it down to the fact that when jenkins runs "ansible-playbook" it's not issuing a "sudo". I tested it in the cli directly. If I run ansible-playbook without sudo, it attempts to connect as ssh. When I run it with sudo, it will attempt as winrm. Same behavior as jenkins.

​

So my question is, how do I get the ansible plugin to issue a sudo to ansible-playbook? There is the sudo check box in the plugin, but that is deprecated for version 2.6 and up. There is become but based on the documentation, that's switching user context on the target, not elevating ansible-playbook to sudo status.

I can always make a shell command but doing it via plugin would be nice.

https://redd.it/g1sc0l
@r_devops

How to analyse logs from docker in my dev environment?

Hey guys. Been feeling quite a bit of pain with log analysis from distributed web applications.


I'm trying to find a way to pipe my docker logs to a tool that would allow me to at least highlight words, maybe filter lines based on regex. Ideally would aggregate multiple log streams based on session ids and stuff to reconstruct a timeline.

Any ideas?

Thank you!

https://redd.it/g1s85p
@r_devops

How much are you paying for datadog compared to your total AWS/cloud costs?



https://redd.it/g1n9a3
@r_devops

What hasn't been Dockerized?

During the quarantine I have been using my time to learn Docker, working towards my certification. But I also want to show some examples on my GitHub of me taking existing applications and building images. I started searching through various projects but almost every repo I looked at already supported Docker deployment. Anyone have tips of finding apps that haven't been Dockerized yet?

​

Cheers

https://redd.it/g2fnru
@r_devops

Using github actions - should I have separate workflows for CI and CD?

I'm a bit confused as I've seen so many conflicting examples of github actions workflows where in one workflow the example shows **testing, build & deploying** all in one and others show **testing & build** as one and **deploy** as separate one.

 

My understanding is that:

* CI: You continuously merge your code into master, running a build and test job on it to make sure everything is ok before merging.

* CD: You continuously deploy your app.

 

Therefore, should I have one workflow called `continuous-integration-workflow.yml` and another called `continuous-deployment-workflow.yml` or have it all in one single file?

https://redd.it/g1pkch
@r_devops

Getting bored of DevOps consulting, what do?

My career started with a company brand new to cloud and I was apart of the team that drove some major transformations. Mostly Ops, a little coding, and a lot of communication between teams to help everyone align on the journey. Since then I've specifically worked at similar orgs and helped them get to (or at least take a step in the direction toward) where they want to be which has lead to becoming a DevOps Consultant for AWS.

So far things have been going well but clients usually aren't bringing in consultants unless somethings wrong so I mostly deal with dumpster fires. Looking forward I'd like to work somewhere that's _already_ practicing DevOps at a high level and simply be an engineer or help in a different way than reducing internal silos (which is 90% of DevOps). I'm curious where other consultants here have ended up.

Amazon's the first FAANG company I've worked at and I may just end up transferring to an internal SWE role which doesn't sound terrible and should check all the boxes but I'd like to hear other's experience on where to go after consulting gets old. I think dream job would be writing golang on a "tools" team for pipelines or prod ops but I don't have any perspective on what it's like.

https://redd.it/g2fm28
@r_devops

An Experiment in Streaming: Bytecode Continuous Deployment

Once one starts their journey in data streaming, one starts to discover a lot of applications beyond just the standard Extract-Transform-Load pattern.

The traditional model to deliver a new version of a Java application is to stop the process, deploy the new JAR/WAR, and start the process again. This directly results in downtime: in this day and age, most companies frown upon such downtime as it directly translates into a loss of revenue.

This link is about how to use streaming to continuously deploy bytecode to a running JVM without needing to stop it.

https://hazelcast.com/blog/an-experiment-in-streaming-bytecode-continuous-deployment/

https://redd.it/g2hefq
@r_devops

API gateway/Ingress controller with Keycloak SSO support

Hi All.

We are deploying our microservices on AWS EKS cluster. Till now, we have deployed couple of containers including Keycloak (for auth) successfully.

Now, we want to deploy a ingress controller which needs to be:

* Open source
* Works well with Keycloak with SSO support

Initially, we chose Trafeik, it works well normally but has very limited support for keycloak SSO. NGINX keycloak SSO seems to be part of its enterprise version.

We also tried using keycloak-gatekeeper but it seems too much to have a separate container for each microservice. Also, it did not work with Traefik.

We want a way where SSO can be handled at Ingress controller level itself. Can anyone please help here to choose the correct Ingress controller or might be a way to do Keycloak SSO with Trafeik?

Any help will be greatly appreciated.

https://redd.it/g2gxdu
@r_devops

Git Hooks - Automating Secrets Detection

I find git hooks to be fundamental in automating security processes and finding potential vulnerabilities.

An interesting blog post not by me on git hooks. An interesting argument that post-receive hooks are the most important, I've always relied on pre-commit hooks.

[https://blog.gitguardian.com/git-hooks-automated-secrets-detection/](https://blog.gitguardian.com/git-hooks-automated-secrets-detection/)

https://redd.it/g2fi25
@r_devops

Service in AWS/Azure for getting randomized Domain Names (via terraform or other IAC Tools?)

Hi,
sorry for the headline, it seems i cannot wrote without upper case letters there. for my question:
for starting up tests or shortlived infrastructure it would be EXTREMELY helpful to have an component in azure/aws to request random domain names which can be mapped to certain public IPs which i want to set.. Do you know if there's such a service within Azure or AWS? i do not want to use another dns provider i would like to have it WITHIN azure/aws.

https://redd.it/g2gu5d
@r_devops

What to do after Docker

Hi, so I am currently making a website as a side project and I want to make a CI/CD pipeline to deploy the app. My current set up hosts my code on github. I have 3 branches :dev, staging, master. I run my tests on pull request and build the Docker containers on push request on staging and master branches. This is all done with github actions and my containers are uploaded to docker-hub. I then have my docker-compose file which uses the previously built containers and set environment variables and various things. I host my website on the smallest DigitalOcean VPS. The current set up is a nginx server reverse proxying to web server in a docker container. I figured I would do this so I could maybe do "load balancing" to update my site, so there is no downtime I can spin up the new container and then shutdown the old. I am using the Laravel framework which is in PHP, with a db, nginx as the web server and redis(not sure if really useful as everything is on the same machine?).

Now I'd like to achieve deploying new versions without having to log in my VPS hosting the containers and if possible do it without any disruption to the service. I also sometime have migrations to run on a database and I'd like that to be automated as well, but I don't see how to do it. They are run via a command.

I know there is Kubernetes that can do it, but it looks a bit overkill and complex to set up. Devops looks pretty intimidating to be honest so much stuff and I don't know what I am doing. Would there be an easier alternative? I am also running into the issue that I will have to maintain 2 docker-compose set up for production and dev? Is that something normal?

https://redd.it/g2fcg5
@r_devops

When should I use Configuration Management Tools (e.g. Chef and Ansible)

Based on my research, I wonder where these tools fit today and how they are used. Since a lot of applications are moving to containers and Kubernetes I'm struggling to understand what's the current state these tools.

https://redd.it/g2f7ad
@r_devops

What is CORS for?

CORS (Cross-Origin Resource Sharing) can be complex to understand and to set up. [Here is an explanation of what is for](https://www.padok.fr/en/blog/cors), and how to configure it on Symfony and Django applications. If you have a CORS tutorial on other framework technologies it would be great for me, please share it with me.

https://redd.it/g2e691
@r_devops

Monitoring advice

Hello, I'm looking for advice for a monitoring solution.

My experience as a devops is reasonable as I'm a backend engineer that has take on devops tasks by necessity.

Our current stack is deployed using AWS ECS and we have mostly node.js app and some python.

We also would like to monitor tasks that are done asynchronously, currently using celery.

We also have a media server where connectivity is very important.

We currently have no alerts, and we have to go to cloudwatch and scroll forever when something goes wrong.. in prod it's almost useless as it's lost amongst http logs.

Basically we would like a unique endpoint to monitor, logs, apps, networking.

I'm a big fan of open source soultion but since I'm alone to do devops tasks, I don't want to spend too much time keeping it operational. So I'm ok with commercial solutions.

I've been looking at datadog and promotheus but I'm not sure which way to go. Also I've read reviews about datadog apm integration that can make your apps crash...

Thank you in advance for any help you would give me !

https://redd.it/g2aduq
@r_devops

BUILD Virtual machine image for multiple cloud providers

Basically, I'm looking for a way to build images or ideally just a single one of Linux and maybe Windows which will work for most cloud providers.

The image should work the same on AWS, GCE, Azure but also on some smaller providers like DO, Scaleway, OVH ...
I guess I could tinker such a thing together but maybe some kind of recipe already exists.
I'm also not entirely sure how feasible this would be or if the underlying architectures etc. would be too different

https://redd.it/g2c115
@r_devops

Code that shares docker images

Title maybe wrong as I’m not sure if this approach has a name.

Just wondering if I can get some opinions on a comparison between the two setups.

For context I started a new job and we are using a micro services architecture with containers with kubernetes for orchestration.

In my past work places we’re we used micro services we decoupled all the services and they had their own repositories and container images.

At my new job they use the same images for many services and adjust the entry point according to which service they would like to run.

These are the advantages/disadvantages I could see with this shared images approach.

Benefits:
- less repeat shared resources for code
- reduced cost, less repos

Disadvantages:
- harder to manage (all code in repo may not be related to particular service your troubleshooting)
- bigger docker images
- increases redeploying of services that haven’t changed (ci/cd will redeploy a service if image has changed but code may not have for that particular service inside the image)

I was just wondering if people had come across this approach before and what people’s opinions are on it?

I personally am not a fan of it but maybe someone could enlighten me.

https://redd.it/g1pfb3
@r_devops

Infrastructure as code vs. platform as code

With infrastructure as code (IaC), you write declarative instructions about compute, storage and network requirements for the infra and execute it. How does this compare to platform as code (PaC) and what did these two concepts develop in response to?

[https://jaxenter.com/infrastructure-as-code-platform-170465.html](https://jaxenter.com/infrastructure-as-code-platform-170465.html)

https://redd.it/g1owaw
@r_devops

What method/tool do you guys use to store/use your secure access credentials to a Cloud provider.

How do you guys are managing to get access to your Cloud provider in a secure way?. Let's say, to configure EC2's with Ansible or provision VPC's with Terraform or any other tool .

For personal stuff I have been using environment variables which can be tedious and sometimes I store the aws credentials in .aws/ which obviously is not secure in any way so when not lazy I add to it the password manager tool "Pass" .

I've been thinking to step into Hashicorp Vault + Raft + [Keybase.io](https://Keybase.io) but I am curious of what are you guys using for a simpler or better approach.

I also considered the own vendor Cloud provider Key management service but not sure if the price and lock-down to an specific provider will work long term for me.

Anyway, what about you? .

https://redd.it/g1o2xj
@r_devops