Drone Deployment Event

I am trying to trigger an event after deployment to Kubernetes by sending a GitHub webhook to Drone. Has anyone done this?

https://redd.it/emfw67
@r_devops

Show devops: My fully open source SAST scanner

After years of fighting with slow, expensive and cumbersome commercial SAST tools, (and after being let go by my ex-client!), I decided to build my own based on available oss tools. It's time to present sast-scan

https://github.com/AppThreat/sast-scan

The scanner uses sane defaults for a number of languages and frameworks. I have started adding SARIF conversion feature and this helped integrate the analysis with VS code, Azure DevOps and so on. There is also GitHub action integration.

I have few ideas in mind to make it useful for modern DevOps teams. Would appreciate your comments and support to make this project useful for everyone.

https://redd.it/emctw4
@r_devops

Different Docker images with the same image ID but from different repos

I ran into a bit of an issue today again with docker image ID's being the same for different docker images. This caused the auth service to not start right since it was thinking it was pulling the auth image but actually was starting the user image up.

​

Now both of these are built using Jenkins and I've checked and checked, the repo names are correct in the pipelines as well as that there is different tags which come from the commit hash.

​

What might have happened here and what can I do to prevent it happening again? This is a bit scary since this could take down production, lucky this time it was in staging.

​

<ID>.dkr.ecr.us-east-1.amazonaws.com/auth 24e38c0efca9 ec57f009d2a6 2 days ago 114MB
<ID>.dkr.ecr.us-east-1.amazonaws.com/user 94a52a1e93fd ec57f009d2a6 2 days ago 114MB

https://redd.it/emfkhb
@r_devops

Aspiring DevOps Engineer and IT Professional's Personal Domain Name Ideas/Examples?

Hey Guys,

I'm currently studying and practicing to make a transition into a career as a Site Reliability Engineer (DevOps) and I'm currently an intermediate Systems & Network Administrator. One thing I'm studying is middleware such as NGINX and I'm learning how to deploy/configure such and also HTTPS certificates. I'm going to buy a domain so that I can host a personal wordpress site (for testing/practice but eventually to advertise my skills, bio, and work portfolio). What are some good ideas or examples of an IT Professional's personal domain name? I was thinking *MyInitials*-IT.net? Is that dumb? Should I just make it my *full name*.TLD? Curious what IT professionals use in the real world.

https://redd.it/emezdb
@r_devops

ansible line in file

I have about 10 debug in my task. i am using line in file. I want to know is there a way to have all debug msg have line each. or any other way. I have 10 variables in line parameter and i have to keep going right to add more variable.

Ex. - name: Tcp Ip

debug:

msg: "{{ ansible \_inventory }}"

line: {{ ansible\_inventory}}

https://redd.it/eme5ks
@r_devops

How do you get notified of important CVE?

I'd like to know ASAP whenever a critical CVE is found, but I'm not willing to poll actively the CVE website. However, they don't seems to have a notification mechanism. How do you do so?

https://redd.it/em8ns2
@r_devops

Automating SQL queries

I just started my first role as a devops engineer and one of my responsibilities is running approved SQL queries against our production databases and returning the results (if any) to the team that requested it. I'm trying to find a way to automate this but don't know where to start. I'm used to configuration management via Ansible and don't wan't to put one-off SQL queries into a playbook in Github. There has to be a better way, right?

https://redd.it/emi6o8
@r_devops

Running containers on Google compute engine

Does Google compute engine auto pulls latest image from registry just like Google kubernetes engine ?

https://redd.it/ema5km
@r_devops

Packaging Woes

Hey Guys,

&#x200B;

Just posting to get a little more insight on how other companies operate with their continuous integration.
At present I am trying to integrate chocolatey packaging for Windows & IIS services, then use puppet to update software on a few of the QA servers before running tests (not unit tests). Unfortunately chocolatey is limited to semver v1 so I would be limited to using BUILD ID as some sort of tracking upgrade mechanism.

I would look to mount the chocolatey packaging inside of the build process and push to a repository where testers can tell puppet what to install, to build new tests.

I can't help but think there's simply a better way to do this though.

https://redd.it/em9358
@r_devops

How to standup/provision/maintain Kubernetes infrastructure and processes in 2020?

Hello r/DevOps,

I will be joining a company whose goal is to re-architecture how we provision and maintain our kubernetes infrastructure. They're open to embracing new tools (Datadog, Gitlab, Jenkins) in order to modernize their architecture in a way that other companies have found success in.

Having said that, I would like to know what tools and technologies you guys would use to standup/provision/maintain a Kubernetes environments in 2020.

The only requirements is that the cluster itself needs to be managed by GKE on Google Cloud. Aside from that, the companies is placing no requirements as to what technologies and tools to use to modernize our Kubernetes cluster.

&#x200B;

Here's what I'm planning:

&#x200B;

**CODE HOSTING**

I'll be looking at Gitlab. It has a CI/CD baked in as well as a container registry and some basic Kubernetes management. I wonder whether we should use [GitLab.com](https://GitLab.com) or host our own inside our kubernetes cluster via their helm chart.

&#x200B;

**CLOUD RESOURCE PROVISIONING**

Everywhere I look I see Terraform as the king of IaC. I have experience with Terraform and I just love it. I wonder whether you utilize Terraform Cloud to manage infrastructure changes in PRs or just use the GCP storage backend like I have done in the past.

&#x200B;

**PIPELINES**

If we embrace Gitlab, I plan on using Gitlab CI/CD and configure their runners to run our build, test and deploy jobs on Kubernetes.

&#x200B;

**DEPLOYMENTS**

Looking to automate deployment via Gitlab CI/CD and Helm 3.

&#x200B;

**SECRET MANAGEMENT**

I have experience with checking in our secrets via SOPS in our repo but I've been looking at Vault recently and wonder if that's a even better approach to managing our secrets.

&#x200B;

**CERTIFICATES**

cert-manager to manage our cluster certificates.

&#x200B;

**INGRESS**

Looking to stick with nginx-controller but have been looking at traefik. Does anyone have any input here?

&#x200B;

**MONITORING**

Datadog seems to be king here. Planning to deploy Datadog as a daemonset in my nodes for monitoring and observability across all application layers.

&#x200B;

**ALERTING**

Here, I have no idea. I have used VictorOps but I have no clue as to whether other services offer better packages.

https://redd.it/emmk3a
@r_devops

What’s new in Helm 3.0?

Helm is a package manager that simplifies installing and managing Kubernetes applications.

Helm 3.0 was released on Nov 13, 2019. The main attraction of this release was the removal of the Tiller server, which deployed inside the Kubernetes cluster and interacts directly with the API server to manage Kubernetes resources. With Tiller gone, the security model for Helm is entirely simplified. Helm 3 supports the latest security, identity, and authorization features of Kubernetes.

The following are the capabilities of the new version;

**Elimination of Tiller**

Helm 3.0 has a client-only architecture with the client still called Helm. It operates similar to the Helm 2 client, but the client interacts directly with the Kubernetes API server. No more middle man. Removing tiller helps to increase cluster security.

**Automated Helm state creation**

In Helm 3.0, we don’t need to install Tiller so initializing Helm is obsolete. In other words, removal of the helm inits, and automated creation of Helm environment.

**Release storage changed**

In Helm 2, releases are stored as ConfigMaps in the cluster under the Tiller namespace.

But in Helm 3.0, releases are stored as Secrets by default and storage is in the namespace of the release.

**Changes in Chart architecture**

In Helm 2, chart dependencies are declared in requirements.yaml.

Chart dependencies are consolidated in Helm 3, moving the dependency definitions to Chart.yaml. The lock file has also changed to Chart.lock.

**Library charts**

The common or helper chart provides a reuse pattern for chart definitions in Helm.

A new ‘type’ of the chart was introduced to the chart.yaml and it can be either application or library.

* Application is the standard chart, which creates release artifacts during rendering.
* Library charts are declared as a dependency directive in Chart.yaml and are managed like any other chart, except that they are not installed.

**Chart value validation**

In Helm 3, values passed to a chart can be validated against a JSON schema. This validation is beneficial to avoid setting incorrect values and help improve chart usability.

**Changes in Chart repository**

The Helm Hub is introduced as a central catalog for Helm chart repositories. Local and stable repositories are no longer added by default.

**Improved release upgrade strategy**

The Helm 2 release upgrade used a two-way strategic merge patch.

In the case of Helm 3, the release upgrade uses a three-way strategic merge patch. Helm considers the old manifest, cluster live state, and the new manifest when generating a patch.

**Over-hauled CRD (Custom Resource Definition) support**

Helm 3 simplified Helm CRD support. The crd-install hook is removed and replaced with a crds sub-folder in the chart folder. Helm 3 does not try to provide full CRD support. It just provides support to install a CRD in a simple fashion.

Helm 3 comes along with a rich set of competence which changed it to a new set of capabilities, a one-off.

https://redd.it/emlt0p
@r_devops

Ansible facts/gather facts

i cant find these facts: base clock, turbo clock, tomcat and weblogic app, EOSL, storage type, server type, list of app on server, number of instances of each app, last audit, backup and brought date, and planned decommission.

https://redd.it/emavd6
@r_devops

Debugging with scratch containers

Hey guys,

I am trying to move towards scratch containers in the company I work for. The Dev team I work with have a requirement to occasionally debug e.g. Redis from the POV of the running container, what options are out there to enable this shell access whilst still ultimately running the application in a scratch container

https://redd.it/em7xrz
@r_devops

Devops vs Sysadmin Jobs? What are all the differences?

Hi folks,

I've seen few open positions for Devops and Sysadmin remote jobs. What do you think is the main difference between them in terms of responsibilities?

https://redd.it/em7bdn
@r_devops

CD Pipeline for Flask + DB

Hey everyone,

I'm working on how exactly to architect a continuous delivery pipeline for a small project that will probably grow in scope over the next year. At the end of the day, it's a basic web server (Flask) with a relatively independent database.

I've got both components set up in their own docker containers and configured with a docker-compose script for local development. While I have experience with docker and deploying a single application on a server manually, I would like to set up a pipeline through GitLab to keep the server updated. I'm planning to deploy on Linode but I'd like to figure out a pipeline for the continuous delivery piece before I get too far into manually configuring a server.

From my experience, deploying a new set of code should just be a matter of logging in, pulling the latest code from git, re-building/updating containers, then restarting the services. I would prefer zero downtime but it's not a necessity at this point. Is there any way to automate this within a GitLab runner? I've heard of them but have never used them, so this may be an uninformed question.

I've also heard of Ansible, Chef, Jenkins, and a bunch of other CI/CD services, but I have 0 experience with any of them. Are there any that work particularly better with docker? Do any of these solve this problem easier than others? In terms of scalability, are any better if I needed to scale?

If you've got any advice on the docker configuration, VPS configuration, or general DevOps, let me hear it! Thanks in advance for the help!

https://redd.it/em4fdk
@r_devops

I feel so stupid

I work as a DevOps engineer. I’ve been at a startup for 10 months. Before this I was a QA Engineer for 2 years. I face so many problems that literally short circuit my brain and I just can’t figure it out in less than a day. Easy stuff like setting up Gitlab Ci pipelines

I know it shouldn’t take that long but it does. Honestly I just feel so inadequate and stupid and like I maybe moved too fast.

Has anyone else had these feelings and how did you fix them? I want to be smarter, sharper, but I just can’t.

https://redd.it/em1m9r
@r_devops

Developer - DevOps all in one server.

I know this is overkill, I work with a giant team very much Tribe, mentality scrum + XP development cycle.

I have a personal project I am working and trying to put some of those procedures into my process (even though its just me) to manage my project. I am building a widget. I have all the scrum stuff in place but want to pretend this is a real product and set up a DevOps server. I have a relatively new machine and was thinking I would use HyperVisor to create the following VMs,

1. Team City
2. Database server with all the databases are.
3. Development web server - [dev.foo.com](https://dev.foo.com) accessibly only locally.
4. Stage web server - [stg.foo.com](https://stg.foo.com) accessibly only locally.
5. Test web server - tst.foo.com accessibly only locally, potentially going live if I get there.
6. Production web server - [www.foo.com](https://www.foo.com) accessible locally or on the web.

Is this crazy or stupid? TC would be configured to draw from different branches in some automated some requiring user intervention.

&#x200B;

Should I do this on Windows 10, or use my MSDN license to get some server brand of Windows?

&#x200B;

Any help would be appreciated.

https://redd.it/em3ibt
@r_devops

Devops engineer vs an organization that truly practices DevOps?

Mini rant I suppose.

There are books like The DevOps Handbook and Accelerate which teach us the art and practice of what DevOps is and what it’s supposed to be. However most organizations don’t do this. They typically just hire DevOps engineers and think “now we’re doing DevOps m”. It doesn’t seem like management or leadership are interested in what it means to truly practice DevOps at an organizational level.

Is anyone else experiencing this? What do you do to cope with it and “accept” it?

https://redd.it/em3cp3
@r_devops

noob that would like to put some order to my workplace

I'm extraordinarily new to devops, I barely know some stuff and I would like to help my workplace have a better order on implementing software/updates.

in here we don't use docker or git, so I was thinking of how to learn git so I can have branches and deploy those on the server, I mean, there are lot's of tutorials on GIT and how to commit stuff and all that, but how do you use git to push things to a server, how to automate test, the best practices for branches and all that.

also, if I would like to do continuos deployment (which I assume is updating the server), is it possible to do it in one of those servers that use cpanel and all that?

is there like a roadmap to do all this?

https://redd.it/emw0on
@r_devops

One or many repos for a single project with several lambdas?

Currently working on a serverless project including two lambdas, going to be adding a third here real quick. I thought initially it would make since to have a repo per lambda but only one cloudformation template so it's not really working out to be as separate as I had thought it would be. I see a couple of options:

1. All-in-one - all code in a single repo with single build process and cloudformation template for depolyment/updates
2. Semi-split - current state of things, one repo for "main" part of project with cloudformation template and second repo for second lambda. Currently the single template references a build artifact from the second lambda build process.
3. Full-split - One repo per lambda with code and smaller "atomic" template, and another for the project template with the "atomic" templates included as well as the documentation and misc.

&#x200B;

I would love to hear about your thought, other options, best practice documentation?

Thanks

P.S. If this would be better over on r/AWS just let me know.

https://redd.it/emvumo
@r_devops