Gitlab and Chef CICD Pipeline

I have to create a CICD pipeline within Gitlab that calls a recipe from an external Chef Node server. So basically the recipe on the Chef Node server is updated and once that takes place I will then use Gitlab to call on that recipe and run it within the pipeline. Anyone know if it is possible to do or a better way to go about it?

Thanks!

https://redd.it/g4yrk8
@r_devops

gitlab ci flow newbie question

I am newbie to DevOps..I have some queries in the gitlab ci flow.Please help me clarify the queries below.

Hw gitlab ci works?

1. Developer creates a new module in his branch.He tests and everything is fine. So he pushes the code to gitlab remote branch

2. He creates an MR so he can get review comments and then the code can merged to main code in master branch

3. The other team members review the code and finally the code is good to be merged to master. -->Will this be merged with master ?When do the gitlab ci pipeline will run?

4. The conflicts with the master branch are resolved and merged —>if its already merged to master, then other developers may pull this code even before it was run by pipeline.? And what is the role of release tags here?

5. This triggers ci [ ](https://pipeline.So)pipeline and the ci runner will do whatever instructions given in gitlabci file.It will build, test and deploy the code

6. Some of these steps could even be manual. Like deploy can be manual so it will wait for approval from someone before deploying.

https://redd.it/g4u5xt
@r_devops

Verifying plugin integrity in Jenkins

Hi all,

I noticed Jenkins fetches its plugins using `http` as defined in [update-center.json](https://updates.jenkins.io/current/update-center.json). Not an issue if the integrity is checked. However, I tried to find the checksum without success.According to [JENKINS-46428](https://issues.jenkins-ci.org/browse/JENKINS-46428) they verify the integrity using a sha1 checksum in base64 format, but I cannot find it. I checked several plugins without success.

If someone could point to documentation or something related which can be used to verify that statement would awesome.

https://redd.it/g4tog8
@r_devops

Newbie to Chef

Hello,

I currently work on an devops infrastructure team (primarily with Gitlab and Artifactory) and I would like to learn CHEF in order to help with my pipeline build. What would be the best way to learn chef "quickly" and can you please provide resource links or suggestions.

Thank you!

https://redd.it/g50z3u
@r_devops

Using Azure Function Apps to process Blob Storage event triggers (CLI + javascript)

I recently found myself needing to processes files uploaded to Azure Blob Storage. Initially I was a bit frustrated by the fact that all the blogs and tutorials out there seem to be written for Visual Studio users coding in C#. This tutorial provides a step by step explanation for hooking up a serverless Azure Function App written in Javascript to Azure Blob Storage using only CLI tools. The Function App framework provides a mechanism for reading and writing to Blob Storage storage without needing to make SDK API calls using the function input and output bindings. This is a very handy piece of innovation that I hadn't seen before.

[https://www.filemage.io/blog/azure-functions-blob-trigger-javascript/](https://www.filemage.io/blog/azure-functions-blob-trigger-javascript/)

https://redd.it/g4wg91
@r_devops

Staging bloated with broken microservices

Hi everyone !

In my company (400 devs), we have around 100 microservices that are communicating through gRPC and brokers like Kafka and RabbitMQ, on top of Kubernetes. We have 4 staging environments, but everything is always broken because everyone is testing on them.

​

* Data in the databases is always weird
* Microservices are sometimes broken and in crashloop mode and we can't test our services which have dependencies on them

​

As Head of Platform, I would like to implement 10 to 20 staging environments (one for every tribe or squad), but would like to know if you have any insights on this topic.

​

Thanks!

https://redd.it/g4owzc
@r_devops

What is default deploy placement in docker swarm?

I noted one thing about some of the compose files.

Some of the services constrainted the deploy placement to node manager, but other services has it unspecified.

Does it mean that these services can go anywhere?

https://redd.it/g4uiqr
@r_devops

kapacitor alerting

I am using influxdb, telegraf and grafana. I would like to use kapictor to generate an alert if no data is coming for a host. Is this a good use case for kapictor?

https://redd.it/g4soh9
@r_devops

comparing tsddb

Among Metrictank , biggraphite and m3db which tsdb is the best in terms of availability, scalability and handling more metrics per second.

https://redd.it/g4q8lk
@r_devops

problems to solve with a monitoring tool

Posted this in /sysadmin, but thought that the crowd here might have a different take.

Currently doing some research to understand the most valuable features in a monitoring tool. These could be things you take for-granted, or table stakes, but nevertheless, mission critical things that make your life much easier. I'm currently evaluating my options and want to get everyone inputs from experience.

https://redd.it/g4mcpn
@r_devops

What are some good side projects to work on?

I'm currently working as a junior SRE and DevOps engineer. I want to get better at this. That being said alot of my current work depends on services created by developers, I don't really write full applications.
that being said, what are some side projects I can get into that don't involve having to write a robust application? I'm interested in learning about things monitoring, networking, ci/cd, load testing, configuration management etc.

https://redd.it/g4ndm6
@r_devops

[Packer] How to use a Kickstart file via vsphere-iso

Hey, I'm having a heck of a time getting this to work and I feel like I'm missing something basic and I'm not sure how to proceed.

ESXI: 6.5
OS: Centos7

I'm trying to get Packer to build an image for using the kickstart.cfg file I'm either getting error messages about not finding the kickstart file, or going straight to gui.

​

From the json file:

"variables": {
"password": "",
"ks_file": "[TXDataStore02]kickstart.cfg",
"iso_paths": "[TXDataStore02]CentOS-7-x86_64-DVD-1810.iso"
},

[[ removed a ton of basic vm configuration ]]

"floppy_files":[
"{{ user `ks_file` }}"
],
"boot_wait": "5s",
"boot_order":"disk,cdrom",
"boot_command": [
"<esc>",
"<wait>",
"linux inst.ks=hd:fd0:/{{ user `ks_file` }}",
"<enter>"
],
"convert_to_template":true

after running it I get this in esxi:
Floppy Drive 1: Image \[TXDataStore02\] example-centos/packer-tmp-created-floppy.flp


That obviously isn't "Kickstart.cfg" and it explains why it can't find the kickstart file.


So, how do I use a kickstart file with the vSphere-ISO builder?


Thank you,
Josh

https://redd.it/g4iqq4
@r_devops

Major outage at VictorOps!

**VictorOps Service Disruption: Actively Investigating -** Apr 19, 2020 - 17:22 MDT

[https://status.victorops.com/incidents/39nq3xqr6kt6](https://status.victorops.com/incidents/39nq3xqr6kt6)

&#x200B;

Update - **Resolved (total outage - 2 hours)**

After monitoring our remediation efforts surrounding this incident, we are moving this incident to a resolved status.

Brief Incident Overview: From approximately 4:30 PM (Mountain) to 5:35 PM (Mountain) on Sunday (04.19.20), we experienced service disruption challenges that may have adversely affected the overall VictorOps end-user experience.

We sincerely apologize for any unintended inconvenience this issue may have caused.

If you have any immediate questions surrounding this incident, please contact the VictorOps Support Team at: [[email protected]](mailto:[email protected])Posted 20 minutes ago. Apr 19, 2020 - 19:17 MDT

https://redd.it/g4iksz
@r_devops

nested bind mount on efs

Is it possible?

I'm using aws efs and trying to mount --bind another directory under the efs mount.

/etc/fstab example:

```
<EFS_URL>:/ /mnt nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,\_netdev,noresvport 0 0

/source/dir /mnt/nested none defaults,bind 0 2
```

If I mount the efs on another instance, /mnt/nested appears empty.

From searching online it seems that it's possible but only if I have access to /etc/exports on the nfs server.

Is there any other way?

https://redd.it/g4eb28
@r_devops

How do you do monitoring/observability for dev/staging environment?

If you are using APM tools like DataDog, NewRelic - do you also use them for staging environment? How do you optimise performance at staging - or the task of optimization is left to the production env. as long as things are working fine?

If you use APM tools for staging/dev environment also - doesn't it increase your cost significantly?

https://redd.it/g5hw7y
@r_devops

I started linting my Ansible playbooks with Drone and wrote about my experience. Hopefully this can help someone!

If you use Ansible, you've probably stumbled across some tutorials/info from [Jeff Geerling](https://github.com/geerlingguy). In an ongoing YouTube series called [Ansible 101](https://www.youtube.com/watch?v=SLW4LX7lbvE&list=PL2_OBreMn7FplshFCWYlaN2uS8et9RjNG), Jeff briefly mentions automated [linting](https://en.wikipedia.org/wiki/Lint_\(software\)) of his Ansible playbooks.

I was curious about this, and went down the rabbit hole over the past week or so. I had never done this before and thought it was a really good topic to expand on, so I wrote a two-part series about Ansible, linting, and [Drone](https://drone.io/), on my blog.

Part 1 - [How and why you should lint your Ansible playbooks](https://loganmarchione.com/2020/04/how-and-why-you-should-lint-your-ansible-playbooks/)
Part 2 - [Linting Ansible playbooks using Drone](https://loganmarchione.com/2020/04/linting-ansible-playbooks-using-drone/)

I'm by no means an expert in this area, but just wanted to document some of my thoughts, hurdles, and configurations in the hopes that it would help someone else!

https://redd.it/g5ina3
@r_devops

How to Survive as a only devops in a company without devops culture?

Hi, I've been in a devops role for 2 years now(after graduating from college) So far I've been doing a lot of things so I have a good breadth of the field but I guess I'm missing some in depth experiences(especially after reading about other devops who do stuff like messing with kernel and other in depth stuff)

To give you some background I accidentally became devops after I applied to cloud data engineer position after college. But that was actually devops(cloud engineer) job. I learnt a lot about AWS, mostly managing data lake and data pipelines. Later I was transferred to a more of a team building web-app .I was responsible for writing some docker images but actually again most of my work was with AWS.

This brings me to 8 months ago when I was approached by recruiter for a large company. They were launching their first app to the cloud so that was an exciting project for me. Well, we designed the cloud native application and now it's deployed (only dev/test environments). These were very intense months but now I kinda have nothing to do until company decide to go for production.

Now, I'm faced with couple of problems. 1. I don't have anything to do until we go to production. And it's still not sure when we plan to go live( I made some optimizations in pipelines and Terraform in this free time) 2. Since the company doesn't have any devops culture. My manager actually expects me to do everything from being responsible for deployments to being responsible for all infra. This is a bit annoying because all day I receive requests from developers e.g.: why is the build failing? why I can't ssh into the instance, etc. This actually wastes a lot of my time. 3. In my previous job I had other devops colleagues from who I might learn however now since I'm the only one I'm not sure how to keep up with all the AWS news and also I want to learn more about other aspects( The pipelines I wrote are quite mediocre IMO but I just don't have enough ci/cd experience.

Sorry for a long post and perhaps someone will be able to relate and help me to navigate this path. I went to a devops days conference last year and it seems that this industry has almost 0 junior devops engineers. I quite like my job just struggling to navigate in this complex field

https://redd.it/g5fywk
@r_devops

yamlscript, An easy to use NPM package that lets you write and run command line scripts in a YAML format

Hello 🙌🏻,

I Hope everyone is safe out there, I have made a small NPM package that lets you run a script of commands using a YAML file, the package allows you to load the scripts to be used again, also you can define which commands run in series and which commands run in parallel, there is a log file for each run for tracking.

I would like to hear your thoughts and your opinions.

Link for the GitHub repo: [https://github.com/mohamed1refaie/yamlscript](https://github.com/mohamed1refaie/yamlscript)

https://redd.it/g5kvsn
@r_devops

Newbie to Checkmarx code scans

We are building a CICD Pipeline at work using Gitlab, Chef and Artifactory. Within Chef we have to implement checkmarx code scans but I have never used it before. Looking for the easiest/quickest method or resource to learn how to use/read checkmarx code scans.

Thanks in advance!

https://redd.it/g5hti2
@r_devops

Tutorial: How to power your GitLab Auto DevOps pipelines with Crossplane

**Introduction**

Most of the times the CI pipeline’s automated test phase requires a database to be provisioned so that the business application can be tested in its entirety.

In this guide we will provision a managed Postgres database on the Google cloud platform using Crossplane and GitLab’s Auto DevOps pipeline as part of CI.

A little bit about GitLab’s Auto DevOps, GitLab Managed Apps and Crossplane in the subsequent sections.

**GitLab Auto DevOps**

GitLab provides a feature called Auto DevOps which automatically creates CI/CD pipelines for your source code repository. It is an opinionated pipeline with reasonable defaults. The pipeline can be customized by way of a yaml configuration. Auto DevOps facilitates standardisation and consistency of the CI/CD pipelines.

&#x200B;

**GitLab Managed Apps**

GitLab provides a feature of a One-Click Install of an application onto a Kubernetes cluster connected to a repository. Applications such as Prometheus, Cert-Manager or the newly added Crossplane control plane can be installed via the Managed Apps dashboard.

&#x200B;

**Crossplane**

Crossplane is an open source multi cloud control plane. It allows you to provision cloud services the kubectl way by exposing them as Kubernetes resources by the use of CRDs and controllers. Crossplane connects to the cloud provider using service account credentials and orchestrates creation of services via the cloud provider APIs. This helps developers and administrators alike in provisioning and configuration of these services using Kubernetes resources hence, allowing a uniform interface for those working on the platform. It follows the model similar to a StorageClass and a PersistentVolumeClaim where an administrator could define the class of services available to be consumed and the developers request for a claim. [read full post here](https://www.infracloud.io/auto-devops-crossplane/)..

https://redd.it/g5hjat
@r_devops

Is it normal to grind LeetCode for a DevOps interview?

I am applying for a DevOps position (AWS shop) and I was expecting the only coding to be python scripting and Jenkins/CloudFormation. Its an entry level position so I was told it was expected for me not to know too much Jenkins (I'm pretty good at Python) - but the interviewer said that for my interview make sure they can see my screen/my IDE for the coding portion of the interview for "code review".

Is this normal? Should I be trying to learn fizzbuzzes?

Thanks

https://redd.it/g5rvg6
@r_devops