Gitlab CI/CD option to skip Auto-Pipeline

Hello, there.

I’m using gitlab for my CI/CD task. But whenever I’m pushing something to the repo, gitlab is starting the pipeline. To avoid this, I know one method to add ‘ci skip’ in git push command. But without explicitly mentioning this, can we avoid auto pipelining in Gitlab?

https://redd.it/fkmj73
@r_devops

Tips for a software developer new to implementing CI/CD?

This is gonna be a little long, I hope you can read til the end.

I'm a software developer. We have been building MEAN stack apps, and until recently, everyone in our team was fine with the deployment process — using bash scripts. But during a meeting, someone asked if there was a way to have some sort of a blueprint or template for setting up a server (which is basically manually installing a myriad of dependencies on the server and configurations — someone wrote a bash script for this after the meeting) and making the deployment process easier/faster (not having to ssh into the server to run the script for deployment on our own machine — what are the cons?).

I have had some experience with Docker, but only because I had to use it for my local environment for coding — I have never built anything from scratch before, nothing extensive.

We don't have a dedicated DevOps team yet, and the project we are working on isn't a full-scale app with many complex processes, it's relatively small. But since we had a bit of downtime, we decided to start looking into Docker and CI/CD to help streamline the processes for the meantime, no strings attached.

I was able to package a very simple Angular/Node.js app using Docker, and deploy it to a server using a GitLab runner. Now they have decided that it would be a good thing for the team to learn how to streamline the current processes we have using these new technologies.

I definitely have always been a coder, but now there is an opportunity to build a new skill or two, but I am a total noob. I am definitely eager to learn, but I am kinda lost and feel overwhelmed, as I have no plans in becoming a master at this, but just wanna have enough knowledge I guess so we can streamline our processes, from reducing the effort for setting up a server to deployment.

Like, what if we won't have to manually install or write a bash script to install all those dependencies every time, we can just simply prepare more readable Dockerfiles and just run containers? And have one-click deployments? Can I actually sort of put limitations on this? Like, I don't have to learn or dive deep into x y z for what I'm trying to do? What if..... should we just stick to bash scripts?

I tend to develop FOMO when learning a new technology :( I always feel like I have to learn everything about it in order to be knowledgeable or productive with it, so I try to learn everything I can, but I know that isn't practical. But for this one, how do I put limitations?

Also if you have any other insights, like if you think Docker is worth it or not in this case, etc, they'll be appreciated.

https://redd.it/fkh1z7
@r_devops

New to DevOps and have a question regarding a CI/CD cycle...

Hi everyone,

So, currently I'm a Sys Engineer on a local IT/Ops team. Over the past two years I've been working pretty heavily with PowerShell and fell in love with it! I tired to automate everything under the sun and succeeded most of the time while of course failing hard and learning from it. Fast forward to about 6 months ago, I really wanted to improve our small team and start building around the idea of automation and infra as code. So, I built out a Jenkins instance and took some scripts I had built for automating email notifications to users when their password expiration was coming up, cleaning up AD stale objects, WSUS updates and cleanup, etc. and started hosting them there and running on schedules. I then realized that keeping scripts on my local dev machine wasn't the best so I learned Git and started storing the scripts there and polling my Git repo in Jenkins instance. I've recently been building out a Chef server / workstation and working to integrate config management! Super happy with everything, except...

As embarrassing as this sounds, I would really like to start using Jenkins not just for one off freestyle projects but really use pipelines in a true CI/CD fashion, but I am having no luck figuring out a scenario in which I would use it? I know that probably sounds so dumb. I can totally understand how our Platform team in work uses it since they have a pipeline that goes from creating a build to testing to deploy, and that makes sense. But I'm trying to figure out how I could benefit from a CI pipeline with what I'm doing say building Shell scripts for various production reasons... I thought maybe a pipeline with say a Pester script and if that passes run a Git add/commit and push to repo? Again, maybe this is a silly question but I just for the life of me can't think of a way to use CI to make what I'm doing in Dev more streamlined or if that's simply not how this works. Thanks everyone!

https://redd.it/fkz7nn
@r_devops

Glueing tools together

Wanted to find out a bit more about what you guys use day to day in terms of your tooling.

At my work we rely heavily on EC2 with Nginx on the edge, then ALBs to EC2. This pattern is mostly for our legacy apps.

Then we have a whole bunch of microservices built with API gateway and lambda (with Dynamodb, SNS/SQS, RDS etc)

And recently we’ve moved some legacy over to ECS again fronted by ALBs (and same nginx on the edge)

All of this infrastructure is managed with a single php tool which is a wrapper around cloudformation, chef and packer.

We have a separation of:
- environment (dev, test, live)
- project (services, magento, core)
- repository (web, cron, servicename)

And we have a git repository in the following format: “environment-project-repository” (e.g. live-magento-cron) , yes we run magento!

Inside this repository is everything needed to create this environment, cloudformation, chef etc.

This setup seems to work for us quite well, really interested to see how others have solved the IaC problem.

https://redd.it/fkz1gy
@r_devops

Trying to locate a website for tools

There was a site that was similar to the XebiaLabs periodic table. I have been reviewing my old bookmarks and trying to Google it . Would appreciate any assistance.

Thanks

https://redd.it/fl190j
@r_devops

OpenShift is simply VMware for containers

Our company is looking to write new applications and deploy them to the cloud (Azure in our case). We have two camps, one for OpenShift the other for using PaaS offerings in Azure. We have a history of challenges in relation to managing the lifecycle of our OS, middleware and application (we are behind, a long way behind in some cases).
I see running OpenShift in the cloud (with no direction to deploy on-premises), as simply another abstraction layer on top of an abstraction layer, it has some great capabilities, I am just struggling differentiating from where we are now with where we will be in 5 years, ie the same place with more technical debt.
Help me change my mind..

https://redd.it/fl258j
@r_devops

Newrelic.com exploiting covid to sell their product, and it’s in poor taste.

“We know that keeping your business running smoothly is top of mind during this time of uncertainty with the spread of COVID-19.

It's more important (now, more than ever) to keep your underlying applications and Infrastructure reliable and to maintain a quality experience for your customers.”

I mean, WTF. Maybe I’m being too sensitive but this is bad taste.

https://redd.it/fkdczq
@r_devops

Any good reference material for learning to make helm charts?

My company is finally considering a move to container based infrastructure instead of the VMs. My first task is to containerize our apps most have official containers but I’m looking for a way to automate deployment. I tried using terraform and that worked in my rancher lab but not on our production rancher cluster since the generic kubernetes provider has no concept of a project and tries to create everything in the default project which I currently have no access to. My thought was to try helm instead. I’m open to other suggestions if anyone has a better idea.

Also how would one mount dynamic content to a config map at runtime? An example is logstash I want each of my pods to have a different configuration. I know I can do this all manually and it works fine but I’d like a way to automate the process. ATM the configurations are generated using ansible which for a logstash vm would copy everything over through a template. I’ve modified the logstash play to generate a directory with the config files.

Still new to the K8s game. I’m fine with standard container concepts but having a hard time figuring out how to apply the same knowledge to the kubernetes way of doing things.

https://redd.it/fl3ff6
@r_devops

Automated dependency updates without having Github

I am hosting all my code on Github and make use of the features dependabot offers (https://dependabot.com/), but in the past months I kept thinking about the same questions:

* What about people using Gitlab/Bitbucket or self-hosting their code (e.g. for compliance reasons). How do they do automated dependency updates?

* And I have had the cases where I wanted to have an automated pull-request, and thus an automated build, whenever e.g. a new node.js version is published to have always up-to-date applications/docker images and so on. What I did in that case, was to simply do a daily/weekly travis run where I would pull the newest node 12.x version and build everything. This works, but is not really elegant. Especially for softwares that gets many releases or rarely any.

What is your take on those two questions? How do you solve those problems?

The reason I am asking is because I am currently building a website to solve that problem for myself and wanted to see whether other people also might be interested in that solution (or maybe there is already an existing solution which I just don't know of)? The basic idea is that I notify developers over a channel (webhook?) whenever a software they uses get an update. I now got some time on my hands, so why not use it productively :D

https://redd.it/fkcr71
@r_devops

How hard will it be to crack Devops engineer job at Google products for example consider YouTube in this case. For a Devops job with 3 years experience requirement.



https://redd.it/fl4nwv
@r_devops

"Interview Question: What are some challenges you faced in your role or project, and how did you solve it?" What are the interviewers really looking for when they ask this question? What do they want you to talk about?

I have faced many challenges in my previous role as a Linux Systems Engineer, but I don't know which ones are relevant for the role?

https://redd.it/fkcoxt
@r_devops

PSA: Your friendly neighborhood OSS Project probably has fun stuff for you

Hi everyone,

Looking at all the "what should I learn" posts and people stuck at home. I just wanted to point out that lots of OSS projects are still ticking away and always looking for new contributors, etc. I work in Kubernetes and wanted to share some of our meetings for the week for those of you looking to get started:

First off on Thursday there's a public community meeting, these happen monthly and are a good 10k foot view of what's happening in the project: [https://youtu.be/04FiOd-bG8c](https://youtu.be/04FiOd-bG8c)

We also have 2 "Office Hours" sessions tomorrow for people to ask questions about Kubernetes, all skill levels are welcome, this covers problems that users actually have so I tend to learn a ton just by reading the chat: [https://discuss.kubernetes.io/t/kubernetes-office-hours-for-18-march/10116](https://discuss.kubernetes.io/t/kubernetes-office-hours-for-18-march/10116)

https://redd.it/fk7vvo
@r_devops

Automate Unit Tests Coverage Reports and Dependency Updates for Your Repositories.

Automation is one of the core topics that can influence and enhance the software engineering and software development processes. The software development life cycle consists of several repetitive tasks. Some of them occur on a monthly basis while others can occur more often on a daily, hourly, or event basis. For instance, creating the release branches and deploying to production environments is a routine task that needs to be executed on every release (depending on the release cycle this could be a daily, weekly, monthly, or yearly task). Another example is execting the unit tests of the software application. This task needs to be executed each time a change is integrated with the codebase to make sure the application stays healthy.

[Mor info](https://medium.com/@wshihadeh/automate-unit-tests-coverage-reports-and-dependency-updates-for-your-repositories-41be1b858eb3)

https://redd.it/fkb83d
@r_devops

Impact of Covid-19 and remote work on DevOps??

Have people started to see a big infrastructure impact following the Coronavirus outbreak? I am in Paris, the entire country of France is locked down and everyone has been told they must work from home. This was announced Friday night so most companies have had no time to think about DevOps for remote work.I know yesterday Google Teams crashed due to high demand of people working from home.I also wonder what the security impact will be, for instance secrets being shared via email if people can't access secure networks.


What have people experienced? what do you predict will happen? How can we be prepared?

https://redd.it/fk82uv
@r_devops

Debating job offer that would change a lot of things in my life.

​

Sorry for yet another post about my personal career but I have a new decision to make.

​

I currently live in a LCOL city with my wife and 15mo daughter. We bought a nice house for $300k and my parents live a couple blocks away and babysit for us whenever. I work remotely for a startup that is HQ in a major city. I earn 145k per year and some bonus.

​

I've been concerned about company performance and how much runway we have so have done a lot of interviews over several months without any offers. In the last few weeks I received two offers and need to get back to them ASAP. The first is a mostly remote job that is based in a town about 1.5 hours from my house. The job would be mostly remote with me driving out there for meetings a few times a month. They pay under-market and it would be about a 10% pay cut from where I'm at currently, but the company is on good ground otherwise. I tried to negotiate higher but it wasn't happening.

​

The second, more recent offer, is from Microsoft in Redmond (Seattle suburb if you didn't know). Its an SRE position on a special team and the earnings taking into account stock award, etc would be in the realm of 210-220k per year. They offered relocation assistance etc. So that is a significant increase in pay.

​

We lived in Seattle for many years and know it well and love many things about it, and miss some of our friends there. But traffic is atrocious, I would be required to work on-site 95% percent of the time probably, and commute to the office etc. We would have no grandma to watch our baby around the corner.

​

I'm really having trouble deciding if a 45% pay raise is worth the traffic and time away from probably the only kid I'm going to have. Seattle is a much more exciting place to be but as a working parent I'm not sure if that even matters that much.

​

I do want to further my career and save more money. And getting a great job offer in this crazy time of covid-19 feels very strange indeed with so many of my friends struggling to make ends meet.

​

Thoughts?

https://redd.it/fk80p1
@r_devops

Automated VM Deployment Tools

Hi, Windows OS, vSphere hypervisor and ansible. What would you recommend for simple os deployment from an image? I'd like to have a simple interface like Ansible Surveys so some lower sysadmins can make selections to deploy VMS based on secure templates. packer and terraform? Ansible playbooks? Any other options I'm missing?

https://redd.it/flcnzg
@r_devops

Create AWS ECS task definition and service in terraform but deploy updates (changing the docker tag that it deploys) outside terraform from CI sequence?

Does anyone have a nice way to do this? We have it working but it's annoying because making changes to the task definition or service causes weird diffs, you can kinda ignore_changes on certain fields but that has to be like commented out to make changes...

I started looking into AWS CodeDeploy as a different way to deploy... would that help with this at all?

https://redd.it/fleac9
@r_devops

Are devops positions remote friendly(er)?

I'm a jr full stack JS developer and have thought of adding Kubernetes to my study/project list as I hear devops is in demand. Probably Ansible, Terraform and related after that. My goal is full time remote work.

Do you guys think devops pairs well with remote work?

Or would you advise I learn back end cloud computing? I'm already ok at back end so I imagine porting that skillset to AWS should be straightfoward.

https://redd.it/fl9b8h
@r_devops

Building Private Clouds

A friend of mine did a really great [podcast](https://packetpushers.net/podcast/day-two-cloud-040-building-and-operating-a-private-cloud/) interview (click the link) that I thought all y'all would enjoy. I think the self-service features of "cloud" are one of the core principals that DevOps is all about, so I hope this lands on this crowd right.

https://redd.it/flgkxz
@r_devops

Looking for HA storage solution for Minio

Hi there!

​

Sorry if this is not the best place to post this, but it kinda seemed fitting. We're running Minio and do a lot of read/writes on it. It is currently not HA, and distributed minio works very slow once a node goes down, so I started looking for HA solutions for the file system. I tried using GlusterFS but it couldn't handle the load, and caused a 200 CPU ready. NFS is my next bet, but I saw that the only good way to make NFS highly available is with Pacemaker and only with 2 servers, which is kinda wired and not where I want to go.

Has anyone faced this issue before? It runs on Kubernetes but is currently bound to a single node with local storage. We don't currently have a supported NFS or anything like that in the environment so I cant bound the persistent volume to anything aside the local storage.

https://redd.it/flhomd
@r_devops

values.yaml: unable to parse YAML: error converting YAML to JSON

Quick question. Trying to map multiple ports to a container in a helm chart but getting

>\[ERROR\] values.yaml: unable to parse YAML: error converting YAML to JSON: yaml: line 37: mapping values are not allowed in this context
\[ERROR\] templates/: cannot load values.yaml: error converting YAML to JSON: yaml: line 37: mapping values are not allowed in this context


values.yaml

`service:`
`ports:`
`name: udp-graphite`
`port: 2003`
`target_port: 2003`
`protocol: udp`

`name: tcp-graphite`
`port: 2003`
`target_port: 2003`
`protocol: tcp`

`name: pickle-graphite`
`port: 2004`
`target_port: 2004`
`protocol: tcp`

`service.yaml`

`{{- if .Values.service.ports }}`
`apiVersion: v1`
`kind: Service`
`metadata:`
`name: {{ include "carbon-relay-ng.fullname" . }}`
`labels:`
`{{- include "carbon-relay-ng.labels" . | nindent 4 }}`
`spec:`
`type: {{ .Values.port_type }}`
`ports:`
`{{ toYaml .Values.service.ports | indent 4}}`
`selector:`
`{{- include "carbon-relay-ng.selectorLabels" . | nindent 4 }}`
`{{- end }}`

​

Not sure what the deal is there. From all the other examples i've seen on the web says that should be possible. using helm version

`version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}`

https://redd.it/flfeh2
@r_devops