Migrating to Gitlab from Jenkins: how?

We are thinking about moving to Gitlab from Jenkins because Jenkins has a lot of bugs recently and the plugin situation is chaotic. Questions we have:

\- we store our code in GitHub currently: can Gitlab work with code stored in Github or we have to migrate everything to Gitlab? Which is a misery, I guess. Not the process itself, but new tools, new environment for all the developers, etc.

\- how can we build a specific subfolder in a repository instead of building the whole repo? Let's say we have 10 products in a single github repo in 10 separate folders, we have 10 Jenkins jobs for these and if we change the code in a folder, only the associated Jenkins job is started, nothing else. Can we do the same in Gitlab? If yes, how?

Thanks for the answers in advance.

https://redd.it/ll04zb
@r_devops

New DevOps engineer

Hi, I am a new DevOps engineer with hands on experience on couple of volunteer projects in Ontario, Canada. I am currently working in QA domain and now looking for new opportunities in DevOps. Can you all please share your day to day typical activities as DevOps engineer and I'll really appreciate if someone could share interview questions/suggestions for DevOps. Thanks.

https://redd.it/lnp2c4
@r_devops

Data Base Automation - How or which tool are you using?

I am in charge for DevOps and Cloud in my Company, we already created all apps devops pipelines, but now we are stucking our velocity because changes in infrastructure, but Data Base Automation we are no Clue how to solve it (CI/CD with automatic Rollback)....

https://redd.it/lnq04g
@r_devops

The Four Key Metrics of Devops

We held a discussion with a Cloud Architect who outlined these as the four key metrics of Devops:

1. Deployment Frequency
2. Lead Time For Changes
3. Time To Restores Services
4. Change Failure Rate

The first two metrics measure velocity, the last two metrics measure stability. Here's the full discussion

https://youtu.be/ep-guKZK468

https://redd.it/lnokcc
@r_devops

Questions about 3 DevOps certificates.

Which of the following three certificates is the easiest and which is the hardest?

Which one is most useful in conjunction with AWS skills?

\- ITIL 4 Foundation certification

\- Certified Jenkins Engineer

\- HashiCorp Certified: Terraform Associate

https://redd.it/lnmov3
@r_devops

Is anyone here running Flockport in a production environment?

I've always found Flockport interesting but I've never taken the time to play with it.

Just curious if anyone here is using it or has used it in the real world. What is your experience with it?

https://redd.it/lnn9qk
@r_devops

Settting up a rate-limited docker repo?

I'm wondering if any of the current docker repository solutions currently have any built-in way to handle rate-limiting. I expect this is a unique case, but a growing one and I'm just brainstorming a solution to try to move it towards broader use by making a business case for further openness.


Ideally, checking if there is an updated container would not be rate-limited or at least limited differently from downloading layers of a container.

https://redd.it/lnm0o1
@r_devops

Anyone know how I can POST a Binary of JSON files (zip, tar) to API Gateway so Lambda can unzip and process those JSON files?

Id like to grab a group of JSON which can be mutated in Lambda and pushed into DynamoDB. Thanks :)

https://redd.it/lnuga9
@r_devops

Can I forward Nginx logs to an API - How can I do this?

I want to process nginx logs of multiple machines. I thought it would be nice to forward the data to an endpoint of an API where I can parse the log and save it to a databse. How would you tackle this?

https://redd.it/lnkc4l
@r_devops

I'm a junior DevOps Engineer at my company. How do I lose the Junior?

Basically the title.
Last position I had was a sysadmin where I did mostly everything there was to do. I have studied CS, but I don't like programming very much, unless we are talking about scripting. I use bash, PS and python for my scripts.

I am working in a team of two, myself included. My colleague claims he has 9+ years of DevOps experience, but I feel I can learn nothing from him (long story and I also don't like to focus on him now). We have an external consultant and that guy is awesome. I'd love to be like that one day.

Currently we are working on Azure and it is planned that this year we're getting the AZ-102 (developer) and AZ-400 (devops) certificates. This will help ofc, but I also need to gather some more exp. with Docker & Kubernetes. We don't yet use VM automation, so that's not a priority.

I have bought some courses for CKA and CKAD. I have started with CKA, but I feel that one is not the right for me, since what we mostly do has to do with helm, tls and k8s. I'd like to learn this stuff too. Some good sources are appreciated, maybe in connection to Azure, so that I may use the newly gained knowledge asap.

Thanks :D

tldr: need to be better at helm, azure and k8s and also lose my junior in the title. What do I do?

https://redd.it/lnazlq
@r_devops

SRE vs. Platform Engineering

Over the past decade, engineering and technology organizations have converged on a common set of best practices for building and deploying cloud-native applications. These best practices include continuous delivery, containerization, and building observable systems.

At the same time, cloud-native organizations have radically changed how they’re organized, moving from large departments (development, QA, operations, release) to smaller, independent development teams. These application development teams are supported by two new functions: site reliability engineering and platform engineering. SRE and platform engineering are spiritual successor of traditional operations teams, and bring the discipline of software engineering to different aspects of operations.

https://blog.getambassador.io/the-rise-of-cloud-native-engineering-organizations-1a244581bda5

https://redd.it/lnhwkb
@r_devops

Should DevOps Toolchain contain Azure KeyVault

Basically what the title says. In your opinion, should a tool like Azure KeyVault be in a DevOps Toolchain?

https://redd.it/lnh5er
@r_devops

Infrastructure for hosting a web scraper that scrapes huge quantities of data? (Interview Q)

Hey guys, I’ve been given an interview question to complete over the next few days which I’m a little stuck with. Basically - I’ve been asked to design the infrastructure for hosting an internal web scraper (the code for the scraper has already been written). Have to create a diagram and name the technologies (Docker/AWS/HAProxy etc) and explain my decisions. It’s a little out of my depth and I’m wondering if anyone has any resources or tips where I could learn more about infrastructure design? I know I’ll need lots of databases and probably a load balancer to divide up work between worker nodes/servers and maybe a load balancer between those and the databases? I just want to learn a bit more about the specifics so that I can design something that makes sense! I know it’s a very open ended question and there is an infinite amount to learn - but any examples or central ideologies would be great! Thanks in advance :)

https://redd.it/lnb3eh
@r_devops

training question (employer paid vs PTO)

I'm a senior developer for a major national consulting company, billing 100% for a project (years & years). My employer offers FT employees seasonal certification bootcamps - DevSecOps, AWS, Azure, etc, which are usually several days and then the exam. Senior mentors from my company do the training in-house. The company pays for the test but they don't cover all the time for attending the bootcamp -- they do half and you have to take vacation time for half. (Since these certs are not-required it's legal to require PTO for training.)

I'm just curious how common this is. In my past jobs w/ non-consultant IT departments, the company covered all time & costs of training -- as a perk for the employee, and I imagine also because it benefits them to have better-qualified staff. This seems kinda cheap to me, considering the training certs are relevant to the work & tools we use on the project.

What are your experiences? Ever have to use PTO for your skills training?

https://redd.it/ln2om4
@r_devops

What are the disadvantages of going cloud-native?

So, I think my previous post about the benefits of going cloud-native (https://www.reddit.com/r/devops/comments/lkbx9e/what\_cloud\_native\_is\_really\_good\_for) was entertaining and certainly useful. My main take-away is that with cloud-native you design your software to make the best use of a public cloud infrastructure - with all the benefits that [public cloud infra\] entails, such as scaling up and down, deploying when and where you needed it, etc. All the other benefits mentioned (e.g. speed) can be realised without "cloud-native" in my view.

But surely cloud-native has its drawbacks too. Off the top of my head, I'd say performance overhead and dependance on a rather limited number of public cloud service providers.

Other views?

https://redd.it/lky489
@r_devops

Introduction to GitOps on Kubernetes with Flux v2

I wrote a blog post about GitOps on Kubernetes with Flux v2.

Wanted to share - maybe this is useful to someone.

Personally, I really enjoy using Flux. What are your experiences?

https://redd.it/lobnqf
@r_devops

Best way to learn Linux?

I've been looking at improving my core skills like networking and Linux. I was thinking about using LA playgrounds, installing Linux as dual boot on my laptop, renting a VPS etc...

Has anyone got any good recommendations?

https://redd.it/lob2ck
@r_devops

Watch Kubernetes Experts Fix Broken Kubernetes Clusters Live

I’ve launched a new series of episodes called Klustered. These episodes feature myself and a guest from the Kubernetes community attempting to fix some Kubernetes clusters. These clusters are also broken by community members 😀

We know nothing upfront. The first episode was very fun. Episodes will be live on YouTube every Thursday. Best week we have clusters broken by Jason DeTiberus and Justin Garrison.

I hope you enjoy

https://youtu.be/teB22ZuV_z8

https://redd.it/lo7a8v
@r_devops

Monitoring 5,000 nodes

Hello.

I’m curious what solutions a community like this employs for the following scenario:

We’re looking to put about 5,000 Linux boxes across America inside of stores. They serve an important purpose and will be more or less 5,000 of the same image. This is a big increase in scale for us as our existing Linux server footprint is roughly 1,500.

We currently use Zabbix but I find it lacks in scalability and supportability.

The support will require cross collaboration between Linux OS support, database support, and application developers, so I am looking for a solution where these disparate teams can write their own monitoring and alerting solutions for their use-cases relatively easily (definitely a challenge to do with Zabbix).

I’ve been thinking about Sensu but I am interested in hearing other options/experiences here.

https://redd.it/lo9l76
@r_devops

How do you trace root cause analysis on your microservices



Hey guys trying to gain some inspiration to rethink how can I make this process less horrible in my own life

Seems to me that everyone is using the same method when doing root cause analysis (on dev/staging/prod envs), Plugging it all to some ELK, Using Kiali/Other tool for specific MS log trailing.

The process is usually something like getting some first order cause like a request failing -> finding where it started -> going to the Log trailing tool(Kiali etc.) finding the exception -> getting the trace id -> search in Kibana with trace id -> move through massive number of lines -> find next stacktrace on another MS -> repeat until finding root cause.

This is of course when you even have a stack trace that gives you more info, what if it is some authorization issue between services or some other DevOps tools in the stack (istio etc.)

Tools like datadog/splunk show the request trace and status but this doesn't solve the long root cause analysis in most of the cases

Hope you guys have something better in practice =)


Thanks in advance

https://redd.it/loawxb
@r_devops

Unit Testing a Jenkinsfile

Thought I would share I figured out how to unit test a Jenkinsfile.

https://clintmod.medium.com/unit-testing-a-jenkinsfile-55eaa6a2e797

https://redd.it/loeeno
@r_devops