We recently wrote about why and how you could migrate from Jenkins Freestyle to Multibranch pipeline.

This article talks about the reasons, advantages and challenges we faced while migrating from Freestyle Job to Jenkins Pipeline. It contains a reusable Jenkinsfile template.


Sharing the link in the comment section.

https://redd.it/lyd1ge
@r_devops

Can someone help me understand where Ansible fits in a CI/CD pipeline?

Hello. I'm fairly new to DevOps and I'm trying to teach myself various tools such as Terraform, Jenkins, Packer, Docker, Kubernetes, etc. However, I'm not quite sure where Ansible fits in a CI/CD pipeline on AWS. Ansible feels...redundant, almost useless. It seems like 99% of the pipeline is already being handled by Terraform, Packer, Jenkins, and Kubernetes.

Can someone please enlighten me on where Ansible comes into play? How do you use Ansible? What am I missing? Thanks.

https://redd.it/lyc8cr
@r_devops

MLOps podcast series

Hello, everyone. My co-host and I are running a multipart series about MLOps on our podcast Delivery, Interrupted. We interview several MLOps experts and discuss everything from how to get started with machine learning in your business to CD practices in a ML world. There are more episodes to come on the subject, so now’s a good time to start listening.

Delivery, Interrupted podcast

https://redd.it/lyde57
@r_devops

What is most challenging about DevOps in real life?

Part of my job is to help people with DevOps and I started making a list of questions for people to consider. Because DevOps is so application specific, there is no standard answer - although all the Hello World tutorials out there would lead us to believe otherwise.

I think the most complicated technical topic is one of deployment promotion, how to move something from dev to staging, etc. to production

I have people tell me they prefer to click a UI to deploy to a "target" environment. I ask them well, what version of your code is running on `uat` right now? Which they cannot answer…
I personally prefer having long running branches per environment, e.g. dev, staging, production…
but people new to DevOps (or git outside of a UI?) struggle with managing long-lived branches and merge conflicts.
but people get stuck on who promotes or merges changes from one branch to another.

Sometimes I feel like this is because people haven't defined these processes, which for me is akin to never planning for failure. It needs to be defined. Sometimes I feel like people just struggle with git. In a previous job where I mentored developers, people would ask for help and I tried to explain out loud to them as I debugged, but often I myself was lost in the IDE. When I switched to a terminal window, many were lost.

I think the most challenging general concept is governance, but that isn't just DevOps specific. It's hard to figure out a RBAC model and then to map that both to cloud and automation. But I think I have that one figured out in terms of how to explain it.

What do you think is most challenging? I'm asking because I like the hard problems and figuring out a way to teach people to help them understand.

In my work I encounter a very specific group of people. I was wondering what other people "in the real world" (and not my work bubble), what do you see?

https://redd.it/lyfwyu
@r_devops

Transitioning in to DevOps from being a QA Tester

I just wanted some guidance on what your doing so far in terms of Devops. What tools/tech stack you are using.

Am currently transitioning in to DevOps so your advice would be very useful.

So far i have created a road map which is the following : 

Programming Language : Python , Shell Scripting
Linuxs Administration :
Terraform , Cloudformation - Infastructure as code
AWS , GCP - Cloud Platforms
Alerting , Monitoring and Auditing - Cloudwatch Alarms , Cloudwatch Logs , Kibana , Grafana
CI/CD - AWS CodePipeline , Jenkins
Containerisation - Docker with ECS/EKS
Centralised configuration management - SSM Parameter Store
Software management/Configuration management - Ansible
Testing : Automated testing for terraform , docker


Pretty much I have covered Python , AWS and now moving on to Terraform. I guess I have python and AWS commercial experience , however I do not with terraform or ansible. Advice would be appreciated from fellow DevOps engineers.

https://redd.it/lyitqz
@r_devops

tagging strategy

When building code (Python,C++, and Java), I always compile locally and run tests locally. When I am ready I commit to Git.

In my CI (Jenkins), it automaitcally builds the software and tests.

My question is, how do you tag? Do you have your CI tag or once you merge to master you tag there? I feel when I tag with CI, I don't have much control of the tag number -- I use the build number. But when I tag it manually and push the tags, I feel it can become cumbersome and error prone (not following a standard).

How do you tag?

https://redd.it/lyd41s
@r_devops

6 Pitfalls to Avoid while Implementing Continuous Delivery

Continuous Delivery (CD) is now ubiquitous across the industrial enterprises as the key DevOps practice for delivering software speedily, safely, and sustainably to the end-users i.e. customers. Yet, as per the state of DevOps 2019 report by Puppet, CircleCI, and Splunk, a survey response by more than 1000 engineers/DevOps or SREs/ITOps Managers suggests that nearly 60% of the organizations are still medium and low performers while implementing continuous delivery.

Delving deep into the areas that affect the performance of Continuous Delivery for greater insight into the problems led us to summarize the six pitfalls that are essential to avoid by the DevOps teams while implementing Continuous Delivery. Be it managing security and audit, orchestrating with a plethora of DevOps tools for your CI/CD pipelines, or contemplating automation for automated risk analysis,

https://redd.it/ly92j3
@r_devops

Online newspaper CI/CD pipeline

Not quite sure it's 100% devops but if someone has so idea to help to understand this, it would be nice (as a Computer Science student).

What the workflow usually look like for an online newspaper like The New York Times in order to deploy / update an article?

Like once a journalist finished to write an article, what's going on after that before to be available on the platform?

https://redd.it/ly8xtt
@r_devops

Advice for introducing DevOps practices to a company

Hi all,

Apologies if this is a repeat post.

I've recently joined a company as their sole DevOps engineer and I'm after advice on how I can implement some best practices without stepping on any toes, the current processes in place are very manual and there is no consistency to any of the work carried out.

For context, I've come from a much larger software company where I was spoilt with tools and best practices, as well as other engineers to bounce ideas off of. DevOps mentality was baked into everything we did as a company.

I guess I'm just feeling quite overwhelmed with the task at hand due to the sheer scope alone, I'm expected to work across both ongoing projects and new projects therefore I never have any time to do any housekeeping of our internal infrastructure as the new projects take precedence.

Any advice on even just a starting point would be greatly appreciated.

Thanks!

https://redd.it/lylfvx
@r_devops

What is the easiest way to push docker-compose on AWS ?

I have dockerized applications using docker-compose files (not single image Dockerfile). I need to push them to AWS. Using github-actions as CI.

I used ECS in the past for Docker, but docker-compose is not available and you need to use aws specific files which is not practical.

I don’t think ElasticBeanstalk handles docker-compose.

What is your pipeline for docker-compose apps on AWS ?

https://redd.it/lyhi98
@r_devops

How to query a docker repo on an artifactory instance for a list of available images

I have a docker repo on a jfrog/artifactory server. Is there a way Is there a way to query all the images in the repo and return a yaml file?

https://redd.it/lypytb
@r_devops

Change container instance from DEFAULT cluster to ANOTHER_CLUSTER in AMI Linux 2 ecs optimized

Good afternoon folks,

The title is self-explanatory.

I need to change a container instance that was created in DEFAULT cluster. I tried everything with no success.

Things that i tried:

1 - delete the content of checkpoint file used by ecs-agent and restart the ecs-agent container. I did it right after create ecs.config file with my target cluster name.

2- Insert the configuration ecs.config in user data of my launch configuration

EVERY ATTEMPT CREATES A CONTAINER INSTANCE IN DEFAULT CLUSTER :((((

https://redd.it/lyjsnz
@r_devops

Looking to switch into devOps from cybersecurity, what should i focus on?

Ansible seems like of no use to me -.-

View Poll

https://redd.it/lyfjcc
@r_devops

Automating provisioning of additional tenant infrastructure

Say you have a multi-tenant web application using the database per tenant approach. And each tenant requires it’s own S3 bucket as well. I can see how one would use IaC (Terraform) to bootstrap/deploy this initial application (zero tenants). But if you wanted to be able to dynamically create a new tenant automatically (from a website sign up form), what strategies or IaC tools would be used?
I feel like this is a common enough problem but I can’t find much specific information on it (or I’m not using the right search terms).

https://redd.it/ly4cw7
@r_devops

How do single tenant systems backup customer uploaded data?

I am working on a fun/learning project which is pretty much a shitty CMS that is mainly about learning and implementing best practices for DevOps/SRE and am looking for help on a design thing related to backups and DR.

Let's say I have a single-tenant CMS core that runs in a Kubernetes pod and there is the option for the user to upload files like images to the site/service which is used to display to end-users. Ideally, I'd have some way of backing these up but still keeping them secure and able to be restored in the event of a DR situation.

How can I have it so someone could upload an image that is used on the Kubernetes pod to set traffic but is still backed up and usable in the event of a disaster?

https://redd.it/ly0c02
@r_devops

Self-hosted heroku-like solution?

I’m looking for a solution where I can play around with some ideas and possibly convert some of them into paid apps. Heroku is great, but it puts unused apps to sleep and tends to get very pricey when trying to scale.

Is there a self hosted solution where I could spin up multiple NodeJS apps on a Digital Ocean droplet and assign a subdomain to each? I was looking at Flynn but development on it seems to have stalled. Something like bunny shell looks like what I want, but $50/month (on top of the digital ocean droplet) is out of my budget for experiments.

https://redd.it/lxyxfs
@r_devops

Helm chart repository on GCP

Hi,

What's everyone using to store their helm charts on GCP? Just Google Artifact Registry?

https://redd.it/lyrwii
@r_devops

Benchmarking Fluentbit vs Fluentd

Hi All, I want to benchmark the performance and resource usage of fluentd vs fluent bit? My use case is for the edge environment, we are of fluent bit designed for edge and IoT environment with limited resources available. We like to validate this by benchmarking these two tools on our servers. I am running these as daemon sets on the k3s cluster. Any tools if you guys have in mind to achieve this, that would be very helpful?  Thanks

https://redd.it/lyq4i3
@r_devops

Email Me When My Site is Down

Imagine you launch your website and out of the blue your site goes down and you have no idea. Yikes! Well, that's where https://github.com/Salaah01/website\_pinger/ comes in!

I had my website go down a few days ago and only noticed when I hardly had any traffic come onto my site https://www.bluishpink.com. And so, I've written a little bit of Python that will email me from now on whenever the site is down that is triggered by a shell script.

Just a little code you can use to email yourself if your server is down!

https://redd.it/lyplw7
@r_devops

New developer trying to understand

Hey all! I'm a very green developer (was moved into the space by way of developing an 'application' for a team which saved a ton of money). Being that a 'DevOps' team was literally formed around myself and a few individuals doing development/scripting/ coding type work, I am wondering if the functionality of our team is more just lipservice to DevOps/the company just deciding that 'the world is doing DevOps so should we'. I would like to better understand how many products, development platforms, and services a single developer typically supports. Does a typical life look like a project request team? IE: one development team services multiple other teams requests for new products/services? Or is it more like you support a single application, service, or Product your company provides, and you make sure that it stays running, optimizing it and developing new features?

https://redd.it/lxc6in
@r_devops

How do you tune for performance/diagnose bottlenecks on a server when you develop on a local machine?

I'm built a micro service architecture that works pretty good on a single workstation machine. When I deploy to a larger server and run things at full speed(it ingests data from different sources) I start to run into different bottlenecks due to CPU allocation, memory and disk iops. For example one of the services is elastic search which uses JVM and can have issues with back pressure if things go to fast so it has to be tuned.

I haven't dockerized things and know I need to. Up until this point i have been ftp'ing files to the server, running a bash script that starts all the services, with a follow up command to begin processing data.

Looking for advice/direction on how to best troubleshoot bottlenecks/tune for performance on a server, when the code is written on a dev machine.

Thanks in advance, I'm new to devops but am trying to learn fast.

https://redd.it/lxbtq5
@r_devops