A Devops Dissertation Idea

Hi Reddit,

I'm approaching the end of my DevOps course and am required to write a dissertation covering a relevant / interesting research question.

My idea is to compare multi-cloud IaC tools (terraform, pulumi, ansible) in terms of performance / ease of development etc.

I'm trying to think of a good use case to implement with each of these tools on AWS, Azure and GCP. Wondering if any person with experience in the field has a novel idea for this? I have no background with cloud platforms so any suggestions would be appreciated 🙂

https://redd.it/nggjz6
@r_devops

CI/CD ideas for DevOps student

Hi!

I have been studying DevOps technologies for last three month. And now I need to choose what final project should I deploy. It's some kind of final task that I need to do to getting to the next course. But I don't have any ideas what to deploy.

I know only Spring/PetClinic, but I don't know java and I have not much time to learn it. Maybe somebody knows some open source project which I will be able to deploy with CI/CD instruments like Jenkins, Ansible and Terraform.

Thanks for advices!

https://redd.it/ngd2ss
@r_devops

Is running a Database like MySQL in a docker container any good?

Hello DevOps Enthusiasts

I am looking for some guidance here, what would be the best practice to run a database - directly on the host or in a container?

Host seems to be the answer to me, but I would like to know if anyone running in containers for production environments and are there any pitfalls/benefits doing this.

Thanks

https://redd.it/ngce5d
@r_devops

The configuration system that has to exist

I feel like I'm taking crazy pills. At work, we've got configuration scattered in various systems. Some application configuration is set in puppet (hiera) which we use for the bulk of our system configuration management. Secrets are mixed in there, so access for view or edit by devs is off limits. Some configuration is set in a git repo that holds a tsv file, which is actually more useful than you might think. Other configuration is scattered among various code repos like you might expect.

I am looking for a configuration to bind them all together, and this is what I'm looking for:

management of secrets (vault would be ideal)
version control of all config (need to know who changed what when, rollback, etc.)
usable by both puppet (hiera) and various other tools (cli / REST api / etc.)
flexible acls/permissions to various parts of the tree

I feel like I'm missing the forest for the trees, because I know I'm not unique, but I can't find this system. Consul or etcd is nice for making configuration available, but doesn't have the sort of version control I want (I'm also not really in need of service discovery). Git repos with structured config might work with hiera, but doesn't have the kind of api or secret management I'd like. The closest I came to ideal was Jerakia (https://jerakia.io/) but this project looks to be pretty much abandoned.

Can someone point me towards a system that can do what I need? Any clues to put me in the right direction would be great - I just need a push.

https://redd.it/ngm8rc
@r_devops

Logging the 12 Factor App Way

So the 12 Factor App methodology recommends logging everything to standard out and using log routers to send logs to their final destination. I definitely like the idea, it abstracts the dependency of the logging technology out of the application, removes the need for the developer to be familiar with logging technology, makes logging more portable, etc.

Opinions of the 12fa aside, I'm looking for some good examples of the "log routers" they are referring to. I see them reference LogPlex (which is depricated) and fluentd on the 12fa website but I'm curious to seewhat else is out there and what experiences others have had with this approach.

Thanks in advanced!

https://redd.it/ng8nac
@r_devops

I am in a DEVOPS role, and can't code.

Good Morning,

First time poster. I recently as of December got a new role within my company. I went from being Tier 2 support for network escalations and a team technical lead to a Devops role.

I got this role because I know networking (10+ years experience, CCNA, Security+) So I am now 6 months in. I am a good fit I was actually loaned to this team for half a year and they wanted to keep me so I can do the job. But I want to grow and go above and beyond.

Python will be a huge asset and I want to learn it but I have no idea where to start. I have learned to bash script as part of my role (testing devices and automating tests) So I am not a complete noob.

Where can I start learning Python? I have tried to use guides online and libraries like paramiko / netmiko but they don't work well in my environment due to SSH encryption differences. So instead of going down that route I want to learn it as it pertains to Networking Automation etc and not rely on unsupported libraries and copy and pasting code.

https://redd.it/ng9451
@r_devops

smee.io to forward webhooks to Jenkins behind firewall?

Trying to setup webhooks between Jenkins and Bitbucket cloud, there's several proxy services out there but smee is the only one that appears to support selfhosting. Anyone have experience with it specifically or recommendations for another way around this issue?

https://redd.it/ng8wxo
@r_devops

Workflow for CircleCI and Terraform

I have recently begun to use the Terraform API to be able to trigger workflows from our CICD CircleCI. One of the obstacles we are trying to overcome is preventing the entire Terraform workflow from Circle from triggering with every single commit. We are utilizing filters, and the apply will only run on the main branch, but we want to be sure that Circle is not being abused/run with every commit on the backend.

We are currently using filters to prevent the workflow from running a Terraform Apply unless it is a merge into a master/main branch; however, we want our users to be able to test their Terraform. Right now in their development branches, the workflow is triggered with every commit. We want the workflow to be run only when intentionally "specified" (Ideally we would want this to be done from git command line, not logging into the circle CI interface and running the workflow). This way a user can make 10 commits to a branch before they test without triggering 10 workflows behind the scenes.

I thought about using branches with prefixes which trigger the terraform plan workflow in Circle, but that wont prevent a user from making half a dozen commits to that branch and triggering half a dozen workflows. It doesn't look like tagging is best method of triggering the workflow either. Is there a best practice or a similar use case that you all are aware of for this?

https://redd.it/ng6z77
@r_devops

How can I become a really bad software developer?

Hello! I want to share with you a true my story.

This has got to take the cake in terms of the nightmares that I’ve seen working with developers in the past. The year is 2014 and our firm was just assigned a large Sharepoint project for an intranet redesign. We are in the middle of the project with design being completed, initial development checked off and we are close to ending the final phase of development before going into UAT.

Over the course of this whole project I had two developers who were completely bashing heads the whole time. They disagreed on everything right from technologies, development stacks, best practices, etc etc. Just a bad situation all round from an ease of work standpoint. It just wasn’t working!

The development director/project manager at the time had tried to contain the madness by assigning them different modules to develop and basically stay out of each other’s way. As with all things in life however, sometimes you have to deal with things you don’t like, and there came a time where these two individuals absolutely had to work together on a single module. We expected there to be absolute and utter chaos but nothing like what ended up happening.

It’s 7:30am on a Monday morning and my project manager calls me absolutely freaking out on the phone. It’s incoherent gibberish so I tell her to slow down and tell me what’s going on. She tells me that a bulk of the code has been completely re-written without approval and version control (Team Foundation Server in this case) has been messed with in a way that doesn’t seem to allow recovery (atleast at the outset) to revert to the previous work. She tells me she spoke with the technical architect and they both agreed it’s a serious problem with the potential to derail the entire project. I tell them I’m rushing in to the office for a meeting for us to figure out what’s happening.

A half hour later I’m jumping in a cab to book it to the office. Walk in and everyone has a look of dread on their face. I sit down with the TA & PM and we start going through what’s been going on. Apparently it looks like over the weekend someone went in and deleted the bulk of the codebase and rewrote many of the critical modules in Javascript. Now I’m not a developer, but I was made to understand that Sharepoint SDKs are primarily written in C#/.NET and this individual had stripped it all out and rewritten into JS. Even before looking at the logs we knew who to grab.

9:30am comes along and in walk the two developers. I tell them to meet us in the conference room to figure out what the hell is going on. Turns out the JS developer had felt so strongly about the codebase that he went in and had been rewriting the modules for a while on his own time for a while. He was able to wrap up the majority of the work and over the weekend had time to go in merge it in and delete everyone else’s stuff. He seemed proud of accomplishing this because according to him this was the right way of doing it and we didn’t know what we were doing. While this was going on, the rest of us sat there listening with our mouths down to the floor. Now I’ll give him this much, he was an amazing Javascript dev and probably one of the best developers I’ve ever worked with from a skill standpoint, but he was terrible to work with because he had no soft skills or interpersonal understanding of how to act in the workplace.

Not only did this guy have no respect for the process, his coworkers, or the company, he completely jeopardized the project and opened all of us up to some serious liability (think millions of $$$ in lost revenue/paybacks if it failed). Besides that he messed with the version control in a way that prevented an easy reverting to the old codebase, so that was malicious as well. Ultimately, my boss made the decision to not fire him (don’t know what he was thinking), but the guy ended up quitting a short while later anyways.

So want to be a really bad…. wait, no… absolutely god awful developer? Do what this guy did! I promise you you

will forever piss off anyone that works with you and potentially derail your career.

https://redd.it/nfzs9d
@r_devops

Difference between Microsoft Azure Security Center and Azure Sentinel

Many Cloud Engineers often fail to get the difference between Azure Security Center (ASC) and Azure Sentinel. These two products look very comparative at first and both are offered by Microsoft to secure your Azure infrastructure to the best of their abilities. There are a few fundamental explanations behind this confusion and in this article, we will have a closer look at what makes these two stand apart from each other.

Azure Security Center vs. Azure Sentinel

Azure Security Center is a security management framework offered by Microsoft to Azure clients. It helps the Azure infrastructure by giving visibility and authority over the security of Azure sources such as Virtual Machines, Cloud Services, Azure Virtual Networks, and Blob Storage. 

Whereas, Azure Sentinel is a cloud-native, security information event management (SIEM) and security orchestration automated response (SOAR) solution provided by Microsoft to assist clients with a birds-eye view across a certain project.

Security Management

With Azure Security Center (ASC), you can deal with your cloud security to help prevent any cyber-attacks and misconfiguration by strengthening your security for various responsibilities deployed in Azure or on-premises. When discussing cloud security management, we are referring to three significant factors: 

• Visibility

• Monitoring

• Compliance 

Azure Security Center extends its security management activities to counter the latest risks on cloud platforms to ensure against cyber-attacks for workloads deployed in Azure, on-premises, or third-party cloud services, for example, GCP, AWS, and so on. 

But with Azure Sentinel, we can have smarter security management and risk management for alert detection, risk visibility, proactive monitoring, and threat response for cutting edge and refined cyber-attacks. 

You can also have Azure Security Center enabled in your membership to receive security alerts to Azure Sentinel from Azure Security Center. Azure Sentinel leverages Machine Learning (ML) and AI (Artificial Intelligence) to make threat monitoring more brilliant. Azure Security Center can generate alarms for various sorts of resources deployed – taking your security a step further.

Issues & Challenges

Azure Security Center tends to solve the following security issues and challenges: 

• Consistently evolving workloads: While users can accomplish more on the cloud, the workloads keep changing constantly. ASC takes care of all the dynamic workload by itself.

• Progressively complex attacks: As users run their jobs on the public cloud, attacks are increasing. Doing so could open them to more weaknesses if they don't follow best security practices. Azure Security Center can help deal with that task. 

• Shortage of security skills: A high number of safety alerts and cautioning frameworks can overpower security administrators, particularly if they're not experienced and skilled enough. Be that as it may, Azure Security Center can help administrators deal with such attacks and threats.

Whereas, Azure Sentinel deals with the following security issues and challenges: 

• Automation and Orchestration: Sentinel supports automated threat responding frameworks called "playbooks". Playbooks, based on Azure Logic Apps, set up a series of procedures to run when the situation arises. Administrators can make their playbooks using the Logic App tools. 

• Deep Analysis of Issues: An amazing element of Sentinel is the ability to do "hunting" and deep analysis of issues. It shows triggered alerts’ explanation. In this way, the administrator seeing it can appoint the case to somebody with proper reasoning.

Use cases of Azure Sentinel

• In Microservices architecture Application logging will flood the activity/event logging with various types of logs from various Azure resources. Sentinel will be handy when we need to build intelligent threat alert system using those tons (GB/TB) of logs

• Institutive graph helps to analyze / investigate threats 

•

Sentinel allows to build automation to respond on threat detection and takes necessary action to prevent it further. E.g. if number for 401 (Unauthorized) errors are increased then it can automatically block specific Ips 

Read More : https://www.taliun.com/difference-between-microsoft-azure-security-center-and-azure-sentinel

https://redd.it/ng6t4l
@r_devops

Weaveworks Flux and Security

A bit new in looking at GitOps and Flux for Kubernetes.

In the recent classes I've taken, there are two approaches to using flux.

As a tool to automatically deploy updated containers. The A Cloud Guru site was using this as their examples. You make a change, such as a version/tag change, and the new container is deployed automatically.

As a tool to maintain cluster configurations. The DevOps Guy on youtube brought this one up and it's an interesting idea. I currently maintain cluster configurations via git and ansible playbooks but being able to automatically update a cluster configuration can be beneficial.

My question though is related to security. Several of my configurations have passwords or more recently, SSL certificates (CA, key, and cert). One of the problems is the data needs to be formatted or is part of a yaml file vs used by a container. For example, one of the passwords is in an embedded data file used by spring-boot so needs to be part of the overall string. Certificates are also formatted with 6 spaces before each line.

I read about the bitnami secure secrets CRD but I don't think it would apply in these use-cases. Bitnami's tool seems to be something where a container/pod mounts a Kubernetes secret vs an embedded string.

https://redd.it/ng64z6
@r_devops

Measuring Software Quality Using Quality Metrics

The tempos of the software development process are growing every hour. Amazon uses to deploy software updates through their Apollo deployment service every 11.7 seconds. Etsy has a fully automated deployment pipeline that does about 50 deployments a day.

With deadlines becoming tougher every day, the product quality requirements are growing as well. Under these conditions, maintaining and constantly improving the product quality becomes a matter of primary importance.

In this article, we pay attention to the importance of software quality management. You will learn about the quality metrics used for assessing the software performance and ways to maintain the quality on the proper level. We also discuss the best practices of maintaining the software quality that the Jelvix team follows during product development for our customers.

https://redd.it/ng2ouo
@r_devops

Pulumi, do you use it and what's your preferred lang?

Just checking out Pulumi and yeah, I am slightly more motivated to get into this than Terraform. Are you happy with Pulumi and in what lang are you using it?

https://redd.it/nfyntd
@r_devops

To all experienced devops, how would you break apart microservices to support colocations?

I'm in a bit of a bind and am hoping the devops Jedis out there can guide me in the right direction.

Tldr: Currently saturating my 940/35mbps line, but have access to 1gb symmetrical fiber. Trying to figure out if my microservices with a message queue of 10k+ per second will work going from fiber to my connection or if I need to create a service to aggregate/de-aggregate messages because of the high volume.

What my setup looks like:
I built a hobby project in golang using nsq(similar to rabbitmq or kafka) as a message queue for microservices. Currently it runs locally on 2 machines. Machine 1 gets data from the internet and does heavy processing, it then sends 100-1000 of smaller messages(~1-250kb) messages to machine 2 which holds the messages and writes them at regular intervals to databases like mongodb and elastic search. Machine 2 then sends new info to machine 1 and the cycle repeats.

The problem:
I have 4 more machines available to use, however with only 1 machine fetching data I have already saturated 1/3 the download and 100% of the 35mbps of the upload; adding more hardware just increases latency not data throughput.

What I would like to do:
I have a family member with a 1gb symmetrical fiber that is happy to let me use their internet full time and stick a couple machines there. I would like to setup all 5 machines there which report back to the database machine at my house.

What I need guidance on:
Right now I'm passing 100-1000 messages (1-250kb each) per second through my message queue between 2 local machines. But I think if I make it 5x bigger and try passing 5000 messages per second(~400mbps)from the symmetrical fiber to my house there will be issues with data loss/communication because of how many there are. Is this right, wrong or does it depend?

What I think might be a solution:
Create an additional microservices which aggregates the messages at the fiber location, makes a zip file, uploads it to my home database machine, unzips it, parses the messages and then sends them off again locally.

Side notes:
Because of the volume of data and processing requirements, cloud is not an option. There are a dozen message queues that are tightly coupled between the database and the worker machine in a feedback loop which can't be undone. I'm already aggregating messages at optimal points and am writing to the DB in batches. Family member's house is a 70 min drive one way and I really don't want to mess around with driving back and forth trying to get it working initially. Both of us also have static ip's.

Thoughts, suggestions, ideas?


Thanks in advance, this is completely unknown territory for me and every little bit helps.

https://redd.it/nfzqz2
@r_devops

Cache MySQL database locally

Looking for a solution where I could have a remote MySQL database and keep its replica locally, in case remote database goes down results would be taken from locally cached database.
I was thinking that ProxySQL is capable of doing that? But after enabling cache and turning off remote database it just spits errors that the backend is unreachable.

Any ideas?

https://redd.it/nh41aw
@r_devops

Confused on how to write my tagging stage on Jenkins script

Hi all, I am working on the tagging stage of my pipeline. I am confused on how to get the following and then append them so I can tag on bitbucket:

1.version# in package.json
2.build number
3.branch name

https://redd.it/nh52sd
@r_devops

How to deploy a multi-container app?

Hi there, newbie developer here.

I am developing a learning application with Docker containers, with the idea of deploying the frontend React container to Vercel and the three backend containers (node API, postgres and redis) to Digital Ocean, using docker-compose and github actions for my deployments.

So far, I have only deployed single containers to Heroku with a couple simple commands and I have dabbed a bit with Nginx, but I have no clue of the procedure for deploying this kind of multi container (and multi host?) apps. I have read (very little) about Kubernetes, but it feels overkill and overcomplicated for what I want to do and it makes me think it might not be the usual way of doing these things.

Any tips on the steps to follow or a starting point to being my research?

Cheers =)

https://redd.it/nh4ijy
@r_devops

Getting a repeatable build, every time

Hey DevOps fans, I spent a lot of time writing this article about best practices for managing build scripts in a growing organization. I'm hoping it would help someone be better at build engineering.

It's basically a collection of tips and tricks we learned over the years about how to make use of Makefile, Dockerfile, and Bash to make scripts understandable and repeatable.

Curious what you think! Feedback on how to improve the article is most welcome!

Article --> Getting a repeatable build, every time

https://redd.it/nh393b
@r_devops

Setting up server from scratch for hosting multiple web applications?

I am a developer but I have to setup a linux server from scratch for hosting dockerized web applications along with infrastructural things like ELK, Databases, Agents, etc. At the top of my head it was k8s but I am interested to know what others would suggest.

https://redd.it/ngxvvc
@r_devops