GitOps with Flux and Kustomize

Hi DevOps,

Long time Redditor, but I finally decided to separate my personal account from a professional account.

Anyway, I have been writing medium posts for some time now and thought I would share them with this sub.

As of late, I have been really into GitOps. When troubleshooting problems in production, I find that the problem lies in one of three categories.

* code
* environment config
* data

If I'm doing things right, environment config related issues can be easily remedied because when my K8S cluster syncs with my source of truth, I'm back to my desired stated.

​

I wrote this article so others can try it out. Hopefully, it will help.

[https://medium.com/@airwavetechio/gitops-with-flux-and-kustomize-24dfabb8bc5d](https://medium.com/@airwavetechio/gitops-with-flux-and-kustomize-24dfabb8bc5d)

​

Thanks for reading.

https://redd.it/gdf877
@r_devops

GitLab CI vs jenkins? thoughts?



https://redd.it/gdbdrn
@r_devops

DevOps On Kubernetes responsibilities

DevOps Engineers working on kubernetes what are your roles and responsibilities?
Do you write the kubernetes helmet charts or developers do that? What parts should be responsibly of devops and what should be taken care by dev other than writing application code?

https://redd.it/gdb9tp
@r_devops

Question about running MongoDB HA on K8s

I've got a kubernetes cluster running and I've deployed MongoDB using bitnami/mongodb helm chart which created 1 primary and 2 secondary pods, mongodb service that points to the pod labeled primary and mongodb-headless service that points to all mongo pods.

If an app uses the mongodb service to write to the database then it will work only if the primary pod is up, if the primary pod goes down one of the secondaries will become primary but the service won't point to that newly elected primary since it uses labels and the app won't be able to access the database anymore. If an app use the mongodb-headless service then it will sometimes get load-balanced to a secondary pod and won't be able to write.

So my question is how can I make MongoDB always writable even if the main primary pod goes down?

https://redd.it/gd9lq0
@r_devops

Python Azure Serverless [Article]


Wrote the article, which gives an example of using Azure Python Function for obtaining Azure Datacenter and Office 365 IP addresses.

[Article link](https://github.com/groovy-sky/azure/tree/master/func-parse-cloud-00#introduction)

[Obtained data](https://strgy5exht4o56pkq.z6.web.core.windows.net/)

https://redd.it/gdcc0o
@r_devops

Free Early Access to New No-Code Data Apps Builder

Hey guys and gals!

A while ago together with my team, we developed a new no-code visual development platform for data-driven applications. We are hoping to launch it soon, and we've opened up an early access program for the most curious and enthusiastic.

With our platform, you can turn your data into useful tools by developing web apps 5-10 times faster than you would with traditional coding. It's especially great for working with multiple data sources via API and building rich UI interactions.

We developed it to save time when building web apps for our clients, so we made it easy to use but very powerful. Now we want to launch it to the public and we need your help.

If you are interested in giving our platform a try, drop a comment below and I'll send you the invite. We are looking forward to hearing your feedback and suggestions, and to see what you build with it!

Hope to see you there!

https://redd.it/gdbnnj
@r_devops

Question on designing Architecture

I am currently  working as a devops engineer, mostly working/supporting on tools during deployments and troubleshoot during downtime, but never really got a chance to design anything. 


Our techstack includes Jenkins, spinnaker, sonarqube, docker, Ansible, k8's and the list goes on. There are numerous tools and new one's coming out daily. With these many tools and technologies, if you have to architect a plan when on boarding an application. How do you plan it? What would be your suggestions to someone who is new to designing an architecture.


Thank you.

https://redd.it/gdbhzl
@r_devops

What Infrastructure as Code is and how Terraform fits into this concept

New free video course on Terraform, part 1 just released: https://www.youtube.com/watch?v=P0FUfo-Xdg4

https://redd.it/gdbg4u
@r_devops

GitOps and immutable infrastructure

What practices do your teams use to enable immutable infrastructure with CI/CD and GitOps?

If you make a change to your application, do you recreate your entire stack or just your services, leaving network consistent from deploy to deploy? What about changes to just your stack? Do you redeploy everything as well? How do you handle artifact generation to ensure the same stack and application artifact between different stages of deployment?

https://redd.it/gdnsps
@r_devops

Performance Testing With Puppeteer Cluster

Did a lil blog post on stress testing your infrastructure with headless browsers using Puppeteer, Buildkite and Ansible. Includes working code examples for you to try at home.

[https://stackchat.com/blog/puppeteer-cluster-performance-testing](https://stackchat.com/blog/puppeteer-cluster-performance-testing)

https://redd.it/gdkrrl
@r_devops

How to Get Your Digital Data Game on with DataOps

Have a look at the following article in [devops.com](https://devops.com) which covers:

* How enterprises can maximize the value of their data
* The best way to build a data optimization strategy
* How IT leaders can bring their staff up to speed on the latest data optimization techs and practices
* Leading data optimization challenges

You can read the article [here](https://devops.com/how-to-get-your-digital-data-game-on-with-dataops/)

https://redd.it/gdev6f
@r_devops

Custom Web Application vs. DXP

Hello!

I am not sure if this is the best community to post this, but I am hoping to get some guidance and opinions on the info below.

The company I work for is looking to create a custom web application to serve as a client portal and internal database to help us work faster and more efficiently.

High Level Info About the Project:

· Client portal for project management to exchange documents, sign documents, see past projects, associate products we can offer with projects

· We need to create custom decision trees and database query results based on the attributes of a client project

· Back end management of client projects for our team that allows them to log data about the client’s project and flow through the right questions to ask, things to remember, etc.

· Database of vendors we have access to that is searchable and filterable to quicken the time it takes our team to prepare proposals, find appropriate products, etc. (this part is internal only)

· Integrate with HubSpot

· Eventually we would like the client portal to have dynamic presentation capabilities that pull products in different views (part of fleshing out the database portion)

I have been working with custom web application development companies up to this point. One of them suggested a DXP as an out-of-the-box solution that could have custom modules added on to achieve our goal. I am starting to explore this option (Liferay, Core Dna, Site core, Salesforce), but am a little lost as far as customization capabilities go for these software offerings.

My questions are:

1. Is a DXP worth exploring?
2. Are there any that come highly recommended?
3. Are there any specific CMS or DXP products that would meet our needs to a client portal and database?
4. How should we look for a developer who is best suited for this type of project?

https://redd.it/gdamjg
@r_devops

OneDev 3.1 - super easy and high performance GitLab alternative

As requested by many users of OneDev, we added git over SSH and yaml based CI spec support in this version. Checkout [https://github.com/theonedev/onedev](https://github.com/theonedev/onedev) for more info.

https://redd.it/gebwf7
@r_devops

What is more stressful? DevOps or Software coding? (Software Developers)

Just curious.

https://redd.it/ge9jim
@r_devops

The basics of SQL injection and testing for them as an ops engineer

Hi, I wrote a post about the basics of SQL injection because I wanted to get to the bottom of it.
I'm mainly an Ops guy, and recently with all these "SecOps" talks, I thought I should get myself familiar with the basics and how to test my company's applications.

It's also an attempt (probably futile) of having developers pay "wholesome" attention to their own applications. Maybe some will...

[https://dev.to/prodopsio/sql-injection-for-developers-2pi](https://dev.to/prodopsio/sql-injection-for-developers-2pi)

Would love to hear any thoughts and feedback.

Cheers.

https://redd.it/ge1gm1
@r_devops

Very new to Jenkins and maven builds. Please help out

We have a project in git that is built either as snapshot or maven release. They are uploaded to different directories in our artifactory. We have the option to set profiles before build is triggered. I can see when a maven release is done, the profile name is tagged along to all uploaded artifacts. However, in snapshot builds, this does not occur. What can I do?

https://redd.it/ge142v
@r_devops

What are the best AWS devops courses?

What are the best AWS devops courses? I am not sure why, but most AWS courses are not enough hands-on. Ideally, everything we see on the screen should be put in code so that we don't need to be familiar with the UI on different AWS services and deploy everything to AWS and configure it by entering a simple command line to execute some scripts. Is there any course like that or is it too much to ask?

https://redd.it/gebcg3
@r_devops

Real DevOps vs Wanting to do DevOps (upper management faults)

I work at a company that doesn't really do DevOps. We basically support Corporate (internal tools, email, etc) and Production (customer / revenue generating stuff.) with some automation.

When I look at DevOps job posting, I mainly see that it concentrates on Production (based on the above assumption) and NOT corporate back end stuff. For me it's very frustrating when management wants us to do pipelines and Inf as Code but has no plan to hand off the corporate stuff because the other team that should be doing it doesn't take it on and it falls back to me. So I get told to do IaC and I am currently trying to create a Terraform Module, but then I get pulled into stupid crap with Office 365 because no one else manages it. Has anyone been in this situation and how did you get management to realize if they are really serious about IaC, they need to start leading how teams should be developed?

https://redd.it/ge08m8
@r_devops

[Article] Putting Security into The IaC Pipeline

Everyone talks about shifting security "left".. but how can you do that and what does it mean? Well in essence it's building security checks/policies/controls into the development life-cycle as early as possible, rather than bolting on security tools in production. It means validating code templates and doing security checks as early and often as possible. One great example of that is scanning IaC templates for misconfigurations and issues before build cloud infrastructure. Here is an article that I wrote on this topic and would like some additional feedback if possible for sure:

[https://medium.com/@fernando0stc/putting-security-into-the-iac-pipeline-4de98f88ad24?source=friends\_link&sk=d628526e4758cde258901447c7453f4b](https://medium.com/@fernando0stc/putting-security-into-the-iac-pipeline-4de98f88ad24?source=friends_link&sk=d628526e4758cde258901447c7453f4b)

https://redd.it/ge8bsg
@r_devops

Let me tell you why I think GCP is better than AWS and you tell me where you agree or disagree.

​

>This post is not just a rant about my predilection for GCP. I genuinely want to read your opinions, especially those of you who have used both platforms like me (I have about two years of experience with each platform). And if you feel I have said something that you think it's wrong or not factually accurate, also please let me know, I am happy to be educated or informed in a civil
and constructive manner. I have no affiliation with Google whatsoever, this is entirely my opinion based on my perceptions of the merits of both platforms. I am also conscious that there are more tools and services that I could ever expect to use in a lifetime so it is entirely plausible I am missing areas where AWS by far outshines GCP due to my limited experience or missing even more areas where GCP is better.

​

* Where do you agree?
* Where do you disagree?
* Can you provide examples of your own where AWS or GCP was better than the other?

​

**Ikea for Cars**

If AWS and GCP were both car companies and you wanted to purchase a car, AWS would give you the wheel, a chunky verbose manual and the keys and then tell you to go to twenty different shops they also own to get the rest of the components and ask you to put them together yourself the best you can. Sure, maybe you can hire a service and get tools to automate this part, but it still falls on you to assemble these components together and maintain the automation.

The experience of GCP on the other hand is more like collecting the car keys and driving off from the parking lot, with the option of dismantling and customising the car if you wish, but the default is a fully built functioning car so you can achieve your objectives, which is driving around, not assemble the car.

My first experience working with AWS, before I had much to compare it to, was brief and I didn’t like it; I felt the interface and the way tools and settings were organised was counter-intuitive and weird. 

For example assigning a static ip to a server was just bizarre, I kept looking for ways to assign the static ip without knowing that it was meant to be called elastic ip and hidden away in a separate set of menus. Then these elastic ips were part of a different pool of ips than the ones that were assigned dynamically. To my dismay I had to stop a production server to change the ip and also change the DNS pointing to that new ip, this was because my predecessor hadn’t assigned an static ip to the server, my bet is that he probably gave up after ten minutes trying to figure out that it was called an elastic ip.

My second experience working with AWS was after a year and a half working with GCP and now by comparison I really couldn’t stand AWS, it took me a few months to get accustomed back to use it and I remember that in my first few weeks I actually considered quitting and just accepting roles with GCP.

It’s not that AWS is harder to use than GCP, it’s that it is needlessly hard; a disjointed, sprawl of infrastructure primitives with poor cohesion between them. A challenge is nice, a confusing mess is not, and the problem with AWS is that a large part of your working hours will be spent untangling their documentation and weeding through features and products to find what you want, rather than focusing on cool interesting challenges.

Let’s just go over a few of the things that make AWS such a pain to use and how it compares with GCP. 

**Accounts vs Projects**

One of the first differences that strikes you when going from GCP to AWS is accounts vs projects. In GCP you have one master account/project that you can use to manage the rest of your projects, people log in with the google account and then you can set permissions to any project however you want. So you can have a dev project, a production project, etc. All of this works out of the box and there is absolutely nothing additional for you to do. 

In AWS you have accounts, and each account has a separate set of users. There are

ways to connect these accounts so your user has permissions on other accounts. One way of doing this is creating a master users account and then adding roles that can be assumed in all other accounts by this master account. 

This is not only a pain to set up, it’s very painful to use as well. For example when using terraform scripts you need to coordinate multiple roles across several modules if you need to work across multiple accounts. 

**Command Line Interface Tools (CLI tools)**

Let’s just compare what you have to do in order to use GCP cli compared to AWS provided we are using 2FA and a couple of different projects/accounts. 

In GCP after you [install the Google SDK](https://cloud.google.com/sdk/docs/quickstart-linux), all you need to do is run gcloud init, which redirects you in the browser to a Google login page. Here you can login with your two factor authentication (which if you have an android phone is as easy as unlocking the phone and pressing okay) and you are done. Your login session is attached to your Google session so when you kill this session you are logged out— very simple.

In AWS you need to create a token that you can use to login with your CLI, simple enough, right? But now we want to use two factor auth, and this is where the fun begins.

After you login with your token you then need to [create a script](https://github.com/asagage/aws-mfa-script) to give you a 12 hour session, and you need to do this every day, because there is no way to extend this.

Okay, but that’s not a big deal, you say, after all it’s just a code that you need to input once a day and you can get on with your day after that.

But wait, there is more! If you need to assume roles in another account, you need to create yet [another script](https://github.com/Integralist/Shell-Scripts/blob/master/aws-cli-assumerole.sh) that creates another profile for you to use.

That’s one step plus two scripts, plus many steps in between. And sure, you can automate much of this or use someone else’s tools you find online (that you most likely will need to tweak), but why? Why do we have to do so much work to use AWS? Why can’t AWS abstract away this pain away from you in the way that Google has done?

**Web User Interface**

If using the CLI is too painful for you, you can always log in to the portal and use their user interface, although I don’t recommend you do this for everything, in fact I recommend you use it the least possible and only for reference and to check status of your services.

AWS interface looks like it was designed by a lonesome alien living in an asteroid who once saw a documentary about humans clicking with a mouse. It is confusing, counterintuitive, messy and extremely overcrowded. 

I can’t even count the times I’ve gotten lost or stumped in the AWS console, sometimes over the most stupid details, like missing that there was a next button hidden on a weird corner. Or trying to use search bars that can only search prefixes (WTF?)

But the biggest frustration I have from the AWS console is how you are always overwhelmed with scores of settings and options you need to fill in before actually provisioning anything.

One example that comes to mind is when someone at work said we should use codebuild/codedeploy to replace Jenkins for ECS deployments. The first engineer tried, he got stuck, the second engineer tried, he got stuck, I tried for hours and I got stuck… in the end I just gave up for lack of wanting to spend any more time on a tool that doesn’t seem to be that popular for CI/CD that I thought was meant to make life easier. 

Amazon seems to be particularly terrible at interfaces in almost all of their products though. For example in my Smart TV the Netflix app works flawlessly and is intuitive to use whereas the Amazon Prime app is an abomination, you are constantly accidentally pressing the wrong button or getting lost or the subtitles are often out of sync. 

In a rant that a Google engineer who had worked at Amazon [wrote a while back](https://gist.github.com/chitchcock/1281611) he explained the issue with