DevOps On Kubernetes responsibilities

DevOps Engineers working on kubernetes what are your roles and responsibilities?
Do you write the kubernetes helmet charts or developers do that? What parts should be responsibly of devops and what should be taken care by dev other than writing application code?

https://redd.it/gdb9tp
@r_devops

Question about running MongoDB HA on K8s

I've got a kubernetes cluster running and I've deployed MongoDB using bitnami/mongodb helm chart which created 1 primary and 2 secondary pods, mongodb service that points to the pod labeled primary and mongodb-headless service that points to all mongo pods.

If an app uses the mongodb service to write to the database then it will work only if the primary pod is up, if the primary pod goes down one of the secondaries will become primary but the service won't point to that newly elected primary since it uses labels and the app won't be able to access the database anymore. If an app use the mongodb-headless service then it will sometimes get load-balanced to a secondary pod and won't be able to write.

So my question is how can I make MongoDB always writable even if the main primary pod goes down?

https://redd.it/gd9lq0
@r_devops

Python Azure Serverless [Article]


Wrote the article, which gives an example of using Azure Python Function for obtaining Azure Datacenter and Office 365 IP addresses.

[Article link](https://github.com/groovy-sky/azure/tree/master/func-parse-cloud-00#introduction)

[Obtained data](https://strgy5exht4o56pkq.z6.web.core.windows.net/)

https://redd.it/gdcc0o
@r_devops

Free Early Access to New No-Code Data Apps Builder

Hey guys and gals!

A while ago together with my team, we developed a new no-code visual development platform for data-driven applications. We are hoping to launch it soon, and we've opened up an early access program for the most curious and enthusiastic.

With our platform, you can turn your data into useful tools by developing web apps 5-10 times faster than you would with traditional coding. It's especially great for working with multiple data sources via API and building rich UI interactions.

We developed it to save time when building web apps for our clients, so we made it easy to use but very powerful. Now we want to launch it to the public and we need your help.

If you are interested in giving our platform a try, drop a comment below and I'll send you the invite. We are looking forward to hearing your feedback and suggestions, and to see what you build with it!

Hope to see you there!

https://redd.it/gdbnnj
@r_devops

Question on designing Architecture

I am currently  working as a devops engineer, mostly working/supporting on tools during deployments and troubleshoot during downtime, but never really got a chance to design anything. 


Our techstack includes Jenkins, spinnaker, sonarqube, docker, Ansible, k8's and the list goes on. There are numerous tools and new one's coming out daily. With these many tools and technologies, if you have to architect a plan when on boarding an application. How do you plan it? What would be your suggestions to someone who is new to designing an architecture.


Thank you.

https://redd.it/gdbhzl
@r_devops

What Infrastructure as Code is and how Terraform fits into this concept

New free video course on Terraform, part 1 just released: https://www.youtube.com/watch?v=P0FUfo-Xdg4

https://redd.it/gdbg4u
@r_devops

GitOps and immutable infrastructure

What practices do your teams use to enable immutable infrastructure with CI/CD and GitOps?

If you make a change to your application, do you recreate your entire stack or just your services, leaving network consistent from deploy to deploy? What about changes to just your stack? Do you redeploy everything as well? How do you handle artifact generation to ensure the same stack and application artifact between different stages of deployment?

https://redd.it/gdnsps
@r_devops

Performance Testing With Puppeteer Cluster

Did a lil blog post on stress testing your infrastructure with headless browsers using Puppeteer, Buildkite and Ansible. Includes working code examples for you to try at home.

[https://stackchat.com/blog/puppeteer-cluster-performance-testing](https://stackchat.com/blog/puppeteer-cluster-performance-testing)

https://redd.it/gdkrrl
@r_devops

How to Get Your Digital Data Game on with DataOps

Have a look at the following article in [devops.com](https://devops.com) which covers:

* How enterprises can maximize the value of their data
* The best way to build a data optimization strategy
* How IT leaders can bring their staff up to speed on the latest data optimization techs and practices
* Leading data optimization challenges

You can read the article [here](https://devops.com/how-to-get-your-digital-data-game-on-with-dataops/)

https://redd.it/gdev6f
@r_devops

Custom Web Application vs. DXP

Hello!

I am not sure if this is the best community to post this, but I am hoping to get some guidance and opinions on the info below.

The company I work for is looking to create a custom web application to serve as a client portal and internal database to help us work faster and more efficiently.

High Level Info About the Project:

· Client portal for project management to exchange documents, sign documents, see past projects, associate products we can offer with projects

· We need to create custom decision trees and database query results based on the attributes of a client project

· Back end management of client projects for our team that allows them to log data about the client’s project and flow through the right questions to ask, things to remember, etc.

· Database of vendors we have access to that is searchable and filterable to quicken the time it takes our team to prepare proposals, find appropriate products, etc. (this part is internal only)

· Integrate with HubSpot

· Eventually we would like the client portal to have dynamic presentation capabilities that pull products in different views (part of fleshing out the database portion)

I have been working with custom web application development companies up to this point. One of them suggested a DXP as an out-of-the-box solution that could have custom modules added on to achieve our goal. I am starting to explore this option (Liferay, Core Dna, Site core, Salesforce), but am a little lost as far as customization capabilities go for these software offerings.

My questions are:

1. Is a DXP worth exploring?
2. Are there any that come highly recommended?
3. Are there any specific CMS or DXP products that would meet our needs to a client portal and database?
4. How should we look for a developer who is best suited for this type of project?

https://redd.it/gdamjg
@r_devops

OneDev 3.1 - super easy and high performance GitLab alternative

As requested by many users of OneDev, we added git over SSH and yaml based CI spec support in this version. Checkout [https://github.com/theonedev/onedev](https://github.com/theonedev/onedev) for more info.

https://redd.it/gebwf7
@r_devops

What is more stressful? DevOps or Software coding? (Software Developers)

Just curious.

https://redd.it/ge9jim
@r_devops

The basics of SQL injection and testing for them as an ops engineer

Hi, I wrote a post about the basics of SQL injection because I wanted to get to the bottom of it.
I'm mainly an Ops guy, and recently with all these "SecOps" talks, I thought I should get myself familiar with the basics and how to test my company's applications.

It's also an attempt (probably futile) of having developers pay "wholesome" attention to their own applications. Maybe some will...

[https://dev.to/prodopsio/sql-injection-for-developers-2pi](https://dev.to/prodopsio/sql-injection-for-developers-2pi)

Would love to hear any thoughts and feedback.

Cheers.

https://redd.it/ge1gm1
@r_devops

Very new to Jenkins and maven builds. Please help out

We have a project in git that is built either as snapshot or maven release. They are uploaded to different directories in our artifactory. We have the option to set profiles before build is triggered. I can see when a maven release is done, the profile name is tagged along to all uploaded artifacts. However, in snapshot builds, this does not occur. What can I do?

https://redd.it/ge142v
@r_devops

What are the best AWS devops courses?

What are the best AWS devops courses? I am not sure why, but most AWS courses are not enough hands-on. Ideally, everything we see on the screen should be put in code so that we don't need to be familiar with the UI on different AWS services and deploy everything to AWS and configure it by entering a simple command line to execute some scripts. Is there any course like that or is it too much to ask?

https://redd.it/gebcg3
@r_devops

Real DevOps vs Wanting to do DevOps (upper management faults)

I work at a company that doesn't really do DevOps. We basically support Corporate (internal tools, email, etc) and Production (customer / revenue generating stuff.) with some automation.

When I look at DevOps job posting, I mainly see that it concentrates on Production (based on the above assumption) and NOT corporate back end stuff. For me it's very frustrating when management wants us to do pipelines and Inf as Code but has no plan to hand off the corporate stuff because the other team that should be doing it doesn't take it on and it falls back to me. So I get told to do IaC and I am currently trying to create a Terraform Module, but then I get pulled into stupid crap with Office 365 because no one else manages it. Has anyone been in this situation and how did you get management to realize if they are really serious about IaC, they need to start leading how teams should be developed?

https://redd.it/ge08m8
@r_devops

[Article] Putting Security into The IaC Pipeline

Everyone talks about shifting security "left".. but how can you do that and what does it mean? Well in essence it's building security checks/policies/controls into the development life-cycle as early as possible, rather than bolting on security tools in production. It means validating code templates and doing security checks as early and often as possible. One great example of that is scanning IaC templates for misconfigurations and issues before build cloud infrastructure. Here is an article that I wrote on this topic and would like some additional feedback if possible for sure:

[https://medium.com/@fernando0stc/putting-security-into-the-iac-pipeline-4de98f88ad24?source=friends\_link&sk=d628526e4758cde258901447c7453f4b](https://medium.com/@fernando0stc/putting-security-into-the-iac-pipeline-4de98f88ad24?source=friends_link&sk=d628526e4758cde258901447c7453f4b)

https://redd.it/ge8bsg
@r_devops

Let me tell you why I think GCP is better than AWS and you tell me where you agree or disagree.

​

>This post is not just a rant about my predilection for GCP. I genuinely want to read your opinions, especially those of you who have used both platforms like me (I have about two years of experience with each platform). And if you feel I have said something that you think it's wrong or not factually accurate, also please let me know, I am happy to be educated or informed in a civil
and constructive manner. I have no affiliation with Google whatsoever, this is entirely my opinion based on my perceptions of the merits of both platforms. I am also conscious that there are more tools and services that I could ever expect to use in a lifetime so it is entirely plausible I am missing areas where AWS by far outshines GCP due to my limited experience or missing even more areas where GCP is better.

​

* Where do you agree?
* Where do you disagree?
* Can you provide examples of your own where AWS or GCP was better than the other?

​

**Ikea for Cars**

If AWS and GCP were both car companies and you wanted to purchase a car, AWS would give you the wheel, a chunky verbose manual and the keys and then tell you to go to twenty different shops they also own to get the rest of the components and ask you to put them together yourself the best you can. Sure, maybe you can hire a service and get tools to automate this part, but it still falls on you to assemble these components together and maintain the automation.

The experience of GCP on the other hand is more like collecting the car keys and driving off from the parking lot, with the option of dismantling and customising the car if you wish, but the default is a fully built functioning car so you can achieve your objectives, which is driving around, not assemble the car.

My first experience working with AWS, before I had much to compare it to, was brief and I didn’t like it; I felt the interface and the way tools and settings were organised was counter-intuitive and weird. 

For example assigning a static ip to a server was just bizarre, I kept looking for ways to assign the static ip without knowing that it was meant to be called elastic ip and hidden away in a separate set of menus. Then these elastic ips were part of a different pool of ips than the ones that were assigned dynamically. To my dismay I had to stop a production server to change the ip and also change the DNS pointing to that new ip, this was because my predecessor hadn’t assigned an static ip to the server, my bet is that he probably gave up after ten minutes trying to figure out that it was called an elastic ip.

My second experience working with AWS was after a year and a half working with GCP and now by comparison I really couldn’t stand AWS, it took me a few months to get accustomed back to use it and I remember that in my first few weeks I actually considered quitting and just accepting roles with GCP.

It’s not that AWS is harder to use than GCP, it’s that it is needlessly hard; a disjointed, sprawl of infrastructure primitives with poor cohesion between them. A challenge is nice, a confusing mess is not, and the problem with AWS is that a large part of your working hours will be spent untangling their documentation and weeding through features and products to find what you want, rather than focusing on cool interesting challenges.

Let’s just go over a few of the things that make AWS such a pain to use and how it compares with GCP. 

**Accounts vs Projects**

One of the first differences that strikes you when going from GCP to AWS is accounts vs projects. In GCP you have one master account/project that you can use to manage the rest of your projects, people log in with the google account and then you can set permissions to any project however you want. So you can have a dev project, a production project, etc. All of this works out of the box and there is absolutely nothing additional for you to do. 

In AWS you have accounts, and each account has a separate set of users. There are

ways to connect these accounts so your user has permissions on other accounts. One way of doing this is creating a master users account and then adding roles that can be assumed in all other accounts by this master account. 

This is not only a pain to set up, it’s very painful to use as well. For example when using terraform scripts you need to coordinate multiple roles across several modules if you need to work across multiple accounts. 

**Command Line Interface Tools (CLI tools)**

Let’s just compare what you have to do in order to use GCP cli compared to AWS provided we are using 2FA and a couple of different projects/accounts. 

In GCP after you [install the Google SDK](https://cloud.google.com/sdk/docs/quickstart-linux), all you need to do is run gcloud init, which redirects you in the browser to a Google login page. Here you can login with your two factor authentication (which if you have an android phone is as easy as unlocking the phone and pressing okay) and you are done. Your login session is attached to your Google session so when you kill this session you are logged out— very simple.

In AWS you need to create a token that you can use to login with your CLI, simple enough, right? But now we want to use two factor auth, and this is where the fun begins.

After you login with your token you then need to [create a script](https://github.com/asagage/aws-mfa-script) to give you a 12 hour session, and you need to do this every day, because there is no way to extend this.

Okay, but that’s not a big deal, you say, after all it’s just a code that you need to input once a day and you can get on with your day after that.

But wait, there is more! If you need to assume roles in another account, you need to create yet [another script](https://github.com/Integralist/Shell-Scripts/blob/master/aws-cli-assumerole.sh) that creates another profile for you to use.

That’s one step plus two scripts, plus many steps in between. And sure, you can automate much of this or use someone else’s tools you find online (that you most likely will need to tweak), but why? Why do we have to do so much work to use AWS? Why can’t AWS abstract away this pain away from you in the way that Google has done?

**Web User Interface**

If using the CLI is too painful for you, you can always log in to the portal and use their user interface, although I don’t recommend you do this for everything, in fact I recommend you use it the least possible and only for reference and to check status of your services.

AWS interface looks like it was designed by a lonesome alien living in an asteroid who once saw a documentary about humans clicking with a mouse. It is confusing, counterintuitive, messy and extremely overcrowded. 

I can’t even count the times I’ve gotten lost or stumped in the AWS console, sometimes over the most stupid details, like missing that there was a next button hidden on a weird corner. Or trying to use search bars that can only search prefixes (WTF?)

But the biggest frustration I have from the AWS console is how you are always overwhelmed with scores of settings and options you need to fill in before actually provisioning anything.

One example that comes to mind is when someone at work said we should use codebuild/codedeploy to replace Jenkins for ECS deployments. The first engineer tried, he got stuck, the second engineer tried, he got stuck, I tried for hours and I got stuck… in the end I just gave up for lack of wanting to spend any more time on a tool that doesn’t seem to be that popular for CI/CD that I thought was meant to make life easier. 

Amazon seems to be particularly terrible at interfaces in almost all of their products though. For example in my Smart TV the Netflix app works flawlessly and is intuitive to use whereas the Amazon Prime app is an abomination, you are constantly accidentally pressing the wrong button or getting lost or the subtitles are often out of sync. 

In a rant that a Google engineer who had worked at Amazon [wrote a while back](https://gist.github.com/chitchcock/1281611) he explained the issue with

Amazon and Bezos not understanding interfaces (or is it human interaction?)

>Jeff Bezos is an infamous micro-manager. He micro-manages every single pixel of Amazon’s retail site. He hired Larry Tesler, Apple’s Chief Scientist and probably the very most famous and respected human-computer interaction expert in the entire world, and then ignored every goddamn thing Larry said for three years until Larry finally — wisely — left the company. Larry would do these big usability studies and demonstrate beyond any shred of doubt that nobody can understand that frigging website, but Bezos just couldn’t let go of those pixels, all those millions of semantics-packed pixels on the landing page. They were like millions of his own precious children. So they’re all still there, and Larry is not.

GCP’s user interface is on the other hand very intuitive to use and whenever you want to provision anything you are given [sane defaults](https://en.wikipedia.org/wiki/Convention_over_configuration) so you can deploy anything in a couple of clicks, I have never gotten lost using GCP or needed to consult a million pages of documentation to find out what I needed to do.

This however does not mean that GCP is taking away from you the power to configure things to an intricate detail, it just means they are giving you an example of a working configuration that you can then tweak to your purposes.

There are also other things you can do from the UI in GCP that either work really badly in AWS or are non-existent. For example you can easily open a terminal and ssh into any instance you have spun (provided you set permissions for it) and it works really well.

Another feature you have in GCP that I absolutely LOVE is the ability to view the CLI command that would do whatever settings you have in the console. That makes learning the cli so much easier, it’s far better than scouring the net for examples on how to do anything or trying to make sense of AWS’s gorgeous documentation…

**Documentation**

You can forgive the documentation in AWS being a nightmare to navigate for being a mere reflection of the confusing mess that is trying to describe. Whenever you are trying to solve a simple problem it is far too often you end up drowning in reference pages, the experience is like asking for a glass of water and being hosed down with a fire hydrant.

Great documentation is contextual, not referential. If you wanted to learn how to cook a dish, you don’t want someone to point you to a list of ingredients, you want a recipe describing how to use them, and this is where AWS documentation too often fails; it exhaustively describes everything that they have, but they don’t always do a good job at putting the documentation into context.

To be perfectly fair to whoever is tasked to document anything in AWS, it is a lot harder to document something that’s confusing and messy than something that’s simple to use. Extensive and overly verbose documentation is often a sign of complicated and over convoluted software or processes, so in this sense Google Cloud already has an advantage to begin with.

The documentation in GCP is generally clear and concise, and while it may not always be perfect I generally found it useful and to the point. If you want other good examples of great documentation look at DigitalOcean — they are great. 

**GKE vs EKS**

If your intent is to use Kubernetes, don’t even bother with AWS, their implementation is so bad I can’t even comprehend how they have the gall to call it managed, especially when compared with GCP

In GCP if you want to spin a cluster, no problem, just a couple of clicks and you are there. The defaults are easy and sane and the entire product feels very cohesive with all the ugly, tedious bits abstracted away from your experience. 

With GKE you don’t need to join the nodes, you don’t need to plan for an upgrade of these nodes either, it’s done automatically or with a couple of easy clicks, and this does not mean you are sacrificing complexity. You can customise a lot, but when presented with sane, simple defaults, it’s a

lot easier to understand a product that when being overwhelmed with a barrage of options and trying to figure out how everything fits together as it’s the case with EKS.

Spinning an EKS cluster gives you essentially a brick. You have to spin your own nodes on the side and make sure they connect with the master, which a lot of work for you to do on top of the promise of “managed”

And yes, I know that there are official terraform modules that take care of most of this work for you and make the job a lot easier and there is also a tool called eksctl developed by weave works which is great, but these aim to simplify a complex solution that should have been abstracted away by AWS by design, not rely on others to make sense of the mess with complex scripts and tools.

Even if you use those tools to create your automation on top of AWS, the fact remains that there are a lot of moving parts underneath that you will always be responsible to orchestrate and make sure that are working and up to date. eksctl for example uses cloudformation templates in the background.

**Product Overload**

At the time of writing this, there are 169 AWS products compared to 90 in GCP. AWS has been around for longer and therefore they have more offering, and in good Amazon spirits, they constantly and aggressively are expanding this offering to give you more of what you may need (and a lot of what you don’t need)

This sounds like a good thing, until you start seeing the amount of half cooked products or [near duplicates they have](https://www.forbes.com/sites/janakirammsv/2018/01/08/aws-service-sprawl-starts-to-hurt-the-cloud-ecosystem/#20aee5455c1f). One good example is Parameter Store and Secret Manager, the latter offers almost identical functionality except for a couple of extra features that you need to pay extra for, which begs the question, why not consolidate these products to avoid confusion and time wasted by the users investigating which one they should be using?

GCP on the other hand has fewer products but the ones they have (at least in my experience) feel more complete and well integrated with the rest of the ecosystem, and choosing one product over other doesn’t become an agonising choice that requires extensive research (okay, you still need to research, but not nearly as much)

I used to mock Apple in the past for how limiting they were and how very few features they had compared to Windows and Linux Distros until I started using a Macbook, it was then that it became so clear to me that having an opinionated approach on a few products and tighter integration of the various components often yields a far superior and more stable experience which is similar to the experience that I am having with GCP vs AWS. GCP gives you less, but what it gives you is far better integrated, simple to use and works better than its AWS counterpart — so unless you have a very compelling feature in AWS that you are missing in GCP, you should seriously consider picking GCP.

**AWS is a lot more Expensive**

AWS charges substantially more for their services than GCP does, but it also has another hidden cost attached to it; expertise and manpower. 

With GCP, a relatively inexperienced engineer in platforming tools can pick it up and get his work done in a relatively short time because most of the tedious tasks of piecing all the parts together have been done by Google already. Also using GCP is substantially more intuitive so even if you have no previous experience on cloud platforms you will pick it up very quickly.

A task that may take you a day or less to do in GCP, you may spend a week to do the same thing in AWS. One example I can give here is Endpoints. I was working with a terraform cluster installation and I wanted to restrict outbound traffic to the internet. The problem is that if you do this then you are also cutting off traffic to AWS, in order to address this problem you need to set up endpoints. Endpoints essentially allow you to connect to AWS via the AWS intranet as opposed to the internet (Don’t ask me why cloud providers don’t do this