So, how are you guys releasing your artifacts?

Similar to another enquiry I've made in the past about what Artifactory software you are using, but a more general information gathering one this time - how are you releasing artifacts and getting them to your customers?

Our shop isn't a full SaaS (yet), so when we do CI/CD we push changes to our own live/customer environments where we can develop against. Our artifacts strategy at the moment is to release the files on our Jira Board in the announcements page to our respective customers. I want to try and improve this as it's been there since god knows when and it feels like a better process should exist.

Our customers take the artifacts we release and give them to there own QA team who do there own CD process after they've quality checked and raised any tickets we might have missed, followed by pushing them to there own live environment we don't control.

We have a container registry where I've been trying to push changes into our docker image and have our environments refresh that everytime there's a change to our image, but it's just experimental and I want to know how everyone else is doing it.

https://redd.it/epxmjt
@r_devops

Scaling docker container horizontally benefits if on same host

What are the benefits of scaling a service horizontally when they run on the same host? Say two instances of a microservice instead of one.

Wouldn't the resources that could have been used for the single microservice just get split up and used for the second, giving no real benefit. Like instead of having one car go 100km/h now there are two cars each going 50km/h ?

https://redd.it/eq8n66
@r_devops

How to automatically ensure pipeline compliance

So, let's say that we want to ensure that all software pipelines put the code through SAST testing (or any other important step), how do we as DevOps engineers ensure that the pipelines for 100 or 1000+ all confirm to business requirements? What's to stop one developer removing a crucial step in the pipeline and another approving this and it therefore not going noticed for a very long time? or, possibly more likely, what's to stop a developer taking a template pipeline and simply removing a step on the initial pipeline commit?

For context we are currently using gitlab CI but are looking at potentially moving to cloud build, I hope the answer to this is platform agnostic (but assume it won't be)

https://redd.it/epyi34
@r_devops

GuardDuty: The request is rejected because the caller is not authorized to call this API

I have an AWS GuardDuty stack. Inside I have one resource:

IPSet:
Type: "AWS::GuardDuty::IPSet"
Properties:
Activate: True
DetectorId: !Ref Detector
Format: TXT
Location: s3://mybucket/ipset.txt
Name: MyIPSet

I create the bucket:

▶ aws s3 mb s3://mybucket/
make_bucket: mybucket

List stuff in the bucket etc, all works fine.

On deploying the stack:

{
"StackId": "arn:aws:cloudformation:us-east-1:111111111111:stack/test-stack/28801e90-390b-11ea-9d33-0ab1b9b87aaa",
"EventId": "ThreatIntelSet-CREATE_FAILED-2020-01-17T09:25:10.136Z",
"ResourceStatus": "CREATE_FAILED",
"ResourceType": "AWS::GuardDuty::ThreatIntelSet",
"Timestamp": "2020-01-17T09:25:10.136Z",
"ResourceStatusReason": "The request is rejected because the caller is not authorized to call this API. (Service: AmazonGuardDuty; Status Code: 400; Error Code: InternalServerErrorException; Request ID: b70503ba-3598-4f3e-97be-bd6ebaf58b94)",
"StackName": "test-stack",
"ResourceProperties": "{\"Format\":\"TXT\",\"Activate\":\"true\",\"DetectorId\":\"4eb7d968df4f4f7db0be0711b0c027d3\",\"Location\":\"s3://mybucket/threatset.txt\",\"Name\":\"fooorg-audit-ThreatIntelSet\"}",
"PhysicalResourceId": "",
"LogicalResourceId": "ThreatIntelSet"
}

I've seen this answer here in Stack Overflow

[https://stackoverflow.com/questions/48404164/accessdeniedexception-caller-is-not-authorized-to-call-api-using-aws-guardduty/57725109#57725109](https://stackoverflow.com/questions/48404164/accessdeniedexception-caller-is-not-authorized-to-call-api-using-aws-guardduty/57725109#57725109)

But no use. Very frustrated! Any thoughts on what else it could be?

https://redd.it/epy9mp
@r_devops

Does Microsoft have their own implementation of microservices?

[https://en.wikipedia.org/wiki/Microservices#A\_comparison\_of\_platforms](https://en.wikipedia.org/wiki/Microservices#A_comparison_of_platforms) shows two implementations of microservices: Spring Cloud and Kubernetes ecosystem.

Does Microsoft have their own implementation of microservices too? [https://dotnet.microsoft.com/learn/aspnet/microservices-architecture](https://dotnet.microsoft.com/learn/aspnet/microservices-architecture) seems to mean that they can create a microservice by deploying a service inside a docker container. Maybe use Docker Swarm to coordinate multiple containers?services? Is this Microsoft's platform/implementation of microservice architecture?

https://redd.it/eqb22x
@r_devops

Kafka is no longer the best choice thanks to Apache Pulsar

Recently on HackerNews someone asked why I was moving off of Kafka and I wrote a somewhat lengthy comment.

This inspired me to do a presentation at a meetup about how Apache Pulsar has come to replace Kafka and how today there's not really a reason to be on Kafka unless you've invested into it heavily already.

Here is the talk:
https://youtu.be/jLruEmh3ve0?t=130

https://redd.it/eppwrp
@r_devops

Application/Service Inventory Query

Between Dev, Test, and Prod we have a large number of On-Premises servers (Physical and VM). Running various applications like databases, web servers etc.

The trouble is working out what is what. For example, I need to make a change to Customer A's web server, I first need to discover the hostname of the web server.

Now I know in AWS we just query the tags of the instance, easy peasy. But how would we go about this for a physical environment?

Would appreciate some guidance on how this could be acheived.

End result: I want to quickly ask some appliction/service where Customer A's servers are, and filter from there.

https://redd.it/epddoj
@r_devops

What software engineering methods are monitoring tools like DynaTrace using to get such detailed metrics?

Sorry if this is the wrong sub, was thinking about posting it in /r/programming or something similar, but it strikes me as a devops-y question. How do DynaTrace and other monitoring tools get metrics and info such as:

* Request routes with response times and tracing
* JVM metrics
* Memory, CPU, etc metrics of hosts and containers
* DB queries and other database information
* All kinds of other information we use for analysis, alerting, etc.

I've implemented tools like DynaTrace and Prometheus at work, and I appreciate how useful they are, but I'd like to understand more about the software engineering that goes into building tools like these. How do they get these metrics? What methods are they using to format and store them? I know I can use top or free to check mem/cpu, or run jstat to look at JVM metrics if I'm on a Linux host, but how are these monitoring tools doing it and making the data easy to understand?

​

I realize this might seem kind of broad, or might be a "duh" question to some people, but I'd really like to learn more about this. A lot of the monitoring tools I use are almost like a black box/magic to me, they just work and I want to know what's happening at the code level for them to accomplish what they're doing.

https://redd.it/epcbz5
@r_devops

Learning to DevOps with RPi?

Hi all,

Been following this thread for a few months now and it's been great, especially the monthly Getting into DevOps thread. I have just finished my first week as a DevOps engineer. I've come from being a developer so a lot of the Linux and networking stuff is completely new to me. Been an intense first week!

Was just wondering... I have a raspberry pi 4gb I bought a while back that I've been meaning to do something with, but was wondering if I could get some suggestions as to what kind of things I could use it for? Specifically to help me learn the fundamentals of DevOps and improve my overall skills.

To be more specific, my day to day includes server monitoring, testing, deployment of new servers and infrastructure, etc. General DevOps-y stuff!

Any suggestions most welcome 😁

https://redd.it/eqchuo
@r_devops

Is this an opposite ways to the microservice architecture?

Martin Fowler said in his Patterns for Enterprise application Architecture 2004 that

> while many things can be made transparent in distributed objects, performance isn't usually one of them.
Although our prototypical architect was distributing bjects the way he was for performance reasons, in act his design will usually cripple performance, make the system much harder to build and deploy, or, usually, do both.

> we get to my First Law of Distributed Object Design: Don't
distribute your objects!

> How, then, do you effectively use multiple processors? In most cases the way to go is clustering (see Figure 7.2). Put all the classes into a single process and then run multiple copies of that process on the various nodes. That way each process uses local calls to get the job done and
thus does things faster. You can also use fine-grained interfaces for all the classes within the process and thus get better maintainability with a simpler programming model.

Is it to prefer a big program and multiple replica processes running the program?

In microservice architecture, is it to partition a big program into multiple small programs, and run them in different processes?

Are the above two ways of architecture opposite to each other?

Thanks.

https://redd.it/eqb6vz
@r_devops

Help needed with Canonical MAAS CLI

I've been trying to create a boot-source-selection for CentOS in MAAS, but MAAS spits out not found as stdout.

I only want to use MAAS CLI to select CentOS7 image as well as Ubuntu 18.04 ( which already is selected and imported ).

https://redd.it/eq9k5v
@r_devops

Implementing DevOps through GitHub Actions for Xamarin iOS & Android

This sub reddit has some awesome posts from the past:

[AWS Solution architect exam and notes](https://www.reddit.com/r/devops/comments/ckpbhl/just_passed_the_aws_solutions_architect_exam/), [Getting Started with DevOps](https://www.reddit.com/r/devops/comments/aqc8cj/interested_in_getting_into_devops_start_here/), [DevOps Interview Questions](https://www.reddit.com/r/devops/comments/dj9ffy/a_collection_of_devops_interview_questions/)

For years DevOps has intrigued me. I always wanted to apply DevOps in mobile apps, and GitHub Actions made it really accessible for me. Once GitHub actions started supporting MacOS runners and "Xamarin", I decided to dive in and figure it out. But I wasn't able to figure it out, getting my most downvoted StackOverflow question, and also no proper response on the GitHub community forums.

After months of trying, I was able to finally get the GitHub Actions working with Xamarin Android & iOS solutions! It helped me learn a lot more about DevOps in general.

I wrote an article on what I learnt, with a video walkthrough, and it became really popular! If you guys are interested

[https://levelup.gitconnected.com/using-github-actions-with-ios-and-android-xamarin-apps-693a93b48a61](https://levelup.gitconnected.com/using-github-actions-with-ios-and-android-xamarin-apps-693a93b48a61) (Unmetered link: [bit.ly/30crWpc](https://bit.ly/30crWpc))

The article was curated by Medium in their "Programming", "iOS Dev" and "Android Dev" categories. In all this, I became more optimistic of the future realized that there's still a divide in Software Development and DevOps, that's continuing to shrink, and there's also a huge demand for developers to learn and improve!

https://redd.it/eq5vg5
@r_devops

What is Serverless ?

Welcome everyone,
In this video, we will talk about serverless which is one of the most widely used paradigms in the world of cloud computing.
Don't hesitate to share your thoughts.


PS: it's an educative video
Thanks !!
https://youtu.be/9mZV2ZJBbHM

https://redd.it/eq77az
@r_devops

Open source OPA for Terraform project

Hi all,

CTO of [Fugue](https://fugue.co) here - we just launched a new open source project at [https://github.com/fugue/regula](https://github.com/fugue/regula) for doing security policy checking of Terraform templates using [Open Policy Agent](https://www.openpolicyagent.org/). There are numerous rules available today, covering SGs, VPCs, IAM, EBS, KMS, etc. and we'll be adding a lot more in the future. We plan to extend to other infrastructure as code tools like CloudFormation as well. We'd also love to get contributors to the project!

https://redd.it/eq3rmj
@r_devops

How do you use manual actions in your CircleCI pipeline?

We have manual actions on E2E tests as they take quite some time and you want to run them just when needed. And then on the master branch we have manual actions that approve the deploy to production.

How about you?

https://redd.it/eq37tn
@r_devops

Route 53 DNS taking long to take effect

I'm trying to create a subdomain on AWS Route 53. I'm following exactly what the docs say:
1) Create a hosted zone for the domain
2) Create a hosted zone for the subdomain
3) Copy the NS records of the subdomain and make a new record set in the hosted zone of the domain
4) Make an Alias Record to point to the service I desire in the hosted zone of the subdomain
Following this I noticed that I have to also change the NS record of the domain with the nameservers associated with the domain.
It' been twelve hours since I have done this and I cannot query my subdomain. Does DNS changes take long to take effect or did I just miss out something?

https://redd.it/epc34r
@r_devops

Self-registering compact k3OS clusters to Rancher Server via cloud-init

Here's some k3OS content for you all :P


[https://medium.com/@jamesdavidbohrman/self-registering-compact-k3os-clusters-to-rancher-server-via-cloud-init-d4a89028c1f8](https://medium.com/@jamesdavidbohrman/self-registering-compact-k3os-clusters-to-rancher-server-via-cloud-init-d4a89028c1f8)

https://redd.it/epbl16
@r_devops

Want to Play with Terraform for AWS without using Credit Card

Is there a way to play with terraform for AWS without using any credit card? I am fine with terraform working on some mock AWS playground. Does anyone know about this?

https://redd.it/eqknwf
@r_devops

In what extend (if at all) do you use pull requests in your job?

I recently got a new job on witch I support devopswise a product along with another fellow. Although the devs use PRs we dont use them at all. Whenever for instance an update to a script is needed we just push directly to develop branch, we then validate the functionality on a staging environment, and finally release a new tag on master branch.

https://redd.it/eqlkmh
@r_devops

What are alternatives to microservices?

I have heard that some people don't use microservices any more. But they don't say what they are using. What are alternatives to microservices? What make microservices fall out of favor?

In [https://www.reddit.com/r/devops/comments/eqb6vz/is\_this\_an\_opposite\_ways\_to\_the\_microservice/](https://www.reddit.com/r/devops/comments/eqb6vz/is_this_an_opposite_ways_to_the_microservice/), I said that I wrote Martin Fowler's book written in 2004, and he said he favors replicating processes to run the same program, over partitioning a program and distributing its parts to run in different processes and locations. Is his way common nowadays? Is combination of his way and partition way (e.g. microservice) common nowadays?

https://redd.it/eqg233
@r_devops