Best tool for managing deploying and running application on Windows EC2 instances?

Heyo, I'm wondering if anybody has some suggestions on a tool for deploying a package on Windows EC2 instances when they startup and then ensuring the application always runs on subsequent startups. We use instance refresh for version updates so it wouldn't need to do any update management, just ensuring the first deploy happens and then if the instance reboots the application needs to autorun. Also looking for something not strictly coupled to AWS, no codedeploy or anything like that. We don't use Chef or Puppet or Ansible so I'm not sure if one of those would be the right option here?

To give an idea of where we're currently at, when an instance spins up, we use a userdata run script to download and deploy the package and then create links in the startup programs folder, followed by a reboot to get them running (we've tried to run them directly through the userdata, but they end up backgrounded and unkillable that way and that causes a mountain of other headaches). It takes up to 10 minutes with this for new instances to be usable and that is... really bad, haha.

We're also using Packer for our AMIs and Terraform for our general infrastructure. I don't think we'd want to build a new AMI with the package on it each deployment tho, so ideally something that we could install on our base AMI if that exists?

https://redd.it/kur6xy
@r_devops

Need study partner for starting with Devops

I want to get started with Devops and particularly Azure devops. I am hoping to talk to someone who can help me start out with it. Like maybe build a pipeline and automate it.
I actually do not have any idea at all even though I read a lot of theory on DevOps. Hoping someone who has a little bit more practical knowledge than me to help me out.
Thanks in advance!

https://redd.it/kujcf9
@r_devops

Porter: Open source Heroku that runs on a Kubernetes cluster in YOUR OWN cloud provider.

Hey r/devops!

tl; dr - Porter is a Kubernetes powered PaaS that runs in your own cloud provider.

Repository: https://github.com/porter-dev/porter

Platform as a Service (PaaS's) like Heroku are great and affordable until your app grows out of it and you start paying 5x the amount of money you would pay to AWS/GCP for the same infra. Porter brings the Heroku experience to Kubernetes without overhead (you don't even need to know k8s at all) but gives you the full flexibility of k8s when you need it. You also directly pay your cloud provider for hosting without paying the expensive middleman cost like you do on Heroku.

It's built on top of Helm and is compatible with standard Kubernetes management tools like kubectl, so it's ready for mature DevOps work from day 1.

This is how it works:

1) Porter spins up a Kubernetes cluster and image registry in your own cloud provider with just one click. (Currently we only support AWS but will be supporting GCP and DigitalOcean soon)

2) Build and push your docker images to the provisioned registry.

3) The images you have pushed will show up on our dashboard. Deploy them on Porter with a few clicks - no need to write a YAML file!

If you already have a Kubernetes cluster you want to use, you can also link up your own instead of provisioning through Porter. You can give it a go by following the README on the repository. The project is still in early stage, but we are moving at turbo speed so please leave us a star if you'd like to keep an eye on our progress!

https://redd.it/kuqpqs
@r_devops

How do you manage your sets of Dockerfile ?

Hello :-)

How do you guys handle your dockerfiles ? Nowadays, there is a lot of base image, sandbox, CI agent,...

\- Do you use a monorepo ? One repo per Dockerfile ?

\- What kind of check do you do on a built image ?

I took the time to handle my dockerfiles in a more structured way : https://github.com/romainPrignon/dockerfiles

​

Do you have any feedback ?

​

PS: It is written in js, but the same principle apply with Python, go,...

https://redd.it/kubofe
@r_devops

Could not find command "cheflicense". Why does it do that after it says the product license was accepted?

Anyone know what this means?

\+---------------------------------------------+ ✔ 1 product license accepted. +---------------------------------------------+

Could not find command "\_chef_license".

What's that mean? I googled it looking at a Github error, but it was not relevant to my situation.

https://redd.it/ku9az3
@r_devops

What DevOps tools are a must for v1 release of a SaaS app?

I'm releasing the 1st version of my SaaS app (first app I built as well, so no prior experience).

What tools should I absolutely include in the 1st release? Currently I only have sentry for error tracking. Are there other tools that you'd consider essential to have in place from day 1?

From my research online I've read about:

\- Log trackers (graylog, papertrail, elastic)

\- APT (graphite, prometheus, splunk)

\- System monitoring(sensu, zabbix, incinga)

\- CI/CD pipelines (jenkins, travis)

\- Visualizations (grafana)

\- Security (vault, firewalls, EPP)

Help me get a feel for what actually matters. Thanks.

https://redd.it/kv16ys
@r_devops

Hi team need some setup suggestion for Hashcorp vault HA.

Hi team is there any vault HA setup comes under open source for Hashicorp vault or it will only available in enterprise versions.

https://redd.it/kv1k96
@r_devops

SAST vs SCA

If the average organization is developing apps with 80% OSS but only using a Static to scan for vulns... does that mean 80% of the code in it’s apps are exposed to hackers? (I know that’s pretty broad question but any insight/thoughts would be appreciated)

https://redd.it/kv33ee
@r_devops

University students in STEM degrees often ask me these questions. Would love to get this community's insights into them.

1. What’s driving the demand for DevOps engineers and how does one become a DevOps engineer?
2. Is there a shortage of DevOps engineers? If so, are there estimates on how many we are short of?
3. What are the skills required of a DevOps engineer? Please elaborate in terms of - but not limited - to the following:
1. Platform familiarity
2. Programming/scripting languages
3. Configuration management
4. Provisioning and deployment
5. Security
6. Integration
7. Communication/team management
4. Are there certification programmes for DevOps engineers? If so, what are these and should DevOps engineers or aspiring DevOps engineers go for those certifications? Why and why not?
5. What is the typical career path for DevOps engineers, if any?
6. How are DevOps teams usually organised? Who are the members and what are their responsibilities e.g. software developers, systems architects, QA engineers etc.?
7. Are there any roles that are not usually seen as DevOps roles but are instrumental to the success of DevOps teams? E.g. user experience engineers and non-technical roles.
8. Skills and experience notwithstanding, what does it take for a DevOps engineer to succeed?

https://redd.it/kv1yfh
@r_devops

Development for Infrastructure in 2021

Hey All,

One of the trends I saw in 2020 was that infrastructure pros and sysadmins need to code. Sure, we’ve seen the “just knowing how to code” part for years now, but something changed drastically.

The need to “think” like a developer and understand development concepts.

I started out my career as a Sysadmin and moved into the development space later on. I thought it was really interesting and I sort of saw the “writing on the wall”.

So, how can one break into the whole “think like a developer” thing?

1. First is theory. You’re going to hear phrases like “immutable vs mutable” and “idempotent”. If you’re new to development, this may sound like gibberish.
2. The second is source control. If you’re already writing code or plan on deploying application code, you’ll need to understand source control.
3. The third is code editors and IDEs. Believe it or not, people are still writing code in Notepad. A good code editor and change the game for you.
4. The fourth is understanding how to write code like a developer. You may write small PowerShell or bash scripts, but how about taking it a step further with things like unit testing?
5. Step five is CICD. In the beginning, CICD was targeted towards application code. Now, it's almost a requirement for any organization to use CICD to deploy infrastructure code.

Last month I posted a thread here that got a lot of love. It was about what I expect in the future and a series to help people get there.

Going From Infrastructure to Developer Is A Reality : devops (reddit.com)

I just finished up the series and it's called "Development for Infrastructure". If you're interested, definitely feel free to check out the playlist!

https://www.youtube.com/watch?v=u-0T-JN0GZc&list=PL8iDDHqmj1oWbbqlVwdZT9dElTyd9JRb4

https://redd.it/kv4qrw
@r_devops

Collect Custom Metrics in AKS

Custom metrics are a large part of monitoring software. I wrote a blog post on how you can define and query custom metrics in the Azure Kubernetes Service (AKS). Read more here!

It utilizes Prometheus metrics under the covers, so creating a new one or implementation an existing Prometheus exporter is necessary.

https://redd.it/kv5fh4
@r_devops

Wide Scale Deployment with/without internet

So we have approximately 800 or so “kiosk” devices in the field (Windows) - some have internet connectivity, some do not; are only connected via VPN to our server infrastructure.

We currently push updates to this software via ManageEngine Desktop Central, which does work, and works quite well, however we want to go the next step and automate from a pipeline to these deployments.

From the DesktopCentral API, there doesn’t seem to be any way to register a new package programmatically, and/or schedule deployments.

So I guess the question is 2 fold: does anyone know if there are secondary api’s or ways to control DesktopCentral programmatically, or alternatively, can anyone recommend a good deployment tool that can handle internet and non internet connected endpoints, programmatic control, and doesn’t cost stupid amounts of money.

Thanks!

https://redd.it/kv2a88
@r_devops

Using GPT-3 for plain language incident root cause from logs

Disclosure: I work for Zebrium.

We are seeing some pretty cool results using OpenAI GPT-3 with a summary of incident log events (generated by ML), to produce a plain language incident root cause description. Read the blog here.

https://redd.it/kva8a3
@r_devops

Any team leaders or managers out there?

If you are one, can you confirm my suspicions?

I've noticed a theme the more I chat with people about DevOps, cloud-native, K8s etc. It seems like people issues are tougher than technical issues.

They accept that the tech will constantly change and make their life tough. But the harder part seems to be getting the team aware of the change and to drive movement in affected activities.

Essentially, making sure all bases are covered seems to be an increasingly difficult moving target.

Is there any substance to this?

https://redd.it/kuz43q
@r_devops

Digger - get instant URLs and Terraform for your microservices on AWS

Hey r/devops! We've been working on this for a while, built an alpha and looking for feedback.


Developers today have great tools to quickly launch small projects without thinking of infrastructure (Firebase, Vercel, Heroku). But these tools don't work for teams. Big tech companies that can afford dedicated platform teams tend to build self-service tools for developers on top of AWS / Azure / GCP to launch new services and manage environments. But smaller teams who can't afford it are out of luck. If they have DevOps expertise in the team then they'll write a lot of repetitive Terraform, and if they don't they'll often struggle for weeks learning all the AWS concepts and make lots of mistakes.


We thought this is wrong, and built Digger


Digger manages your cloud account, allows to create apps and microservices from templates (can be custom), generates and runs Terraform, and manages environments. So developers get modern Vercel-like experience while DevOps engineers still retain full control. Starting on AWS with Digger is just as simple as on Heroku, but cheaper and you get a future-proof stack with DevOps best practices.


What do you think?

https://redd.it/kvd989
@r_devops

Pyroscope - continuous profiling tool to help debug performance issues. Would love some feedback!

Hi r/devops

At my last job I had to deal a lot with performance issues on the backend and I found profiling tools to be very helpful in figuring out where the bottlenecks occur. But the problem is that it’s often pretty hard to replicate exact situations that happen in production environment. So I figured why not profile my apps 24/7 in production — that’s how Pyroscope was born.

It’s open source and it currently works with Go, Python and Ruby apps. Here’s the link: https://github.com/pyroscope-io/pyroscope

Would love to get any feedback or hear from you all on how you do performance analysis for your apps.

https://redd.it/kvdqgd
@r_devops

How Canada's Exposure Notification App (Covid Alert) runs on AWS infrastructure.

AWS wrote a blog post regarding the infrastructure that I'm working on.

https://aws.amazon.com/blogs/publicsector/keeping-canadians-safe-protecting-privacy-covid-alert-app/

You can check out the code here:
https://github.com/cds-snc/covid-alert-server

And the terraform for our staging environment here:

https://github.com/cds-snc/covid-alert-server-staging-terraform

https://redd.it/kvfifq
@r_devops

Logging Platform

Everything generates logs, but not everything needs to be retained.

Is there a logging platform/option that will allow logs to stream, retain for a nominal amount of time, let us say 1 hour. If there are issues (insert detection pattern here - security, 5xx, other defined patterns) retain the logs of that stream for say 10 minutes before the event and 10 minutes after and then apply the log retention rule above.

Essentially you'd not have the entire log, just the snippets from when there was a detected issue and one would not have to worry about the retention based pricing?

https://redd.it/kveioo
@r_devops

Pushing More, Smaller Rocks as a Team

OK, I've read about Amazon releasing deployments every day and I'd love to take that concept and get more projects in concurrent pipelines, with staggered releases so that we're moving faster, breaking very little and recovering quickly too.

Working on building out a workflow from development through testing and customer feedback.

If you've been able to deploy more frequently than weekly what are some bottlenecks have you found and best practices to remedy them?

We're now acquainted with working remotely, should we stay remote? Have you found slow-downs from it?

Have any lesser-known tools that have helped you out to achieve releasing 2-3+ times per week?

https://redd.it/kvgg2v
@r_devops

Google Cloud Run, Terraform Cloud and GitHub actions for continuous delivery of a Ruby on Rails application

In this video, we are going to learn how to deploy an application in Cloud Run, how to use GitHub actions, Terraform Cloud, and how to integrate everything together in a real example: https://www.youtube.com/watch?v=JxBSXbmf4F0&feature=youtu.be

https://redd.it/kuy6bt
@r_devops

dEVOPS when company doesnt develop software from scratch?

Hi guys, just wondering if you guys have heard of, or have experience seeing devops principles and its automation practices being implemented outside companies that produce software. For example I work for a medium sized grocer that sells through online and traditional brick & mortar channels. We changed focus to Cloud SAS offerings and building integrations mainly and now are trying to implement a devops culture/practice. Have any of you heard of or experienced successes/challenges in this kind of scenario? Maybe not directly mapping to mine but similar context at least?

https://redd.it/kuxxek
@r_devops