Deployment scripting is not continuous delivery. Agree?

I was just discussing with my friend that most of the companies are actually not doing DevOps the right way. Well, nobody knows what is the right way still but I know that deployments through scripting that is maintained over time is not continuous delivery and it isn't the path to DevOps.

What if the guy who knows this type of deployment leaves one day?

I think most of the companies say they are doing continuous delivery and deployment but they aren't.

Well, even most of the service/tool providers say they are a CI/CD tool company but what they do is just CI. The CD part is missing in most cases.

Do you agree on this? Open for discussion.

https://redd.it/rpgqzo
@r_devops

Can someone ELi5 webhooks

I understand that they POST data usually within a JSON payload. I've been going through some docs for various tools (lakeFS, Prefect) and everything seems to support posting webhooks, but I can't find anything about receiving them? Am I missing something?

I'm trying to achieve CI/CD data pipeline.

Step 0 - There is some data in a cloud bucket

Step 1 - Batch job runs and fetches data from somewhere, but that data is pulled into an isolated data branch in LakeFS

Step 2 - *this is where I think webhooks should be used* As part of a pre-commit hook lakefs needs to send data saying "hey there's new data on this branch" to a databricks cluster which then says "sure let me run these data quality checks"

Step 3 - Assuming the quality checks pass it databricks says "you're good to go, all checks cleared" and then that data is commited and then merged. (or some pre-merge check could be done too for conflict resolution)

Does one still need webhooks when using something like Prefect/Airflow anyways? The "state" information passes between tasks, and data can be explicitly passed between tasks. I'm confused I guess as to whether an orchestration tool is sufficient for data pipelines? But I'm confused as to why lakeFS has webooks.

​

Anyone willing to explain webhooks in the context of data engineering?

https://redd.it/rph18d
@r_devops

Azure Terraform deployment with GitHub workflow and approval

Refer to my latest blog on GitHub Actions

https://rakesh-suryawanshi.medium.com/azure-terraform-deployment-with-github-workflow-and-approval-dc67b36dd9ce

https://redd.it/rpjggw
@r_devops

Azure IaC code deployment with terraform and GitHub Actions

https://youtu.be/aAOpq-HqTSo?sub\_confirmation=1

https://redd.it/rpjf65
@r_devops

Anyone else seeing tons of GitHub / GitHub Actions workflow failures?

Our GHA workflows have been failing for hours.

Also: https://www.reddit.com/r/github/comments/rpkgya/does\_someone\_else\_experience\_issues\_with\_github/

https://redd.it/rpkqgd
@r_devops

Modernize Your Legacy System The DevOps Way

“Modernizing the legacy IT infrastructures has become a necessity to keep pace with digital transformation and disruption as no organization can afford to weigh down under the burdens of traditional IT systems by becoming outmoded in the contemporary world.” 

https://redd.it/rpks7f
@r_devops

Lab recommendations

I started as a sysadmin then moved into management. I'm looking to move back to working hands on with tech. DevOps seems like a good fit because I have a strong scripting and dev background and have been managing a private cloud and some Azure resources for the past 5 years.

When learning about virtualization I would build servers at home and run services like Plex, OwnCloud, SabNZBD and the likes. This was both educational and useful for some hobbies.

I want to practice building and deploying in AWS and Azure, but am looking for inspiration for the services that I can deploy in that lab environment. Any recommendations?

https://redd.it/rpoznu
@r_devops

Moving into Azure DevOps

Hi All,

I am moving from a Level 3 Support role to a Azure DevOps role

Would anyone have recommendations on where to start my new learning path and what would be the fundamentals to learn and what order would be best to learn them?

I am already very confident with Azure from a administration and implement aspect

https://redd.it/rpq08a
@r_devops

How did you guys learn networking

Hi All,

I am into azure. Sometimes I feel the networking stuff is too hard to understand not azure but networking in general..
I do know about IP address subneting and stuff but haven’t studies anything particularly for networks
Any network gurus here can you suggest me a path to learn networking so that it can be useful for my cloud journey ( preferably practical and short)

https://redd.it/rpspmu
@r_devops

Should and Can I move from networking to Devops after 9 years behind a cisco router?

Hello everyone! I want to shift from my current Routing and SD-WAN day to day work to DevOps. For me it is more engaging and exciting. Also from what I hear, the job postings are more and the pay is somewhat better. There is also the option for remote-work. I want to hear opinions from people who shifted to this as well. And will I make it as a much older candidate entering as a fresh in something this big?

https://redd.it/rptd38
@r_devops

Terraform - Import existing infrastructure or recreate everything?

You have been given full control of your companies infrastructure (≈10 lambda functions, 2 RDS and 5 S3 buckets, all across 2 VPCs), and must manage it with Terraform. Which of the options would be best?

View Poll

https://redd.it/rpt9mw
@r_devops

A hashicorp vault plugin for ephemeral Jenkins users/api tokens

Hey folks!


I thought some of the folks here would be able to take advantage of a new vault plugin I've been working on. I recently had a need to have auto-expiring API tokens for Jenkins so I developed this plugin for vault to have a cleaner solution. Hopefully it helps some others here as well.


https://github.com/circa10a/vault-plugin-secrets-jenkins

https://redd.it/rpxstq
@r_devops

Managing an ec2 in 2022

Hi,
I work in a relatively small shopping oriented startup.
We have a mobile app with ~10k users and a back-office system which should handle similar load to the app.
We are expecting to grow steadily over the next year, probably to a max of 100k users in the best case.
The user traffic is quite steady during the day (with only a small percent of users active at any given time) and non-existent at night (good time for maintenance stuff).
We built our stack all in on aws with amplify including appsync, dynamodb, lambda etc.
In practice, the mobile app and the backoffice are using separate amplify stacks.
While the mobile app has benefited greatly from this stack, and graphql in particular, the backoffice system has become a pain to work with as the team grows and the usecases are getting more complex. Dev velocity is affected negatively, and simple product requirements often entail unnecessarily complex solutions.
As a result, we have been considering going back to the basics with a simple ec2 instance, and a monolith backend deployed on it, probably running on docker, 2016 style.
A big difference compared to 2016 is that there are good IaC solutions to help maintain this type of stack, simple as it may be.
We also considered ecs/fargate - but our bad experience with amplify discourages us from going into another aws rabbit hole.
I think that we can probably scale on a single ec2 instance over the next year while increasing dev velocity significantly, and we can reconsider scale when the time comes.
I do however have some reservations, in particular going back to an older type of stack (2016 style), and having to manage an ec2 in terms of patches, permissions etc (I have seen aws ssm can ease some of the pains).
I'd be glad to get some opinions on this topic, and open to hear other alternatives.

Thanks!

https://redd.it/rpv6m0
@r_devops

Kafka best practices in production

Many organisations utilise Kafka to enable data pipelines between applications and micro-services.
With its wide adoption and its integration into enterprise-level infrastructures, DevOps teams often face challenges ensuring that this open source application is deployed at scale in a secure and reliable way.
This blog explains the key considerations for setting up Kafka in production.

https://inder-devops.medium.com/kafka-best-practices-lessons-learned-by-inder-431dc5fafd3b

https://redd.it/rprwox
@r_devops

Cheapest GPU cloud instances for Machine Learning inference

I'm looking to find the cheapest available cloud providers for inference purposes.

More specifically, I need some GPU with CUDA cores to execute the inference in a matter of a few seconds.

I've tried DigitalOcean, GenesisCloud and Paperspace, with the latter being (slightly) the cheapest option - what they offer is pretty much the same and doesn't change much for me (OS, some CPU cores, some volume space and some bandwidth).

​

I don't think I can use spot instances, since [from what I know\] I would need to spin off the instance and the API every time it is shut off, and that wold take quite a bit of time, so I don't think it's an appropriate solution.

​

tldr: cheapest GPU instance to rent from any cloud service, for ML inference purposes;

https://redd.it/rpr94p
@r_devops

What would you do if you didn't have to collect a paycheck, still wanted to do DevOps, but weren't sure about working a standard 9-5 job anymore?

Let's say you found an old USB drive containing 30 BTC from an even older tech conference and suddenly had the ability to retire early (not that I did, just an example), or that you suddenly had enough passive income from, say, real estate investments or something similar. What would you to keep learning in the field, stay current, and contribute back to the community? Just curious.

https://redd.it/rq2xw8
@r_devops

What causes a kafka broker or consumer to crash

What are the main causes of crashes in kafka consumers and brokers ? The question can be expended to all servers why can't me give a server enough ressources to never ever crash ?

https://redd.it/rp36cw
@r_devops

Automating manual gitlab CI/CD Steps?

https://sanderknape.com/2019/02/automated-deployments-kubernetes-gitlab/#building-docker-in-gitlab shows how to build a Gitlab CI/CD pipeline for deploying to Kubernetes.

One manual set of steps involves:

1. creating a Gitlab Service Account YAML for k8s access
2. `kubectl describe secret gitlab-service-account-token-....` to get the token
3. create secret env var in gitlab CI/CD Settings from step 2's output
* That token is then referenced [here](https://gitlab.com/kevinmeredith/ci-cd-test/-/blob/master/.gitlab-ci.yml#L44) on this line, `kubectl config set-credentials gitlab --token="${SECRET_SERVICE_ACCOUNT_USER_TOKEN}"`

In the spirit of automating everything and infra as code, how can these manual set of steps be automated or source controlled?

https://redd.it/rq6tfn
@r_devops

What IDE do you use at work?

I started using atom a few years ago and my supervisor recommended VSCode since it uses less ram and is faster. I love VSCode and completely switched last year and love all the extensions and capabilities it has. Recently, I started looking into switching to neovim + tmux for work since it’s faster and has a lot of customization.

My problem stems from that I develop on WSL for work and Ubuntu at home and like to use my mouse to copy and paste and move around within VSCode. I don’t think you can use your mouse within neovim + tmux. Plus the integration of extensions for ansible, terraform and what not is not the best when comparing to VSCode. Maybe it’s my configuration setup and I’m doing it wrong but I am wondering what the Devops community is using for their environment and why.

View Poll

https://redd.it/rpupat
@r_devops

Kubernetes Monitoring

What are you guys currently monitoring in Kubernetes? I’m not looking for products to monitor but rather when you monitor.

Assume on Prem, blade servers. CentOs.

https://redd.it/rqax3y
@r_devops

CHAOS CARNIVAL 2022

ChaosNative is back with Chaos Carnival 2.0 this January 2022!

A 2-day ChaosEngineering conference worth remembering!

With 30+ chaos sessions, [LIVE\] Chaos Panel, and exclusive workshops, this conference is going to be the perfect mixture for SREs, QA Engineers, and Cloud-Native Developers which you do not want to miss!

Register here: https://chaoscarnival.io/register

https://redd.it/rqa5gx
@r_devops