Hands-on examples of observability-driven development
https://tracetest.io/blog/observability-driven-development-with-go-and-tracetest
Based on one of my previous discussions about ODD, I wanted to go into more depth and explain how it works with a code demo using open-source tools like Go and Tracetest. The main point I think is that there are no mocks. Instead, you're running E2E and integration tests against real data. I think the biggest pain point in testing on the back end is the amount of coding you need to do to actually just make the test run. Mocking API responses, setting up credentials and env vars to access different services and databases. It's just a lot of hassle to run an integration test.
Disclosure: I am on the Tracetest team, so I'm passionately not disinterested in what you think about the whole ODD movement.
https://redd.it/10gab31
@r_devops
https://tracetest.io/blog/observability-driven-development-with-go-and-tracetest
Based on one of my previous discussions about ODD, I wanted to go into more depth and explain how it works with a code demo using open-source tools like Go and Tracetest. The main point I think is that there are no mocks. Instead, you're running E2E and integration tests against real data. I think the biggest pain point in testing on the back end is the amount of coding you need to do to actually just make the test run. Mocking API responses, setting up credentials and env vars to access different services and databases. It's just a lot of hassle to run an integration test.
Disclosure: I am on the Tracetest team, so I'm passionately not disinterested in what you think about the whole ODD movement.
https://redd.it/10gab31
@r_devops
tracetest.io
Observability-driven development with Go and Tracetest
Hands-on tutorial covering observability-driven development, how to develop microservices with Go & how to run trace-based tests with Tracetest.
mOVING FROM Puppet To Ansible - A few questions around structure and config drift.
So we're on Puppet right now - it's old, out of date, but at the core of everything we do.
We'd like to move to Ansible, which a lot of us are familar with, and I think is the better path forward for us as we're moving a lot of things to the cloud.
Now I have a few thoughts/questions for which I don't have an exact answer for:
1: Configuration Drift
We can make a playbook, chuck it into gitlab, have a pipeline run it...but then what?
What if someone makes a config change on the box but not in git? (it WILL happen)
Puppet runs every 45 minutes or so, without using Ansible Tower, how are people doing this?
Something like Rundeck?
An "Ansible Master" server at each DC running cron jobs every hour?
2: Structure or hierarchy of our Playbooks/Roles, with multiple DCs
There will be quite a few common roles that ALL server will need:
NTP, Security/SSH settings, Log rotation, Log shipping etc etc
Do we just create a playbook for each server type/location, chuck in the "Common" roles and then the app/location specific role into that playbook?
Seems like #2 could get messy quick with lots of servers, doing the same thing over multiple DCs.
e.g. I might want to only affect the mail servers at DC1 today, and then DC2 tomorrow, and DC 3,4,5 & 6 later...but now that means I got 6 versions of the same role to maintain?
EDIT: Damn text editior FORCES YOU TO BE IN CAPS EVEN WHEN YOU'RE NOT SO THE TITLE LOOKS LIKESHIT..
https://redd.it/10gc90g
@r_devops
So we're on Puppet right now - it's old, out of date, but at the core of everything we do.
We'd like to move to Ansible, which a lot of us are familar with, and I think is the better path forward for us as we're moving a lot of things to the cloud.
Now I have a few thoughts/questions for which I don't have an exact answer for:
1: Configuration Drift
We can make a playbook, chuck it into gitlab, have a pipeline run it...but then what?
What if someone makes a config change on the box but not in git? (it WILL happen)
Puppet runs every 45 minutes or so, without using Ansible Tower, how are people doing this?
Something like Rundeck?
An "Ansible Master" server at each DC running cron jobs every hour?
2: Structure or hierarchy of our Playbooks/Roles, with multiple DCs
There will be quite a few common roles that ALL server will need:
NTP, Security/SSH settings, Log rotation, Log shipping etc etc
Do we just create a playbook for each server type/location, chuck in the "Common" roles and then the app/location specific role into that playbook?
Seems like #2 could get messy quick with lots of servers, doing the same thing over multiple DCs.
e.g. I might want to only affect the mail servers at DC1 today, and then DC2 tomorrow, and DC 3,4,5 & 6 later...but now that means I got 6 versions of the same role to maintain?
EDIT: Damn text editior FORCES YOU TO BE IN CAPS EVEN WHEN YOU'RE NOT SO THE TITLE LOOKS LIKESHIT..
https://redd.it/10gc90g
@r_devops
reddit
mOVING FROM Puppet To Ansible - A few questions around structure...
So we're on Puppet right now - it's old, out of date, but at the core of everything we do. We'd like to move to Ansible, which a lot of us are...
"I Know So Much Stuff I Learned Over The Years I Forgot Half Of That By Now?"
I feel like my brain has a limited capacity to remember stuff I dont repeat from time to time.
As a DevOps/SysOps/SysAdmin w/e I had so many tools I had to learn how they work over the years that I lost track of half of them..
Example 10 years ago was using puppet. Could write configurations 1b1, it was super easy to understand and now I would have to remind myself most of it.. coz Im using mostly GA..
Am I just a bad engineer or the tools change so often from company to company its just impossible to remember all of them ? Maybe some ppl can/ or most ?
Just curious whats the other ppl experience in this regard.
https://redd.it/10gfegd
@r_devops
I feel like my brain has a limited capacity to remember stuff I dont repeat from time to time.
As a DevOps/SysOps/SysAdmin w/e I had so many tools I had to learn how they work over the years that I lost track of half of them..
Example 10 years ago was using puppet. Could write configurations 1b1, it was super easy to understand and now I would have to remind myself most of it.. coz Im using mostly GA..
Am I just a bad engineer or the tools change so often from company to company its just impossible to remember all of them ? Maybe some ppl can/ or most ?
Just curious whats the other ppl experience in this regard.
https://redd.it/10gfegd
@r_devops
reddit
"I Know So Much Stuff I Learned Over The Years I Forgot Half Of...
I feel like my brain has a limited capacity to remember stuff I dont repeat from time to time. As a DevOps/SysOps/SysAdmin w/e I had so many...
Monitoring stack demo using Grafana, Loki & Mimir
Wanted to share a demo/tutorial with everyone on how get started with a monitoring stack using grafana, loki and mimir with prometheus metrics & promtail log sender:
[https://github.com/wick02/monitoring](https://github.com/wick02/monitoring)
I also created a [video demo](https://www.youtube.com/watch?v=KPqbA7ys24o) of it working on a mac m1 along with a few of my old colleagues cloning it with no issues reported. I have around 6-7 years helping maintain logs and metric backends and this is my second video on Grafana which is available on [Grafana's youtube channel](https://www.youtube.com/watch?v=AgV5DoWcY6I&t=1544s) from a meetup in 2017.
**Goals of this repo:**
* To trim down to the very basics of each service, to isolate them from each other so you can pick and choose what you want to use from the demo.
* I've configured it in such a way where you can scale it in a cloud environment and to give something to the developers.
* It's not dependent on keeping volumes on the machine, so you can use something like Amazon ECS without managing the volumes and use spot servers to help cut costs.
* It's not a lot of code or configuration, it uses a lot of existing tutorials already but made in such a way that I think anyone with some operational experience can use and get started with.
* It's also built in a way where the metrics are pushed to an S3 like backend using minio so you can keep and persist all the logs and metrics.
* Lastly, it uses Tenant IDs, so you can isolate offenders if you need to use this as a massive shared service for the company by rate limiting them until they stop sending you too many metrics/logs as we all are accustomed to see when we manage these type of backends.
* Since it is simple to spin up a Mimir or Loki cluster with a design like this, you could make multiple clusters and isolate components away even further
I hope someone out there finds this useful. I hope to add Tempo in the future along with a terraform deployment process for this stack.
https://redd.it/10gfu0t
@r_devops
Wanted to share a demo/tutorial with everyone on how get started with a monitoring stack using grafana, loki and mimir with prometheus metrics & promtail log sender:
[https://github.com/wick02/monitoring](https://github.com/wick02/monitoring)
I also created a [video demo](https://www.youtube.com/watch?v=KPqbA7ys24o) of it working on a mac m1 along with a few of my old colleagues cloning it with no issues reported. I have around 6-7 years helping maintain logs and metric backends and this is my second video on Grafana which is available on [Grafana's youtube channel](https://www.youtube.com/watch?v=AgV5DoWcY6I&t=1544s) from a meetup in 2017.
**Goals of this repo:**
* To trim down to the very basics of each service, to isolate them from each other so you can pick and choose what you want to use from the demo.
* I've configured it in such a way where you can scale it in a cloud environment and to give something to the developers.
* It's not dependent on keeping volumes on the machine, so you can use something like Amazon ECS without managing the volumes and use spot servers to help cut costs.
* It's not a lot of code or configuration, it uses a lot of existing tutorials already but made in such a way that I think anyone with some operational experience can use and get started with.
* It's also built in a way where the metrics are pushed to an S3 like backend using minio so you can keep and persist all the logs and metrics.
* Lastly, it uses Tenant IDs, so you can isolate offenders if you need to use this as a massive shared service for the company by rate limiting them until they stop sending you too many metrics/logs as we all are accustomed to see when we manage these type of backends.
* Since it is simple to spin up a Mimir or Loki cluster with a design like this, you could make multiple clusters and isolate components away even further
I hope someone out there finds this useful. I hope to add Tempo in the future along with a terraform deployment process for this stack.
https://redd.it/10gfu0t
@r_devops
GitHub
GitHub - wick02/monitoring: Get a monitoring system up and rolling easily with a few steps
Get a monitoring system up and rolling easily with a few steps - wick02/monitoring
Feedback Request: TCO Calculation for Apache Kafka
I'm working on calculating the total cost of ownership (TCO) for tools like Apache Kafka to determine when to build vs. buy.
I'd love your feedback -- what am I missing? What did I underestimate/overestimate? How can I improve this?
First, the criteria to consider when calculating TCO:
Up-front costs
software cost & licensing, if applicable
learning & education
implementation & testing (including data migration costs)
documentation & knowledge sharing
customization
Ongoing costs
direct infrastructure costs (e.g., hosting & storage)
backup infrastructure costs (e.g., failover & additional AZs)
supporting infrastructure costs (e.g., monitoring & alerting)
maintenance, patches/upgrades, & support
feature additions
Team & opportunity costs
hiring to replace the engineers now working with the new software
time spent on infrastructure that could otherwise be spent on core product
Now, an example using the above criteria:
Desired specs for our example deployment (I picked one of the smaller Heroku plans):
Capacity: 300GB
Retention: 2 weeks
vCPU: 4
Ram: 16GB
Brokers: 3
Assuming an engineer has an all-in comp package of $200k/yr (this would obviously be different in every situation, for every geo), year one would look like:
||Building (on AWS)|Buying (Heroku)|
|:-|:-|:-|
|software cost & licensing|$0|$21,600|
|learning & education|$7,692 (2 eng \ 1 week)|$3,846 (1 eng * 1 week)|
|implementation & testing|$15,384 (2 eng * 2 weeks)|$7,692 (1 eng * 1 week)|
|infrastructure costs (see above specs)|$12,117.60|$0 (included in software cost)|
|supporting infrastructure costs (monitoring, etc.)|$1,200/yr|$1,200/yr|
|maintenance, patches/upgrades|$15,384 (2 eng * 2 weeks spread throughout the year)|$7,692 (1 eng * 2 weeks spread throughout the year)|
|Year 1 TCO|$51,777.60|$42,030|
Directionally, this example seems correct.
What do you think? What am I missing? What did I underestimate/overestimate? How can I improve this?
Thanks!
https://redd.it/10g9bk2
@r_devops
I'm working on calculating the total cost of ownership (TCO) for tools like Apache Kafka to determine when to build vs. buy.
I'd love your feedback -- what am I missing? What did I underestimate/overestimate? How can I improve this?
First, the criteria to consider when calculating TCO:
Up-front costs
software cost & licensing, if applicable
learning & education
implementation & testing (including data migration costs)
documentation & knowledge sharing
customization
Ongoing costs
direct infrastructure costs (e.g., hosting & storage)
backup infrastructure costs (e.g., failover & additional AZs)
supporting infrastructure costs (e.g., monitoring & alerting)
maintenance, patches/upgrades, & support
feature additions
Team & opportunity costs
hiring to replace the engineers now working with the new software
time spent on infrastructure that could otherwise be spent on core product
Now, an example using the above criteria:
Desired specs for our example deployment (I picked one of the smaller Heroku plans):
Capacity: 300GB
Retention: 2 weeks
vCPU: 4
Ram: 16GB
Brokers: 3
Assuming an engineer has an all-in comp package of $200k/yr (this would obviously be different in every situation, for every geo), year one would look like:
||Building (on AWS)|Buying (Heroku)|
|:-|:-|:-|
|software cost & licensing|$0|$21,600|
|learning & education|$7,692 (2 eng \ 1 week)|$3,846 (1 eng * 1 week)|
|implementation & testing|$15,384 (2 eng * 2 weeks)|$7,692 (1 eng * 1 week)|
|infrastructure costs (see above specs)|$12,117.60|$0 (included in software cost)|
|supporting infrastructure costs (monitoring, etc.)|$1,200/yr|$1,200/yr|
|maintenance, patches/upgrades|$15,384 (2 eng * 2 weeks spread throughout the year)|$7,692 (1 eng * 2 weeks spread throughout the year)|
|Year 1 TCO|$51,777.60|$42,030|
Directionally, this example seems correct.
What do you think? What am I missing? What did I underestimate/overestimate? How can I improve this?
Thanks!
https://redd.it/10g9bk2
@r_devops
reddit
Feedback Request: TCO Calculation for Apache Kafka
I'm working on calculating the total cost of ownership (TCO) for tools like Apache Kafka to determine when to build vs. buy. I'd love your...
Script or software that automatically populate specific profile in ~/.aws/credentials
cat \~/.aws/credentials
default
awsaccesskeyid = xxxx
awssecretaccesskey = yyyyy
foo
awsaccesskeyid = xxxxx
awssecretaccesskey = yyyyy
awssessiontoken = zzzzz
Every time I need to run `aws sts assume-role --role-arn arn:aws:iam::123456789012:role/xaccounts3access --role-session-name s3-access-example` then manually edit \~/.aws/credentials `foo` profile. I was wondering if there software or script that does it automatically for me?
https://redd.it/10ggej1
@r_devops
cat \~/.aws/credentials
default
awsaccesskeyid = xxxx
awssecretaccesskey = yyyyy
foo
awsaccesskeyid = xxxxx
awssecretaccesskey = yyyyy
awssessiontoken = zzzzz
Every time I need to run `aws sts assume-role --role-arn arn:aws:iam::123456789012:role/xaccounts3access --role-session-name s3-access-example` then manually edit \~/.aws/credentials `foo` profile. I was wondering if there software or script that does it automatically for me?
https://redd.it/10ggej1
@r_devops
reddit
Script or software that automatically populate specific profile in...
cat \~/.aws/credentials [default] aws_access_key_id = xxxx aws_secret_access_key = yyyyy [foo] aws_access_key_id =...
Hands-On: Kubernetes Gateway API With APISIX Ingress
A tutorial on using the new Kubernetes Gateway API with Apache APISIX Ingress. This is a hands-on walkthrough that you can follow on your own.
Read: https://navendu.me/posts/kubernetes-gateway-with-apisix/
https://redd.it/10gnldc
@r_devops
A tutorial on using the new Kubernetes Gateway API with Apache APISIX Ingress. This is a hands-on walkthrough that you can follow on your own.
Read: https://navendu.me/posts/kubernetes-gateway-with-apisix/
https://redd.it/10gnldc
@r_devops
Navendu Pottekkat
Kubernetes Gateway API With APISIX Ingress
A hands-on tutorial on using the new Kubernetes Gateway API with Apache APISIX Ingress.
Why do some SaaS have multiple subdomains for each business domain?
What is the logic of this? I feel like it adds complexity. Only thing I can think of is BFF (Backend for Frontends) architecture. Essentially each frontend app getting their own api gateway.
Examples:
Shopify
* Auth screens and anything to do with accounts is on accounts.example.com
* Admin Dashboard has admin.example.com
* storefront entirely different domain and subdomain. hello.myshopify.com (the customization makes sense, since its public facing)
I want to know the benefits and logic of having an architecture like this. Security reasons? Increases complexity quite a bit I feel. Like JWT coming from account.example.com, but then also valid on admin.example.com.
I see Jira does this: start.atlassian.com id.atlassian.com, yourname.atlassian.net
https://redd.it/10gpiaw
@r_devops
What is the logic of this? I feel like it adds complexity. Only thing I can think of is BFF (Backend for Frontends) architecture. Essentially each frontend app getting their own api gateway.
Examples:
Shopify
* Auth screens and anything to do with accounts is on accounts.example.com
* Admin Dashboard has admin.example.com
* storefront entirely different domain and subdomain. hello.myshopify.com (the customization makes sense, since its public facing)
I want to know the benefits and logic of having an architecture like this. Security reasons? Increases complexity quite a bit I feel. Like JWT coming from account.example.com, but then also valid on admin.example.com.
I see Jira does this: start.atlassian.com id.atlassian.com, yourname.atlassian.net
https://redd.it/10gpiaw
@r_devops
reddit
Why do some SaaS have multiple subdomains for each business domain?
What is the logic of this? I feel like it adds complexity. Only thing I can think of is BFF (Backend for Frontends) architecture. Essentially each...
Internal tooling ideas?
I am interested to hear the kinds of internal tooling people have created. Is there a tool you have made that had a significant impact on your team or organisation?
https://redd.it/10gsy86
@r_devops
I am interested to hear the kinds of internal tooling people have created. Is there a tool you have made that had a significant impact on your team or organisation?
https://redd.it/10gsy86
@r_devops
reddit
Internal tooling ideas?
I am interested to hear the kinds of internal tooling people have created. Is there a tool you have made that had a significant impact on your...
backstage.io common Issues and Pitfalls
We are a \~500 developers organization, we are planning a project to onboard Backstage, in order to improve our developer experience, and better enable our developers.
What are the common pitfalls, gotchas, and issues we should be aware of when onboarding into Backstage? Any common plugins we should look into? Any popular tooling that is used with Backstage?
Any thoughts and feedback would be really great!
https://redd.it/10guddj
@r_devops
We are a \~500 developers organization, we are planning a project to onboard Backstage, in order to improve our developer experience, and better enable our developers.
What are the common pitfalls, gotchas, and issues we should be aware of when onboarding into Backstage? Any common plugins we should look into? Any popular tooling that is used with Backstage?
Any thoughts and feedback would be really great!
https://redd.it/10guddj
@r_devops
backstage.io
Backstage Software Catalog and Developer Platform
Backstage is an open source developer portal framework that centralizes your software catalog, unifies infrastructure tools, and helps teams ship high-quality code faster.
It seems like we are always in diapers. The anxiety of never knowing what we are doing and the future...
Reading posts here like this or this makes me feel better about how I always keep forgetting things or making a fool of myself in interviews after years of experience because I either forgot something basic from disuse or I haven't used some shiny new tool enough.
I always manage to do a good job, but sometimes I feel I am perpetually stumbling my way there.
Do you feel we are in a transition period?
I feel we live in a post Big Bang era of DevOps (and tech in general) where there are just thousands of tools that do similar things, many of them betaish, alphaish - half cooked. Perhaps the future will have fewer, but more stable and user friendly options, and we will probably be delivering internal developer platforms with these tools... and a lot of ChatGPT and AI tools :-)
Any predictions about the future along these lines?
https://medium.com/@nandovillalba/devops-engineer-perpetually-in-diapers-a2b125d5906c
https://redd.it/10gslt1
@r_devops
Reading posts here like this or this makes me feel better about how I always keep forgetting things or making a fool of myself in interviews after years of experience because I either forgot something basic from disuse or I haven't used some shiny new tool enough.
I always manage to do a good job, but sometimes I feel I am perpetually stumbling my way there.
Do you feel we are in a transition period?
I feel we live in a post Big Bang era of DevOps (and tech in general) where there are just thousands of tools that do similar things, many of them betaish, alphaish - half cooked. Perhaps the future will have fewer, but more stable and user friendly options, and we will probably be delivering internal developer platforms with these tools... and a lot of ChatGPT and AI tools :-)
Any predictions about the future along these lines?
https://medium.com/@nandovillalba/devops-engineer-perpetually-in-diapers-a2b125d5906c
https://redd.it/10gslt1
@r_devops
Reddit
From the devops community on Reddit
Explore this post and more from the devops community
But really, why is all CI/CD pipelines?
So I've been deep in the bowels of our company's CI processes the last month or so, and I realize, everyone uses the idea of a pipeline, with steps, for CI/CD. CircleCI
These pipelines get really complex - our main pipeline for one project is ~400 lines of YAML - I could clean it up some but still, it's gonna be big, and we're about to add Playwright to the mix. I've heard of several orgs that have programs to generate their pipelines, and honestly I'm getting there myself.
My question/thought is - are pipelines the best way to represent the CI/CD process, or are they just an easy abstraction that caught on? Ultimately my big yaml file is a script interpreted by a black box VM run by whatever CI provider...and I just have to kinda hope their docs have the behavior right.
Am I crazy, or would it actually be better to define CI processes as what they are (a program), and get to use the language of my choice?
https://redd.it/10gzdqg
@r_devops
So I've been deep in the bowels of our company's CI processes the last month or so, and I realize, everyone uses the idea of a pipeline, with steps, for CI/CD. CircleCI
$$$, Buildkite <3, GHA >:( .These pipelines get really complex - our main pipeline for one project is ~400 lines of YAML - I could clean it up some but still, it's gonna be big, and we're about to add Playwright to the mix. I've heard of several orgs that have programs to generate their pipelines, and honestly I'm getting there myself.
My question/thought is - are pipelines the best way to represent the CI/CD process, or are they just an easy abstraction that caught on? Ultimately my big yaml file is a script interpreted by a black box VM run by whatever CI provider...and I just have to kinda hope their docs have the behavior right.
Am I crazy, or would it actually be better to define CI processes as what they are (a program), and get to use the language of my choice?
https://redd.it/10gzdqg
@r_devops
reddit
But really, why is all CI/CD pipelines?
So I've been deep in the bowels of our company's CI processes the last month or so, and I realize, everyone uses the idea of a pipeline, with...
"Accredited" DevOps Training/Cert Courses?
Does anyone know of any courses offered to prep for eventual certification in either SRE or CI/CD (DevOps Institute certs) that are considered to be "accredited"?
Seems that's the only way I can use my education funds at work and the usual programs I go to (AWS Coursework, Cloud Academy, etc...) don't see to mention anything about being accredited.
Thanks!
https://redd.it/10gxw59
@r_devops
Does anyone know of any courses offered to prep for eventual certification in either SRE or CI/CD (DevOps Institute certs) that are considered to be "accredited"?
Seems that's the only way I can use my education funds at work and the usual programs I go to (AWS Coursework, Cloud Academy, etc...) don't see to mention anything about being accredited.
Thanks!
https://redd.it/10gxw59
@r_devops
reddit
"Accredited" DevOps Training/Cert Courses?
Does anyone know of any courses offered to prep for eventual certification in either SRE or CI/CD (DevOps Institute certs) that are considered to...
fargate with react
Hi all!
I have some deploys in ECS (fargate) a frontend and a backend. The frontend is in a public subnet while the backend is private. The frontend uses react with axios, and initially I wanted to use service discovery, but.. I just forgot about the fact, the requests are made from client side not server side. So even though my service discovery work fine, it's useless.
Was wondering if this issue could be bridged using API Gateway or not ? Not very familiar with APIG.
Other thing was to deploy another middle server which would act as a gateway, but still, not the best solution because I have to hardcode the host in the react app, so I'd need a sub/domain for this as well to not depend on the IP.
Looking for a solution, or any ideas if you have... I have some apps and wondering how could I solve the issue the possible easiest and cost effective way.
https://redd.it/10h0asx
@r_devops
Hi all!
I have some deploys in ECS (fargate) a frontend and a backend. The frontend is in a public subnet while the backend is private. The frontend uses react with axios, and initially I wanted to use service discovery, but.. I just forgot about the fact, the requests are made from client side not server side. So even though my service discovery work fine, it's useless.
Was wondering if this issue could be bridged using API Gateway or not ? Not very familiar with APIG.
Other thing was to deploy another middle server which would act as a gateway, but still, not the best solution because I have to hardcode the host in the react app, so I'd need a sub/domain for this as well to not depend on the IP.
Looking for a solution, or any ideas if you have... I have some apps and wondering how could I solve the issue the possible easiest and cost effective way.
https://redd.it/10h0asx
@r_devops
reddit
fargate with react
Hi all! I have some deploys in ECS (fargate) a frontend and a backend. The frontend is in a public subnet while the backend is private. The...
Resume review - support engineer > cloud/Devops
Hey guys I’m currently a support engineer working on moving into a cloud engineering/ devops job I’m still working on updating my projects and adding new ones I found interesting while reading the cloud resume challenge guidebook and please note I’m still making adjustments to this while I keep learning and putting that knowledge to work in these projects any feedback would be greatly appreciated
https://imgur.com/a/exec3S0
https://redd.it/10gmx90
@r_devops
Hey guys I’m currently a support engineer working on moving into a cloud engineering/ devops job I’m still working on updating my projects and adding new ones I found interesting while reading the cloud resume challenge guidebook and please note I’m still making adjustments to this while I keep learning and putting that knowledge to work in these projects any feedback would be greatly appreciated
https://imgur.com/a/exec3S0
https://redd.it/10gmx90
@r_devops
Imgur
Post with 5 views.
Do any CI/CD systems allow for retrofit of cross-cutting concerns?
I’m wondering if any CI/CD systems allow this, or even if it’s a good idea. The thought is wanting to implement the same type of cross-cutting concern across all or most CI/CD pipelines across an org.
If you know what you need in advance it’s easier to build it in from the start in a reusable way. But, we invariably never predict the future. As examples, say later on you realize you want to implement security/vulnerability scanning, or audit/metrics aspects (such as tracking of DORA metrics). Is it a feasible thought of trying to implement this once in the CI/CD system itself, or are you stuck manually updating hundreds or thousands of pipelines?
https://redd.it/10hg862
@r_devops
I’m wondering if any CI/CD systems allow this, or even if it’s a good idea. The thought is wanting to implement the same type of cross-cutting concern across all or most CI/CD pipelines across an org.
If you know what you need in advance it’s easier to build it in from the start in a reusable way. But, we invariably never predict the future. As examples, say later on you realize you want to implement security/vulnerability scanning, or audit/metrics aspects (such as tracking of DORA metrics). Is it a feasible thought of trying to implement this once in the CI/CD system itself, or are you stuck manually updating hundreds or thousands of pipelines?
https://redd.it/10hg862
@r_devops
reddit
Do any CI/CD systems allow for retrofit of cross-cutting concerns?
I’m wondering if any CI/CD systems allow this, or even if it’s a good idea. The thought is wanting to implement the same type of cross-cutting...
Secret Management Across Environments / Vault
My team is growing, and we're running into an issue now where managing secrets is just getting too crazy. It's happened several times where a secret deployed to our integration environment doesn't exist in production and it halts the release. We want to do a release to staging? Forget it... We'll have to wade through all the secrets added since the last deploy.
I was thinking of creating a tool that allows uploading secrets to our environments, but whenever you upload a secret to one environment it forces you to specify it for all of them. Then I realized this is too common of a problem and surely there is a better solution.
1. I started looking into Vault. I'm not sure what to think of it though. I also still don't feel like I'm getting it. It sounds like Vault wants you to deploy an instance of it per environment, instead of having a single instance over all environments. If I have an instance of Vault for every single environment, it seems like Vault doesn't really solve my problem.
2. I'm not an ops guy, but this is going to fall to me to champion it. I'm not really finding a full explanation of how this is all going to work together on GCP.
3. If a secret expires from vault (because that's a thing apparently?), how does the server get a new value? Is retrieving the new value manual or automatic?
4. Is it ok to deploy Vault to a serverless environment like CloudRun? This means that the container won't get CPU cycles unless there is an active request. Will this cause me issues?
https://redd.it/10hipa7
@r_devops
My team is growing, and we're running into an issue now where managing secrets is just getting too crazy. It's happened several times where a secret deployed to our integration environment doesn't exist in production and it halts the release. We want to do a release to staging? Forget it... We'll have to wade through all the secrets added since the last deploy.
I was thinking of creating a tool that allows uploading secrets to our environments, but whenever you upload a secret to one environment it forces you to specify it for all of them. Then I realized this is too common of a problem and surely there is a better solution.
1. I started looking into Vault. I'm not sure what to think of it though. I also still don't feel like I'm getting it. It sounds like Vault wants you to deploy an instance of it per environment, instead of having a single instance over all environments. If I have an instance of Vault for every single environment, it seems like Vault doesn't really solve my problem.
2. I'm not an ops guy, but this is going to fall to me to champion it. I'm not really finding a full explanation of how this is all going to work together on GCP.
3. If a secret expires from vault (because that's a thing apparently?), how does the server get a new value? Is retrieving the new value manual or automatic?
4. Is it ok to deploy Vault to a serverless environment like CloudRun? This means that the container won't get CPU cycles unless there is an active request. Will this cause me issues?
https://redd.it/10hipa7
@r_devops
reddit
Secret Management Across Environments / Vault
My team is growing, and we're running into an issue now where managing secrets is just getting too crazy. It's happened several times where a...
Do you use Intune?
What do you think about Intune? Are you using it? Or do you have specific reasons to not to? Are you using something else? Is Intune fitting into a a complete DevOps solution for your job? I'm curious how common Intune is for people into devops.
https://redd.it/10hkl0e
@r_devops
What do you think about Intune? Are you using it? Or do you have specific reasons to not to? Are you using something else? Is Intune fitting into a a complete DevOps solution for your job? I'm curious how common Intune is for people into devops.
https://redd.it/10hkl0e
@r_devops
reddit
Do you use Intune?
What do you think about Intune? Are you using it? Or do you have specific reasons to not to? Are you using something else? Is Intune fitting into...
Git merge from development to production
The content of our file in the dev branch
>server: dev-server
parameters - 200|300
we change the parameters to 200, which needs to be moved to production. However the server portion should not be changed while merging. I am wondering how can/ways to achieve this using git?
prod configuration
>server: prod-server
parameters - 200|300
I know if we do a merge this will change the server portion as well.
As a side note we maintain git for maintaining the server configuration files for a Data quality software tool. This repository contains the configuration/files that is needed for that tool to be deployed properly.
There is shell script which takes this code from git and deploys to the server where the tool is hosted. After deployment, post a restart of the server, the changes will take into effect to the tool.
https://redd.it/10hl665
@r_devops
The content of our file in the dev branch
>server: dev-server
parameters - 200|300
we change the parameters to 200, which needs to be moved to production. However the server portion should not be changed while merging. I am wondering how can/ways to achieve this using git?
prod configuration
>server: prod-server
parameters - 200|300
I know if we do a merge this will change the server portion as well.
As a side note we maintain git for maintaining the server configuration files for a Data quality software tool. This repository contains the configuration/files that is needed for that tool to be deployed properly.
There is shell script which takes this code from git and deploys to the server where the tool is hosted. After deployment, post a restart of the server, the changes will take into effect to the tool.
https://redd.it/10hl665
@r_devops
reddit
Git merge from development to production
The content of our file in the dev branch >server: dev-server parameters - 200|300 we change the parameters to 200, which needs to be moved to...
Containers
Nowadays 2023 and going forward, I'm pretty new to containers and currently learning, is worth at all learn Docker concepts or even use Docker within an Orchestration technology as Kubernetes? I know that Docker manages containers for Apps mean while the container Technology can be swap for another one like RKT or CRI-O.. I'm wondering if as of today 2023 is Docker a solid proposal for example for green field projects? .. I haven't heard too much about RKT or CRI-O ... Seems like the buzzword in regards of containers is Docker but I'm here asking you... Any feedback appreciated thanks !
https://redd.it/10hja1t
@r_devops
Nowadays 2023 and going forward, I'm pretty new to containers and currently learning, is worth at all learn Docker concepts or even use Docker within an Orchestration technology as Kubernetes? I know that Docker manages containers for Apps mean while the container Technology can be swap for another one like RKT or CRI-O.. I'm wondering if as of today 2023 is Docker a solid proposal for example for green field projects? .. I haven't heard too much about RKT or CRI-O ... Seems like the buzzword in regards of containers is Docker but I'm here asking you... Any feedback appreciated thanks !
https://redd.it/10hja1t
@r_devops
reddit
Containers
Nowadays 2023 and going forward, I'm pretty new to containers and currently learning, is worth at all learn Docker concepts or even use Docker...