any pentester that you would recommend?
preferably modern ones and can pentest desktop and mobile
https://redd.it/11su6uv
@r_devops
preferably modern ones and can pentest desktop and mobile
https://redd.it/11su6uv
@r_devops
Reddit
r/devops on Reddit: any pentester that you would recommend?
Posted by u/linux_n00by - No votes and no comments
An adventure with SLOs, generic Prometheus alerting rules, and complex PromQL queries
I'm working on a library called Autometrics that makes it easy to add metrics to a code base and recently worked on support for SLOs/alerts. We ended up with a solution that enables us to have a single set of Prometheus recording/alerting rules that will work for any autometrics-instrumented project and the libraries use some fun label tricks to enable specific rules.
I wrote up a blog post about this experience here in case others are interested: https://fiberplane.com/blog/an-adventure-with-slos-generic-prometheus-alerting-rules-and-complex-promql-queries
https://redd.it/11svszk
@r_devops
I'm working on a library called Autometrics that makes it easy to add metrics to a code base and recently worked on support for SLOs/alerts. We ended up with a solution that enables us to have a single set of Prometheus recording/alerting rules that will work for any autometrics-instrumented project and the libraries use some fun label tricks to enable specific rules.
I wrote up a blog post about this experience here in case others are interested: https://fiberplane.com/blog/an-adventure-with-slos-generic-prometheus-alerting-rules-and-complex-promql-queries
https://redd.it/11svszk
@r_devops
GitHub
Autometrics
Easily add metrics to your code that actually help you spot and debug issues in production. Built on Prometheus and OpenTelemetry. - Autometrics
Best Tools for Kubernetes Local Development: A Comprehensive Guide
Testing your code before deploying it to a production environment is vital.
One of the ways to do this is to develop and test your code using a local Kubernetes cluster.
In this article, Shingai shared the best tools for Kubernetes local development 👇
https://blog.getambassador.io/best-tools-for-kubernetes-local-development-a-comprehensive-guide-3577d351d31e?utm\_source=Social-Media&utm\_medium=Reddit&utm\_campaign=SM
https://redd.it/11sx8un
@r_devops
Testing your code before deploying it to a production environment is vital.
One of the ways to do this is to develop and test your code using a local Kubernetes cluster.
In this article, Shingai shared the best tools for Kubernetes local development 👇
https://blog.getambassador.io/best-tools-for-kubernetes-local-development-a-comprehensive-guide-3577d351d31e?utm\_source=Social-Media&utm\_medium=Reddit&utm\_campaign=SM
https://redd.it/11sx8un
@r_devops
Medium
Best Tools for Kubernetes Local Development: A Comprehensive Guide
Discover the top tools for Kubernetes local development, how to use them, and why you need this solution for your projects. As a…
Can I get help with my resume?
Hello everyone!
I am asking for help as it is the first time I've had to update my cv since before I graduated and I'm really struggling.
I've done quite a few task in my current job and I don't even know how to group them together or if I should try to summarized everything in 3- 4 points , tbh I feel it's looking awful and it's just a word salad so I haven't finished it in case I was wasting my time and I had to completely change it.
Also, I have some TA experience in grad classes and I did an internship as a data engineer should I add them to my cv considering I will be searching for devops job?
Any advice or criticism would be greatly appreciated !
https://imgur.com/a/rJMuydN
Thanks!
https://redd.it/11sxglh
@r_devops
Hello everyone!
I am asking for help as it is the first time I've had to update my cv since before I graduated and I'm really struggling.
I've done quite a few task in my current job and I don't even know how to group them together or if I should try to summarized everything in 3- 4 points , tbh I feel it's looking awful and it's just a word salad so I haven't finished it in case I was wasting my time and I had to completely change it.
Also, I have some TA experience in grad classes and I did an internship as a data engineer should I add them to my cv considering I will be searching for devops job?
Any advice or criticism would be greatly appreciated !
https://imgur.com/a/rJMuydN
Thanks!
https://redd.it/11sxglh
@r_devops
Imgur
Discover the magic of the internet at Imgur, a community powered entertainment destination. Lift your spirits with funny jokes, trending memes, entertaining gifs, inspiring stories, viral videos, and so much more from users.
Why is identity federation better than an alternative?
This is something I've yet to find a good answer on. Identity federation is supposedly to best zero trust way of handling automation, but how is this the case? I agree that a single account with access to everything is bad, but that's essentially what a "federated account" that's able to assume other permissions is as well.
I see the only difference being that assuming other permissions requires an extra token to pass around, and you can segment the tokens and permissions better but is that all there is to the benefits? So instead of leaking that one single account and access with it, you also have to leak the password for the assumed identities with actual permissions?
If the main account has access to read the secrets required to assume permissions, did anything really change security-wise? Perhaps the attacker must now figure out what permissions to assume? Is there some secret to the main accounts permissions that makes this really good in comparison to some alternative?
https://redd.it/11t03qu
@r_devops
This is something I've yet to find a good answer on. Identity federation is supposedly to best zero trust way of handling automation, but how is this the case? I agree that a single account with access to everything is bad, but that's essentially what a "federated account" that's able to assume other permissions is as well.
I see the only difference being that assuming other permissions requires an extra token to pass around, and you can segment the tokens and permissions better but is that all there is to the benefits? So instead of leaking that one single account and access with it, you also have to leak the password for the assumed identities with actual permissions?
If the main account has access to read the secrets required to assume permissions, did anything really change security-wise? Perhaps the attacker must now figure out what permissions to assume? Is there some secret to the main accounts permissions that makes this really good in comparison to some alternative?
https://redd.it/11t03qu
@r_devops
Reddit
r/devops on Reddit: Why is identity federation better than an alternative?
Posted by u/NUTTA_BUSTAH - No votes and no comments
Is this a "manage up" or "shut up" situation with a new devops manager?
Hey /r/Devops, how's it goin? You good? Good.
I need some help, or would like for some feedback on something. I'm an IC (individual contributor) again after a few years as a team lead, while I was effective and my teams had really good outcomes and my subordinates had really encouraging and positive things to say about my leadership style when I left the company...
...I came to realize being a workplace leader was asking more of me than I wanted to give to it and it was taking a serious mental toll, so I departed my last job and took a sabbatical for a few months.
Found a new job not long after reactivating the job search and have been very happy where I am over the last year and a half as an IC on an embedded platform team. Instead of one devops "team", we have practitioners partitioned out to different development pods in the organization (best I can tell it's very similar to the spotify team model) working as platform engineers.
The problem is my new manager, who now manages all of PlatformEng whereas previously we reported to the manager of the pod we were assigned to. He's been at the company only a few months now and brought with him a very "us versus them", and high-blame mentality. Specifically blaming dev teams for almost everything.
I don't hold such an adversarial opinion of the developers I support, but the new manager's tone and statements are constantly full of accusation towards them whenever we're reviewing the previous night's PagerDuty incidents during our standups, 1:1's between he and I, and sometimes it even leaks into sprint planning with the rest of the platform team.
Which I have two problems with:
a) Many times now the blame he's applying for why something broke or why a certain deployment incident happened are demonstrably and provably incorrect on many levels, but are consistently targeted at the teams we support. This has had the result of people chasing down red herrings to fix certain things under his orders.
b) if I don't step in and curtly change the topic back to our 1:1 agenda or if on our PE meetings if no one says anything about time left, he absolutely will chew up an entire meeting ranting about what other teams or managers are doing or aren't doing to his satisfaction.
He and I have a perfectly fine and so far healthy working relationship, I can ask for things and have confidence he'll act on it, if there's a blocker he's so far been consistent about doing something about it, so I got nothing to really say about that specific side of the coin, but the other side...I just don't find very helpful.
What do you do here?
Manage up? How do I do this tactfully, but effectively?
Shut up.
tl;dr - New manager wants to blame developers for everything instead of finding ways to solve our shared problems, has the effect of frequently taking team members off task, and prevents project planning because he wants to rant for the entirety of 1 on 1's and other meetings.
https://redd.it/11szz1o
@r_devops
Hey /r/Devops, how's it goin? You good? Good.
I need some help, or would like for some feedback on something. I'm an IC (individual contributor) again after a few years as a team lead, while I was effective and my teams had really good outcomes and my subordinates had really encouraging and positive things to say about my leadership style when I left the company...
...I came to realize being a workplace leader was asking more of me than I wanted to give to it and it was taking a serious mental toll, so I departed my last job and took a sabbatical for a few months.
Found a new job not long after reactivating the job search and have been very happy where I am over the last year and a half as an IC on an embedded platform team. Instead of one devops "team", we have practitioners partitioned out to different development pods in the organization (best I can tell it's very similar to the spotify team model) working as platform engineers.
The problem is my new manager, who now manages all of PlatformEng whereas previously we reported to the manager of the pod we were assigned to. He's been at the company only a few months now and brought with him a very "us versus them", and high-blame mentality. Specifically blaming dev teams for almost everything.
I don't hold such an adversarial opinion of the developers I support, but the new manager's tone and statements are constantly full of accusation towards them whenever we're reviewing the previous night's PagerDuty incidents during our standups, 1:1's between he and I, and sometimes it even leaks into sprint planning with the rest of the platform team.
Which I have two problems with:
a) Many times now the blame he's applying for why something broke or why a certain deployment incident happened are demonstrably and provably incorrect on many levels, but are consistently targeted at the teams we support. This has had the result of people chasing down red herrings to fix certain things under his orders.
b) if I don't step in and curtly change the topic back to our 1:1 agenda or if on our PE meetings if no one says anything about time left, he absolutely will chew up an entire meeting ranting about what other teams or managers are doing or aren't doing to his satisfaction.
He and I have a perfectly fine and so far healthy working relationship, I can ask for things and have confidence he'll act on it, if there's a blocker he's so far been consistent about doing something about it, so I got nothing to really say about that specific side of the coin, but the other side...I just don't find very helpful.
What do you do here?
Manage up? How do I do this tactfully, but effectively?
Shut up.
tl;dr - New manager wants to blame developers for everything instead of finding ways to solve our shared problems, has the effect of frequently taking team members off task, and prevents project planning because he wants to rant for the entirety of 1 on 1's and other meetings.
https://redd.it/11szz1o
@r_devops
Reddit
r/devops on Reddit: Is this a "manage up" or "shut up" situation with a new devops manager?
Posted by u/wtfsoda - No votes and 11 comments
Courses for Cloud Computing
Hey there
My previous job was running with on-premise infrastructure and I was doing plenty of different things - provisioning VMs, creating and managing Kubernetes clusters, writing applications (backend & frontend), etc… In general I am very up to date with new technologies - and have good background in the entire hardware and software stack.
In a few weeks time I will be moving to a company that works only on cloud-computing infrastructure, and not using on-premise hardware and infrastructure anymore. I am not at all concerned about my ability to work with those environments, but I do want to learn about how to use cloud computing well.
What is the best way in your opinion to earn that knowledge and experience in the few weeks I have prior to starting that job? Any particular courses that you recommend?
Thanks a-lot in advance.
https://redd.it/11t1pm3
@r_devops
Hey there
My previous job was running with on-premise infrastructure and I was doing plenty of different things - provisioning VMs, creating and managing Kubernetes clusters, writing applications (backend & frontend), etc… In general I am very up to date with new technologies - and have good background in the entire hardware and software stack.
In a few weeks time I will be moving to a company that works only on cloud-computing infrastructure, and not using on-premise hardware and infrastructure anymore. I am not at all concerned about my ability to work with those environments, but I do want to learn about how to use cloud computing well.
What is the best way in your opinion to earn that knowledge and experience in the few weeks I have prior to starting that job? Any particular courses that you recommend?
Thanks a-lot in advance.
https://redd.it/11t1pm3
@r_devops
Reddit
r/devops on Reddit: Courses for Cloud Computing
Posted by u/xColdWar - No votes and 2 comments
Learning Git
As an Ops guy I haven't really used Git to its full potential, but I feel as though I'm missing out on a lot of features to make my own workflows more efficient.
I'm looking for a way to learn more about the more advanced concepts withing risking nuking my own environments (although probably the best way to learn).
Is there any online (interactive) course / labs you'd recommend, or a book with practical hands-on examples?
https://redd.it/11t5dlz
@r_devops
As an Ops guy I haven't really used Git to its full potential, but I feel as though I'm missing out on a lot of features to make my own workflows more efficient.
I'm looking for a way to learn more about the more advanced concepts withing risking nuking my own environments (although probably the best way to learn).
Is there any online (interactive) course / labs you'd recommend, or a book with practical hands-on examples?
https://redd.it/11t5dlz
@r_devops
Reddit
r/devops on Reddit: Learning Git
Posted by u/sbbh1 - No votes and no comments
Developer Dashboards?
Hello!
What's people's experience with developer dashboards? Minimally something that can show CI status across different branches, but could also pull in test status from, e.g., TestRail, show code coverage stats/trends, maybe have links to autogenerated (in CI) API docs for code, etc.
Thanks!
https://redd.it/11t5cf6
@r_devops
Hello!
What's people's experience with developer dashboards? Minimally something that can show CI status across different branches, but could also pull in test status from, e.g., TestRail, show code coverage stats/trends, maybe have links to autogenerated (in CI) API docs for code, etc.
Thanks!
https://redd.it/11t5cf6
@r_devops
Reddit
r/devops on Reddit: Developer Dashboards?
Posted by u/godfool - No votes and 7 comments
What do you really think of your daily standup?
Most posts about daily standup/daily scrum are polarized- it's either the most important meeting of the day or a total waste of time. I'm a developer turned founder, and I'm trying to find out what the majority really thinks of daily standup and share the results. Share your opinions in Typeform. There's a thank you at the end. https://y90utja3h0v.typeform.com/to/xlCGiutr
https://redd.it/11t7i3x
@r_devops
Most posts about daily standup/daily scrum are polarized- it's either the most important meeting of the day or a total waste of time. I'm a developer turned founder, and I'm trying to find out what the majority really thinks of daily standup and share the results. Share your opinions in Typeform. There's a thank you at the end. https://y90utja3h0v.typeform.com/to/xlCGiutr
https://redd.it/11t7i3x
@r_devops
Typeform
What do you really think of your standup?
What do engineers think about their daily standup? Take this survey and receive a coffee on us (ie $5).
Is this possible to generate unique certificate for different subdomains based on root cert?
I have a case with multiple hundred virtual desktops shared with user, all of these desktops will have a subdomain, like desktop1.mysite.com, desktop2.mysite.com , etc. Assuming I have a CA certificate generated for mysite.com. Is this possible to generate the separate certificates for these subdomains, keeping in mind that user will have access to the private key? May I do it by myself programmatically, using openssl for example?
https://redd.it/11t00q9
@r_devops
I have a case with multiple hundred virtual desktops shared with user, all of these desktops will have a subdomain, like desktop1.mysite.com, desktop2.mysite.com , etc. Assuming I have a CA certificate generated for mysite.com. Is this possible to generate the separate certificates for these subdomains, keeping in mind that user will have access to the private key? May I do it by myself programmatically, using openssl for example?
https://redd.it/11t00q9
@r_devops
Devops start
Hi there,
Is this a good career to transition to out of being an IT support specialist? Does anyone have some personal insight into how long/difficult the road is to get into this career path?
Thanks so much!!
https://redd.it/11tbpcc
@r_devops
Hi there,
Is this a good career to transition to out of being an IT support specialist? Does anyone have some personal insight into how long/difficult the road is to get into this career path?
Thanks so much!!
https://redd.it/11tbpcc
@r_devops
Reddit
r/devops on Reddit: Devops start
Posted by u/Malaine1 - No votes and no comments
Experience working and using Managed Service Providers?
Hey all, was just wondering if anyone has experience redistributing or selling MSP's through partnerships. Saw Stellar Cyber released a partner program for users and thought I'd potentially give it a shot.
I know there's others like SecureFirst, Fortinet, SecureWorks etc. If anyone has experience with any of these platforms whether you use them for your company or work as a partner I'd love to know more, thanks!
https://redd.it/11tcd5g
@r_devops
Hey all, was just wondering if anyone has experience redistributing or selling MSP's through partnerships. Saw Stellar Cyber released a partner program for users and thought I'd potentially give it a shot.
I know there's others like SecureFirst, Fortinet, SecureWorks etc. If anyone has experience with any of these platforms whether you use them for your company or work as a partner I'd love to know more, thanks!
https://redd.it/11tcd5g
@r_devops
CRN
Open XDR Vendor Stellar Cyber Debuts Partner Program For Resellers
Open XDR vendor Stellar Cyber announced its first channel partner program for resellers.
Thank you, WeaveWorks!
Not sponsored or something.
I started using fluxcd heavily three months ago and immediately fell in love with it. The way it is designed, it’s speed and it’s robustness is very cool. Seeing how CD really becomes CD without all the hassle of managing infrastructure and pipelines is a true game changer.
I recently started implementing tools to optimize our automated testing, because we are planning to implement flagger using k6 load testing. After a couple of days I figured that all these new tools (testkube, tracetest, etc.) need an own management k8s cluster.
I was wondering: how can I install an EKS cluster quickly, without terraform? That’s when I discovered another tool: eksctl. Made by WeaveWorks. Again. I won’t explain why I love it, but believe me when I say it’s great. Best things: flux integration and IRSA management.
So now my cluster is running and my apps use pvc. Some of them also need Aurora DBs. I was wondering: how can I configure and create the Auroras using GitOps ways? That’s when I encountered crossplane. I installed it, wrote some XRs and XRDs and liked it initially, but it installing thousands of CRDs made the cluster unresponsive. The helm controller of flux even started crashing. So I decided to remove it. But I love the concept and idea of deploying infrastructure with k8s as orchestrator while using GitOps and started searching a little. That’s when I discovered another component: tf-controller. Made by WeaveWorks. Again.
I just deployed a s3 bucket and the speed of it is so great! I will try to deploy a rds cluster like I did with crossplane already to see how it compares tomorrow.
But: I just want to say thank you already. You can’t believe how much easier your genius makes my everyday life. Everything you guys do and develop has such a high quality and design. You are the Apple of DevOps for me. I can’t thank you enough for the work you have done. I am a big fan and if there is a chance to get any insights about the way you decide on designing your apps and tools, please let me know. Your ideas are perfect and just what our industry needs. Thanks <3
https://redd.it/11t8tbp
@r_devops
Not sponsored or something.
I started using fluxcd heavily three months ago and immediately fell in love with it. The way it is designed, it’s speed and it’s robustness is very cool. Seeing how CD really becomes CD without all the hassle of managing infrastructure and pipelines is a true game changer.
I recently started implementing tools to optimize our automated testing, because we are planning to implement flagger using k6 load testing. After a couple of days I figured that all these new tools (testkube, tracetest, etc.) need an own management k8s cluster.
I was wondering: how can I install an EKS cluster quickly, without terraform? That’s when I discovered another tool: eksctl. Made by WeaveWorks. Again. I won’t explain why I love it, but believe me when I say it’s great. Best things: flux integration and IRSA management.
So now my cluster is running and my apps use pvc. Some of them also need Aurora DBs. I was wondering: how can I configure and create the Auroras using GitOps ways? That’s when I encountered crossplane. I installed it, wrote some XRs and XRDs and liked it initially, but it installing thousands of CRDs made the cluster unresponsive. The helm controller of flux even started crashing. So I decided to remove it. But I love the concept and idea of deploying infrastructure with k8s as orchestrator while using GitOps and started searching a little. That’s when I discovered another component: tf-controller. Made by WeaveWorks. Again.
I just deployed a s3 bucket and the speed of it is so great! I will try to deploy a rds cluster like I did with crossplane already to see how it compares tomorrow.
But: I just want to say thank you already. You can’t believe how much easier your genius makes my everyday life. Everything you guys do and develop has such a high quality and design. You are the Apple of DevOps for me. I can’t thank you enough for the work you have done. I am a big fan and if there is a chance to get any insights about the way you decide on designing your apps and tools, please let me know. Your ideas are perfect and just what our industry needs. Thanks <3
https://redd.it/11t8tbp
@r_devops
Reddit
r/devops on Reddit: Thank you, WeaveWorks!
Posted by u/Financial_Comb_3550 - No votes and no comments
Terraform automation with GitHub and GCP Workload Identity Federation
This is how I automate IaC following the least privilege principle with GitHub and Google Workload Identity Federation. Hope you find it useful...
The workflow will run terraform plan and apply base on the event triggering the workflow, and based on that will use a dedicated service account to allow us to strictly follow the least privilege principle. If the workflow is triggered by a
https://youtu.be/DMwl9WcSAL8
https://redd.it/11t5r2z
@r_devops
This is how I automate IaC following the least privilege principle with GitHub and Google Workload Identity Federation. Hope you find it useful...
The workflow will run terraform plan and apply base on the event triggering the workflow, and based on that will use a dedicated service account to allow us to strictly follow the least privilege principle. If the workflow is triggered by a
pull_request event the workflow will execute the step terraform plan with the tf-plan service account. If instead it is triggered by a push against main it will execute the apply step using service account authorised to manage the resources in GCP.https://youtu.be/DMwl9WcSAL8
https://redd.it/11t5r2z
@r_devops
YouTube
Automate Terraform with GitHub and GCP Workload Identity Federation
In this video we are going to create a workflow in GitHub that allows you to manage infrastructure changes.
The workflow will run terraform plan and apply base on the event triggering the workflow, and based on that will use a dedicated service account to…
The workflow will run terraform plan and apply base on the event triggering the workflow, and based on that will use a dedicated service account to…
Prometheus Push Architecture
I know Prom is extremely opinionated and only does pulls. That's fine, I understand where they're coming from.
But I have some highly mobile devices (think Raspberry Pis or phones) that may not be connected at all times. So whatever application they're running, metrics can't be collected at all times.
Another use case might be egress-only networks. So if you can't set up VPNs to your edge devices, they can only push out to a well known Prom endpoint.
Therefore I want to push metrics (and queued metrics) instead.
Is pushgateway still the way to go? Or are Prometheus "extensions" like Thanos?
https://redd.it/11t3yyv
@r_devops
I know Prom is extremely opinionated and only does pulls. That's fine, I understand where they're coming from.
But I have some highly mobile devices (think Raspberry Pis or phones) that may not be connected at all times. So whatever application they're running, metrics can't be collected at all times.
Another use case might be egress-only networks. So if you can't set up VPNs to your edge devices, they can only push out to a well known Prom endpoint.
Therefore I want to push metrics (and queued metrics) instead.
Is pushgateway still the way to go? Or are Prometheus "extensions" like Thanos?
https://redd.it/11t3yyv
@r_devops
Reddit
r/devops on Reddit: Prometheus Push Architecture
Posted by u/zachlab - No votes and 3 comments
How to run jenkins pipeline jobs in parallel which call the same downstream job
I am a beginner to jenkins and making groovy scripts for pipelines, I want to trigger a downstream pipeline in parallel for all the files inside a folder given by the user... below is the sample code I wrote:-
​
def GLOBAL_RELEASE_NUMBER
def GLOBAL_BUILD_NUMBER
pipeline {
agent { label 'centos7-itest' }
options {
timestamps()
buildDiscarder(
logRotator(
daysToKeepStr: '100'
)
)
ansiColor('xterm')
}
parameters {
//some parameters
}
environment {
// For python3
}
stages{
stage("setting environment") {
environment {
//setting up environment
}
steps {
script{
// deciding build number and release number
}
}
}
stage("Clone repo & replace variables & call my pipeline") {
steps {
withCredentials([
//credentials
]){
cleanWs()
deleteDir()
git branch: "${params.branch}", credentialsId: 'jenkins-user-key-vcs', url: '[email protected]:some_repo/devops.git '
script {
sizingFiles = []
def branches = [:]
def counter=0
if (params.sizing_directory.endsWith(".yaml")) {
sizingFiles.add(params.sizing_directory)
} else {
sh(
returnStdout: true,
script: "find ${params.sizing_directory} -type f -name '*.yaml'"
).trim().split('\n').each { sizingFile ->
sizingFiles.add(sizingFile)
}
}
for (def sizingFile in sizingFiles) {
echo "Processing ${sizingFile}"
sh """
sed -i 's/{{[[:space:]]*user[[:space:]]*}}/${params.test_user}/g;
s/{{[[:space:]]*owner[[:space:]]*}}/my_team/g;
s/{{[[:space:]]*dept[[:space:]]*}}/team/g;
s/{{[[:space:]]*task[[:space:]]*}}/sizing/g;
s/{{[[:space:]]*SoftwareVersion[[:space:]]*}}/$GLOBAL_RELEASE_NUMBER-b$GLOBAL_BUILD_NUMBER/g' ${sizingFile}
cat ${sizingFile}
"""
branches[counter] = {
stage('yb'){
build job: "Myteam/myPipeline",
wait: false,
parameters: [
text(name: 'sample_yaml', value: readFile(file: sizingFile)),
string(name: 'branch', value: "${params.branch}")
]
}
counter+=1
}
I am a beginner to jenkins and making groovy scripts for pipelines, I want to trigger a downstream pipeline in parallel for all the files inside a folder given by the user... below is the sample code I wrote:-
​
def GLOBAL_RELEASE_NUMBER
def GLOBAL_BUILD_NUMBER
pipeline {
agent { label 'centos7-itest' }
options {
timestamps()
buildDiscarder(
logRotator(
daysToKeepStr: '100'
)
)
ansiColor('xterm')
}
parameters {
//some parameters
}
environment {
// For python3
}
stages{
stage("setting environment") {
environment {
//setting up environment
}
steps {
script{
// deciding build number and release number
}
}
}
stage("Clone repo & replace variables & call my pipeline") {
steps {
withCredentials([
//credentials
]){
cleanWs()
deleteDir()
git branch: "${params.branch}", credentialsId: 'jenkins-user-key-vcs', url: '[email protected]:some_repo/devops.git '
script {
sizingFiles = []
def branches = [:]
def counter=0
if (params.sizing_directory.endsWith(".yaml")) {
sizingFiles.add(params.sizing_directory)
} else {
sh(
returnStdout: true,
script: "find ${params.sizing_directory} -type f -name '*.yaml'"
).trim().split('\n').each { sizingFile ->
sizingFiles.add(sizingFile)
}
}
for (def sizingFile in sizingFiles) {
echo "Processing ${sizingFile}"
sh """
sed -i 's/{{[[:space:]]*user[[:space:]]*}}/${params.test_user}/g;
s/{{[[:space:]]*owner[[:space:]]*}}/my_team/g;
s/{{[[:space:]]*dept[[:space:]]*}}/team/g;
s/{{[[:space:]]*task[[:space:]]*}}/sizing/g;
s/{{[[:space:]]*SoftwareVersion[[:space:]]*}}/$GLOBAL_RELEASE_NUMBER-b$GLOBAL_BUILD_NUMBER/g' ${sizingFile}
cat ${sizingFile}
"""
branches[counter] = {
stage('yb'){
build job: "Myteam/myPipeline",
wait: false,
parameters: [
text(name: 'sample_yaml', value: readFile(file: sizingFile)),
string(name: 'branch', value: "${params.branch}")
]
}
counter+=1
}
}
parallel branches
}
}
}
}
}
}
The issue is when I trigger this pipeline with a folder containing 2 yaml files, I notice that the the job is triggered for first file and first completes the job and then goes to the next file to run the job for this file. I want to run all the jobs in parallel hence I gave the "wait:false" for the individual jobs. Can someone point out if what I am doing wrong?
https://redd.it/11thwql
@r_devops
parallel branches
}
}
}
}
}
}
The issue is when I trigger this pipeline with a folder containing 2 yaml files, I notice that the the job is triggered for first file and first completes the job and then goes to the next file to run the job for this file. I want to run all the jobs in parallel hence I gave the "wait:false" for the individual jobs. Can someone point out if what I am doing wrong?
https://redd.it/11thwql
@r_devops
Reddit
r/devops on Reddit: How to run jenkins pipeline jobs in parallel which call the same downstream job
Posted by u/PerceptionGreen4566 - No votes and no comments
Devops interview types?
Hey fellow devops folks, what has been you interview type like? Take home assignment or some leetcode programming live session or drilling devops technical knowledge of various tools?
https://redd.it/11t3rp2
@r_devops
Hey fellow devops folks, what has been you interview type like? Take home assignment or some leetcode programming live session or drilling devops technical knowledge of various tools?
https://redd.it/11t3rp2
@r_devops
Reddit
r/devops on Reddit: Devops interview types?
Posted by u/tofumanboykid - 1 vote and 6 comments
Any alternative to Redshift for streaming data from Aurora for analytics?
Hey guys,
So we use Redash to run lot of analytics queries on Aurora and we are seeing its limitations now.
The obvious choice is to use Redshift since our DBs are AWS Aurora.
But was wondering if there is a better alternative to look for?
We are thinking BigQuery, ClickHouse and Snowflake.
Anybody has any experience with this?
Our requirements are:
* Connector for streaming data from Aurora. Ideally real-time.
* Connection with Redash
* Ruby ActiveRecord gem
BigQuery, Clickhouse and Redshift have ActiveRecord gems for connection, but Snowflake doesn't.
But Redshift seems the only option if we need real time streaming of data.
​
Thank you.
https://redd.it/11tjo1a
@r_devops
Hey guys,
So we use Redash to run lot of analytics queries on Aurora and we are seeing its limitations now.
The obvious choice is to use Redshift since our DBs are AWS Aurora.
But was wondering if there is a better alternative to look for?
We are thinking BigQuery, ClickHouse and Snowflake.
Anybody has any experience with this?
Our requirements are:
* Connector for streaming data from Aurora. Ideally real-time.
* Connection with Redash
* Ruby ActiveRecord gem
BigQuery, Clickhouse and Redshift have ActiveRecord gems for connection, but Snowflake doesn't.
But Redshift seems the only option if we need real time streaming of data.
​
Thank you.
https://redd.it/11tjo1a
@r_devops
Reddit
r/devops on Reddit: Any alternative to Redshift for streaming data from Aurora for analytics?
Posted by u/theBeeprApp - No votes and no comments
How often do you do deployments at your startup/company? A poll (version 2)
Just to get a feel for how DevOps/SRE culture has impacted the deployment frequency at various companies/startups for your PRODUCTION environment.
And just to clarify it means "how often do you deploy 1 particular selected component", not if you have one artifact and you need to deploy it to hundreds of prod environments.
​
Thank you very much for your answer!
View Poll
https://redd.it/11tktk3
@r_devops
Just to get a feel for how DevOps/SRE culture has impacted the deployment frequency at various companies/startups for your PRODUCTION environment.
And just to clarify it means "how often do you deploy 1 particular selected component", not if you have one artifact and you need to deploy it to hundreds of prod environments.
​
Thank you very much for your answer!
View Poll
https://redd.it/11tktk3
@r_devops
Reddit
r/devops on Reddit: How often do you do deployments at your startup/company? A poll (version 2)
Posted by u/Bubbly_Penalty6048 - No votes and no comments