Integration of IBM WebSphere with Jenkins

Hello Everyone,
And I am working as Devops Trainee we have been assigned a task in which we have to deploy .war in IBM WebSphere thought pipeline script and I am unable to integrate Jenkins with WebSphere.I need help in integration and what steps we have to write in pipeline script.Our IBM WebSphere is installed in Windows Platform (localhost)

https://redd.it/136am41
@r_devops

Build Static Application Files Into Container or Download at Runtime?

Not sure what container building best practices are, I've always just been a consumer in the past.

Our current docker container qa build process for an asp.net application is that all of the app files are downloaded from a NAS in the ENTRYPOINT powershell script. There is a RUN powershell script that preps IIS but that's all. Everything else is done during ENTRYPOINT . The version grabbed from the NAS is set by environment variable and the container image is versioned separately from the application. All application files are stored in a volume and overwritten by another pull when the environment variable is changed.

In the past I've always used containers that are versioned the same the application and volumes for just the state/userdata.

Is there benefits to downloading files at runtime as opposed to building them into the image? I suppose it keeps the image smaller.

https://redd.it/1367hmj
@r_devops

How do i keep track for each of the step on the entire process

Basically i'm offering a service that would transform an asset (video, image, and audio). The entire process consists of a user uploading an asset to s3, this would then trigger a lambda function then trigger another lambda function then trigger another lambda function. I would like to keep track of what step is the user currently in and display it into their mobile screen real-time. Any leads on what should i look into?

​

https://imgur.com/a/51Zon7G

https://redd.it/136fkn8
@r_devops

1000 100% OFF coupons: DevOps course for small companies and individuals

Hello everyone,

I've made a DevOps course covering a lot of different technologies and applications, aimed at startups, small companies and individuals who want to self-host their infrastructure.
To get this out of the way - this course doesn't cover Kubernetes or similar - I'm of the opinion that for startups, small companies, and especially individuals, you probably don't need Kubernetes. Unless you have a whole DevOps team, it usually brings more problems than benefits, and unnecessary infrastructure bills buried a lot of startups before they got anywhere.

As for prerequisites, you can't be a complete beginner in the world of computers. If you've never even heard of Docker, if you don't know at least something about DNS, or if you don't have any experience with Linux, this course is probably not for you. That being said, I do explain the basics too, but probably not in enough detail for a complete beginner.

Here's a 100% OFF coupon if you want to check it out:

https://www.udemy.com/course/real-world-devops-project-from-start-to-finish/?couponCode=FREEDEVOPS2305KOQYV

Be sure to BUY the course for $0, and not sign up for Udemy's subscription plan. The Subscription plan is selected by default, but you want the BUY checkbox. If you see a price other than $0, chances are that all coupons have been used already.
You can try manually entering the coupon code because Udemy sometimes messes with the link.

The accompanying files for the course are at https://github.com/predmijat/realworlddevopscourse

I encourage you to watch "free preview" videos to get the sense of what will be covered, but here's the gist:

The goal of the course is to create an easily deployable and reproducible server which will have "everything" a startup or a small company will need - VPN, mail, Git, CI/CD, messaging, hosting websites and services, sharing files, calendar, etc. It can also be useful to individuals who want to self-host all of those - I ditched Google 99.9% and other than that being a good feeling, I'm not worried that some AI bug will lock my account with no one to talk to about resolving the issue.

Considering that it covers a wide variety of topics, it doesn't go in depth in any of those. Think of it as going down a highway towards the end destination, but on the way there I show you all the junctions where I think it's useful to do more research on the subject.

We'll deploy services inside Docker and LXC (Linux Containers). Those will include a mail server (iRedMail), Zulip (Slack and Microsoft Teams alternative), GitLab (with GitLab Runner and CI/CD), Nextcloud (file sharing, calendar, contacts, etc.), checkmk (monitoring solution), Pi-hole (ad blocking on DNS level), Traefik with Docker and file providers (a single HTTP/S entry point with automatic routing and TLS certificates).

We'll set up WireGuard, a modern and fast VPN solution for secure access to VPS' internal network, and I'll also show you how to get a wildcard TLS certificate with certbot and DNS provider.

To wrap it all up, we'll write a simple Python application that will compare a list of the desired backups with the list of finished backups, and send a result to a Zulip stream. We'll write the application, do a 'git push' to GitLab which will trigger a CI/CD pipeline that will build a Docker image, push it to a private registry, and then, with the help of the GitLab runner, run it on the VPS and post a result to a Zulip stream with a webhook.

When done, you'll be equipped to add additional services suited for your needs.

If this doesn't appeal to you, please leave the coupon for the next guy :)

I hope that you'll find it useful!


Happy learning,
Predrag

https://redd.it/136gc7d
@r_devops

Flyte - is it bad?

Just wondering if anybody has experience with Flyte and/or thoughts.

I've used other orchestration platforms before like Prefect and had a much easier time setting that up.

Flyte just feels unwieldy and auth is a clusterfuck. Also docs don't seem to have been updated for over half a year which is worrying. But maybe I'm just dumb.

Also, Lyft might be close to bankruptcy and I'm not sure if using their open source library is the right way forward as a result of that. Especially considering the company wants to offer this platform as a service to clients.

I've tried offering alternatives technologies to higher ups and they want Flyte. Obviously I should just deliver the deployment but I can just foresee how fucked I am in terms of maintaining this thing.

https://redd.it/136gncf
@r_devops

Transition to devops

Hello everyone,

I’m currently trying to transition from a data technician role to devops. I have 6 years experience as a telecommunications tech and 4 as a data tech. Currently my main responsibilities have been managing a site with 12 NVR’s (Dell servers). Have experience troubleshooting all types of networking problems. Also have experience with VM’s

I’m currently learning python, Linux as well as NGINX, AWS and plan to learn Docker, Kubernetes, Ansible, Jenkins,

Currently I have a A+ and AWS SAA cert and working on my AWS Sysops and Developer certs. I also took a couple courses on how backend Networking and engineering works when it comes to web apps.

The plan is to create a portfolio of projects using different technologies to showcase hands on experience and skills.

just want to know if my current track and experience is good enough to get a shot at a devops role.

https://redd.it/136is5f
@r_devops

I meshed Microsofts Bing AI technology into Instagram with Scripts

{ Sophie_Eloise.ai } Is the user name on Instagram

https://redd.it/136jnzs
@r_devops

fair price for up work contract

greetings

a client of mine asked me to set my price for the following task, I am new to up work and still trying to build up my profile, what do you think is the fair price for this job

We have 5 APIs that are connected. We want to deploy all of these into the k8s cluster in the most automated way possible.

​

Required Skills

Microservice architecture

Container orchestration

Kubernetes

Terraform

Docker

.Net

Linux

GitHub Actions

SQL Server

Cloud infrastructure

CI/CD pipelines

=====

Setup Docker images for all microservice apps

Setup Docker Registry

Setup Docker image for the database

Setting up database backup over FTP

Setup Kubernetes cluster

Setup Load balancers

Run the images using Kubernetes

create the necessary CI/CD pipelines

you will be asked to explain your code files by the end of the contract.

we will use Cloud Computing Services | VMs, Kubernetes, Storage | Linode for hosting

https://redd.it/136ks4m
@r_devops

How to open source code from a private monorepo

Hey r/devops!

I wanted to share an article I wrote about our experience open sourcing code from a private monorepo.

Check it out here!

https://redd.it/136pj21
@r_devops

The Hitchhiking Contributer’s Guide to Onboarding Docs in CI/CD

https://stateful.com/blog/guide-to-onboarding-docs-in-ci

It's all too commonplace in too many projects to obfuscate development tasks & workflows behind a complex Makefile or collect them in an extensive script list inside a package.json. How often are those files your entry point for understanding how to do anything inside a repo? Are you sure they always work?

Projects that initially invested in building out contributing guides eventually become outdated as contributors change workflows without updating the developer documentation accordingly. This drift of docs and code does not just slow us down; it often causes outright frustration and contributor churn.

A CI/CD pipeline is not much different than any code contributor to your software project. It requires some prerequisites to set up and an entry command to run a workflow like testing or deploying the app. What’s neat about pipelines is that they are a gateway for developers to ship code. If the project fails to build or tests fail, the CI/CD system turns red, preventing the code from floating under the radar and silently making it into main.

Wouldn't it be nice if we could teach our automated CI/CD pipelines to harness our developer docs and follow the same exact steps as any new employee or contributor?

You might think this could be an excellent job for chatGPT, but no! Plain vanilla automation to the rescue! We can run snippets of code from within markdown files with Runme anywhere, no AI (yet! 😉).

https://redd.it/136r2xl
@r_devops

A Guide to Privilege Escalation with AWS Identity Center (formerly known as AWS SSO)

https://www.cloudquery.io/blog/aws-priv-esc-identity-center

We did research on AWS's Identity Center (SSO) on paths to privilege escalation, the underlying IAM, and how to secure Identity Center. AWS Identity Center is AWS's recommended method of managing workforce access to AWS accounts and resources in an AWS Organization. (I'm the author).

https://redd.it/136nxx1
@r_devops

AWS Permission Bouncers: Letting Loose in Dev, Keeping it Tight in Prod

Hi there, I wrote a blog post that y'all may be interested in. It discusses how to manage cross-account IAM permissions for different teams. Would love feedback!

https://www.noq.dev/blog/aws-permission-bouncers-letting-loose-in-dev-keeping-it-tight-in-prod

https://redd.it/136tnkd
@r_devops

remote connection to GCE Vm machines

Not sure if this is appropriate sub, but how do you guys connect to your vms within GCE with no external interfaces? For now we just have openvpn server through which most of the devs connect. I know that there is Cloud Interconnect and Cloud IPSec VPN native solutions, but they all assume org on-premises network which we don't really have since all of us work remotely. What are best practices here? Thank you in advance and if this isn't appropriate sub feel free to delete.

https://redd.it/136pmjo
@r_devops

Sysadmin/DevOps Engineer without Terraform experience looking for relocation feeling hopeless advice needed.

Hi everyone.

I am Sysadmin or DevOps Engineer without Terraform experience. I have 4 years of experience. I spend 2 years in on-premise environments doing mostly dealing with CI/CD pipelines, deployments and general infrastructure and apps support, after that 2 years I joined a startup from its first moments as a only guy for infrastructure and build infrastructure based on AWS, EKS, RDS. However I did not have an environment to use Terraform collaboratively, and mostly used console and eksctl when building.

In summary, I have knowledge about cloud, kubernetes, databases, ci/cd, helm, monitoring, little bit of gitops and even understand how terraform and terragrunt works and wrote the same infrastructure i created in terraform but i have no real experience.

I need to relocate and even though i pass first 2-3 interviews, I get rejected due to having no real experience with terraform. Unfortunately i have reasons to relocate, if it is not going to happen, I will have to seek asylum after my visa expires.

I am thinking to getting certifications of Terraform and AWS however I don't know it will help or not.

What will be the next steps that can help?


Thanks.

https://redd.it/136yccz
@r_devops

Deploying Similar Applications on Kubernetes

We started our K8s journey in 2022 Q3. We have a dev cluster. The stack is pretty well fleshed out with 3rd party software. We have CI and CD working with a test application that is similar to our 1st party software. I could manually create the manifests for our applications like I have for this test application, but I'm thinking about this from an automation perspective.

Our applications are similar in that they made up of the same K8s resources: Namespace, Deployment, Service, ServiceMonitor, NetworkPolicy, PodDisrutionBudget, and so on. I wonder if we can use a tool to template the resources and deploy these applications all using the same template.

Is this a use case for Helm? Could we use a single Helm chart to create the manifests for all similar applications?

https://redd.it/136zmd0
@r_devops

OWASP ZAP and Jenkins

Has anyone made OWASP ZAP work with Jenkins to trigger DAST security scans automatically? Or is there another way to make sure I am scanning everything?

https://redd.it/136zhlx
@r_devops

How do you scale Longhorn in k8s?

Hi all,

My current research is how to implement Longhorn in DigitalOcean managed kubernetes environment and below I need to use DigitalOcean Volumes. All is good until I wonder about the scaling. I have more than 10TB of data that is being generated/used by 10-15 pods and the tendency is the data and pods to grow.

So how do Longhorn scales? How he can automatically increase it's volume? How does it work under?


I really appreciate any help you can provide.

https://redd.it/136pguj
@r_devops

tool for pushing AWS costs in analytics tool like Metabase, Looker etc..

Hi devops community,

Need some advice/suggestions on a new tool I've written, the scope for this tool if it's made open source for the community etc..

Context:

Our CFO, and finance team has no access to AWS console but there came a requirement for them to visualise the costs data in Metabase so they can see the monthly and daily costs for each AWS service and can see and prepare internal cost reports for AWS INFRA, as we use Metabase for our analytics purpose they asked me if this can be pushed to Metabase.

Currently, I couldn't find any tailor-made open-source solution for this, although Vmware has something called CloudHealth tool that does something similar but it's super expensive.

So using the AWS SDK for Go I was able to write a tool that gets the cost data daily, push to Mysql DB and from there Metabase can read it and the analysts team can perform the required action on this data via Metabase.

A snippet of the output is give below: https://ibb.co/2y3DQqC

So my ask is how helpful is this tool per your opinion?
If I’m able to helmify everything and put this in a Github repo as open source so anyone can install it, will it help anyone with a similar-use case ? What's the scope for this project?

Can this be converted into an OSS product idea?

Need your valuable feedback folks :)

https://redd.it/13779d7
@r_devops

Tips on keeping track of complex code...?

I dont think i had this problem during college, but since ive joined a company and started working with very very big and complex code.. i have a very hard time remembering "where things were" especially when reading code.

For example there are functions that create objects and this points to there and that function actually calls this and that and blah blah blah.... u need to connect all this..

I just get really frustrated keeping track of these intermediate info.. and when it gets complicated my brain kinda shuts down. I think you need to be able to some sort of remember where things were or at least save some in your memory to keep on going and read code.

Has anyone struggled with this before? Are there any tips that made you better at this? Such as practicing some thinking patterns, tools or writing it down.

Also does this get better by time? I think some people are just good at this compared to other people.

https://redd.it/1376knz
@r_devops

DevOps Learning Series Infrastructure as Code (IaC): The Foundation for Automated Infrastructure Management

Series index

It's been a couple of weeks since the last post, and I'm excited to dive deeper into the world of DevOps. Today, we'll be discussing Infrastructure as Code (IaC), a critical component of automated infrastructure management in DevOps practices. We'll explore the benefits, key concepts, popular IaC tools, and walk through some specific examples to give you a good beginning to start understanding this essential aspect of DevOps.

What is Infrastructure as Code?

Infrastructure as Code refers to the practice of managing and provisioning infrastructure resources (such as networks, servers, storage, etc) using code and version control systems, rather than relying on manual configurations or ad hoc scripts. IaC enables organizations to treat their infrastructure in a similar manner to their application code, thus ensuring consistent, repeatable, and automated deployment of environments.

Key Benefits of Infrastructure as Code

Adopting IaC brings several significant benefits to organizations, including:

1. Consistency and predictability: IaC enables organizations to maintain a standard, version-controlled infrastructure configuration, which minimizes configuration drift and ensures consistent deployment across environments.
2. Faster provisioning: IaC allows for rapid and automated provisioning of infrastructure resources, reducing the time and effort required to set up new environments.
3. Increased collaboration: By managing infrastructure code in a version control system, development and operations teams can collaborate more effectively, share knowledge, and establish a single source of truth for infrastructure configurations.
4. Enhanced security and compliance: IaC allows organizations to enforce security best practices and regulatory requirements throughout the infrastructure lifecycle, improving security and auditability.

Popular Infrastructure as Code Tools

There are several IaC tools available, each with its own strengths and trade-offs. Some of the most popular IaC tools include:

1. Terraform: An open-source IaC tool by HashiCorp that enables provisioning and management of infrastructure resources across multiple cloud platforms through declarative configuration files.
2. AWS CloudFormation: An AWS-specific IaC service that allows users to define, manage, and provision AWS resources using JSON or YAML templates.
3. Azure Resource Manager (ARM) Templates: A native IaC solution for Microsoft Azure that enables users to define, deploy, and manage resources through JSON templates.
4. Google Cloud Deployment Manager: An IaC service for Google Cloud Platform that automates the creation, deployment, and management of resources through YAML configuration files.

Example: Creating and Managing Infrastructure with Terraform

To better understand IaC in action, let's take a look at a practical example using Terraform. Imagine you're working on a web application project that requires a load balancer, two web servers, and a database server. With Terraform, you can create a configuration file (using HashiCorp Configuration Language, or HCL) to define these resources and their dependencies.

Here's a simple, and untested, example of what your Terraform configuration file might look like:

resource "awsvpc" "example" {
cidrblock = "10.0.0.0/16"
}

resource "awssubnet" "example" {
vpcid = awsvpc.example.id
cidrblock = "10.0.1.0/24"
}

resource "awssecuritygroup" "example" {
vpcid = awsvpc.example.id

ingress {
fromport = 80
toport = 80
protocol = "tcp"
cidrblocks = ["0.0.0.0/0"]
}
}

resource "awslb" "example" {
name = "example-lb"

internal = false
loadbalancertype = "application"
securitygroups = [awssecuritygroup.example.id]
subnets = [awssubnet.example.id]
}

resource "awslbtargetgroup" "example" {
name = "example-target-group"
port = 80
protocol = "HTTP"
vpcid = awsvpc.example.id
}

resource "awslblistener" "example" {
loadbalancerarn = awslb.example.arn
port = 80
protocol = "HTTP"

defaultaction {
type = "forward"
targetgrouparn = awslbtargetgroup.example.arn
}
}

module "web" {
source = "./modules/webserver"
instancecount = 2
securitygroupid = awssecuritygroup.example.id
subnetid = awssubnet.example.id
}

resource "awsdbinstance" "example" {
allocatedstorage = 20
engine = "mysql"
engineversion = "5.7"
instanceclass = "db.t2.micro"
name = "exampledb"
username = "admin"
password = "password"
vpcsecuritygroupids = [awssecuritygroup.example.id]
subnetgroupname = awsdbsubnetgroup.example.name
}

resource "awsdbsubnetgroup" "example" {
name = "example"
subnetids = aws_subnet.example.id

tags = {
Name = "example-db-subnet-group"
}
}

In this example, we define an AWS VPC, subnet, security group, load balancer, target group, listener, two web servers (using a module), and a database instance. By running `terraform init` and `terraform apply`, Terraform will create and manage these resources for you, handling dependencies and updates as needed. If you need to make changes to your infrastructure, simply update the configuration file and run `terraform apply` again. Terraform should calculate the necessary changes and apply them accordingly.

Conclusion

Infrastructure as Code is a foundational concept in the DevOps landscape, allowing organizations to automate the provisioning and management of infrastructure resources while maintaining consistency, predictability, and security. By embracing IaC and incorporating it into their DevOps workflows, organizations can streamline their operations and foster greater collaboration between development and operations teams.

​

Further Reading:

1. ***Infrastructure as Code: Managing Servers in the Cloud***. O'Reilly Media, Morris, K., & Kordyban, K.
2. ***Terraform: Up & Running***. O'Reilly Media, Brikman, Y.
3. Terraform - **What is Infrastructure as Code (IaC)?**
4. Amazon Web Services - **AWS CloudFormation.**
5. Microsoft Azure - **Azure Resource Manager (ARM) Templates.**
6. Google Cloud - **Google Cloud Deployment Manager.**
7. **Terraform Commands (CLI).**

​

I hope this more in-depth look at some Infrastructure as Code tools has been informative and helpful. As always, feel free to share your thoughts, experiences, and questions in the comments below!

https://redd.it/137bwv9
@r_devops