containerized java development to deployment pipeline

Anyone here work in a java shop that are building and deploying web apps on Tomcat that is willing to share the entire process, just high level, from the actual developer environment all the way to production deployment?

I'd be especially interested in anyone who is developing in, and/or deploying to, containers.

Our developers all work locally on Windows with Eclipse today, they deploy to staging/test/qa/uat themselves and then we have an ops team that manually deploys to production. You can see the problems with this already, I'm sure.

https://redd.it/ztmepl
@r_devops

how hard would it be for a data engineer to become a devops engineer?

I work as a data engineer, and do a lot of development in python, SQL, and command line scripts. I also am very familiar with AWS.



I'm pretty familiar with infrastructure as code, where I have used Terraform and Cloudformation.




How hard would it be for me to apply for devops jobs? All devops jobs in my area are pretty senior it seems, and I have been working in data engineering for 4 years.

https://redd.it/ztm601
@r_devops

Should i study for Software Engineer or AWS re/Start?

I have no idea what each of these do, but i need to change a career , and found a school with these 2 choices. My creteria for selection:

1)be able to work from home

2) have decent salary

3) not overstress from work.

​

Any advice is appreciated

https://redd.it/zu0s88
@r_devops

Beginner DevOps

I’m maths graduate student who is trying to get into learning all about DevOps. I need advice on where to start from. I want some recommendation on some online courses or books for me to start with.

https://redd.it/zu08d4
@r_devops

Am I doing it right?

I am new to DevOps (intern) and I am trying to secure/improve how we handle config files. We deploy java projects which needs a config.properties file.

This config file contains username and passwords of databases. For now, we are storing these files on our server and move them to tomcat at the time of deploying via jenkins, which is not a secure way.

I was thinking of introducing hashicorp vault and integrate it with Jenkins. So the flow would be:

​

get secret from vault at the time of build > parse config file using python (to update credentials) > deploy it to tomcat

​

Please tell me if this is the right way of doing it or if there are other better ways?

https://redd.it/zttqnj
@r_devops

Paying $7847.42/mo in unused Reserved Instances?

I'm the CTO of a web3 company (I know lol) and I recently found myself in a bit of a predicament with AWS. We were given $100,000 in AWS credits to use for building out our infrastructure, and as someone who doesn't have a lot of experience setting up AWS accounts, I turned to the AWS cost explorer for guidance. I didn’t pay enough attention to our costs (which is probably exactly what AWS wants).

The cost explorer recommended that we purchase 1-year reserved instances, so that's what we did. At the time, we were on credits and didn't really worry too much about the architecture – looking back, that was a pretty stupid move.

We ended up purchasing the following instances: r6i.xlarge, c4.large, t3a.xlarge, m5.xlarge, m5.xlarge, m5.xlarge, t3a.xlarge, m5.xlarge. At the peak, these were all being fully utilized.

However, things changed when the crypto market crashed and we saw a large dropoff in traffic (around 60%). As a result, our traffic dropped from 10,000 requests per minute to 4,000 requests per minute, and my EKS cluster also dropped in the number of instances it needed.

Here's a breakdown of the unused hours for each instance:

r6i.xlarge: 34,944 reservation hours unused
c4.large: 353 reservation hours unused
t3a.xlarge: 10,680 reservation hours unused
m5.xlarge: 9 reservation hours unused
m5.xlarge: 4 reservation hours unused
m5.xlarge: 108.55 reservation hours unused
t3a.xlarge: 19,141.32 reservation hours unused
m5.xlarge: 911.68 reservation hours unused

(see screenshot here https://i.imgur.com/2WmfP10.png)

I was left with a bunch of reserved instances that I wasn't using, and I was still paying for them even though they weren't providing any value. I tried talking to AWS support to see if they could help me solve this issue, but I didn't make any progress. I tried reaching out via email, phone, and even through the AWS console, but I didn't get any help.

I had 6 months on the reserved instances and this was our biggest expense at the company. We were able to reduce our other costs quite a lot, but this sucked bc we weren’t using any of the instances at all.

I know I'm not the only one who's experienced something like this, so I wanted to share how we fixed it.

I had already tried selling the reserved instances on the marketplace (no one bought them) so had resigned myself to paying the costs.

One of our investors recommended a SaaS company (I don’t want to name them bc promotion) that would buy and sell RIs on our behalf. Lowkey thought this was too risky and didn’t want to give them access to our AWS accounts but in the end, we decided to run a POC to give it a try. To be safe, we talked to one of their competitors as well (trying to learn from my mistakes to take better precautions lol).

In the end, we decided to go with the first tool for a few reasons: more favorable contract terms, better insights and reporting, and a product roadmap that fits what we think our needs will be in the future. Would recommend making sure that any tool you use in this space fits your needs in both the near future and the long-term time horizon - I can see this not being the case for many companies with more stable compute.

It took this company around 20 days to offload the reserved instances that we had on hand and they also were able to get us better terms with their 3-yr RIs. They charged us XX% for the savings that they created which is not ideal but still came out to more than we were getting on the one-year reserved instances. At the end of the day, I/we would be in a much better spot if we had taken more time to buy the right instances and had better prepared for the crypto crash but if you are in a similar situation then this might be helpful.

TLDR: I bought a lot of 1 yr reserved instances that went underutilized shortly after. I found a solution to offload the reserved instances, but not as soon as I would have liked.

PS: Sorry for my rant

:)

https://redd.it/ztnbxy
@r_devops

What's going to be the next big thing after DevOps?

Hey guys, a question that just sprung up in my head - today there's a lot of hype around DevOps, but what will be the next big thing?

https://redd.it/ztbpod
@r_devops

Need Help in selecting a CICD tool AWS CodePipeline Suite or Gitlab

Learner Here, Starting a small project and would like to learn and implement CICD for a project .

Need some help in deciding a CICD tool for getting things started for a web app project which relies almost AWS Infra (Server less). So would like to hear some thoughts on which tool should I be considering for a small team (3) and benefits and pitfalls of AWS toolkit over Git lab.

Would like to hear some insights from people using these tools in daily work and know more about ease of setting things up and cost wise

https://redd.it/zu7zmc
@r_devops

How to calculate hardware requirements for a hypervisor?

hello guys,

Wish everyone is doing okay.

I would like to setup a baremetal hypervisor that would virtualize \~25 vm for 25 users.

The vms mostly would be used for sending email , working with docs and will have software package like microsoft office,adobe etc and all of them running Win 10 Enterprise.

How to calculate minimal hardware requirements for hypervisor server and what should one take in consideration, are there any better ways?

https://redd.it/zu7wsu
@r_devops

Billions of unnecessary files in GitHub

It seems a lot of people who use #GitHub don't know about .gitignore or they misunderstand how to use it. This leads to billions of unnecessary files: generated files, 3rd party installations, caches etc. to be added to git and GitHub.

Not only is this a waste of space and a waste of network bandwidth when you clone a project, it also makes tracking the real changes more difficult.

https://dev.to/szabgab/billions-of-unnecessary-files-in-github-i85

https://redd.it/ztfayw
@r_devops

Advice needed for someone willing to get into Devops

Hello.

I've been working on sysadmin role for almost 8 years, mostly Windows Servers, Hyper-V, Networking and Firewalls stuff, had some certifications in the past, MCSA, CCNA, some firewalls certifications and so on, have played a little with Azure and O365, nothing too deep. All that time been working for a small consultancy company with small business customers. Before that I've worked as web developer with PHP, so I think have a decent programming logic.

Now I am trying to migrate to a "DevOps role" (I understand that there is no such thing as a role and I also understand the DevOps culture, thanks to this sub), so I have changed job to a bigger company (9000+ employees) with an internal dev team, multi-cloud environment (mostly OCI) and lots of oportunities to put in practive some DevOps tools, but right now there is no such "devops role" in this company, I am hired as a midle Infrastructure Analyst.

So I started learning some basics stuff (linux mainly), and after 6 months I already deployed some automation using Ansible and Powershell, developed a web app using Python and Flask that integrates with Terraform to provision VMs to our local VMware Infrastructure (VxRail), learned Azure and got two certifications (AZ-104 and AZ-400), and was studying for CKA to learn Kubernetes (passed on first attempt), I like sutdying for certifications because this way I can have something like a study plan.

But now comes the point where I currently am, with this little experience I can't even be called to an proper DevOps role interview, I know there are lots of tools and expertise that I still have to learn and this gets me overwhelmed, I am planning on learning Terraform and Jenkins deeper and study for AWS Certifications (Sysops and Devops) but I can't really feel secure that this is going to give somewhere because I feel that I am always running from behind, feeling like I have lost the timming.

Sorry for the long post and thanks to anyone that read it.

https://redd.it/zto2yx
@r_devops

Should we be afraid of AI and a more advanced ChatGPT automating all engineers out of a job within the next 10 years?

Basically the title

https://redd.it/ztot6t
@r_devops

Alerting rules "libraries, compendiums, or bundles:" where to find already-written, useful alerting rules for Prometheus, especially for the AWS Cloudwatch Datasource metrics?

Prometheus has no out of the box alerting rules. The list of such "libraries of useful rules" I have so far is:

1. https://awesome-prometheus-alerts.grep.to/ (many examples, my primary reference)
2. https://alex.dzyoba.com/blog/prometheus-alerts/ (examples, links to other pages)

Does anyone have a source for Cloudwatch alerting rules, or did you write them all by hand? Any other lists of pre-written rules for other exporters and/or Data Sources? I've decided on Alertmanager instead of using Grafana Alerts, so I'm not really looking into Grafana Alerts.

https://redd.it/zsvasl
@r_devops

is eBPF the cool thing now?

Lately I am getting a lot of popups on media prophesying the rise of eBPF era and the doom of sidecars approache.

Any thoughts or experience about this?

https://redd.it/zucclf
@r_devops

December Challenge: Kubernetes Certified within 3 weeks

I would like to share my victory with the community when it comes to a challenge I set for myself for December. I am currently a Junior DevOps engineer. I have been working for 1 and a half years total and I decided to challenge myself to get all the 3 Kubernetes certifications (CKA, CKAD and CKS in this order) within 3 weeks.


My challenge (so my studying as well) started on December 1st and then I had my CKA on the 5th, my CKAD on the 12th and the CKS on the 19th. I managed to pass all of them and I also wrote a bit about the experience I had on my blog:


\- CKA: https://mirceanton.com/posts/2022-12-06-my-cka-experience/
\- CKAD: https://mirceanton.com/posts/2022-12-13-my-ckad-experience/
\- CKS: https://mirceanton.com/posts/2022-12-20-my-cks-experience/

https://redd.it/zumyqi
@r_devops

A question that puzzles my mind, how can a cloud server like gdrive transmit data without congestion? even the best SSD would get slow if there are multiple reads and writes at the same time.

As the title says,
A file from gdrive could get more than 10 connections at the same time, and that single drive might be hosting tons of other data too which might be active too, so I don't see how one drive could survive hundreds of simultaneous data access.
Unless maybe active data above the threshold could be moved temporarily into multiple drives without us realizing it?


Thanks!

https://redd.it/zuob2s
@r_devops

'Best' Build System Language & Interface?

So in my workplace we have two teams (broadly comprising application side and operating system side for embedded control systems). We both contribute to a sort of internal 'platform' of scripts for various parts of build, test and analysis on internal products we deploy to.

The operating system side have implemented a lot in 'make' (and I mean not just interface but a lot of logic and parallelisation within the makefiles) - and frankly it makes my head spin and only the one person on the team can understand the bizarre ways they've stretched the capabilities of make to its limits. On the application side we have primarily implemented in PowerShell (mainly because it is a good language for integrating with Azure DevOps pipelines and maintaining our own Windows machines), with an 'interface' maintained with lots of one liner batch scripts. However, whenever we have a new hire or someone from the dev teams who are dipping their toes in DevOps, they rarely know any powershell and need to learn a lot from scratch.

Basically we've started butting heads over implementation and it's starting to become very counterproductive. Some things we've both implemented in our own way because we haven't been able to align which is a pretty big waste of time.

Ideally in the new year I would like to propose a rewrite in something we can both agree on. I was wondering what languages and build systems people think are clear and easy enough to maintain while still being powerful. Is there really a benefit to using a proper build framework over a bunch of script files? For reference, some of our code is in C, some is in Java (Eclipse plugins for an internal version of Eclipse we produce), some MATLAB scripting, but mostly Simulink models (yes an interesting mix I know). So there's quite a lot of calling external executable tools on binary files (particularly MATLAB).

I'd be interested in what people have done in their workplaces and how it has worked out for them?

https://redd.it/zuezi1
@r_devops

What's the best resource to learn EKS (paid or free)?

Hi,
I am looking for a course which covers everything needed to setup a production ready EKS cluster.
Thanks

https://redd.it/zugcu8
@r_devops

Best/low maintenance devops toolchain for basic sass?

I'm looking for a new devops stack, I got pretty deep into k8's, and I feel like it's just too much overhead. It is cool, and does a lot of great stuff, but it seems like there are other solutions that are better for basic sass app type stuff.

I start a decent amount of projects, and so I'm looking for something that's fairly low maintenance. I know nothing is 0 maintenance, but the less the better. I do lean towards AWS, but I wouldn't mind flexibility in different cloud providers as well. If a solution is really elegant though, only supports one provider, i could get behind that.

I've been doing a little research, and these are some of the options I'm considering.

1. App engine/Elastic beanstalk with terraform
2. cloud formation, with lambda
3. CDK, with lambda and api gateway
4. serverless with CDK
5. SAM
6. Some other combination of the above technologies.

The only requirement I have is that I should be able to develop offline. I can't stand having to be online and deploying things in order to develop / test.

Currently, I'm kind of leaning towards serverless with CDK (if I can use CDK offline). Anyone have any experience with that? Did you have any long term issues or drawbacks?

I typically also use hasura with postgres as well, so I don't want some premade firebase type solution

https://redd.it/zumr7u
@r_devops

Looking for insight into this DevOps books

Hello everyone one i was looking for reviews and searching these books and came here looking for more thoughts on this

https://www.fanatical.com/en/bundle/dev-ops-bundle-3-rd-edition

That's all thank you!

https://redd.it/zuzzug
@r_devops