What Techs Should I Use While Moving from a Monolith To Microservice Architecture?

I know there's no right answer, I'm just looking on some input to use in helping me create arguments for which techs to use. My company has developed a very large application in a monolith using an early version of Grails (2.3.11). This has caused a lot of issues, including not being able to really upgrade Grails in any way even though the current version in 2 major version ahead, and there is some HUGE reworking and additions of really nice features we've never gotten to use. That and a couple other issues have caused us to want to slowly adopt a microservice-based architecture instead (We know we'll never be able to really FULLY move the app all the way to microservices, but splitting out as much as we can will help a lot).

We've always just built (either manually or through Jenkins) the .war file and uploaded it to EBS. I'm trying to create a game plan for moving towards microservices that sets us up for success as early as possible. Right now my gameplan is to start with containerizing the current application and deploy it that way, then it becomes easy to create the "first" microservice in another container and not be doing a whole lot of building it in the main app and potentially causing issues at one time.

I wanted to know if anyone had any good articles or stories of their own on moving from a monolith to microservices that were built similarly. What opinions do you have on using EBS even with microservices? Should we move to all EC2 instances early on? Should we try something like EKS? Do you like Docker over K8s or the other way around for some reason? Open to all information.

https://redd.it/ntcmqm
@r_devops

No, you can't do it better in Python or bash (challenge)

Doing DevOps today? 99% either abuse bash or abuse a general purpose programming language.

bash does not meet any modern expectations from a programming language: syntax, error handling, data structures

General purpose languages such as Python, Ruby, etc are not domain specific enough to have the desired facilities.

Here is small example of straightforward solution to a small problem: list all CloudFormation stacks that are managed by the given CodePipeline.

Your are welcome to prove me wrong about this particular example - come up with at least as clear and as concise solution in either bash or Python. I think it can not be done.

https://redd.it/ntipck
@r_devops

Domain structure for multi-cloud, multi-environment

Are there any articles/docs that reference planning out a DNS structure for multi-cloud, multi-environment setups? I've tried searching out there using different terms that I can think of, but I'm not finding what I need. Here is the basics of what I'm looking for information:
- How to structure private and public dns structures to support multiple environments (prod,qa,uat,dev, etc), that will run between multiple providers (onPrem, <insert cloud providor here>).
- Would adding several subdomains allow for easier scaling in the future, albeit more work to setup now. Examples below would be CNAMES based on client location.
- public ex1. www.useast1.aws.prod.example.com
- public ex2. www.uswest1.aws.prod.example.com
- by using the above examples of FQDN's is that giving too much information away, with regards to security. Should there be some obfuscation in the names.

Currently, we dont have a need for this complex of a setup, but is it worth the trade off of pre-creating this structure reduce future growth.

https://redd.it/nu36fg
@r_devops

Looking to join DevOps Slack channels.

What channels would you guys suggest?

https://redd.it/nu6361
@r_devops

Any CircleCI experts able to help me? I can't access my context/environmental vars in my docker image (aws-ecr/build-and-push orb)

Hey all, as the title suggests I am trying to build & push a docker image to AWS ECR. Everything works good until the containers try and run! My app is dependent on the ENV vars, which are coming in as undefined.

Couple of notes:

1. I have no problem when I declare export environmental vars locally and test.
2. I have added the environmental vars to the CircleCI context and referenced that context (this works for other processes without problem.
3. I can see the step in CircleCI where the env vars are prepared.
4. Problem is, when I build & run, the containers are not able to access the vars. I have confirmed this with cloudwatch logs on the containers.

Here is the relevant line in my config.yml:

version: 2.1
- aws-ecr/build-and-push-image:
context: CIRCLE
requires:
- Build react app
account-url: AWSECRACCOUNTURL
aws-access-key-id: AWSACCESSKEYID
aws-secret-access-key: AWSSECRETACCESSKEY
create-repo: true
path: ./server
region: AWSREGION
repo: my-socket-app-repo
tag: "latest"

Any help is greatly appreciated! Thanks

https://redd.it/nu0w2x
@r_devops

Is there a service to handl rewriting URLs?

Hi all.

We are trying to reduce some technical debt/remove legacy applications in our ecosystem.

We've come across a simple .NET core app that a previous employee called "Domain Forwarder".

It essentially has a JSON list of hostname/hostname+path redirects and simply 301's.

For example, we have a bunch of legacy domains that have been setup with a rule to send them to a page on our website depending on what domain they are going to. If you go to abc123.com, you might 301 to ourwebsite.com/abc123.

This is setup in Azure on App Services and has all our legacy domains binded to it.

Is there a better tool/service that can handle all this for us? We want to reduce as many apps as possible so would be good if we had a service (preferably within Azure) that can handle forwarding of these domains/paths for us.

&#x200B;

Thanks

https://redd.it/nu35nm
@r_devops

Why government website don't use SSL in some websites? any idea

I saw this case in many places

https://redd.it/nu5ytk
@r_devops

I’m a software engineer that had to learn ops. I am not always sure how to set thinks properly. Any course / tutorial to follow?

Recently, my position made me implement the whole project architecture on AWS (with Terraform).

We are mostly deploying code on Lambda, but we also need to deploy some apps on EC2 instances.

I have successfully deployed an ALB with a single EC2 using docker swarm and Traefik.

However, I never managed to do zero downtime deployment with it. I had to develop a custom bash script to do blue / green deployment. (The script find the next color, deploy it, check the health of the app, switch the Traefik config to redirect traffic to the new color, check that everything is working, shutdown the previous color)

I’m using docker swarm mainly because it’s easier to deploy services, and I’m only deploying a single node app, with a single elastic search and 2 nodes of Traefik.

I feel like I’m doing something wrong, but I cannot point my finger at it.

Docker Swarm tutorial are quitte old, as I feel the majority of people are using Kubernetes.

Any help, tutorial would be awesome! I’m more than happy to share my scripts / terraform config

https://redd.it/nu99ti
@r_devops

How do you keep on top of calendar tasks such as renewing certificates?

How does your team manage calendar related tasks such as renewing certificate, app provisioning profiles etc?

We use Confluence for documentation but I don't think there's a calendar plugin with reminders.

We are thinking of using Outlook.

Wondering if anyone has any better way of managing these things?

Cheers

https://redd.it/ntyf2j
@r_devops

How Can I Keep the Storage Bill Cheap

Hello Reddit!

I have a project that uses about 16TB of bandwidth per month from users downloading files. I would like to keep quick download speeds, but the site does not make enough money to use any big-name brand providers.

Digital Ocean Spaces starts at $5 a month, but with 16TB, it's closer to $300 a month.

Does anyone know any cheaper alternatives or have advice for lowering the bill?

https://redd.it/nttfkj
@r_devops

Hello Guys- Need a lil of your help!

Hii Guys, I have been thinking to contribute to our developer community for a while now, I think creating informational blogs could be helpful to all kinds of members of this community, now I want your little help here, please tell me what kind of blog content would you want me to create?!

As I do not want to create anything that people are already making and is already been chewed on, So please it would be great If I could get some insights from you guys! Thanks in advance!

https://redd.it/ntrqn5
@r_devops

Favorite way to provision and configure ec2 instances?

Hey people,

&#x200B;

I'm looking to deploy some applications in ec2. The applications need to be configured with environment variables and require docker. I'm trying to think what are some of the quickest and easiest ways to provision new ec2 instances, and manage their configuration as a one-off and ephemeral instance?



I'd ideally like to use one tool for the provisioning and configuration, and would like to avoid needing to build out a CD pipeline. What tools are you guys using for provisioning and configuration? Ansible, AWS CDK, terraform?

https://redd.it/nuev4p
@r_devops

Python or Batch script

Hello everyone. I recently got assigned with refactoring some batch scripts.

It included multiple logical part that could be seperated. Downloading dependencies, configuring downloaded dependencies and running some tests. The script was a little bit bloated so i moved dependency managment to maven, and wrote a python script so it would be easier to maintain. But %50 percent of the python script is just calling subprocesses to run tests and copying files.

Im kinda on my own. Company dont care how i do it and they are generally new to best practices to devops.

Most useful part of the python script is the -h option generated by argparse module. And i also made the script independent from the working directory (it takes command line parameters for directories it works on and has good default values)

Am i right to rewrite batch into python in this situation? Batch script was seperated into 7 files totaling at 300ish lines. Python script is 100ish lines

https://redd.it/nuhsug
@r_devops

Interview tips for 1 Year Experience Engineer

Hi folks,

I am working in DevOps since last 1 year and now Tomorrow I have an interview for the position of DevOps Engineer.

Interviewer Company is mostly using AWS as cloud platform and some other DevOps tools

Any key points or hint to focus on for better result ?

https://redd.it/nuj5vv
@r_devops

Learn how the Power Platform now enables professional and citizen developers to participate in DevOps

Watch the latest Fusion Teams Development 101 webinar to learn about DevOps improvements to the Power Platform for professional and citizen developers. Fusion Teams | Microsoft Power Apps

https://redd.it/nuop6r
@r_devops

New to IT, should I consider a career in DevOps?

I'm a 26 yo guy, security studies major (geopolitics, military, international relations, internal security, etc. had some IT stuff but not much), just finished my master's degree and I have some decent job prospects in the local government administration or the law enforcement, but my programmer friend (5 years as a full stack developer, Java pro) said I should consider a career in DevOps (he explained to me what that is and after some digging it sounded interesting) considering my skills

so, my skills include:

• I'm a 7 year Linux user, 3 years Arch Linux, I'm not a sysadmin but I know the basics, like managing services, making my own services, managing accounts and permissions, ssh, making simple bash scripts (for selecting audio sinks and utility stuff like that), htop, ps - e, grep on daily basis, I basically love Linux, especially when it breaks, so I can spend countless hours fixing it in vim, when not using browser I tend to spend time mostly in CLI-apps as it feels romantic, neomutt for mail, cli Spotify client, weechat for IRC, etc.

• so Arch has this user package repository called AUR, I maintained few packages there, that's how I picked up git, pulling, pushing, commits, felt simple,

• I learnt a bit of MySQL, I can set up a new database, operate on data, SELECTs, JOINs, etc., we had a database class, that's where I picked up most of it,

• we were also taught how SSL works, so I know how to create a local certification centre, self-sign my website certificate and use it, learnt on apache, it was pretty cool, obviously the browser won't accept it, but still,

• what was also fun, we were taught how to create a private VPN? between two computers, ipsec racoon tunneling, had to generate asymmetric RSA key pairs,

• I installed OpenWRT firmware on my tp-link router, so it would have SSH capabilities, so I can now SSH to my router from my phone, make router send a magic packet to my PC to turn it on and SSH to my PC if I need something,

• I picked up a bit of Java for fun, liked the concept of object programming, I could create simple CLI apps like phone book or polling app on my own, but I gotta say that when I encounter a Python code (happens often when I use some amateur Linux apps) it feels way more readable,

• my friend sent me some simple project which I had to run with docker-compose up, I was amazed at the technology, frontend website, database and main backend program all on three different containers, I could SSH into each of them as all of them had separate internal IPs and different ports, the technology is neat, fascinating stuff, three separate but interconnected Linux installs!

So I'm not sure if this is stuff every IT graduate does like it's nothing or do I hold some promise? I never worked in IT, so it's okay if I don't fit in, it's more of a hobby for me. Cheers to you all, especially those who contribute to open source stuff!

https://redd.it/nuqgnu
@r_devops

NEW Databricks Zero to Hero | Azure SQL Integration with Databricks | Session 6

\-- https://youtu.be/UzQlmMcvm18 \-- This video is the continuation of the "Databricks Zero to Hero" Series Free crash course. In this video , you can learn and refresh 1. How to connect Azure SQL db from Databricks notebook 2. How to install MS-ODBC driver and pyodbc 3. Insert data into Azure SQL database using pyodbc 4. Validate the data from Azure Portal.

&#x200B;

\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^ Please Subscribe, Like, & Share for more video and to get notified when new video comes⤵⤵⤵ :https://www.youtube.com/c/CloudDataScience?sub_confirmation=1 \^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^ #DatabricksHeroToZero

https://redd.it/nupmvi
@r_devops

What are some steps you run in your CI Pipelines?

Obviously every project has different necessities, and there are some steps that every project should run in CI, but just curious what some of you use. Maybe sharing will give everyone some ideas.

We’re run a pretty basic set of instructions at my company so far, just running unit tests and linting over the app.

https://redd.it/nuszy5
@r_devops

Do I need to know a programming language?

I've been in the devOps/infrastructure field for about 5 years now. Since then, I've been working with AWS using tools (Terraform, Packer, Ansible, Chef, etc) rather than writing code in a traditional language (Python, Ruby, Java, JavaScript, etc). I've never run into any problems with building our infra and getting our services to run smoothly with high-availability. I might have had to write a quick script here and there either in Bash or Python to perform some minor task, but it's nothing challenging.

I'm noticing a lot more job postings are looking for people with strong backgrounds in a traditional language (Python, Ruby, and JS show up a lot), but I don't know why. Are people here writing more "traditional" code to do their work? Should I be doing the same? It's really worrying me because as much as I love the company I'm with, I know that one day I might choose to leave or they might downsize me and I don't want to miss out on opportunities for not knowing how to write a web app in Python or something like that.

https://redd.it/nutcu5
@r_devops

how do you manage database migrations

Does anyone here have a rock solid recommendation for handling database migrations in a robust manner?

I feel like backwards compatible schema updates are the best approach i.e ensure that changes are always compatible one schema version ahead.

However, I don't have any good ideas for how to enforce backwards compatability. Also, would love to hear if anyone hear has fully automated migration upgrades AND rollbacks.

https://redd.it/nuu8bc
@r_devops

How to deploy heroku dyno with nodejs web app AND python script?

So what I have is I'm using nodejs/express for website. I am also using another javascript module called spacy-js. To make spacy-js work, I have to run a python file in the spacy module like "python node_modules/spacy/api/server.py" in 1 terminal and then do "npm start" on another terminal so that spacy works on my website. There is a separate requirements.txt inside the spacy-js module. Now, this website uses nodejs website and also runs a python file inside node_modules. How do I deploy this to heroku?

https://redd.it/nutayw
@r_devops