Help with "Dev side" skills/concepts to learn

I recently interviewed with a place for a devops related internship where it looked like they wanted a person with a more Dev/Programmer background - I have more of an Ops background (Linux/AWS) . I did well enough to get a second round but it really opened my eyes on learning more about the dev aspect and I'm not a traditional computer science major either. I'm planning to put about 15-20 hours of study towards that this week, Python is the language I'm learning and most familiar with. Any important concepts you recommend I should study (OOP, Data Structures & Algo, Databases, Intro to CS, API's etc)?

https://redd.it/qehpti
@r_devops

Launching VM instances only when needed

Hi all,

I am looking for the right tool to launch VM instances on a cloud platform only when I need them. Once launched, they should run a containerized script (let's call it a job) that runs for about 5 to 15 minutes (depending on the input data and hardware) before returning about 100 MiB of data (which could, for instance, just be uploaded to a bucket somewhere by the script itself).

For now, I have worked something out using AWS SageMaker training and processing jobs or GCP AI platform training jobs. My problem is that there a relatively long overhead (~5 minutes) before the script actually starts running, which about doubles the time it takes for me to get my results.

Neither GCP cloud run nor AWS lambda are options for me, since I need a GPU for my jobs to complete in a reasonable time. Renting a VM full-time does not seem like a good solution either since it wouldn't handle parallelization of jobs and it would be idle most of the time.

After spending a couple hours in Kubernetes docs, it is still unclear to me if (a) this could be the tool I need and (b) it is not completely overkill for my needs. So my humble questions to you, reddit, are:

- Is Kubernetes the tool I am looking for?
- Have I overlooked the daunting UIs of GCP and AWS and missed something that would be suited for me?
- Should I hack something using EC2 or GCP CE's APIs to automatically turn on and off GPU-powered VMs on-demand? I am fairly sure that I could do that, but it feels like this is not going to be very robust, scalable or cost-efficient...

Thanks for reading me and in advance for your suggestions.

https://redd.it/qditck
@r_devops

London based Developers for social

Hi everyone,

I am fairly new to software engineering (< 2 years experience) and am trying to take my skillset to the next level. I currently work in a tiny tech start up in Shoreditch where I am pretty much the sole Python developer. I gain most of my knowledge through reading books every day and searching the web. However, I have little to no opportunity to gain informal knowledge and advice from other developers.

I am looking for any London based developers for a meet up for a coffee/ beer after work. Even if you are also fairly new to the field it would still be good to hear your experiences. I am happy to pay in return for picking your brains and sharing knowledge!

Please feel free to give me a direct message and we can arrange. Thanks!

https://redd.it/qdjy6j
@r_devops

Needs Advice: What to do in this situation?



I am a 3rd year student from a developing country, and recently got a part time position doing in a startup, gonna call my self Business Automation intern . I get into this since the one who hired me is doing DevOps based on his professional profile, and that's a path I wanted to take. Now, on the interview with him he told me that we'll do DevOps. Yet, as the day goes by, I'm not doing any of that, just Marketing Automation like integrating a tool with that tool. All are SaaS ones. Building websites, generating leads, designing and writing articles, things like that.


I feel that I'm not gonna grow here as an aspiring DevOps practitioner. An example is, I created a website tab for IT services and he told us that we'll just focus on Business Automation and decided to delete it. Now one of my friend refereed me to a MNC company where I will really do "real devops."

This comes the hard part, I wanted to grow and develop my skills in this company, but what keeps me here, is my gratitude to my current boss since he is the first one who accepted me. I'm not yet 1 month here. Also take note, I didn't have a bond, a contract or something that will keep me from leaving. Just the NDA. Tasks here are fine not too much stress, great team too, but can't grow with my career path.


Any advices, that would be much appreciated.

https://redd.it/qeqll0
@r_devops

AWS Amplify

I have hosted AWS Amplify and it has a frontend URL . That URL is accessible to all. Is it possible that this URL should be accessible to specific IP Address.

If yes. How can it be done??

I already know about access control option in AWS Amplify but it gives Sign ON option i.e. to enter id and password. That I don't want it

https://redd.it/qddr7y
@r_devops

How we are going to monitor our infrastructure

Hi r/devops

I want to share with you how we are currently monitoring our infrastructure.

# Overview

We have various python and JS micro services. All in all I think about

* 10 different custom services

Those services have dependencies like:

* rabbitMQ
* Postgres
* MySQL
* Redis
* MongoDB
* ElasticSearch
* SSO-Service (Keycloak)

All services and dependencies are containerized and used in various clouds always in a managed kubernetes service like AKS and EKS.

# Monitoring Stack

For monitoring, logging and tracing we chose this tech stack:

* Grafana
* Loki (Logging)
* Zipkin (Tracing)
* Prometheus (Monitoring)

The monitoring stack is containerized as well.

We are using Helm charts to roll out every service and those who did not offer to add a service monitor in the rollout process when added them ourselves (custom micro services).

# The Environments

We want to offer the development team and the company 3 environments:

* development
* staging
* production

All of those are integrated with our monitoring stack.

So far we decided to provision for every environment:

* Own kubernetes cluster
* Own monitoring stack (except Grafana, we use the different data sources to choose from -> dev, stage, prod)
* All micro services & dependencies

# Updating

1. Micro service updates happen in a rolling update manner using the rollout processes from helm and kubectl.
2. Infrastructure updates did not happen yet. I think we will simply go into AWS WebUI and hit the "update this cluster to the newest k8s version".

* So no blue-green deployment or canary deployment approach

# Further thoughts

1. Should we keep the environments separated on a cluster level or would you tend to separate on a namespace level?
2. What are your experiences in terms of updating a k8s cluster while the cluster is operating.
3. When did you reach the point where you said that self-managed dependencies is no option anymore and you switched to managed database services, managed queue services.
4. How could we improve the monitoring stack rollout? What I currently see as discuss-able is to rollout for every environment its own monitoring stack. I would rather tend to create a company's "Monitoring as a Service" and every Service that wants to be monitored should consume this single companies monitoring stack.

&#x200B;

**Thanks a lot for reading.**

**I am looking forward to your feedback :)**

https://redd.it/qeu0u9
@r_devops

Any affordable router supported well by ansible modules?

Are there any affordable routers for home and small companies which can be managed well by ansible?

Non community preferred. But in case of community modules PLEASE ONLY ANSWERS BASED ON YOUR OWN EXPERIENCE, NOT INTERNET OPINIONS!

https://redd.it/qdb6rd
@r_devops

Doing algorithm type coding challenge for a DevOps Engineer role which primarily focuses on CI/CD and terraform work. Is that normal?

I just did one, got destroyed lol. While I knew the coding language going in, I did not expect the level of difficulty. It isn't a big deal, but I am just wondering if others have come across this type of exercise for a DevOps role.

https://redd.it/qewltm
@r_devops

Can anyone help me providing some Info regarding the on-prem dc-dr(active-standby) setup?

The portion I am not so sure and confident about is the flipping from dc to dr. How to promote the slave db in the dr to master and use rsync to sync data from dc master to dr master after the flip!

Any info and any real life experience would be appreciated!

Thank you!;)

https://redd.it/qez0pl
@r_devops

Help me out here! Single Instance with multiple VMs in it or just Multiple Instances?

So i was wondering that if i deploy multiple instances with a fronting load balancer will it be more efficient in the terms of internal networking? (suppose if i am using Elastic Cache or Elastic Search or something else) rather than deploying multiple Containers/VMs in a single instance. Idk but i think the second one is more efficient there won't be any bottle neck or something? :/

&#x200B;

what do you guys say? take nodejs into consideration as the main process.

https://redd.it/qeq274
@r_devops

Deploying cloud resources alongside new versions of Kubernetes Deployment

Hi!
Im looking for approaches on deploying cloud resources alongside new versions of my applications in Kubernetes.

The problem im looking to solve is that i have an application dependent on an Azure SQL database (MSSQL). The application does not work against a database which havent ran migrations.


Solving this would look something like this:

Set the current application (and?) database to Readonly by ConfigMap flag mounted as a volyme which will be watched by my application
Provision a copy of the database
Deploy the new version of the application, targeting the database copy and run migrations against it
Probe application, when healthy & ready, delete the old version of the database

The thing I think I need help with is figuring out where deploying new cloud resources should be ran.
One way I can think of is running a Job in K8S which runs Terraform, but I dont know if this is the best approach.

https://redd.it/qeodw1
@r_devops

Anyone here working out of Ottawa? I am contemplating a move from Toronto to Ottawa for a DevOps role, insights would be much appreciated for our industry in Ottawa.

The role is not in government, but would be with a large international cybersecurity firm (private sector). Currently working with a consulting firm in Toronto. On very competitive wages. Love my job, only reason I am contemplating this is because of the reason below.

Reason I want to move: lifestyle in Ottawa fits us better. Proximity to nature, affordable housing, family support nearby, no crazy traffic jams, quieter lifestyle.

Why I am asking: Thinking longevity. I understand Ottawa tech market is not Toronto tech market, but where do you guys see it going in the next 5-10 years?

https://redd.it/qd29r8
@r_devops

Gh-pages project and OVH domain

Hi,

Is anyone here who can help me out with deploying gh-pages project on OVH domain? I really need help with setting it up.

Please write to me on discord: Sowa#9826

https://redd.it/qd445k
@r_devops

How do you make Apache work after transferring the EBS volume to another server?

Because the puppet configs no longer work, because some libraries cannot get installed even if I use a backup server for CENTOS6, I decided to take the EBS volume of another server and use it on my new AWS instance, but it doesn't seem like Apache is working, so what configs should I look to change? I think I saw some Apache config files with the wrong url, but I am not if just changing those is enough. We use nginx as a reverse proxy, Varnish and Apache. Also, any tip on debugging any issue or verifying each component work?

https://redd.it/qd3oa7
@r_devops

A single offline installation package for docker-compose

Hi

I am using docker-compose to deploy my services (including databases).

I need to find a way to save the current state of the containers (docker commit & docker save), archive it into a single package, and have the way to install it in a few clicks on Linux/Windows servers without access to the internet.

Maybe there is some elegant way to do it?

https://redd.it/qcz3j8
@r_devops

Exposing scripts over the network

In 2021, in the age of docker, kubernetes & serverless, is there anyone else still using VMs? Do you ever need to execute and extract the output of some scripts over the network, just like calling an API endpoint? If so, what do you employ for this?

I used to use PyJoJo for this but I feel it's limited. I just discovered Crystal lang a week ago and, since I had this need, I wrote a small tool that I feel is better and helps overcome some limitations where I couldn't pass parameters to PyJoJo.

I feel it might help others that have such a need. Drop a line if interested - it's out there on Github, FWIW.

https://redd.it/qf2bij
@r_devops

Azure networking

Unable to understand azure networking to the core in detail ,
Hi all

Do you guys have any suggestion on where I can learn azure networking in detail
any book suggestions from your side

Also it’s been a while I have been into networking , any refresher courses where I can refresh networking related stuffs
TIA ❤️

https://redd.it/qfbn9w
@r_devops

DevOps Bulletin Newsletter - Issue 23

DevOps Bulletin - Digest #22 is out, the following topics are covered:

DevOps becomes no-code?
Free Resources to become SRE/DevOps Engineer
Interview questions to land an SRE role
How to prevent Kubernetes misconfigurations from reaching production

Complete issue: https://issues.devopsbulletin.com/issues/should-devops-know-how-to-code.html

Feedback is welcome :)

https://redd.it/qfcub6
@r_devops

Maven build in Jenkins | Maven And Jenkins Integration

Maven build in Jenkins | Maven And Jenkins Integration

https://youtu.be/VWCB\_0M2DJc

https://redd.it/qfdc5l
@r_devops

Currently training for a DevOps job but not sure exactly what my actual role will be.

Hi, this is literally my first reddit post so bare with me if I don't understand some etiquette.

Basically, after dropping out of uni I struggled to get on my feet and then suddenly landed on a job as a DevOps engineer, this was a job that seriously interested me and right up my alley with prior skillset. I'm now 2 weeks into my 12 weeks of training and the job as a role has been explained to me but not really the progression in which I'll be working through. So what I'm asking is what do you guys expect I'll actually be doing in the beginning as I feel it would be a daunting task to have all these responsibilities 12 weeks after having 0 experience.

https://redd.it/qfhkr4
@r_devops

Integrate AWS SSO with Github

I am trying to integate AWS SSO with GitHub.

I have added the Sign On URL, Issuer URL and Public certificate from SSO on Github.

But I am stuck in the Application MetaData section in SSO.

For "Application SAML metadata file" I uploaded the file which I downloaded from above "AWS SSO SAML metadata file"

But when I Save Changes. It says "An error occurred, SPSSODescriptor is null."

I even tried entering meta values manually but I can't find ACS URL in the XML.

I am just confused about what to fill in ACS Url and Application SAML audience.

Any help?
Thanks!

https://redd.it/qfkifg
@r_devops