Migrate large size or Mongodb and MYSQL to another server (low bandwidth)

Hey guys,

Any tips about migrating/cloning large size of mongodb databse and MySQL (100gb++) from one server to another.
Also, I’m on AWS and their instance are on China cloud services, so the network connection could be unstable.

https://redd.it/efwtwl
@r_devops

Hashicorp Vault for temporary aws/gcp/az web console logins?

Hey
I work for a team that often has to give Cloud related demos, workshops, etc.. at schools or other organizations. The demo accounts we use for this are usually created manually now, but we'd like to provision these with an expiration date automatically so we don't have to think about credentials being left active.
These are fairly low level workshops so all we need is an ability to generate a few accounts with Console access and limited permissions to spin up a VM etc. On all 3 major cloud platforms.

Hashicorp Vault seemed to be a great platform-independent tool at first glance but it looks like it is more aimed at provisioning service accounts.
Can this be done Vault? Is there a better tool for our use-case?
Thanks

https://redd.it/efvk7s
@r_devops

Single dev startup: Moving from Heroku to ???

The company I work for runs everything in Heroku. It does a good job and allows us to focus on building stuff, but now that the user base is growing so are the costs... So we are considering other options.

For context, our backend consists of a pretty standard Django/DRF monolith plus Celery, backed by Postgres and Redis. I am the sole backend dev and have very ilttle devops experience, so I want to avoid something overly complex or that will use up too much of my time. I am comfortable with Docker, but not with Kubernetes/swarm/etc.

A self-hosted Heroku would be the holy grail I guess, but I don't think it exists. I've used Dokku before and it works great, but doesn't provide clustering/HA. Flynn is similar and supports clustering, but the docs aren't great and the database support is described as "experimental".

So I've gone on to think about more complex alternatives. Maybe a cluster with docker swarm? Maybe managed Kubernetes from GCP or DO? Maybe Beanstalk or similar?

Has anyone made a similar transition or have any ideas on how to do this without embarking on a massive learning journey/taking up all my time?

https://redd.it/efsjo0
@r_devops

OUR SUBREDDIT MADE IT TO THIS BLOG

[https://www.portshift.io/blog/kubernetes-cloud-resources-devops/](https://www.portshift.io/blog/kubernetes-cloud-resources-devops/)

We got a shoutout!

https://redd.it/efuve4
@r_devops

Internal-only TLS certs from Let's Encrypt

I found a project that automates TLS certificates from Let's Encrypt that looked really great! Wanted to share


https://github.com/bradfitz/autocertdelegate

https://redd.it/efkvrw
@r_devops

Job offer to associate devops

Hi guys,

Recently been offered a role to an associate devops role for a msp. It is what I have been looking for, except, its a junior role and the pay is slightly lower than mine (8k aud less, I am on 98k)

I am a sr sys admin, but my work dont have much to do with clouds and they lack the cloud vision to move forward as a cloud company. The offered company has been cloud Consultant in aus for some time now, so big clients like major banks and Airlines.

What would you guys do? Stay with potentially becoming a SE or go to a junior devops role?

https://redd.it/efmq5w
@r_devops

Docker consumes too much of storage space

Hi everyone,
I have been working in DevOps from the past 2 years now. Everyday we face challenges that we somehow manage to resolve, but there are some challenges that still needs to be addressed by me and I think I have researched a lot and haven't got any relevant solution.

I have an Ubuntu EC2 instance on which I do PoC's using Docker. In order to create a container I build a docker image from scratch and then I run that docker image. When I'm done with my PoC I usually remove that container and it's associated image, then during night my cron job runs a script that triggers docker system prune in order to remove dangling images/containers/volumes.

I have observed that docker still is filling up my storage space even after deletion of all the images/containers/volumes. I have also cleared the system cache, but somehow storage space is filling up fast.

Any idea/suggestions on what I'm missing or what the issue is?


Thanks

https://redd.it/efgfnu
@r_devops

Ansible Vulnerabilities 2019

Hey,

Christmas time and ansible does not get unnoticed throw the security team

Read all about it:

[https://blog.firosolutions.com/exploits/ansible-monday-2019/](https://blog.firosolutions.com/exploits/ansible-monday-2019/)

https://redd.it/efftnb
@r_devops

Continuous Infrastructure Testing, how much do you all do it today?

Obviously testing for applications is a no brainer and best practice....there are numerous approaches including TDD for traditional software engineering. But for infrastructure as code I feel like testing gets discussed far less often. To what extent are people testing their infrastructure code? What tools and frameworks if any do you use? What pain points have you run into?

https://redd.it/efdzzl
@r_devops

Upcoming Project - Is Kubernetes the right choice?

I am currently tasked with building some automation around enduser's requesting the provisioning of a long-lived Amazon Linux/Windows Servers in our AWS environment.

These servers will host a wide range of services for customers e.g. file storage, database server, etc. This service is primarily a "We give you the Infrastructure. You do what you want with it".

Would it beneficial to develop the infrastructure around Linux/Windows containers in a Kubernetes cluster via AWS EKS?

Or would it make more sense to spin up long lived EC2 instances and manage them that way?

Thanks!

https://redd.it/efdf0w
@r_devops

Remote Devops/Cloud Engineer Job Possible?

Its been around One year since I have been introduced you cloud(Google cloud platform) since then I have been building my skills at up till now i Knows quiet few things about GCP and Aws other that this I also knows Docker,Linux,And kubernetes planning to take certification of Google cloud Associate Cloud engineer Next month.
My question is,
Is it possible if I have mastered this tools and script in languages I'll land a remote job if I'm fresher and not live in US?

please comment (Thanks)

https://redd.it/ef8ssk
@r_devops

Alerting when a GCP instance fails

This article shows a way to turn Stackdriver logs into mobile notifications (with a non-Google notification tool). Particularly useful when dealing with VM being restarted, and also potentially useful for any other kind of logs-to-notification.

https://medium.com/@cmaster11/alerting-when-a-gcp-instance-fails-c6cc3d60ff7

https://redd.it/egnp01
@r_devops

IAM Role for CloudFormation

I have a requirement to create an IAM Role for CloudFormation. I found a great document in AWS showing the IAM policy needed: [https://aws.amazon.com/blogs/devops/aws-cloudformation-security-best-practices](https://aws.amazon.com/blogs/devops/aws-cloudformation-security-best-practices/) What would be best practice for configuring the IAM role that it should be attached to though?

https://redd.it/eglk5b
@r_devops

Is there a "friendlier" tool to replace saltstack in our infra? (details inside)

Our infra is a combination of cloud servers plus multiple servers at our clients office's that we manage.

Therefore we need to manage both our cloud + "bare metal".

Because of this combination and my past exp, I went forth with SaltStack.

We prepare the client machines in house and then they become salt-minions and managed using salt.

Our cloud is setup as well using saltstack.

\---

The problem?

\- It doesnt feel 2020 (almost), although saltstack does what I need it feels there is room for improvement

\- learning curve. We are hiring and the problem is that we either need to look for hires with existing knowledge of salt or wait for them to catch up.

\- Shouldnt there be a more "user" friendly tool out there?

\- the client machine's run containers only - but we dont have the ability to "see" those machines as simple containers that we deploy to, they are still servers that SaltStack manages.

​

Let me know what you think.

https://redd.it/egq946
@r_devops

Phoenix project: this book still worth buying?

This is a quick question, I'm planning to purchase some books and I was considering this one, it still worth it?

https://redd.it/eggwyg
@r_devops

Could someone please share your insights/thoughts on my IT certification roadmap for 2020-2021?

Hi all,

I have created a roadmap for certifications to make in the 2020-2021 years. Please check the roadmap here: [https://imgur.com/rty1SwS](https://imgur.com/rty1SwS). The **red-color highlighted are the exams I am planning to appear for** and blue color highlighted are the ones I would do side learning but will not take the final exam.

Why do I keep the blue color highlighted ones if not take the exam? because I want to gain knowledge by preparing for free practice tests and going through the exam syllabus hot topics.

**A little bit about myself:** I used to be a Software Developer for the past 5 years. Mostly doing JavaScript, so limited knowledge on computer networking, hardware, and infrastructure side. I must also mention that I have intermediate Linux knowledge as a home Desktop user (not Linux server experience). I recently transitioned to the DevOps Engineer role in my organization. It is an enterprise company with many job opportunities, so I took my chance to upskill and try something I have not been doing. As DevOps Engineer, we work a lot on AWS, Linux, sophisticated server tech, VMWare, etc, etc.

**How is it so far**? I see that I lack Networking knowledge and server-level knowledge. I feel like some colleagues are speaking in Chinese because I can't follow the jargon they use at DevOps work.

**How can you help**? The roadmap posted is something I came upon my own after lurking on this subreddit. I am assuming some of the certifications I want to appear (marked in red) are duplicate or do not fit well with others. For example, I am not sure if RHCSA makes much sense because I run Linux at home and on top, I want to do VCP-DCV. And, I need your help to point out which ones.

**About certifications**: My company is supposed to sponsor for the exams but it depends on how much I can convince my supervisor, but the plan is to do the red ones in 2 years time frame. So, I am hoping it to be doable.

**What is my end goal?** I want to be a professional AWS solutions architect. I thought doing the red highlighted certifications will complement the AWS cert learning.

https://redd.it/egghk5
@r_devops

SPA application deploying independently of backend. Woke up a lot of missunderstanding on my side. Need bunch of opinions.

I'll try to explain clear. But might update the question if it is needed.

I work as a sysadmin/devops guy, freely I can say Junior role.

Our work is outsourced projects and we do not get to build a "product", these are usual jobs in my part of the world, Eastern Europe. Basically some guy that gets the customers and big part of the cake in US sends us some money and our comrade boss gives us paycheck. That said, projects are between 2 - 6 months and mostly are handed of to US client after that, either he spent money on an idea and sell it or just does not care any more.

Having that in mind, we got rather small project as a demo to our new client which will bring us more work afterwards. My though was to try something "new", at least new to us. Since it would be SPA app with additional native mobile work, I thought it would be a good idea to split frontend from backend completely.

My pros were in general that we can have independent deploys and faster deploys. And the project is small enough to test how it will go.

Some things were already known to us, like CORS issues. So that was not a "CON" side. I guess CORS is not an issue.

Me as a devops guy setup simple CI/CD and had all working fine. Frontend was loading from a main domain **OUR-FRONTEND.COM** and backend was **API.OUR-FRONTEND.COM**.

That was all good, but main developer now wanted to try another "new" thing and that is **httpOnly** cookies with same domain. That said my suggestion simply won't work. We reverted back.

**Question:** How can be achieved that frontend can still be developed, deployed and hosted independently of backend this way?

https://redd.it/egi63r
@r_devops

can i use line command in each task ansible?

I have 5 tasks in yaml file including lineinfile task. In line in file task, the line command prints the data in a file. I want to know how to use line command in each task under msg command instead of creating line in file in a separate task.

https://redd.it/egg6oj
@r_devops

What is the best way to answer "Describe your background question" for a DevOps position?

I come from a Senior Linux System Engineer background and would like to transition to a DevOps role. I am having a hard time getting an in-person interview after the initial one-hour phone screen? I do have significant cloud knowledge, I worked 3 years at an IaaS company like Digital Ocean, and have used Python, Bash, CI/CD Pipeline tools, Ansible, Puppet, and Docker, as well as provide T2/T3 support to clients via a ticketing system. What am I doing wrong?

https://redd.it/egsogy
@r_devops

Should I be looking for non-junior roles now?

Hey all!

Currently I graduated a year and a half ago and was very fortunate enough to jump straight into a Junior DevOps role and I've been there since. However, the company I currently work for is moving to another city (In May), and I'm adamant on staying where I am for various other reasons. Now when I'm applying for new roles in May, I would have accumulated just shy of 2 years experience as a DevOps Engineer.

Is it naive/complacent of me to stop looking and applying at Junior roles and apply for normal DevOps engineer positions? Or should I strictly be applying to junior position until a certain point etc.

The only reason I ask is that I want to be sure going through the whole, learning everything from scratch and being taught everything from ground up again (like my current did for me) is the best thing to do in my position.

NB: I know judging experience by years working is not an accurate indication of how good someone is, but the question is just for some general advice for those of you who have been in the industry far longer than I have.

​

Thanks!

https://redd.it/egtumq
@r_devops

Cors issue

I have run into a cors issue that doesn't seem to want to work, I have a Vue.js application which connections to the backend (node.js), front end is \`app.example.com\` and the backend is \`api.example.com\`

The backend has cors enabled on the api gateway which works fine in testing locally which leads my to believe it's an issue with the kubernetes deployment or maybe aws setup.

I am using Kubernetes with the nginx ingress controller to host the api with the front end on s3 as a static site. I have this anotation added to my ingress controller though I added it after creation.

nginx.ingress.kubernetes.io/cors-allow-origin: "https://app.example.com"

​

My ingress controller is an AWS ELB which is created by my ingress resource.

​

Anyone have a suggestion on what I can try? I've done a good bit of google time and came up short past enable cors on the backend (did before issue), enable cors on the bucket (not sure it's needed since I have no issue connecting to the front end just sending to the backend) as well as adding that anotation to the ingress controller and applying that

https://redd.it/egur1f
@r_devops