Deploying cloud resources alongside new versions of Kubernetes Deployment

Hi!
Im looking for approaches on deploying cloud resources alongside new versions of my applications in Kubernetes.

The problem im looking to solve is that i have an application dependent on an Azure SQL database (MSSQL). The application does not work against a database which havent ran migrations.


Solving this would look something like this:

Set the current application (and?) database to Readonly by ConfigMap flag mounted as a volyme which will be watched by my application
Provision a copy of the database
Deploy the new version of the application, targeting the database copy and run migrations against it
Probe application, when healthy & ready, delete the old version of the database

The thing I think I need help with is figuring out where deploying new cloud resources should be ran.
One way I can think of is running a Job in K8S which runs Terraform, but I dont know if this is the best approach.

https://redd.it/qeodw1
@r_devops

Anyone here working out of Ottawa? I am contemplating a move from Toronto to Ottawa for a DevOps role, insights would be much appreciated for our industry in Ottawa.

The role is not in government, but would be with a large international cybersecurity firm (private sector). Currently working with a consulting firm in Toronto. On very competitive wages. Love my job, only reason I am contemplating this is because of the reason below.

Reason I want to move: lifestyle in Ottawa fits us better. Proximity to nature, affordable housing, family support nearby, no crazy traffic jams, quieter lifestyle.

Why I am asking: Thinking longevity. I understand Ottawa tech market is not Toronto tech market, but where do you guys see it going in the next 5-10 years?

https://redd.it/qd29r8
@r_devops

Gh-pages project and OVH domain

Hi,

Is anyone here who can help me out with deploying gh-pages project on OVH domain? I really need help with setting it up.

Please write to me on discord: Sowa#9826

https://redd.it/qd445k
@r_devops

How do you make Apache work after transferring the EBS volume to another server?

Because the puppet configs no longer work, because some libraries cannot get installed even if I use a backup server for CENTOS6, I decided to take the EBS volume of another server and use it on my new AWS instance, but it doesn't seem like Apache is working, so what configs should I look to change? I think I saw some Apache config files with the wrong url, but I am not if just changing those is enough. We use nginx as a reverse proxy, Varnish and Apache. Also, any tip on debugging any issue or verifying each component work?

https://redd.it/qd3oa7
@r_devops

A single offline installation package for docker-compose

Hi

I am using docker-compose to deploy my services (including databases).

I need to find a way to save the current state of the containers (docker commit & docker save), archive it into a single package, and have the way to install it in a few clicks on Linux/Windows servers without access to the internet.

Maybe there is some elegant way to do it?

https://redd.it/qcz3j8
@r_devops

Exposing scripts over the network

In 2021, in the age of docker, kubernetes & serverless, is there anyone else still using VMs? Do you ever need to execute and extract the output of some scripts over the network, just like calling an API endpoint? If so, what do you employ for this?

I used to use PyJoJo for this but I feel it's limited. I just discovered Crystal lang a week ago and, since I had this need, I wrote a small tool that I feel is better and helps overcome some limitations where I couldn't pass parameters to PyJoJo.

I feel it might help others that have such a need. Drop a line if interested - it's out there on Github, FWIW.

https://redd.it/qf2bij
@r_devops

Azure networking

Unable to understand azure networking to the core in detail ,
Hi all

Do you guys have any suggestion on where I can learn azure networking in detail
any book suggestions from your side

Also it’s been a while I have been into networking , any refresher courses where I can refresh networking related stuffs
TIA ❤️

https://redd.it/qfbn9w
@r_devops

DevOps Bulletin Newsletter - Issue 23

DevOps Bulletin - Digest #22 is out, the following topics are covered:

DevOps becomes no-code?
Free Resources to become SRE/DevOps Engineer
Interview questions to land an SRE role
How to prevent Kubernetes misconfigurations from reaching production

Complete issue: https://issues.devopsbulletin.com/issues/should-devops-know-how-to-code.html

Feedback is welcome :)

https://redd.it/qfcub6
@r_devops

Maven build in Jenkins | Maven And Jenkins Integration

Maven build in Jenkins | Maven And Jenkins Integration

https://youtu.be/VWCB\_0M2DJc

https://redd.it/qfdc5l
@r_devops

Currently training for a DevOps job but not sure exactly what my actual role will be.

Hi, this is literally my first reddit post so bare with me if I don't understand some etiquette.

Basically, after dropping out of uni I struggled to get on my feet and then suddenly landed on a job as a DevOps engineer, this was a job that seriously interested me and right up my alley with prior skillset. I'm now 2 weeks into my 12 weeks of training and the job as a role has been explained to me but not really the progression in which I'll be working through. So what I'm asking is what do you guys expect I'll actually be doing in the beginning as I feel it would be a daunting task to have all these responsibilities 12 weeks after having 0 experience.

https://redd.it/qfhkr4
@r_devops

Integrate AWS SSO with Github

I am trying to integate AWS SSO with GitHub.

I have added the Sign On URL, Issuer URL and Public certificate from SSO on Github.

But I am stuck in the Application MetaData section in SSO.

For "Application SAML metadata file" I uploaded the file which I downloaded from above "AWS SSO SAML metadata file"

But when I Save Changes. It says "An error occurred, SPSSODescriptor is null."

I even tried entering meta values manually but I can't find ACS URL in the XML.

I am just confused about what to fill in ACS Url and Application SAML audience.

Any help?
Thanks!

https://redd.it/qfkifg
@r_devops

Cheapest Image/Video storage for my scenario?

Hey all,

I've been suggested to use Backblaze b2, but I noticed a flaw,

My goal to upload a file to a media storage and then store that uploaded file's URL in my database automatically with my API so I can refer to it. Backblaze b2 does not support this, I have to manually go to my storage, find my image link and then store it in my database.

I am deploying with Heroku. I am only using this so user can change their profile picture. It will not require any downloads.

I've heard Wasabi is the next cheapest option? Does that suit my case?

​

​

Here is a past post I made relevant to the subject on this subreddit :)! :

https://www.reddit.com/r/devops/comments/pr9dzh/cheapest\_imagevideo\_cloud\_storage\_for\_2021/

https://redd.it/qfjhuf
@r_devops

AWS secrets manager - Fargate: Can PHP FPM read the environment variables for root?

We currently have a PHP application running on EC2 (PHP7, php-fpm and Nginx) and we use .env file to store our credentials, and RDS is our database.

We dockerized the application and we want to move the application from EC2 to farmgate. We want to stop using .env file to store credentials. We want to use the AWS secrets manager instead.

There are two ways of implementing the AWS secrets manager, one way is to call the AWS rest API like this, but the DevOps prefers to export AWS secrets manager credentials to fargate as Linux environment variables like this I think

The problem is that, if you're running PHP from the terminal, it has access to the root user, so if you do getenv() you can get the correct value, if you run The PHP from the browser, it can only access its own enviroment, which are under www-data.

I asked the Dockerfile maintainers about this and I was told that their images can read the enviroment variables, here, I think the problem is that, the variables are exposed to root but not to www-data

Can I make php-fpm read the linux enviroment variables for the root user? If not, I was thinking of generating .env file dynamically every time the container runs by doing this

RUN printenv | sed -e 's/=/="/' -e 's/$/"/' > .env

The problem with this, is that I need to run that command every time a user adds a secret to the secrets manager and I don't want to recreate the container every time that happens. Any solution to that problem?

https://redd.it/qfms2z
@r_devops

More Experience In Environment Without a Lot of Need?

Not sure if this question is ok here so if not let me know and i'll remove it. So basically i'm an Azure cloud engineer and would like to get more devops experience but the current environment and business in general doesn't seem to have a big need for devops. I'm familiar with docker and ansible and a few of the technologies but only because i use them for personal use at home. We do utilize a good amount of services in Azure but i'm having trouble finding spots where devops would be helpful. For instance we only have about 15 vm's and only spin up a new one maybe every other month or so. I just don't see terraform being very useful for that. Also we don't really have any in house or custom apps. No in house developers. So no real need for app services or even container services or AKS. We have a few small apps but they are all built by a different team using power apps and power automate. I think i'm going at least start looking at ARM templates (or terraform) for different types of vm's to standardize that for future vm's. Any other suggestions of things to look into? I don't want to force new technologies just for the sake of having them if there is no real benefit. Just curious if anyone had any suggestions. It may just be this is an Azure environment that wouldn't benefit from devops tech. Sorry for the long post. Thanks for reading.

https://redd.it/qfowuo
@r_devops

ECS Fargate vs EKS

So, my startup is using ECS Fargate to run applications and it's going really well, it satisfies all our requirements. I mean, its fargate so we barely have to do anything, serverless is just amazing.

But if this is the case then why do people need Kubernetes, I mean apart from the fact that we can SSH into our application and that it's cloud native, I don't see any advantage of using Kubernetes.

I know I might have missed any points but that's point of this post, I want to learn more about this.

I have worked on Kubetnetes(not much though) and I found it pretty interesting and challenging but the startup I work for uses ECS Fargate and tbh it's kinda boring, and I feel like it stopped my growth.

https://redd.it/qfotkk
@r_devops

Learning devops hand-on, out of ideas

How do you do fellow devops engineers?

I come from a backend developer background and trying to learn devops to expand my skillset. I was trying to implement gitops principles but soon ran out of ideas.

What I have done till now:

1. Declared my infrastructure (GKE) using pulumi
2. Declared helm charts for ingress-nginx, cert-manager, external-dns
3. Deployed argocd and defined applications to deploy the above declared helm charts
4. Everything is automated and made easily recreatable

Now what? I was thinking maybe deploy Prometheus and grafana and configure it, can someone suggest me a better path? I want to learn how industry level infrastructure works.

If anyone's willing to go skim through my code to suggest changes:

infrastructure: https://github.com/jellyfish-squadddd/infrastructure

gitops: https://github.com/jellyfish-squadddd/gitops


PS: I do know that some of my secrets are exposed rn, I will fix it by using mozilla/sops tomorrow.

Thanks!!

https://redd.it/qfphn5
@r_devops

Resources on writing Python tests for CI/CD

I’m currently getting my feet wet with and learning DevOps, and read that part of the CI/CD process is to write tests that are automatically executed when code is pushed. Then, if the tests pass, the code is deployed to production.

So I’ve been working on a pet project that is pretty basic: an AWS Lambda function written in Python, API Gateway, and DynamoDB table on the back end, and a static web page with some JavaScript to fetch the API on the front end. I’ve got all this working through SAM and GitHub Actions, but I’m curious as to what tests I should be writing for QA for something this simple. I don’t think I’m grasping it enough and after pondering it for a couple days I wanted to ask if anyone here could point me in the right direction.

The Python is pretty simple: increment a value in the table then return that value. What kind of tests should I write for this??

https://redd.it/qfrn9p
@r_devops

Quality paid DevOps resources or classes

It's near the end of the year and I have a pretty large training budget I need to take advantage of. I was looking for some recommendations on good paid resources or classes. I come from a fairly heavy cloud infrastructure, cyber, and automation background but I'm starting to do more work with containerized environments (Kubernetes variants in both AWS/Azure and a lot of supporting toolsets luke Terraform, helm, docker, Jenkins, Gitlab etc.). I guess I'm looking for some quality resources beyond the typical Linkedin Leaning crap or low-quality Udemy courses and cost isn't an issue. I'm not against certs but I don't really want to manage renewing more certifications beyond what I have already. I appreciate any recommendations!

https://redd.it/qfopsa
@r_devops

Are we really doing CI/CD?.

In this post, We discuss the various spectrum of deployments and what Tailwinds observe with its customers.

https://medium.com/tailwinds-navigator/kubernetes-tip-is-ci-cd-really-popular-83887e1737e0

Do read and share your views/comments.

https://redd.it/qfz3dq
@r_devops

Holon Hackathon 2021

Hey all!

Wanted to invite you to a Hackathon I'm organising at the moment, sponsored by Filecoin and IPFS

Up to 55k in FIL to be won!

Check out this link for all the details https://taikai.network/en/holon/challenges/holon-innovate/ overview

https://redd.it/qfzfbt
@r_devops

Integrate GitHub with AWS SSO

I am trying to integrate GitHub into AWS SSO.

In the "Application Metadata" section that contains Application ACS URL and Application SAML audience. According to this link their values are different (I am taking AssertionConsumerService and EntityDescriptor) and when I upload the metadata file from AWS SSO page, it gives some different values.

​

When I copy values from that link, I get "Your request included an invalid SAML response" error.

When I upload the meta data file, I get "It's not you, it's us. We couldn't complete your request right now. Please try again later." error

​

I am confused about the attribute mapping part too, I have just mapped the default "Subject" to "${user:subject}".

I login on my github from email and on my aws with my username, I think attribute mapping is used for this but not really sure how to do it in console.

​

Not really sure what I am doing wrong here, need help.

Thanks!

https://redd.it/qg1vxz
@r_devops