Google just legit sucks to try and contact regarding Dev to Dev issues.

So, I'm a dev, and I'm trying to contact Google.

So, apologies for the rant.

However, we don't have a specific contact there for our needs. So, I have to legitimately try every single which way to get a live support person that seemingly does not exist.

Apparently, the Android platform is adding pixels to the device instead of changing the pixel location on a browser page.

So, this creates an issue where it causes our platform to refresh. However, this causes a bug if we remove it \[which we plan to do interim\].

So, we're trying to create a trouble ticket for them to address the problem.

***\*Crickets\****

So, meanwhile, this is seemingly common place. So, if you have the same issue where normal interactions cause a window refresh, this is likely the root. Though, sorry I can't get anyone to address it.

https://redd.it/qr40uh
@r_devops

How do you manage resources like servers, accounts, accesses, switches?

I imagine most of you here work on bigger infrastructures with multiple resources involved. How do you manage all that? I hope not by using the spreadsheet or sheet of paper...

Are there any tools out there designed for managing servers, accounts, accesses etc? By managing I mean things like tracking IP addresses of the machine, perhaps MAC addresses or the hostname, the spec of the machine, software that runs on it, logins & passwords, ssh keys, physical location etc.

Same question about using accounts, e.g. 3rd party online accounts like gitlab / github / you name it within the automated, headless operations of the infrastructure. How do you manage credentials, billing etc.?

https://redd.it/qr7nau
@r_devops

sentry.io - almost exceeding transaction limit

I have a new web app and I'm monitoring it with sentry.io. This is my first time using sentry and I don't know a lot about it, nor am I an experienced devop. I implemented sentry to catch errors on my website. I'm now up to 80% of my transaction quota.

I suppose that at some point I may want to look at performance monitoring but I'm not there yet. I don't know how to turn it off, or how I would limit which types of transactions are being monitored. I think my website sees very little traffic. Over a month I'll record maybe 100 transactions a day and every 3 to 7 days I'll get a day with 800 transactions.

​

The email warning I received from sentry suggested I look at this webpage to "manage my event volume": https://docs.sentry.io/product/accounts/quotas/manage-event-stream-guide/?utm\_source=sentry&utm\_campaign=quota\_notification#-how-can-i-see-a-breakdown-of-incoming-events

but I understand very little of these instructions.

I'm hoping someone can get me started in understanding how I can limit the transactions, and also what I might want to do to manage my event stream.

thanks

https://redd.it/qr8fby
@r_devops

Where should Jenkinsfiles be stored using Remote Jenkinsfile Provider Plugin?

My team is in a peculiar situation where 1 jenkinsfile will run different repos. This is due to the portability aspect of each repo and how alike the build process is for each. We didn't want to duplicate the Jenkinsfiles and maintain each one by storing them in multiple different repos. So we are deciding to place the Jenkinsfile in their own repo and just use this plugin Remote Jenkinsfile Provider | Jenkins plugin.

However, what is the best practice when doing this? Should they just be contained with our Jenkins Shared Library repo? Or should they be in a separate repo?

https://redd.it/qran85
@r_devops

multi-redirected web script

Hi GUYS!!!

I want to make a multi-redirected web script without making a trace, from domain to other, can you give me the best practice to use or tell me the best language to create the script with.

Please, it's a serious question and I need your experience to understand!!

THANKS, GUYS!!!

​

You are the best always :*

https://redd.it/qr0qb3
@r_devops

Sandbox environment for evaluating devops candidates?

We are hiring devops folks and I'm looking for a solution where we can use a lab sandbox where I can ask candidates to build out some simple infra and write some code against it. Does something like this exist?

Edit

I'm looking for a kubernetes sandbox where I can ask them to install a helm chart and write a quick python or shell script to query what the helm chart deployed. Nothing to complex

From previous experience we seem to get people who know how to drive cloud GUIs but not so much read docs, put something together and then write a script to use it. Meaning we want equal emphasis on dev/tooling and operations. Out of the people who say they know Jenkins most have not even installed it themselves, less of those used the containerized version and even less are familiar with deploying it to k8s.

When we evaluate devs we give them a simple take home quiz, which some never even run the code they submit and are surprised that we do. We give them a choice of two languages and some of them write it in both.

I'm looking to give devops candidates a way to do the same with but with simple infra.

I was hoping to give candidates a VM so they could

* Install k3s, a one line command
* Install as service in k3s, I really don't care what
* Write a simple script to interact with the service

Not really seeing any of that as rocket science. Then in the technical review we could dig into the how and why.

Edit

Very interesting the responses here, some helpful, some gatekeeping, some passive aggressive

​

https://redd.it/qqvnhv
@r_devops

Gitlab Runner and Maven - Guide With the efficient cache method.

Speed Up your build process.

https://www.bitslovers.com/gitlab-runner-maven/

https://redd.it/qrl4lj
@r_devops

Logs and tracing: not just for production, local development too

We're a super small start-up, but with prior experience of great observability toolchains. It meant we knew what good looked like, but didn't want to spend ages building or maintaining it.

With that in mind, we've started shipping logs and traces into Cloud Trace + StackDriver and been really surprised at how little effort it required to get a great setup going.

Equally, we've all been surprised by how much it changed our local development experience. Going from an error in dev to the trace view and all the logs inline is just one click, which is pretty transformative.

Happy to answer any questions, hope you enjoy the article!

https://incident.io/blog/tracing

https://redd.it/qrkm6n
@r_devops

new to devops job and lost

Coming from a fullstack dev background and trying to learn all the new modern tools for this company. I've never had to use containers and container services, AWS, CI/CD. Since I worked at a startup I was able to get away with all the tech decisions. I spun up VM's, used cheaper clouds without complexity (hetzner, digital ocean), never used containers and am still having trouble seeing the point to them when I can use VM's and configuration/provision tools like ansible and terraform to have the servers exactly how I want. But this new company does seem to have a lot of different environments (i had just dev and prod) but they have like 6. That might have something to do with it.

They have the dev team and devops team separate and both kind of keep to themselves unless one needs something from the other. The dev team uses rails+react and other tools.

We use:
- AWS (EC2, ECS, Cloudformation)

- Jenkins CI/CD, jenkinsfile pipelines

- Docker containers

- splunk for log


- New Relic (for app monitor I believe)


My issues are that I just don't know how to learn these tools, especially AWS. The guides I'm checking out are outdated. I don't know if AWS recently changed but even a 2 year old video on ECS clusters with docker was too outdated to follow, since too many things had changed in the AWS console.

Could someone give me some clarity... or maybe the resources they used to learn these services?

https://redd.it/qrnivn
@r_devops

The DevOps Master Class - Go Behind The Concept



his free course by John Savil, a true Master, is 12-hours of superb material on everything you have to know in order to incorporate devops into your workflow.

It really is a complete and structured resource covering every aspect of DevOps, code included. As it is structured it is meant to be watched in order, but that's just a recommendation.

Starting with the definition of DevOps, Savill shows that it is not just about the tools:

DevOps is the union of people,
process, and products
to enable continuous delivery
of value to our end users

https://www.i-programmer.info/news/90-tools/15006-the-devops-master-class-go-behind-the-concept.html

https://redd.it/qrs3cm
@r_devops

AWS pushing live data and saving it to process regularly?

Here's the situation I'm in.

I get sent a single JSON object with current stock prices, for 2000 assets, every 10 seconds. The moment it comes in, I want to split it up by asset and send the current price for a specific asset to any client listening via websocket for that asset.

At the start of a new minute, let's say 11:45:00, I want to take all the price data I received for an asset between 11:44:00 and 11:44:59, average it, and permanently save the average (actually saving as candlesticks not averages, but that doesn't matter for this example).

Basically, the client connects and I send them the archived candlestick data for the last 1000 minutes. Then I update the client every 10 seconds as new data comes in.

Candlestick data is just stored in RDS with each asset as a table (no particular reason why, just simple). Client sends XHR, web server queries RDS, values are returned, connection is closed. Simple enough.

To get the live data, the client opens a WebSocket connection to API Gateway. I have an EC2 instance receiving the JSON payload that splits it up by asset and stores it in a DynamoDB table called live_price with the asset id as the key. A Lambda function polls the DynamoDB stream, and when there is an update to an asset a client is subscribed to, it passes that data to API Gateway which passes it on to the client. The great thing here is that there can be thousands of open connections to API Gateway and none of that matters. I just need my one little EC2 instance that receives the payload and updates DynamoDB.

Here is the problem. What's the best way to store the live data and pass it to a process that can form it into candlesticks to save?

Instead of a single table in DynamoDB called live_price with {asset_id:price} I could create 2000 tables each called asset_id storing {timestamp:price}. Then every minute I have a worker thread that connects to DynamoDB, get's all the records for the last minute, turns it into a candlestick, stores it in RDS, and then removes it from DynamoDB.

Or I could have my EC2 instance that receives the JSON payload store the last minute of price data in memory. When a new price comes in, it updates DynamoDB. When the minute ends, it transforms the array of prices into a candle stick, saves it to RDS, and then empties the array.

But that doesn't really work. Because if a client connects at 11:44:30 then I need to send them the RDS candlestick data, as well as the raw price data from the last 30 seconds, and then push the new live price as it comes in.

Is the best way to just have 2000 DynamoDB tables and a worker routine that processes and cleans it up every minute?

Or is my entire design flawed from the start lol?

https://redd.it/qrm3vr
@r_devops

Do you guys wind up doing a fair amount of helpdesk for devs?

Hey all, I was just wondering about how much you have to help the dev teams you support. I have ~2 years of devops experience and one year at my current company. I don't really mind helping out the devs if they have issues but I was wondering where I should draw the line.

I've got a few big projects that I'm working on and am having a bit of trouble balancing helping out the dev team with issues and my actual projects I've been assigned.

https://redd.it/qrni69
@r_devops

CI - help needed

Hi everyone! I am developing a web API for a university project and I would like to implement automatic deployment with bitbucket pipelines. I run my vms and DBs on Proxmox in my lan and my team uses Vpn to connect to.

How can I do it? Use SFTP? Is it safe enough? My only firewall is the ISP router and some rules on ProxMox built in FW.

https://redd.it/qruhfr
@r_devops

How do you handle your containers and environment with your dev teams?

Hey guys,

My team and the associated developers are really interested in moving to containers for our custom apps/services. We've spun up a small Kubernetes POC and started migrating a few things over and its been great. What I am curious about is how to disseminate this change to our dev teams. As of right now, they build their stuff, mostly in .NET core/5 and we push via our CI/CD pipelines between our IIS environments and eventually to prod.

When we switch to containerization I am not sure the direction to give our developers and looking for advice. Do we have them install docker locally, have them build/modify their projects and then have them also dockerize it locally? Or maybe they don't bother with it, let them do their thing and work on our CI/CD pipeline to handle the dockerization component?

Anyways, new to this shift and just curious what you guys are doing, what advice you might have.

Thanks

https://redd.it/qs1n2r
@r_devops

#Kubernetes tutorial for beginners | K8s Monitoring and troubleshooting ...

https://youtu.be/jlsxIx2bzsg

https://redd.it/qs3lo4
@r_devops

Issue with VirtualHost settings?

<VirtualHost 127.0.0.1:80>
ServerName staging02.cherry.com



ServerAlias www.staging02.cherry.com



DocumentRoot /home/staging02/version/www.staging02.cherry.com
ServerAdmin [email protected]

SetEnv environment prod
SetEnv project staging02

UseCanonicalName Off
#CustomLog /var/log/httpd/staging02.cherry.com_log combined
#CustomLog /var/log/httpd/staging02.cherry.com-bytes_log "%{%s}t %I .\n%{%s}t %O ."

## User cherry # Needed for Cpanel::ApacheConf
UserDir disabled
UserDir enabled staging02

#<IfModule mod_suphp.c>
# suPHP_UserGroup staging02 staging02
#</IfModule>

SuexecUserGroup staging02 staging02

<directory "/home/staging02/version">
AddHandler php5-fcgi .php
Action php5-fcgi /php5-fcgi-staging02
AllowOverride All


</directory>
<IfModule concurrent_php.c>
php5_admin_value open_basedir "/home/staging02:/usr/lib/php:/usr/local/lib/php:/tmp"
</IfModule>
<IfModule !concurrent_php.c>
<IfModule mod_php5.c>
php_admin_value open_basedir "/home/staging02:/usr/lib/php:/usr/local/lib/php:/tmp"
</IfModule>
<IfModule sapi_apache2.c>
php_admin_value open_basedir "/home/staging02:/usr/lib/php:/usr/php4/lib/php:/usr/local/lib/php:/usr/local/php4/lib/php:/tmp"
</IfModule>
</IfModule>
<IfModule !mod_disable_suexec.c>
<IfModule !mod_ruid2.c>
SuexecUserGroup staging02 staging02
</IfModule>
</IfModule>
<IfModule mod_ruid2.c>
RMode config
RUidGid staging02 staging02
</IfModule>
<IfModule itk.c>
# For more information on MPM ITK, please read:
# https://mpm-itk.sesse.net/
AssignUserID staging02 staging02
</IfModule>
</VirtualHost>

I replaced this with:

<VirtualHost 127.0.0.1:80>
ServerAdmin root@localhost
DocumentRoot /var/www/html
ServerName staging02.cherry.com
</VirtualHost>

&#x200B;

And I was suddenly able to get the default Apache page instead of 503 varnish error, and I was able to go to [staging02.cherry.com/test.php](https://staging02.cherry.com/test.php). So I am thinking there's something wrong with the above config. What can I remove in that config and what values should I enter? I am thinking ServerAdmin is wrong, I should change the IP address among other things, but I am not sure if it would work. Should I remove all ifModule configs? If it worked with simpler VirtualHost settings can we rule out a problem with php-fpm and fcgi?

https://redd.it/qrwtpr
@r_devops

Career advice

Hello everyone, I'll keep this simple I'm a 26 year old having 2 year of experience in AWS and docker and other cloud technologies(not devops). However I am not working from last 1.5 years for few reasons. In this time I have learnt a few other cloud technologies such as k8s, anisible etc. I want to get back to work and I'm applying to several devops related roles for past 2 months but not getting any responses. I don't hold any certifications as of now. I'm only trying for devops roles as this is the domain I want to get started in. How can I improve my situation and get hired?

https://redd.it/qs8cyx
@r_devops

How to avoid being pigeonholed in the job market?

First, to be clear, I’m not actively looking for another job because I’m pretty happy where I am and would rather stick it out until I’m promoted into a salaried SRE role.

tl;dr/BLUF: I have a little over 3 years of experience doing work that was nominally full-stack but 90% back-end in practice. For several reasons, I have no interest in going back to business application/API development. I’ve made this clear to recruiters who have reached out to me but all I get are LinkedIn messages and emails about full-stack and back-end jobs doing just that. The response I often get is that I could start out doing that but make it clear in the interview that I’d eventually want to move into a more devops-related role. But I know that promises are cheap and easy to initially make and that it can be a battle to actually hold a manager to them and if I lost that battle, I’d have effectively locked myself further into a career path I have no interest in pursuing.

The wall of text, for context:

For the past few years, I’ve been in a program for career switchers that gives people actual experience more or less as SREs, but with varying degrees of autonomy. In my day-to-day, I’m fairly autonomous; if I hit a stumbling block on a task, I know how to find the information needed to unblock myself or ask an informed question when the answer can’t be found in internal documentation or with a Google search.

Prior to this job, I had a contract role that was similar but with more responsibility in that my day-to-day duties were basically writing and maintaining code that processed data from that company’s infrastructure. Just prior to my contract ending, I was offered a full-time salaried job there, but declined it to take this job and because the environment there was pretty toxic (think things like engineers and managers sabotaging other engineers and teams for petty reasons). I mention this to provide context about my experience.

I came into my current job with a little more experience than a lot of people who typically get accepted into this program, but my actual job title is extremely entry-level (basically intern++). The reason I even applied for it is that I was having the same problem: I didn’t have enough experience outside of hobby projects for recruiters or hiring managers to be willing to take a chance on me.

The reason it’s been so long and I still haven’t gotten promoted was largely out of my control (think something like having had to take a lot of time off to take care of a sick family member). That has been resolved for a few months and I’ve been able to fully concentrate on work instead of juggling that and shitty life circumstances. I’m deliberately being vague because specificity would out me to anyone at my current job who reads this sub regardless of whether I chose to use a throwaway account, which is why I didn’t bother with that.

While I’m working towards being promoted into a real SRE role, the earliest that could possibly happen is the second half of next year (fuzzing the timeline, but not by much) due to the way promotions are done. In the meantime, I don’t want to have that as my only option and even if I did, it would still be nice to have another offer in hand as additional leverage when it’s time to negotiate salary and equity.

Short of just taking the full-stack roles off of my resume and LinkedIn and effectively making it seem as though I have half my actual work experience and a weird multi-year gap in employment (not to mention being deceptive), how can I start being taken more seriously for the jobs that I want rather than the jobs I’ve had? I have applied for the sort of job I’d rather have, but I think because I don’t have a CS degree, my resume automatically gets screened out by the ATSes at those companies. I have a liberal arts BA but also a fair amount of work experience from having worked as a low-voltage tech for a couple of years and then as a technician at an electronics refurbishment startup before self-teaching for a year and then doing

a coding bootcamp to get into web dev.

I understand it from the POV of hiring managers and recruiters; it’s much easier to hire someone for the skills for which they have an established track record. I don’t blame anyone else for my predicament, but I just want to break out of the cycle even if it means staying longer in a job I took a considerable pay cut to take.

https://redd.it/qsb4wb
@r_devops

Ideas for a side project / blog tutorial showcasing modern SRE/DevOps skill?

Hey all,

I am an SRE at a big company, but I'm narrowly focused on some stuff that isn't necessarily directly relevant to the greater market for SRE/DevOps.

I'd like to write a blog/tutorial series showcasing how to use modern devops practices, CI/CD, infrastructure as code, and some "bleeding edge" cloud services. This is mostly to sharpen up my own skills, perhaps benefit others, and something to show potential employers in the future so I can stay relevant.

I'm less concerned with what the application actually does at this point (a simple "voting" app, or perhaps a mastodon social network or even a wordpress blog), this is more about the tooling and showing how to deploy a multi-region, highly available and fault-tolerant, cloud-native system.

&#x200B;

I'm fairly certain I want to include the following technologies:

1. Dockerized app(s) - running in AWS in multiple regions
2. Probably with AKS, perhaps ECS or "fargate"
3. Infrastructure completely described with Terraform
4. Amazon Aurora/RDS backend
5. Content (images etc) served by CloudFront
6. CI/CD - infrastructure and app to be deployed via Github or perhaps AWS CodeDeploy CI/CD
7. Monitoring/Alerts (Datadog or similar free trial/tier?)

&#x200B;

Looking for suggestions on what you all think are the most "relevant to the industry" services and technologies, preferably which have a reasonable "free tier" or pay-as-you-go so I can minimize the cost of this project.

"Buzzword bingo" is a bit of what I am going for, I want something to show in github (I can't share anything I do at my actual job).

&#x200B;

Thanks in advance for any ideas. What would you include if you were (or have) done something like this? What are the must-haves for experience to be hired as a DevOps/SRE on your team?

https://redd.it/qsg5eq
@r_devops

How prepare for DevOps Role?

I’m a recent graduate who got their first programming job as a Junior DevOps Engineer. Prior to this role the only experience I have with DevOps is with the one course I took in college, which I enjoyed a lot and briefly touched on the technologies that I will be using in this role as well, but that was 2 years ago and I haven’t done much with it since then. I want to be best prepared for this new role as a Junior DevOps Engineer and am wondering what resources I could use to be best prepare myself as I start on December 6th.

TDLR; Starting DevOps role with no experience, how should I prepare for this role?

https://redd.it/qshc6u
@r_devops