Reddit DevOps
270 subscribers
5 photos
31K links
Reddit DevOps. #devops
Thanks @reddit2telegram and @r_channels
Download Telegram
What according to you is a developer environment?

Hello Folks.

I have been hearing about this term for some time now and wanted to understand what does this mean?

In our organization, everyone installs vscode on their laptop, and the required language. Developer environment is kind of ready, is there anything else?..

Pls share your thoughts

https://redd.it/z02e3b
@r_devops
How to organize E2E testing?

I am in an org dealing with financial data (SQL). A lot of the functionality is just CRUD operations but there are some more complex parts such as FIX trading and dynamic PDF generation.

In this environment we have limited unit tests as most functions are to update or select from the database and a lot of calculations are using aggregate functions from the DB.

Truly complex calculations are scripted as functions and they have unit teststhey also dont need to be updated and aren the places wev run into issues where functionality doesn't meet expectations.

Integration/E2E testing is where we've found tests are valuable however if we setup the environment for each test, it'll take forever to test the entire system (800+ routes, 50+ jobs, 50+ reports).

Instead our approach is to create a "data story", throughout the tests, data is created by the testsused and updated as the tests execute. However this does mean at least some test suites are based on others, and that when ealier tests are change they break later tests which starts to become unwieldy.

Thoughts?

https://redd.it/z04aqw
@r_devops
TraceView - OpenTelemetry UI released

We have released an early version of our OpenTelemetry UI "TraceView"

https://github.com/asynkron/TraceViewDeploy

TraceView is intended as a tool for developers, to pinpoint issues in microservice architectures.
There are plenty of good and scalable tools for observability out there, DataDog, Grafana, etc.

But they do tend to focus on DevOps and SRE, e.g. focus on latency, focus on huge logs.

TraceView rather tries to combine and analyze this data to show meaningful views where you can reason about what is going on.

e.g. as a replacement for Jaeger or similar on local dev machines.

Any feedback is welcome

https://redd.it/z05bcc
@r_devops
Please criticize my SaaS architecture

https://drive.google.com/file/d/1JNnqSIbkSikTjtmjQYNtsDHbclL\_Pnot/view?usp=sharing

Monthly budget: 150 USD

I have also used the AWS calculator to sort of determine how much it will cost monthly, see here https://drive.google.com/file/d/1V-ZQrEyBYYl1PRaLJ5L\_SLjOeyRD5U81/view?usp=sharing

Idea: It's a serverless WordPress hosting infrastructure

1. The site owner sends a request to setup WordPress
2. There are two containers in the fargate instance (WordPress and DB container), Data are persisted in the DB
3. EFS for volume mount
4. & 5. Lambda function to deploy the static site to S3
5. The same with 4
6. Managing end-user requests to s3 bucket

​

This is just an high-level overview, I didn't go indepth on security and managing DDOs, however, I welcome all suggestions

GOAL: The goal is to reduce the budget as much as possible to 120-150 Monthly

​

Thank you

https://redd.it/z08vmc
@r_devops
A poor man's API

Creating a full-fledged API requires resources, both time and money. You need to think about the model, the design, the REST principles, etc., without writing a single line of code. Most of the time, you don’t know whether it’s worth it: you’d like to offer a Minimum Viable Product and iterate from there. I want to show how you can achieve it without writing a single line of code.

Read more

https://redd.it/z0a1zx
@r_devops
New to devex and worried my work isn't going to scale -- I could really use some advice; as broad or as narrow as you like

I started as a SWE at my current job, and kept stealing time from other projects to work on CI improvements and process automation. This work was well-received, management asked if I wanted to do DX full time; I said sure. For context, we're at about 30 devs right now.

I'm a department of one, for now, and have huge leeway to set my own "product direction," goals, KPIs, etc. This is really exciting, but I haven't done it before, and I'm not good at it yet. I started by holding meetings with eng management and running a developer experience survey, and came to the conclusion that my team's two biggest problems were broken development environments and tests that were difficult to run locally and flaky in CI.

Since assuming this role officially a few months ago, I've put the majority of my effort toward environment standardization. It hasn't gone particularly well. Major problems include:

* **Bugs in upstream tooling** \-- we're using VS Code's dev containers right now, and it's been a rough ride, with poor environment caching , intermittent port forwarding failures, and disappearing git credentials being particular pain points. We weren't doing great with purely local environments, though, and I'm not sure whether other solutions will be worth the pain of switching.
* **Sloppy custom tooling** \-- we have our own process runner that we wrote to replace Invoker, and a simple deployment UI in Slack, both of which break all the time and have poor DX. Neither was ever meant as a long-term solution, but both are now ingrained enough that switching will be an ordeal. While I'm aware of many other options here -- docker-compose or tilt for task-running; Terraform Enterprise for deploys since we use it for other infrastructure stuff already -- I'm again worried about making a bad bet.
* **Hidden information** \-- I get a lot of inbound about problems people could solve on their own... if they knew where the information was. Some of this is related to the aforementioned weird task runner and its bad DX; a missing dependency or occupied port will just manifest as "the app won't load" and then I have to go spelunking. CI is a big one, too: our integration tests, for instance, record videos and an incredible amount of debugging information but all of that is hidden in the CI runner's UI. Most devs don't even know we record it until I show them in a debugging session; I haven't settled on a good way to surface it.

So, I'm looking for advice from experienced practitioners -- I recognize that I've asked an unhelpfully broad swath of questions, but right now I'm not confident in my ability to decide which problem is most worthy of my attention, so I kind of need to ask about the whole job.

This could be as broad as "strategies for interfacing with devs and determining priority" or "here's how I evaluate tooling," or as specific as "this is a tool for environment management that has consistently worked for me." Basically, anything you would want to know if you were just getting started, moving over from being a SWE.

https://redd.it/yz3g21
@r_devops
Side work contracting

I have a company that is interested in hiring me for some contract work to support their junior DevOps person and also guide the DevOps side of the tech team. It's a small start up with very few people outside tech.

Currently their systems are a mess from what I can tell. So it will be a bit of work to resolve it all.

A bit about me, I have held DevOps or SRE roles for the last 4.5 years working at companies ranging from 30 person start ups to a 6000 person multi billion dollar org.

I currently make about $110 CAD an hour and am trying to come up with a fair rate to charge this prospective company. I would be doing the work after hours of my day job. I tried to look up rates online but came down to a range of $80-250 which I can't really tell where I should ask for in there. Based on what I said, what would be a reasonable range to aim for? I was thinking $150 since that's a bit above my day job but not too high. I was thinking a bit higher than my day job because of no benefits and PTO. Or does it make sense to ask for the same rate as my day job?

https://redd.it/z0d2rw
@r_devops
Tilt vs. bespoke Kubernetes tooling

I've been reading up on Tilt lately, and I think the concept of development in k8s clusters could be really beneficial for our team. As it turns out, somebody else on the team who was unaware of Tilt had recently had the same thought, and he's been building out his own tooling for cluster-based development within our internal dev CLI.

My initial instinct is that we should use an off-the-shelf thing for this since it's available. But inspecting my colleague's code, it seems like the job of actually starting/updating resources is only a small part of what he's put together -- much more of it is devoted to

wrapping that process in a CLI flow that's attentive to the particulars of our application
determining which services to update and when (the precise arrangement of k8s resources is done in Helm templates which are parameterized and triggered from the CLI)

Tilt's UI is phenomenal, and I'm tempted to use it to run our services for that reason alone. But I'm not sure whether there's benefit to going deeper than that. It seems like, in Tilt-world, the logic for what should update and when would go in a Tiltfile, and the interactivity would still go in a higher level tool (since AFAIK Tilt/Starlark has no mechanism for interactive user input). But given that my colleague has already done a lot of work on the orchestration in Go, is there any value in porting it? Or to ask the question a different way, is there anything cool that Tilt does for you that it might not be obvious you even want until you've done cluster-based development for a while?

https://redd.it/z0czs0
@r_devops
Scenario based question for DevOps Engineers out there

Trying to do a challenge I was given, I was wondering how different people would approach responding to questions like these in a real life environment. Just indulge me if you can :)

You need to set up a new product development effort for a new team. The product will be built in Python on AWS. It will be a database backed product with an APIs and an Angular webapp. The product will be containerized and will be deployed in AWS. The DevOps will heavily use Terraform.
What questions do you have about the setup of this product development effort? (Really looking forward to your answers on this)
What are your first steps?
Preferrable to do these steps in Terraform (you can skip this part)

​

Additional questions if you have time to answer (more of what your boss would look out for I guess):

What technologies do you suggest? Are they modern, well explained, practical?
What kind of questions do you ask about the effort? Do those seem like practical, curious DevOps questions?
How do they plan to set up the Terraform? Do they think of remote state? How much console work do they plan? Do they reference the API/CLI? Are they suggesting Terraform workspaces, modules, variables, conditionals, tfvars, etc.?
What are their bootstrap steps? Where do they plan to keep their files and state? Do they plan to use GitOps approaches? If so, how do they set that up?

​

So in essence what kinda questions would you ask to start off and set up when your team gets a new project, keeping in mind you're provisioning with Terraform.

https://redd.it/z0galk
@r_devops
Free Auto-Unseal cloud service for a private Vault instance

Hi all,

As the title says, I am looking for a free way to auto unseal my private vault using any cloud service, I already checked a couple and they already charge for everything which will be a pain to monitor.

Currently I am using this nice project but was wondering if a cloud solution would make more sense. Please advise.

Cheers,

https://redd.it/yx00fb
@r_devops
robocopy and hidden file

Does robocopy.exe copy hidden files by default ?

Do I require to give any parameters/options in the command ?

https://redd.it/ywubzx
@r_devops
What do you guys use for outage monitoring / incident reporting?

Seems like there are a lot of different services out there offering this kind of thing nowadays.

Which ones do you use, and if so, what should I look for when picking one? Any I should check out in particular?

For context, I manage the devops pipeline on a number of projects - smaller teams (under 10) covering a wide range of services.

https://redd.it/z0lcss
@r_devops
Is KodeCloud a good devops platform?

Saw an ad that Kodecloud is running some discounts. But the subscription plans are for 6 months/ 1 year. Has anyone tried the Kodecloud platform to learn devops? How good are their platform?

https://redd.it/ywe9tu
@r_devops
Create Your Own SpeedTest Server. Only Static File, All you need is Nginx.

Free and Open-Source HTML5 Network Performance Estimation Tool Written in Vanilla Javascript and only uses built-in Web APIs like XMLHttpRequest (XHR), HTML, CSS, JS, & SVG. No Third-Party frameworks or libraries are Required. All we need is a static web server like Nginx.

https://redd.it/z0qr0o
@r_devops
Trying to deploy my app with Elasticbeanstalk

Hi guys, so I'm new to AWS and I'm trying to deploy my Django app with elastic-beanstalk. Right now I'm trying to create an environment with my terminal. The following commands I've used:

1. eb create; Then I choose the default environment name and DNS CNAME prefix. I also choose us-east-1(N virgina) for location
2. I choose application load balancer and No for spot fleet requests.

I then get this error;

WARNING: Insufficient IAM privileges. Unable to determine if instance profile 'aws-elasticbeanstalk-ec2-role' exists, assuming that it exists.
Creating application version archive "app-5537-221115164654825241".
Uploading django-blog-v2/
app-5537-221115164654825241.zip to S3. This may take a while.
Upload Complete.
Environment details for: django-blog-v2-dev
Application name: django-blog-v2
Region: us-east-1
Deployed Version: app-5537-221115164654825241
Environment ID: e-vwmssiuvwt
Platform: arn:aws:elasticbeanstalk:us-east-1::platform/Python 3.8 running on 64bit Amazon Linux 2/3.4.1
Tier: WebServer-Standard-1.0
CNAME:
django-blog-v2-dev.us-east-1.elasticbeanstalk.com
Updated: 2022-11-15 21:46:58.561000+00:00
Printing Status:
2022-11-15 21:46:57 INFO createEnvironment is starting.
2022-11-15 21:46:58 INFO Using elasticbeanstalk-us-east-1-347584916308 as Amazon S3 storage bucket for environment data.
2022-11-15 21:47:00 ERROR Unable to assign role. Please verify that you have permission to pass this role: aws-elasticbeanstalk-service-role.
2022-11-15 21:47:00 ERROR Failed to launch environment.

ERROR: ServiceError - Failed to launch environment.

I have two users, Administrator and django\
user. I've tried adding multiple permissions/policies to both users but it's still not working. Anyone knows what to do.?

Administrator: https://imgur.com/MkUU3nR

django_user: https://imgur.com/FmR1mz5

users: https://imgur.com/DuDH7gV

aws-elasticbeanstalk-service-role: https://imgur.com/dy7ygoN

aws-elasticbeanstalk-ec2-role: https://imgur.com/y4Nzwo1

roles: https://imgur.com/yQRlLdg

current config.yml file: https://imgur.com/8pVhfXp

https://redd.it/ywbeda
@r_devops
Rapidly rising costs; bloated over-provisioned infra; and a lack of ownership of cloud expenses among IT teams are increasingly common issues. Did you know that FinOps is the answer to addressing these challenges? Read full blog here:

https://www.umbrellainfocare.com/blogs/finops-best-practices-to-manage-cloud

https://redd.it/z0udo8
@r_devops
Advise for someone going back to work after burnt out

Headed back to work after a month or two of a break. I definitely had a lot going on before leaving the last place (personal and work related) and unintentionally I realise I got burnt at the end of it.

Took sometime, went off digitals and eventually reset myself. Some of it also came from stopping my continuously studying and learning habits (as the grind must go on).

Moving forward my approach has been a bit direct and focus on industry I wanted work in, with team I want to be involved with and the technology I want to look at, so I can build up on my skills for my personal self that I have been practicing and building in my own time.

( My background is Sys Engineer moved to Development and looking after Site Reliability operations. I’m definitely not new to the space but haven’t grown enough grey hair yet)

However, I have just come back into the market and have found myself something good with great opportunity to learn and be involved in.

In this space, I have found learning is part of life, but it concerns me to end up in the same rabbit hole as I did before.

Any advise for someone who maybe had a similar experience or can relate?

https://redd.it/z0ubym
@r_devops
Awesome Black Friday & Cyber Monday Deals For Developers

Hey everyone,

I hope you guys are doing fine.

I'm sharing here some awesome black Friday deals for developers in 2022.

* [Themeselection Admin Templates & UI Kits- 30% Off Storewide](https://themeselection.com/)
* [WPRocket- 30% OFF](https://wp-rocket.iss.one/)
* [Vue School – Premium Vue.js Tutorials – 51% OFF](https://vueschool.io/sales/blackfriday?)
* [Weblium - 30% OFF](https://weblium.com/)
* [Pluralsight - 50% OFF](https://www.pluralsight.com/)
* [Laracast – 45% Off](https://laracasts.com/sales/black-friday)
* [SeedProd - 65% OFF](https://www.seedprod.com/black-friday/)
* [65% OFF on DataCamp Annual Plan](https://www.datacamp.com/promo/black-friday-2022)
* [Udemy Black Friday Deals 2021 Up to 90%](https://www.udemy.com/)

I hope you all find it helpful.

https://redd.it/z0xmg8
@r_devops
AppSec: How to gain full security code scanning coverage of all projects via CI/CD pipelines?

I work for a large old company with over 1000 projects (or apps), projects are split into different domains in Azure DevOps and a bit scattered in GitHub. Currently, we have about 10 percent of projects going through our scanning tools via CI/CD pipeline. We are currently trying to enforce any projects going live to meet our vulnerabilities pass criteria (e.g. public app needs to have all high and mediums fixed, etc). This is a priority, we cannot have any public apps that are not security scanned as it poses a threat to our company. Is there a way to force this in ADO?

How do we ensure complete coverage of all apps, what can we do to enforce developers/devops to add our security tooling into their pipeline and what do you do in your organisation?

https://redd.it/z0xug5
@r_devops
Cyber Threat Advisories Study (Tufts University & University of Edinburgh)

Hello r/devops we need your help!!


I (researcher from University of Edinburgh) am working with a PhD student and their advisor from the Tufts Security & Privacy Lab, and we are conducting a survey to understand where organizations get information about potential cyber threats (i.e Vulnerabilities or even Patching Information). After discussing with a moderator of this community we have been allowed to post here and would love to have your input! We will share any publications that come out of this work with the communities that allow us to post :)


We want to know where you go for this information and what you think of your sources. Our goal is to improve how information about potential cyber threats is communicated, which will make warnings more effective and reduce alert fatigue. The survey will ask where you get cyber threat information and how useful you perceive certain sources to be, along with a couple general questions about your organization’s sector.

You are eligible to take the survey if you manage the security of computer systems (for example, applying security patches) or have held such a job in the last 2 years. Respondents currently located in Mainland China are not eligible. (Respondents in Hong Kong SAR, Macao SAR, or Taiwan are eligible.)

Survey respondents can choose to enter their email address in a raffle to win one of 30 $100 Amazon.com gift cards. We will protect the privacy and confidentiality of any information shared on the survey with the utmost of our abilities. You can see more information about the survey here - https://tsp.cs.tufts.edu/cybersecurity-advisory/


If you are interested, go to https://tufts.qualtrics.com/jfe/form/SV\_e4c9iqHdLJUjRfE to complete the survey. Please contact us at ir\[email protected] if you have any questions about the study and I will also answer any questions I can on this post!

Additionally - please do share this post/survey with any other admins/security personel who work in Vulnerability and Patch Management.

Thank you again for any participation and help given!

https://redd.it/z0z76w
@r_devops