Has anyone switched to a product team from devops? What were your reasons and how are you liking it?

I'm curious if any others have come from building products as a SWE to DevOps, and then gone back. What did you like about each role? What didn't you like?

​

For me it feels like product work is much more flexible with the implementation details, and creating/scaling infra or setting up monitoring and alerts is very rigid, there's not much room for creativity, however, there are moments of pure automation joy when everything works and needs little maintenance.

https://redd.it/13eesqi
@r_devops

How do you deploy multiple preview environments across multiple repos and branches?

Hi. Currently working on creating a better way of managing our preview environments.

Our current setup is having 10 preview environments which can be set to a branch on demand, using a web-ui (Forge). However, as this is kinda unscalable and not dev-friendly, I am working on a way to automate this using Github branches and automatic deployment to AWS.

So, when someone creates a PR on frontend, I will deploy everything needed to AWS (like an S3 bucket). However, I also want it to "connect" to its own, isolated, API (say, an EC2 instance). This seems simple, but we ALSO want to be able to select on which branch that API is checked out, as our devs' workflow is often that they work on api and frontend simultaneously, so we can't just make their frontend PR-deployment connect to api-master.

Does this make sense?

I basically have every building block already done, except for this part - the "workflow" part as we call it.

Would love some insight into this! If you need more info on specifics in order to give a meaningful answer, please let me know!

https://redd.it/13efp40
@r_devops

I had to do mandatory military service which wasted a year of my life (had no access to a smartphone or internet). Have a career fair coming up but I don't even know what kind of jobs to apply for.

######My resume

----
After I graduated from university, I did 13 months of military service, training police officials with my IT skills. No phone or internet access and only a week of vacation monthly. Tried to study but limited access made it unfeasible, so I postponed it till I was done with my service.

Now, I'm currently following the "devops roadmap" that recommends foundational knowledge in Python, Linux, Networking, AWS, and IaaC. I have a solid grasp in python and linux (studied for the RHCSA), and I'm now prepping for the AWS SAA cert while grinding leetcode.


Questions:

1. What kind of jobs do I even apply for? I don't think I'm ready for a developer position. And yet, I don't have the necessary experience for a system administration one either.


2. Should I continue learning the rest of the roadmap?

https://redd.it/13egfeu
@r_devops

DevOps to ‘DevOps’

Yo Dev-Ops cats.


I hope some of you can hear me out and maybe relate to me a little. And if possible offer some advice, or encouragement.

When I got into the industry I started as a software developer. I was doing C#, a little javascript, and a little Java. I was new and young and it was awesome. I could write run-on sentences for days describing how happy it made me. It’s like a toddler describing Christmas to an alien.

Eventually I started learning the Ops side of the project I was on. That was exciting. It was new technology, it was meaningful to my project. I could see a direct impact of what the Ops had on the Dev work. And why they would go hand in hand, praise the DevOps!

Here’s where the storm clouds roll in. Once I learned these Ops skills, like CI/CD, Docker, Docker-Compose, other sys admin things, I was sort of pulled. The company said “this is great! We have a guy who understands this new (ish) technology and can reliably use it for a project!” So they set me loose. I was now doing Ops for every project it felt like. I thought wow this is interesting. But boy did I burn out fast. I didn’t realize that I lost all of my passion and drive because I was missing real DevOps. Not ‘devops’.

I was missing having a meaningful relationship with the project that I was working on. I missed doing Ops for something I was working on. Not Ops for someone else’s problem. This model of taking real budding DevOps programmers and making them ‘devops’ or really just ops guys can’t be something only I went through.

I just don’t know what to do. I feel like I had so much momentum as a developer(and originally more of true DevOps programmer). But I feel like a baby who had his lollipop taken away. Personally I’m tired of feeling bad for myself. I’m going to find a way to break back into a real DevOps role. But I’d appreciate any thoughts, advice, relating to’s, or even ‘this is the wrong place for this bud’.

As for the title I describe this as being a DevOps guy put into a ‘devops’ role looking to return to the DevOps guy. Like Star Wars:

Star Wars 4: A New DevOps

Star Wars 5: Empire ‘devops’ <- I’m here

Star Wars 6: Return of the Dev-Ops



Help a DevOps guy out.

P.S. on mobile, sorry for format issues. Plz comment and I can fix in the morning. Been up all night with an existential crisis (this is it this is the existential crisis).

Edit 1-3: formatting

https://redd.it/13ei26p
@r_devops

Digest #102: Are Terraform’s days numbered? - Same argument that language X may die b/c of new language Y

Its not apples to apples. But these headlines are stupid for a devops bulletin, misleading to some that may not have a grasp on what it means for them, if they are just getting used to TF.

Just b/c Prime goes Monolith, after rapid development and exhausting, enough, mitigation of performance to realize, that its time.

TF doesn't only deploy/use micro services, lambda, step functions and containers.

</rant> I guess. You see this so often on subs like r/AskProgramming

https://redd.it/13d71dy
@r_devops

For a career in DevOps what certification should I acquire?

Hello,

I am new to the field of DevOps. I have AWS Cloud Practitioner certificate and am working on some hands on project with help of ACloudGuru training platform. No professional experience though.

At associate level, what certificate should I apply for Associate Developer Or Associate SysOps Administrator or Associate Solution Architect? Which is more in demand? My company may sponsor the certification exam if I can get it before July. Hence, the hurry 😅

Any other suggestions for a beginner in Aws and DevOps are welcome.

Thanks in advance.

edit: help me choose the best associate level certificate

https://redd.it/13elq9u
@r_devops

Automate creation of Project Agent Pools (VMSS)

Hi all.


Has anyone managed to link up a scale set to a project agent pool via command line?

I have a pipeline that creates the gold image, adds that to a VMSS via Terraform but i would like to finish the workflow by having this scale set added to a scale set pool so they can be used within Devops on other projects. I have tried to google but come up short.

Thanks.

Below is a screenshot of what i am wanting to automate.

https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/media/scale-set-agents/create-agent-pool.png?view=azure-devops

https://redd.it/13en9sp
@r_devops

FYI: GitHub appears to be down at the moment (https://www.githubstatus.com/)

I'm getting HTTP 500s when viewing any repository on the site.

https://redd.it/13enp35
@r_devops

3 straight days of degraded github performance

That's it, I'm bring back the Travis deploy scripts. Suddenly Travis' 90% uptime seems amazing.

https://redd.it/13enths
@r_devops

Leaving job after 6 months

This place is a total mess. 0 documentation, 0 onboarding, 0 work life balance. They completely lied to me during the interview process and their job description was entirely misleading. I've stuck around so far hoping that it's going to get better, but it's only gotten worse. There is 0 work life balance so I'm looking at new jobs.

What do I put on my resume for this place? I've managed to learn a few of their tools here and onboarded myself since their process was entirely non existent but not much more than that...

https://redd.it/13eph8w
@r_devops

Friend & I built a production debugging & monitoring alternative to Datadog, New Relic (based on OpenTelemetry + Clickhouse)

While Github is down, figured it might be some good down time to share what my friend & I have been working on...

We've been building a Datadog alternative to have one place to monitor and debug production apps, in an actually affordable way (Currently 9x cheaper compared to DD).

We’ve previously ran the numbers looking at Datadog for some of our services and realized our Datadog bill would rival our AWS EC2 bills! (and I know we aren’t the only ones with that problem). Yet we also knew it was hard to get the end-to-end visibility we often needed to debug complex race conditions and data-driven edge cases from other vendors.

So we’ve decided to spend time crafting the production debugging product we needed internally, and share it as a viable alternative for others as well.

It’s built on top of OpenTelemetry, Clickhouse and S3. This ensures we’re able to scale indefinitely, with minimal cost, and still have tons of flexibility to build a complex product on top of it all. With it, we’re able to easily tie together charts, logs, traces, and session replays, all in one place.

If this is interesting to y’all - would love to hear what everyone thinks:

https://hyperdx.io/

https://redd.it/13espyq
@r_devops

Needing help with the non-technical side of pipelines.

Please forgive any non devops terms, I’m not a SWE.

I’ve been desperately searching for information on how the CI CD process works from a people/processes perspective. What is the process when code/application fails a test or check?

What I really need to know is what happens when code fails a test? Who is notified? How are they notified? Who is suppose to fix it? How are the failures and fixes typically documented? Is there a person to make final approval before deployment? I realize this will highly vary for different environments and products but is there a standard guideline for this?

I’ve stacks upon stacks information that tells me what x product checks for and how to configure for checks. I’ve got 100 different diagrams that show me how code is created then tested then deployed. But almost none of it goes through the process of code failure and fixes.

I am hoping to find something like this:

1. Developer creates/ updates code
2. Code is ran through tests/checks
3. If Code fails test
A. Developer/Manager/Princess Peach is noticed of failure
B. Failed code is sent back to (original developer? backlog software? Ticketing program? Manual spreadsheet tracking?)
C. Code is fixed by
D. Return to step 2
4. If code passes all tests
A. Code is sent to passed code repository
B. is notified of code awaiting final approval
C. Code is approved and sent for deployment
5. Code is deployed to production environment.

It is possible I’m looking at the whole process wrong and I am open to any feedback on that too. Maybe there is no process? Maybe if the code passes the tests it is just dropped into the production environment automagically and there is zero human intervention?


Thanks for the help.

https://redd.it/13eqkut
@r_devops

Anyone using Terraform in a single production environment with multiple devs that need to test changes?

We're leveraging a single production environment using CI/CD, feature flags, and other best practices. The biggest challenge we have is testing changes via Terraform, whether it's application or infrastructure. The main problem is that dev A is working on something and deploys changes directly to prod (from local) for testing, but dev B also needs to deploy, undoing dev A's changes.

Right now we're just closely collaborating to not step on each other and I can't imagine there's much of a better way.

https://redd.it/13evbwf
@r_devops

What's your opinion on self-hosted Github Enterprise?

Is there feature parity between GE and Github? Is it reliable?
Does it use some of the global Github services and does it go down when Github goes down? (seen that in some of the SaaS offerings)

https://redd.it/13eyvp2
@r_devops

5 hours interview for Principal DevOps Engineer?

I already cleared HM , and HR phone screen.

What is this 5 to 6 hours over different days with different folks in cross functional roles about?

Any guidance? I have done 1 hour ones before usually 3 to 4 panel Including the Hiring Manager.

How to prepare? It's virtual but still kinda daunting and making me nervous.

https://redd.it/13ey6mn
@r_devops

Hey fellow devs! Just curious if any of you freelancers out there are charging for value instead of time when it comes to devops. I’m trying to explore my options and would love to hear your experiences and thoughts on this approach. Thanks in advance for any help!

Hey guys, I run a software dev agency and we specialize in developing solutions and managing client servers. As we've grown, we've noticed our clients requesting more complex systems. Right now, we charge by time but I'm considering switching to a devops model where we charge by value. For instance, setting up a server is a breeze for me with the scripts I have. Are any of you charging fixed fees and can share some pricing ideas? I'm based in the US. Thanks!

https://redd.it/13f1fsc
@r_devops

Opinions on job decision

Would you recommend …

Job A)
- Industry/production environment
- Salary increase 28k, no on-call, same hours
- IT director pushing shift to Linux/OSS
- Phenomenal mindset towards documentation
- There is source control/CI systems implemented but barely used and barely configured yet
- Having the freedom to do things to my preference
- Basically being „the“ guy to introduce DevOps methodology and possibly having a great impact on policies and standards

Job B)
- Finance sector
- Salary increase 40-45k, no on-call, more weekly hours
- Already established DevOps culture
- Highly complex workflows
- Very restricted due to regulations
- Being able to learn from experienced peers
- Basically being „one“ of the DevOps guys


Any opinions are welcome

https://redd.it/13f3opj
@r_devops

How to Approach Kubernetes Infrastructure With Container Sprawl in Mind

Just came across this interesting piece that discusses how to tackle container sprawl in Kubernetes infrastructure. It dives into some key aspects like resource management, logging and monitoring, scalability, security, and networking. I found it pretty useful as I've been dealing with container sprawl issues in my own projects. Curious to hear if any of you have faced similar challenges and how you've addressed them in your Kubernetes environments.

Source: https://www.devopsinstitute.com/how-to-approach-kubernetes-infrastructure-with-container-sprawl-in-mind/

https://redd.it/13f54vg
@r_devops

Oh, it's just Github having their 'routine' daily outages

In all seriousness, what's happening there?

https://redd.it/13f6f56
@r_devops

What would you want to be hired as?

What would you call this role? DevOps (not really a role), SRE... not really, Platform Engineer... maybe but also has some ops.

EDIT: This person is joining a company where others have been doing this for several years, they are not the only person doing it, just the first person dedicated to it that isn't also managing teams, or also doing product development.

This is a startup where we're starting to distribute the hats people wear but still have a ways to go, and it does have a division between ops and dev, though it's getting blurrier every month.

Responsibilities of the role are to help with all of the following but doesn't need to lead/own it:

1. AWS Cloud/Infra management. EC2, RDS, S3, MWAA, IAM using CloudFormation and the console.
2. Person that Operations can escalate to. The Ops team is trained on the application and the data but have limited AWS/Infra and real Observability/tuning experience.
3. Improve observability, ELK, Py Dash and alerting, PagerDuty for example
4. CI/CD Pipeline development and maintenance excluding writing test automation.
5. Manual deployment on areas without automation, until they help setup the automation
6. GitOps
7. Performance tuning and optimizations' in production ranging from concurrency settings for the application to rightsizing AWS Infra
8. Development Env, Docker management
9. Escalation point for Development when they have Infra/Pipeline issues. (not on call)
10. Security implementation around architecture and software, not business risk.

What other responsibilities am I forgetting?

No "On call" time but when DNS messes up and we lock ourselves out of the office like Facebook did, they would be called along with every up from them through the CTO.

https://redd.it/13exmrp
@r_devops

Hashicorp Vault TCP listener

I was wondering if my tcp listner defined in my vault config has to be HTTPS? The reason being is Im trying to utilize a pre-existing nginx (gitlab omnibus nginx) to serve as a reverse proxy for a vault instance running on the same host but every example Im seeing has the TCP listener configured for HTTP. So Im not sure what is the best case for production use.

https://redd.it/13eq2ur
@r_devops