What is the difference between ECR and CodeArtifact?

Title.

https://redd.it/ymsug0
@r_devops

People that have started consulting, how do you get contracts?

Hello,
For those that have started or are starting their own consulting companies, where do you even find contracts? Are you cold calling companies? Are you applying to contract positions meant for individuals?

https://redd.it/ymzvrt
@r_devops

Are there any good GCP monitoring examples. General advice welcome

Hey all! I was recently given the chance to take ownership of SRE at the relatively small company that I work for. We have some long and short term goals but the immediate focus is to start better monitoring practices.

I have done lots of reading and googling and even worked on a few Google provided certs but I learn best by seeing real world examples. I’m not seeing great information about how to apply the metrics provided by google to the “golden signals”.

It’s all a bit overwhelming.

https://redd.it/yn42zi
@r_devops

What are your opinions/feelings on the DevOps mantra of "continuous improvement?" What really drives engineers and engineering leaders?

Re-reading "Accelerate" as I'm trying to dig deeper into the psyche of developers and engineering leaders.

It's clear that engineering leaders are not "built" like Sales or Marketing leaders. These types of leaders are often driven by money, "fame" (for lack of a better word) and are highly competitive.

Whilst my feeling is that eng. leaders are more concerned with the product they're building, and see this as an "art."

What are your thoughts on this? What do you think drives and motivates engineers? How do you think this affects their relationships with the other functions (i.e. Sales, Marketing, Business Dev.) in organizations? Is the mantra of "continuous improvement" a strong enough driver?

https://redd.it/yn3yz8
@r_devops

Tool for visualizing your backend, not just cloud infra

Hey there,

I was wondering if there is a tool that lets you visualize your backend at a higher-level than just cloud. Something that pulls info from my Github + AWS and shows things like:

* what API endpoints a microservice calls?
* what tables a service uses?
* what's the format of the messages passed between different services?

I could then interact with the nodes to make queries like:

* what are the last n calls made from one service to another?
* what are the current waiting messages in an async message queue?

I know there are tools like Cloudcraft, Lucidscale that automatically create diagrams of your cloud infra, but they're usually just limited to cloud-level details (e.g. what kinds of AWS instance a node is running).

Thanks!

https://redd.it/yncqhz
@r_devops

Do Platform Engineers require the same skills as DevOps Engineers? If so, then what's different about them?

DevOps and Platform Engineering all sound the same to me. It's as if DevOps grew up and just turned into a new term, Platform Engineering.

https://redd.it/yn1pqy
@r_devops

Advice on Azure DevOps

Hello everybody!

This is my first time on this subreddit, and I just recently started working as a DevOps Engineer.

I am currently trying to automatically run a pipeline on Azure DevOps, and retrieve the status of the pipeline (succeeded/failed), but I am running into a bit of trouble achieving that.

I was mainly using python to script the process, which would then be used in a lambda function in order to perform a health check on our service.

If any of you are able to suggest any paths to try out, I would be very thankful!

https://redd.it/ymymql
@r_devops

Can someone explain to me, as a total noob, is it possible to horizontally scale an RTMP server/severs with Kubernetes?

Hey guys,

I'm looking to create my first Dev ops/Full stack dev based side project. Basically I have 300 web cam streamers, each with their own "channel" (like twitch or Youtube), that I want my users to react with through a messaging system using web sockets.

I've been studying Kubernetes for a few months and know little about RTMP. I was wondering if it would be possible to horizontally scale RTMP with Kubernetes? And can someone explain to me like I'm 5 some basic things that I'd have to do?

(I'm just trying to research this project for viability)

https://redd.it/yng3x5
@r_devops

a guide for a process delivery from staging to production

Hello all,

I am searching for a guide with in depth description for all the steps for a process delivery to production, i am looking for a guide with also a description about the role of the different intervenants , for example : a prodcut owner make sure the package delivered respond well to the buisness needs, the techlead make sure it respects the technical aspect etc.

Thank you all for your help.

https://redd.it/ymrc37
@r_devops

Github As A Code

Hi,

We are startup looking at enabling Github as a code using terraform. I am trying to implement self service for github org so developers can deploy their repositories and we (devops) can manage entire org. Preferably using some user friendly yaml templates that make sense to end users, not plain HCL.

I can see that someone already did a great job doing that https://github.com/mineiros-io/terraform-github-organization

Has anyone used this product? I was looking for the alternatives but could not find anything decent in github/OSS field with terraform provider.

I could of course just spend a bit ( a lot ) of time and write a lot of resources and wrappers but if someone went through it already and made an effort to do it properly an in an elegant way, that would save me a lot of time and pain.

I would rather avoid writing solution from the scratch if there is a framework out where.

https://redd.it/ymrz92
@r_devops

advice about transitioning into DevOps

Hi!
I'm considering trying to move into DevOps but I'd like some advice from people who work or moved to DevOps themselves.

I'm 25 years old in a moment and nearing 7 years of experience in dba and db development, I have a pretty good grip on Linux (Ubuntu mostly) and I made some projects in collaboration with Jenkins and docker.

I have background as an IT expert as well, and some other programming languages and tools .

What should I learn and know before I try to get into DevOps? I work in a big company and transitioning is potentially an option if relevant.

Thanks!

https://redd.it/ynkeqf
@r_devops

Which free GCP product for Heroku worker dyno?

This might be a newbie's problem but I'm wondering if anyone can provide some direction on it:

Consider the following use case: an app is running on a worker dyno on Heroku 24/7 using their free tier. It’s not a web app that binds to a port and listens for requests sent via a particular HTTP webhook, rather it sits idle and uses persistent connections via TCP sockets to interact with an external server and the server pushes updates to it, hence why it’s run on a worker dyno like a background process and not on a web dyno.

If the user wants to move to GCP, what’s the right GCP product to go for for this use case? Something where the usage is entirely covered by the free tier for the respective product would be preferable. From what I’ve read, I’ve gotten the impression that Google Cloud Run isn't right because it’s more an equivalent for web dynos, not worker dynos.

https://redd.it/ynl9bd
@r_devops

How do you determine in a CI/CD pipeline which build is pushed to prod?

I'm fairly new to DevOps, but I'm excited about learning new stuff!

I'm curious about how to setup a CI/CD pipeline in order to push to multiple environments such as dev and prod. So far we had a single dev env. and the devs pushed their code into a feature-specific branch. After the review a merge was initialized and the CI/CD pipeline started on the main branch. We deployed the artifact into the dev env.

So far so good. Now we have multiple environments, and I would like to know how you guys handle this usually. Do you have specific branches, such as main and dev which each push into the corresponding environment? Or do you use tags?

What's the best practice?

https://redd.it/ynkqc5
@r_devops

How do I pivot more towards DevOps career-wise?

Hi experts,

I had a few years of experience as a BI developer (ETL + Dashboarding mostly) and moved into a new job 9 months ago which is sort of BI-Ops.

I dabbled into three focuses:

1. ETL: Mostly Python + Airflow + BigQuery
2. Internal Doc site: Think a static site generator running on Google Cloud Run with GitHub Action as CICD
3. BI workflow: Think a large BI team that needs a proper git workflow + a lot of GitHub Actions for linting/automatic approving/etc.

I actually led all three projects but the reason I said "dabbled" is because either the project got shelved or I only got to complete a PoC and then it was good enough so nothing complicated (read: good for resume) was needed. After all all three are internal so there is no need to "scale" for production.

From project 2 and 3 I learned a lot about Cloud click ops and CICD stuffs including Terraform, docker and GitHub Actions. However I don't know how to go further down the DevOps road. I'll establish the reasons:

1. My team actually is not supposed to do a lot of Ops, but we do it anyway because the company is ill structured
2. As mentioned all Ops are internal so there is no need to scale
3. We kind of use a lot of tools say Terraform but we never need to set them up. Someone else set it up for us (the real DevOps team I guess) and we simply plugin our code

How can I grow from here and pivot more towards DevOps? I'm missing a lot of skills, e.g. as mentioned I actually don't know how to setup things. Neither do I do any real Dev (Dev in our team simply means writing ETL code in Python and SQL, which I'm already good at). I guess I can learn some of the skills by setting up a home lab (I do have a 32GB server so surely I can learn basic K8S) but is that going to be enough for resume? TBH I don't really have need of those things for myself (at home I mostly dabble with system programming, either low level C or quick Python), so the best I can learn is going to be superficial.

Or maybe I should find a way to move to the DevOps team? It's a bit tricky because the whole team is in another time zone and the company seems to prefer to dismantle and outsource it, so I might need to actually leave and join a new company - and again here comes the uncertainty - is my resume good enough?

What do you think? Thanks in advance.

https://redd.it/yll4te
@r_devops

Trying to find a way to use a docker container in Hashicorp Packer for installing app in an AMI

Hi All,
As of now, I am configuring metadata in BUILDER block and then using PROVISIONER block to configure my AMI,
but what if I need to just use a configuration that is already inside a Docker Container?

Eg, I want to directly use a Docker Container to install an application on my AMI that is being configured using Packer.
Is there a way just to use my docker container for such use case? If Yes, how?
(Best guess ‘by using SOURCE block in packer????)
Or
How can I use a Docker Container in most optimal way while building my AMI?
Please suggest some ideas!

https://redd.it/ykuuep
@r_devops

what should i do as devsecops Engineer

I was recently recruited by this govt organization on the pretext of software development (which I am familiar with). 

However, following the orientation programme, I was assigned the role of devsecops.


Now, because most senior management in government organisations aren't particularly knowledgeable about technology, they rely on various private firms to provide services such as code or infrastructure, each of which has its own devops pipeline, and these guys don't give a damn if you aren't from their own firm.


So, guys, please point me in the right direction, because even though they aren't teaching us much, the expectations are high

https://redd.it/ykunjx
@r_devops

AES Beanstalk not refreshing logs

Coming from heroku I tried to implement my web application on AWS EB. The application runs fine, however the logs are “stuck” at a certain timestamp and are not refreshed, even when I download all logs and restarted the application, the old logs are shown.

Did I run in some quota? Do I have to setup another service?

https://redd.it/ynshjw
@r_devops

Identity and Access management for DevOps tools

I wonder how do I get secure access to all my DevOps tools? Some of these tools may be use my AD or Okta groups to provide access. Nevertheless of these IAM tools I see DevOps folks use shared credentials, share tokens manually. I feel this is a huge security gap. I am curious to learn if every DevOps persona handles shared credential and tokens manually (by choice or the ecosystem they work within) and what is the reasoning behind it?

https://redd.it/ynurfv
@r_devops

Tool for visualizing your backend, not just cloud infra

Hey there,

I was wondering if there is a tool that lets you visualize your backend at a higher-level than just cloud. Something that pulls info from my Github + AWS and shows things like:

* what API endpoints a microservice calls?
* what tables a service uses?
* what's the format of the messages passed between different services?

I could then interact with the nodes to make queries like:

* what are the last n calls made from one service to another?
* what are the current waiting messages in an async message queue?

I know there are tools like Cloudcraft, Lucidscale that automatically create diagrams of your cloud infra, but they're usually just limited to cloud-level details (e.g. what kinds of AWS instance a node is running).

Thanks!

https://redd.it/ynv1pu
@r_devops

Geo-routing with Apache APISIX

Apache APISIX, the Apache-led API Gateway, comes out of the box with many plugins to implement your use case. Sometimes, however, the plugin you’re looking for is not available. While creating your own is always possible, it’s sometimes necessary. Today, I’ll show you how to route users according to their location without writing a single line of Lua code.

Read more

https://redd.it/ynvbq8
@r_devops

keycloak oauth2-proxy configuration

Hi guys,

I'm right now stuck with some configuration I have in my kubernetes. In my lab I want to configure oauth2-proxy to use keycloak as an identity provider. I've everything ready but when trying to login using keycloak it shows a 403 Forbidden error "Login Failed: The upstream identity provider returned an error: invalid_scope"

Pod logs:

[2022/11/03 08:49:31] [oauthproxy.go:752] Error while parsing OAuth2 callback: invalid_scope

08:30:38,734 WARN [org.keycloak.events] (default task-43) type=LOGIN_ERROR, realmId=test, clientId=oauth2-proxy, userId=null, ipAddress=10.50.21.171, error=invalid_request, response_type=code, redirect_uri=https://oauth.test.dev/oauth2/callback, response_mode=query

08:34:11,933 ERROR [org.keycloak.services] (default task-41) KC-SERVICES0093: Invalid parameter value for: scope

I've look for documentation and I don't see why is complaining about the scopes as I've them right.

This is my oauth2-proxy values:

provider = "keycloak-oidc"

provider_display_name = "Keycloak"

cookie_domains = ".test.dev"

oidc_issuer_url = "https://keycloak.test.dev/auth/realms/test"

reverse_proxy = true

email_domains = [ "*" \]

scope = "openid profile email groups"

whitelist_domains = ["test.dev",".test.dev"\]

pass_authorization_header = true

pass_access_token = true

pass_user_headers = true

set_authorization_header = true

set_xauthrequest = true

cookie_refresh = "1m"

cookie_expire = "30m"

And in keycloak I have the oauth2-proxy client created with Groups and Audience mappers.

I see these errors in keycloak:

LOGIN_ERROR

Client oauth2-proxy

Error invalid_request

response_type code

redirect_uri `https://oauth.test.dev/oauth2/callback`

response_mode query

If someone has experience with this and can point me to the right direction and tell me what I'm doing wrong I would be very grateful

Thank you

https://redd.it/ykwmrv
@r_devops