OIDC Proxy Server for AWS/Github Actions

Has anyone created a custom api server that acts as a proxy between Github enterprise org and AWS for adding OIDC provider

​

end goal is to use github actions to use oidc role on aws. however github enterprise org has a VPN in front of it and cannot validate thumbprint through providers (AWS console)

https://redd.it/13q2dc8
@r_devops

"um": GPT-powered CLI Assistant

Hey, fellow Redditors! I'm excited to share with you a CLI tool that we've been working on called `um`. `um` as in "um... what was that aws cli command to invoke lambda function?".

GitHub: [https://github.com/promptops/cli](https://github.com/promptops/cli)

**Motivation**

On average I run roughly 15 shell commands a day. These include a lot of trivial git commands, but there’s the occasional aws cli command to run, or I have to untar a file, or run internal script with positional arguments which I always mess up. And this leads to googling, or reading through the aws cli help, source code, etc. Ultimately I don’t want to remember funky syntax, I just want to say what I want to do and get it done. That's why we developed "um" - just ask questions in plain English right there in the terminal and get the perfect shell command. No more context switching.

**Key Features**

* **Find the right command without leaving the terminal:** ask questions in plain English directly from the command line.
* **(Optional) Index your shell history for sub-second response:** By indexing your history, `um` can suggest previously used commands and personalize generated responses. The index is stored locally on your machine.
* **GPT-powered answers:** `um` uses GPT-4 (by default) to generate command line expressions based on your queries.
* **Context-aware corrections:** `um` learns from your corrections, ensuring that similar questions give you improved results over time.
* **Respecting your privacy:** To protect your sensitive data, `um` uses the excellent [detect-secrets](https://github.com/Yelp/detect-secrets) python library to remove passwords and tokens before indexing commands. Also our OpenAI account is opted out of collecting and using data for training the next versions of GPT.

**How** `um` **works**

`um` first checks the indexed history for sub-second responses. If the command is not found, it uses the generative model to provide command suggestions based on your query. You also get explanations of the suggested commands. Corrections that you make are indexed, to ensure improved suggestions for similar questions in the future.

**Installation**

👉 Please visit our [GitHub repository](https://github.com/promptops/cli) for installation instructions and more details.

**Examples**

$ um list running ec2 instances

📖 aws ec2 describe-instances
➜︎ ✨ aws ec2 describe-instances --filters 'Name=instance-state-name,Values=running' --query 'Reservations[].Instances[].{Instance:InstanceId,Type:InstanceType,State:State.Name}' --output table
💭️ don't see what you're looking for? try providing more context

$ um list git branches ordered by recency

📖 git log
➜︎ ✨ git for-each-ref --sort=-committerdate --format='%(committerdate:relative) %(refname:short)' refs/heads
💭️ don't see what you're looking for? try providing more context

We value your feedback! We are still in the early stages of development, if (when) you encounter any issues or have suggestions for improvements, please let us know. Reply to this post, report in github, or contact me directly.

Thank you for your support, and happy scripting!

https://redd.it/13q48o5
@r_devops

Ansible - how widely used is it ?

Past two organizations / teams I have been with they used ansible but it seemed like rarely. They weren’t always pushing changes to a ton of nodes etc. Probably got used a handful of times a year. New place I’ve seen it used twice in the past 3 years….

It is just the places I have worked at or is it slowly being phased out ?

https://redd.it/13q3oo3
@r_devops

How long do you stay “stuck”

Preface, Junior here - on a relatively smalll team. Only 3 of us and 2 other seniors. One just started parental leave as of yesterday.

Have been stuck on an issue well since yesterday and I have been exploring a few options. Have been running things locally but I keep coming back to square 1. At this point it’s frustrating, on a relatively simple Ask but the requirements have me unsure how to tackle said problem.

Other senior is super busy lately and I’m just frankly not sure when to ask for help/insight, especially since this would be considered a more junior problem.

When do you guys decide it’s time to reach out rather than being stuck for X amount of time longer. Unfortunately Imposter syndrome has me paralyzed on seeking advice/help..

https://redd.it/13q3k4n
@r_devops

My boss keeps committing his creds into git

Today I saw that my boss had created a script, again, that could include his system credentials, I looked at his branch, and sure enough, it does. Last time I cleaned a repo where he did this I had to clean it twice, because he pushed his copy of the branch up again. How do you handle this?

No impersonating him destructively isn’t an option, it’s a sensitive system where I’d have more than normal penalties.

Yes, I am already looking at other opportunities despite only being here almost 3 months.

https://redd.it/13q6fwm
@r_devops

Running tests in dockerfile that depend on ephemeral mongo container, is it worth trying or is there a better way?

So I've got a flask/react app as well as some unit tests for it. the api connects to mongodb. The tests depend on a mongodb connection, and I've currently got it set up to connect to a local mongo container, populate it with some data, then test against that but I'd like to automate the mongodb container creation/population. Except im currently just running docker run -d mongo --network host to start the container and the tests are connecting to mongodb://localhost:27017

Now, from what I've read it is typically not the best practice to do testing in the same step as building the image, so I'm open to alternatives that would be a better idea for this, I'm fairly new to devops. I've read a little about multi-stage docker builds, but still trying to wrap my head around how the above app would be structured or if they're even applicable here.

https://redd.it/13pvy59
@r_devops

Instana synthetic monitoring

Im using the new instana synthetic monitoring feature to monitor my website. I have a smart alert that is created when the test fails 3 times consecutivrly . My website shuts down over night, so the tests fail - is there a way for the tests to run only at a certain time, I.e during the day?

https://redd.it/13pu3og
@r_devops

SONARQUBE LTS 9.9

Hi Team , i have one query we have sonarqube 8.9.8LTS and devloper have written code in java8, now with the sonarqube lts 9.9 it supports java 17 and sonarscanner java 11 or 17 , will this affect as the code written in java 8 will come to scanner where java 8 is not supported . How we can handle this?

https://redd.it/13qc6h6
@r_devops

Unable To Publish Port On Host Machine With Docker

I was trying to build an image from Dockerfile but it didn't worked When I Use

docker run -p 8081:8080 yt-test

But, It Works When I Use Host As Network Interface. On Host Port 8080 (But I Want 8081).

docker run --network host yt-test

This is my Dockerfile

FROM python:3.9

RUN apt-get update && apt-get install -y git

WORKDIR /app

RUN git clone https://github.com/user234683/youtube-local

WORKDIR /app/youtube-local

RUN pip install --no-cache-dir -r requirements.txt

EXPOSE 8080

CMD "python", "server.py"

Any Idea Why This is Happening?

https://redd.it/13qdeie
@r_devops

Running Post-Mortems

Ever wanted to introduce post-mortems to your team or department? Here is the detailed process of how to run them!

https://certomodo.substack.com/p/running-post-mortems

(cross-posted from r/SRE)

https://redd.it/13pvkwt
@r_devops

If an 18yo person applied for a job and had a load of Cloud Provider certs + CKA - what would be your gut reaction?

My 17yo daughter (non-binary) is F-ing up at school caused by the usual stuff of bullying, depression and teenage angst and is unlikely to finish her A levels. She didn't get a high enough maths gcse grade, 1 point off being able to A level Computer Science and selected criminology instead along with Accounting and law.

So I've bought the GCP Architect, Azure Architect, AWS Architect and Mumshads CKA courses and I'm going to sit with her while she goes through them, does the practice and then buy her the exams.

She'll be 18 by the time she sits them and/or has to do retakes.

So if a CV for an 18 yo with no work experience and a load of certs got past the recruiter screening and landed in your inbox, what would be your thoughts/reactions

Edit

What I should have pointed out is that yes she does have an interest in doing it. And that this is a step to getting a bottom of the ladder entry-level job plus the architecture of the providers, their services, how they integrate, gives the high level knowledge of what they are about.

https://redd.it/13ooqr6
@r_devops

What would be the optimal working environment for junior cloud/devops engineers?

Where I come from good devops engineers are rare as (natural) diamonds - and every company is searching for them. I don't really have that much competition but I think my rate of growth could be much better at a larger company.

I'm just some weeks short of becoming intermediate where I work at, finally.
But their lack of an automation/security mindset is probably corrupting my future chances, so:


If you had some experience and want to grow or if you were younger again and had a 2nd chance, what would you actually look for at companies?
What structure, benefits, responsibilities... how large?
I want to have impact and for that I need the right environment.

https://redd.it/13qg6m4
@r_devops

DevOps Conferences -Europe 2023

Hi, guys,

I hope you are all doing well.

I was hoping you could give me some ideas of interesting conferences for DevOps people in Europe happening this year still.

Thank you :)

https://redd.it/13qi5lz
@r_devops

Can Terraform Replace Powershell scripts ?

Hello and sorry for asking this as im not really experienced enough know the answer to this.

Context : as my company as a default setup for the Azure tenant of our Clients and will adjust them afterwards for special "needs", ive created around 6-8 Powershellscripts that will create the User and Groups Management import the basic policies ( Endpoint compliance etc ,thanks github for that one) etc etc.


Now my question is could the same be achieved with a terraform file ?
Would you recommend doing it that way or stick to the PS Scripts ?


Thanks :)

https://redd.it/13qk8ou
@r_devops

Recommended approach for setting up performance testing with Locust to test an EKS cluster?

I initially attempted to install and test Locust on Minikube and then deploy it to our EKS cluster.

However, all documentations I encountered were limited and outdated, which caused things not to work. The official documentation offers a partially functional Terraform snippet for an unspecified service.

Considering our usage of GitHub actions, would it be advisable to create a job that can be manually or automatically triggered to run Locust via CLI? Should I still try to have Locust run on EKS? Is there a recommended approach?

Thanks ahead

https://redd.it/13ql7t6
@r_devops

Support Auth0 in Azure Static Web Apps

Learn how to support Auth0 in Azure SWA for your Blazor WebAssembly application.
Read more…

https://redd.it/13qn1rq
@r_devops

Self heal timeout per app in Argo?

I'm trying to see if it's possible to set a delay on self heal syncing in argo by using an annotation or label or something on a resource directly. It seems like there is a config option to do this across the whole argo installation but I can't seem to find any way to use that option in a more fine grained way. I can't really find an authoritative list of available configuration labels at all to be honest. Most seem like they are scattered across the docs.

Anyone know anything?

https://redd.it/13qkp3r
@r_devops

Advice needed for CI/CD

My friend and I are working on a project that involves a web app, an extension app, a native mobile app, and a desktop app. All these applications share a large amount of codebase, primarily based on React JS, and are structured as a monorepo.

We're looking to set up a comprehensive CI/CD pipeline. Being relatively new to the field, our objective is to have separate staging environments for each of the apps and a pipeline that can test, build, and deploy individual applications as and when necessary. User Acceptance Testing (UAT) is another important component that we wish to include in our workflow.

We've been considering using GitHub Actions for our testing and building phases, Fastlane for mobile app deployments, and potentially integrating Sentry for error tracking.

I would love to hear your thoughts on:

Which CI/CD tools would you recommend for such a setup?
Any best practices for managing CI/CD in a monorepo environment?
Strategies for managing deployments of multiple applications (web, extension, mobile, desktop) from a monorepo?
Recommendations for incorporating UAT testing into our CI/CD pipeline?
Insights on error tracking and monitoring within such a pipeline?
Any advice or insights you could share would be greatly appreciated. Thank you in advance!

https://redd.it/13qj3h6
@r_devops

Can anybody help with a Gitlab / Docker-Compose issue?

Hi, I've created a gitlab pipeline for a laravel project. I'm trying to work on creating a boilerplate docker/laravel setup I can used between each project I use. I know Sail exists but it's too magical & I want something I understand how it works & can configure easily from there.


That being said I build my docker-compose file & all works swimmingly, I can run my tests locally with my database connection working through an artisan entrypoint. However when I run this in my gitlab ci/cd I get:
SQLSTATE[HY000\] [2002\] Connection refused (Connection: mysql, SQL: SHOW FULL TABLES WHERE table_type = 'BASE TABLE')


I've got it to post the logs which shows it creating the test database & got it to show what databases are accesible witht he user credentials I've passed in my env which also return what I expect.


Pipeline looks like this:
image: docker/compose
services:
\- docker:dind
stages:
\- test
variables:
DB_CONNECTION: mysql
DB_HOST: database
DB_PORT: 3306
DB_DATABASE: laravel
DB_USERNAME: matt
DB_PASSWORD: password
before_script:
\- cp .env.example .env
\- docker-compose build --no-cache
\- apk add mysql-client
\- docker-compose up -d
\- sleep 40s
\- docker-compose logs database
\- docker-compose run --rm database mysql -h database -u matt -ppassword -e "SHOW DATABASES;"
test:
stage: test
script:
\- docker-compose run --rm composer install
\- docker-compose run --rm artisan key:generate
\- docker-compose run --rm artisan cache:clear
\- docker-compose run --rm artisan config:clear
\- docker-compose ps
\- docker-compose run --rm artisan test


Docker-compose:
version: '3.8'
services:
app:
container_name: app
build:
context: .
target: php
working_dir: /var/www/html
volumes:
\- ./:/var/www/html
ports:
\- "8000:8000"
depends_on:
\- database
networks:
\- laravel
database:
container_name: database
image: mysql:8.0
ports:
\- "3306:3306"
environment:
\- MYSQL_DATABASE=${DB_DATABASE}
\- MYSQL_USER=${DB_USERNAME}
\- MYSQL_PASSWORD=${DB_PASSWORD}
\- MYSQL_ROOT_PASSWORD=${DB_PASSWORD}
volumes:
\- ./docker/mysql:/docker-entrypoint-initdb.d
\- db-data:/var/lib/mysql
networks:
\- laravel
pma:
container_name: pma
image: phpmyadmin
ports:
\- 9000:80
environment:
\- PMA_HOST={$DB_HOST}
\- PMA_ARBITRARY=1
networks:
\- laravel
jenkins:
image: jenkins/jenkins
container_name: jenkins
ports:
\- "8080:8080"
networks:
\- laravel
artisan:
build:
context: .
target: php
container_name: artisan
volumes:
\- ./:/var/www/html:delegated
depends_on:
\- database
working_dir: /var/www/html
entrypoint: [ 'php', '/var/www/html/artisan' \]
networks:
\- laravel
composer:
image: composer:2.3.5
container_name: composer
volumes:
\- ./:/var/www/html
working_dir: /var/www/html
depends_on:
\- app
entrypoint: [ 'composer', '--ignore-platform-reqs' \]
networks:
\- laravel
volumes:
db-data:
networks:
laravel:
driver: bridge
Any help or point in the right direction would be great. Many thanks!

https://redd.it/13qs3fg
@r_devops

On AWS: Why use EKS instead of ECS?

I'm in a position where I've got to stand up some dockerized services (Airbyte, Kowl, etc.) which need to stay up (so no Lambda).

As I see it, my choices are to use ECS, EKS or good old fashioned Kubernetes. When would you lean towards EKS or Kubernetes instead of ECS? What do those services provide that make up for the added complexity?

https://redd.it/13qtujx
@r_devops

How do you on onboard new engineers?

My team are going through a growth phase in the coming months and I want to prepare some training material for new engineers. I have a bunch of architecture diagrams already there, and some descriptions about each repo, how we build and host. Our cloud environments and accounts. What else would you guys have?

https://redd.it/13r28vf
@r_devops