How do you store your helm charts?

My last gig we just stored the “raw” helm yaml files (not the chart.tgz) in gitlab and we didn’t bother with packing them and storing them in artifactory since all of our charts were internal. I’m at a new gig now and we’re trying to figure out how to store and organize our helm charts. Is there any benefit to pack our helm charts and store them in something like chartmuseum if we’re not planning on distributing them?

https://redd.it/op2zz0
@r_devops

Suggestions on web interface for inserting values into a YAML document

Looking for some ideas on a good approach to solve this. I have a YAML document that describes some stuff. I can use JINJA2 syntax in it to dynamically insert values. I want to build a web interface that a person can access, be presented with a form, and when they click submit, they are given the completed YAML document with their entered values inserted. (Alternatively I may store the document and send it somewhere else later). Bonus points if it can run serverless on Lambda :)

Just looking for some suggestions on approaches to this, or maybe some resources that I can review. Thanks in advance!

https://redd.it/op45yc
@r_devops

DevOps Institute membership....is it worth it?

Any thoughts or experience about it?

https://redd.it/oo63m6
@r_devops

Difference in build/production env?

Hi, I'm writing a web app in Java.

Is is okay to build the app in Jenkins running on docker,

and deploy the artifact on linux VMs in production?

https://redd.it/op5skm
@r_devops

Prometheus Alerts using Grafana vs AlertsManager

Hello Community,


Need guidance on setting up the alerts using alerts manager vs Grafana inbuilt alerts mechanism? What are the pros and cons of using Grafana to set the alerts vs using the alert manager config file directly? I am doing a POC on this and would need help from the community here based on your expertise and experiences.


Thanks,
Ab

https://redd.it/op7cpy
@r_devops

To Devops Professionals - What industries do you work in?

Hello World!

I'm conducting some market research and would like to ask devops professionals:

1. What region of the world are you based?
2. What industry vertical do you work in?
3. Is your dev team empowered to source your own devops tools to get the job done?

I want to find out what non-traditional industries are adopting devops practices successfully and whether they foster a sense of developer empowerment. Any thoughts would be appreciated!

Thanks!

https://redd.it/op1e7d
@r_devops

EMPTY FILE IN GITLAB ARTIFACTS

Sorry if this is the wrong subreddit but I'm having an issue with Gitlab artifacts. We're trying to get test coverage written to a XML file as an artifact. We have marked tests split into two pipelines.

Build:
stage: Build
script:
- *setup-build
- *build
- touch banana.xml # where I write the code coverage collected by pytest-cov
- *push
artifacts:
paths:
- banana.xml
reports:
cobertura: banana.xml

Unit Test:
stage: Test
script:
- *setup-build
- *pull
- docker-compose run $DOCKER_IMG_NAME pytest -m unit --cov-report xml:banana.xml --cov=app --cov-append;
needs:
- Build

Validity Test:
stage: Test
script:
- *setup-build
- *pull
- docker-compose run $DOCKER_IMG_NAME pytest -m validity --cov-report xml:banana.xml --cov=app --cov-append;
needs:
- Build

After the Build and Test stages run, I go to download the banana.xml file from Gitlab but there's nothing in it, even though the jobs say `Coverage XML written to file banana.xml`

Am I missing something with how to get the total coverage written to an artifact file when splitting up marked tests in a gitlab pipeline stage?

https://redd.it/oozjqu
@r_devops

How's your experience with Pulumi?

I have been playing around Pulumi for a while and I have a couple of questions for DevOps developers and for Pulumi users specifically, I hope you don't mind answering them.

What are the challenges that you faced working with Pulumi?
Would you change anything in the tool if you have the power to do so?
Are there any pain points using the tool?
Thank you for your time.

https://redd.it/oow4sd
@r_devops

CircleCI strikes back again: hide per-user charges

Recently, I've decided to review our billing statements on CircleCI and discovered that the user payments disappeared. I was really surprised because when I complained about it to their team earlier this year, I was told that it is not going to change.

I went to their pricing page but it still stated that they charge per active-user. It took me a couple of minutes to find this post: https://discuss.circleci.com/t/performance-plan-billing-update/39502. In short, they hid the per-user charges by mixing it with the charges for running build jobs such that now it is harder to tell how much you are paying for the users. It is still trivial to compute it but it is not visible on the invoice anymore.

This per-active-user charges always annoyed me since we upgraded to Performance plan. It is basically allows them to grab more money from their customers claiming lower costs on their pricing page. They even charge you for a system user triggering "cron" jobs. So you are paying $15/moth for using the "cron" jobs feature. For us, the per-user charge reaches almost 50% of the total bill on an easy month so it is a significant part of the total charge. Hiding it in the invoice seems to be a dirty trick.

Anyhow, I would like to renew the discussion started on https://www.reddit.com/r/devops/comments/d3sqr0/whats\_your\_experience\_moving\_away\_from\_circleci/. What are the best alternatives to CircleCI in 2021?

https://redd.it/oou34v
@r_devops

Migrate from Heroku, Postgres, MongoDB to local server

I had made an application which was running a React frontend on Firebase, Django backend on Heroku, and using MongoDB & Heroku Postgres as database. As I was using Free Tier of all these services, I was approaching their usage quotas.

My university recently allotted me a bare linux machine (with generous specifications) which I could use for hosting my app. I've received SSH credentials for the same. Can someone advise me how I should go about the process?

https://redd.it/opazu5
@r_devops

ChaosNative Workshop

Hello everyone!

The ChaosNative community is glad to invite you to the Cloud-Native Chaos Engineering workshop where you can network with tech geeks around the world. Get yourself exposed to the world of resiliency and reliability. You will be having interactive talks and discussions with our experts about:

1. Hands-on experience with a workshop on Cloud Native Chaos Engineering which introduces you to the basics & fundamentals of Chaos Engineering.

2. A sneak-peek at the latest evolution in the Cloud Native Chaos Engineering world with LitmusChaos 2.0.

3. A demonstration of running Chaos experiments in just a few steps!

Mark your calendars for 12th August 2021, 11:30 AM EST | 5:30 PM CEST | 9 PM IST

Register here: https://chaosnative.com/cnce-workshop

https://redd.it/opblpz
@r_devops

What is IT like in your organization

We struggle with some basic competence and what I would consider core problem solving instincts and process of elimination when it comes to looping in IT. DevOps doesn't have the bandwidth for everything, though we try to automate more and more to not have to put in a ticket to IT.

Is IT maddeningly bad at your organization? I tried to give a terraform and ansible tutorial to a 5 member team and they were clearly struggling with concepts like a variable. We have to write the powershell scripts for them...Just feeling deeply frustrated and fed up.

https://redd.it/ootnoz
@r_devops

I've been working on some programs that interface with various saas services api's, and I need some methodology advice.

I'm just trying to see if my method is correct... I mean it works fine but it feels dirty... but I think it might just be my lack of experience.



So let's say this API wants XML data. What I've been doing is (in python) is something like this:

requestXML = """"""
requestXML += '<?xml version="1.0"?>'
requestXML += "<someRootElement>
requestXML += "<someElement0>{}</someElement0>".format(element1Data)
requestXML += "<someElement1>67890</someElement1>".format(element2Data)
so on and so forth...

I'm just adding in the xml data to this string, line by line, while also formatting them to include the variables I need. Then at the end I have this giant XML request string that I parse with urllib and then send, and its been working fine.


But is this really the best way to do this? I have this suspicion if someone more experienced ever saw this they would cringe lol



Would making a function to do this be any more "right"? House the XML "template" in the function and fill it in with args.


Thanks!

https://redd.it/opdmzu
@r_devops

Tutoring on the side

Just a question out of curiosity - has anyone been successful as marketing themselves as a "devops tutor" on the side? Teaching technologies and concepts on the devops roadmap, holding the hand of folks who are new to IT, or want to learn more about devops specifically?

Thinking of doing some of that in my off time and was wondering what others' experiences have been.

https://redd.it/opdu6n
@r_devops

How to Deploy a Secure API with FastAPI, Docker and Traefik

Putting your API to production comes with securing it with HTTPS and encrypting data transfer: something a lot of people neglect or take for granted.

HTTPS cannot just be turned on by changing a config file: a few steps are required.

In this post, I detail the process of securing a FastAPI app with HTTPS by using Docker and Traefik (the procedure is the same for other types of web applications)

Here's what's covered

A brief introduction to HTTPS: how does it work and why you should care about it?
Building a simple API with FastAPI
Introducing Traefik as and how it can handle HTTPS by integrating with Let's Encrypt and Docker
Deploying on AWS

https://towardsdatascience.com/how-to-deploy-a-secure-api-with-fastapi-docker-and-traefik-b1ca065b100f

https://redd.it/ope9ww
@r_devops

504 gateway timeout on containers

My company just started moving towards containers
Some of our containers throw random timeout errors to multiple users. When looking at my dashboard, those containers don’t seem to be reaching their CPU and Memory limit.
I’m fairly new to all of this and don’t understand why we would get this error if not all resources are used

https://redd.it/ope3iy
@r_devops

Cloud Veterans, really need your adept opinion!

Hi Folks,

I’m in the MVP stage of a solution, which I believe would greatly assist DevOps in performing their daily routine and save hours of work.

Also, shedding more light on what is MatosSphere. It’s an observability and remediation solution that was built for IT, DevOps, and Engineering with the goal of masking complexity, simplifying workflows, and accelerating operational productivity by bringing the most commonly used tools and services to a single integrated solution. We provide complete visibility to your core infrastructure services, by leveraging deep APIs integrations and policy enforcements and allow you to fix common problems while ensuring remediation is a single or no action task.

Appreciate it if you will share your feedback on MatosSphere or on things I can improve. Ping me and I will send you a 2-minute demo.

About me. I’m the Founder of Bayinfotech and CloudMatos (MatosSphere). So having two companies, one SaaS another Cloud Services Firm. Would love to help you with any answers, suggestions or assist you with advice.

Chat with you soon!

https://redd.it/opdv4m
@r_devops

ChallENGES with MOBILE APPS CI/CD?

I am a mobile developer, Android and iOS. And I've used CircleCI, GitHub workflows, and now Tekton to setup CI for my personal/consulting work. When I was working for bigger teams/companies- I experienced that mobile CI setup had lot of challenges and took us sometime working with the DevOps team to get it dialed in. Mostly because DevOps team wasn't aware of nuances of mobile toolchains and worse we ourselves were clueless about setting up our build system to be used in CI environment eg injecting variables and secrets.

 

As DevOps professionals what challenges are you seeing (if any) around mobile CI/CD setup?

 

note: idk why the title is mixed case :/

https://redd.it/opjaow
@r_devops

Need Dedicated Server with Unlimited, unthrottled bandwidth

Hi there, I'm working on a project which requires the streaming of music, and this music streaming could easily exceed the typical bandwidth limits on most regular hosting services. I'm a full-stack JS dev and I had a moderate amount of DevOps experience, but I'm certainly no expert, so I wanted to reach out to the community and ask: Is there a dedicated hosting service where we can have unlimited bandwidth? I know it'll more expensive than a regular hosting service, but that's ok!

Any feedback is much appreciated! Thanks :D

https://redd.it/opkl7l
@r_devops

Rebuilding pipelines for multi-stage Dockerfiles and changes to dev workflow

I’m in the process revising my dev environment and CI/CD pipelines based on a bunch of helpful feedback I recently received.

For instance, I wasn’t using multi-stage Dockerfiles and had some anti-patterns in my CI pipeline that I’m in the process of researching and fixing.

Since I’m implementing multi-stage Dockerfiles, it also affects my dev environment which currently uses Minikube and Skaffold (and a separate Dockerfile.dev for each microservice). This is also causing me to consider better ways of doing some common tasks like running unit tests and managing database migrations.

I’m also considering moving away from Skaffold to something like Devspace, Tilt or Telepresence.

That being said I can’t articulate why I’d need to switch because I have some lingering workflow questions related to unit tests and database migrations. Having answers to those will help me determine if Skaffold will work for me in my new workflow, or if I should look at tools like Devspace, Tilt or Telepresence.

My questions:

First, I know I’d need to target the dev stage in the Dockerfile which, looking at the documentation for Skaffold, would probably involve just doing something like:

build:
docker:
dockerfile: Dockerfile
target: development-stage
buildArgs:
key1: value1
key2: value2

Is this correct?

Second, when it comes to unit tests I know there should be a test-runner stage in the Dockerfile. I’m thinking these should just be run for the microservice that was changed automatically upon save. Thus, there shouldn’t be a need to run something manually, or have a --watch flag, in a separate terminal. Is this correct?

Not sure if this would fall under the test section of Skaffold or Custom Tests. I know that Devspace has Hooks that I would think would allow this.

Third, and lastly, when it comes to running migrations I mostly have an answer for this. At least for DevSpace, it sounds like there would be a “post-sync hook” that would run database model migrations. If not, it looks like you can run something like devspace enter python manage.py migrate to manually do the migrations. Looking at the Skaffold documentation I’m not entirely sure how this would be done or of it would just be using kubectl exec -it on the pod to run migrations after the fact. Can someone elaborate?

Thanks for the help and clarifications on this.

https://redd.it/opm9w2
@r_devops

Starting new role...any advice welcomed.

So after spending the last 12 months taking the jump from traditional Ops guy to a more DevOps orientated role I've been asked to lead the DevOps implementation/provision at my work place.

Although my new title will make alot of people cringe "DevOps Manager" I'm mainly tasked with all things automation, IaC, supporting the dev team and general implementation of the DevOps culture across the business.

I was hoping some good people out there could offer any advice on transitioning into the role, dos and donts and any other experiences they've had with a similar role shift...any advice is much appreciated 🙂

https://redd.it/opm1ex
@r_devops