Passing and creating metrics in Prometheus using Postgres queries

HelloI want to create metrics which I got from Postgres db.So far the metrics appear in Prometheus using this query-exporterThe problem is that both metrics are big integers in Postgres so the values which i`ve got are not the real ones for some reason..This is a code sample for my queries

databases:
db1:
dsn: postgres://........

metrics:
delay:
type: histogram
description: A sample gauge
id:
type: histogram
description: A sample summaryqueries:
query1:
interval: 5s
databases: [db1]
metrics: [delay, id]
sql: SELECT delay AS delay, id as id from table

These are metrics got from prometheus

id_bucket{container="prom-postgres-monitor", database="db1", endpoint="http", instance="0.0.0.0:9560", job="prom-postgres-monitor", le="+Inf", namespace="dev", pod="prom-postgres-monitor-g3g43g34g3g", service="prom-postgres-monitor"} 6141id_bucket{container="prom-postgres-monitor", database="db1", endpoint="http", instance="0.0.0.0:9560", job="prom-postgres-monitor", le="0.005", namespace="dev", pod="prom-postgres-monitor-g3g43g34g3g", service="prom-postgres-monitor"} 0id_bucket{container="prom-postgres-monitor", database="db1", endpoint="http", instance="0.0.0.0:9560", job="prom-postgres-monitor", le="0.005", namespace="dev", pod="prom-postgres-monitor-g3g43g34g3g", service="prom-postgres-monitor"} 0id_bucket{container="prom-postgres-monitor", database="db1", endpoint="http", instance="0.0.0.0:9560", job="prom-postgres-monitor", le="0.005", namespace="dev", pod="prom-postgres-monitor-g3g43g34g3g", service="prom-postgres-monitor"} 0

Most of them are zeros but some real values which i have in postgres are - 10024958860, 10027398870, 10027401148 etc..

What metric option should i use to get some real data - enum, histogram, summary etc..

Also does it matter what is the data type which i have in postgres - for example as i said now is big integer and i`m not shure if Prometheus is ok with this values ?

https://redd.it/nikmlq
@r_devops

High-Performance Short URL generator app built with Golang

Please suggest some feedback or feature you want to see - https://github.com/go-awesome/shortlink

https://redd.it/nihv4e
@r_devops

Simple Bitrise build dashboard

My writeup on simple bitrise build dashboard that can visualise all the branches with its buld statuses. You can see your colleague’s or dependent branch status in a single web page.

https://link.medium.com/zLevB0MWsgb

https://redd.it/nihlx7
@r_devops

Progressively Build an Optimized Docker Image for React Projects

Hi everyone, I'm following up on the series of building Dockerfiles, now with React:

https://www.codingholygrail.com/build-docker-image-for-react-projects

Hope you enjoy and as always please provide me with feedback how you 're deploying React on your container clusters.

ps. I know the vast majority of react apps are being deployed in CDNs and other cloud providers (Vercel, Netlify). If you're using Docker what more steps do you take?

https://redd.it/nid6sk
@r_devops

How do I know if a devOps/SRE career is right for me?

Topic. I am a new grad with a Masters in CS and trying to evaluate my career options. One thing I have been considering for a while is going into devOps/SRE as I'm pretty sure I enjoy working with systems instead of just strictly writing code all day. I am trying to figure out the pros and cons of being in the devOps/SRE space as opposed to just being a SWE, system engineer, or even a security role such as security analyst. Could anyone elaborate on this or point me in the right direction? It would be much appreciated.

https://redd.it/nig1lu
@r_devops

Python Parallel Workflow/Dataflow Framework

Want an easy and powerful python framework for writing parallel devops workflows using just "plain old python"? Have a look at my framework! :)
https://github.com/radiantone/entangle

https://redd.it/niuna3
@r_devops

Best way to run 100+ concurrent jobs on K8s over and over, CI or Parallel processing?

Hi,

Most CI tools are "triggering a workflow, job X then job Y then deploy, integration with many repositories" etc but I feel running 100+ concurrent jobs is more like a parallel processing problem than a CI one.

All the jobs are a single docker image (with sidecar) getting different environment variables at each run.

Kubernetes Job type works but it requires a different job.yaml file for each changed variable and when the jobs finish they need to be cleaned up before resubmitting or new jobs need to be renamed.

I feel there must be a cleaner way to do this.

Thoughts on Jenkins X? Argo Workflow? Tekton CI/CD?Some parallel processing tools on Kubernetes? Just `kubectl run` ? A better way?

Thank you!

https://redd.it/nidvdz
@r_devops

gitlab many projects on one boards

Its posible to create board with tasks from many projects?

https://redd.it/niqwsk
@r_devops

More than monitoring

I don’t want to give my position quite yet but, thoughts on Dynatrace? Pros & cons?

https://redd.it/niq36e
@r_devops

DevOps best practice cheat sheet?

Is there some DevOps best practice cheat sheets available for Software Engineers? Example, not deploying on a Friday before weekend, Keeping Monday reserved for troubleshooting issues generated over the weekend etc.

https://redd.it/nj7xan
@r_devops

Difference between modern dev-op tools Ansible / Vagrant / Packer ...

I'm a developer with almost no background in modern automated dev-op tools. I've been reading on what these tools are about and would greatly appreciate if someone can elaborate by examples why why and when we may need these separate tools?

From how I understand it, Ansible and Vagrant sound almost the same. They both configure and manage VMs.

For Packer, it's not clear to me why we may need this as well. My question is more alluding to why can't we just use Ansible completely to bring about the changes we need on top of a VM? If we use something like Packer, where do we draw the line as to how much should be included in the VM image vs how much Ansible should do with the VM image afterwards?

https://redd.it/njjr6t
@r_devops

tOOLING TIPS FOR LOG PARSING AND FOLLOW UP ACTIONS

Hello friends, comrades, patriots and fine nerds.

I find myself in a position were I need to grab various logs, ranging from Laravel logs, Linux Syslogs, Windows syslogs, logs from own written python scripts and so on so fourth, and depending on what it is, do X or Y action.


I am sure there are great tools out there for this rather than writing my own python parser for each and every log, Prometheus and so on.

However, this is a field in which I am not very familiar so if _you_ could write down some short ideas on how to build something "slightly" more scalable than making 1 python container per log, I'd be most grateful.


Examples:


>Laravel.log throws an REST-API error that Unit 411 has been locked out from $ERP-APP > Make a slack error or trigger PRTG alarm

>Linux syslog throws an error that $FTP_user is using wrong password > Send an email to [email protected]

>$Proprietary_integration_platform throws a log error that $customerFTP is not answering. > Send a text message to /u/guemi and tell him to bitch at someone

>$CustomMadeApp log shows that an automated job has completed > Update a statistics DB that then get's printed to pretty graphs in Grafana so management can look at pretty numbers and statistics

Etc etc etc


EDIT: Uh, sorry guys. I think my caps lock may have been accidentally turned on when I wrote the title. :D

https://redd.it/njgem1
@r_devops

Would an early job switch pigeonhole me?

Hi.

I've been working for a little over a year as a software developer. I'm unhappy with my current employer, so I've been applying for other jobs. One of those jobs was titled as a devops role, which was offered to me.

My understanding is I would be doing a lot of programming in the form of automation. I really like automating things and making people's lives easier, which is what drew me to the position.

Will an early job switch from developer to devops kill my chances of returning to a developer role if I wanted to? I want to explore and try new things while I am still comfortable doing so. I'm young and don't have much as far as commitments go, so I'm thinking why not, but change is always scary so I'm reaching out to hear the experience of others.

Any insight would be appreciated.

https://redd.it/njewm0
@r_devops

DevOps The Hard Way (AWS)

Hey All :)

My name is Michael Levan. I'm an engineer and educator in the cloud and DevOps space. I've been thinking about a new open-source project that can help people create a real-world scenario so they can get the practice to become a DevOps Engineer.

I see a lot of posts on this forum about getting into DevOps and seeing what a day-to-day will look like and truthfully, it's a pretty hard question. If you line up ten organizations and ask them all what DevOps engineers do, you'll most likely get ten different answers lol.

Because of that, I wanted to put together a little something to help out.

This project is still very much in-progress, but here's the line-up :)

AdminTurnedDevOps/DevOps-The-Hard-Way-AWS: This repository contains free labs for setting up an entire workflow and DevOps environment from a real-world perspective in AWS (github.com)

https://redd.it/nj7bf2
@r_devops

What do you want me to know? DevOps Recruitment

I am a tech recruiter in the UK who is primarily hiring in the DevOps market. As experts, I was wondering what you want me to know about DevOps to get you a job effectively?

I normally have a 15-30 min chat before putting you forward for any job, feel free to suggest some questions/answers you think would be useful :)

Cheers!!

https://redd.it/njepjn
@r_devops

is it worth it to create self hosted open LDAP server?



I am working on many projects, private and for costumers where the main goal is to create streaming/analytical infrastructure. Here are different tools that I am using:

* kafka for streaming
* s3, postgresql for storage and analytics
* Grafana and Prometheus for monitoring
* Hachicorp Consul and Vault for data mesh and secrets management
* Nginx as reverse proxy

Now, I am struggling to find some decent concept for centralized authntication/authrization. I was thinking about creating my own LDAP and Kerberos server but it feels somehow wrong.

any suggestions or recommendations?

https://redd.it/njdqzj
@r_devops

I am Challenging myself to master DevOps in 6 months!

https://blog.devgenius.io/i-am-challenging-myself-to-master-devops-in-6-months-534167db988c?sk=96a8264a2be1cb8c94c571aa5041f861

https://redd.it/njsfws
@r_devops

Docker -> Traefik + Keycloak: how to Claim and Check rolenames?

Hi guys,

first, Iam quiet new to devops and docker and i really Like it so far. My questionen which i cant get an simple answer is: how do i claim the roles from the token from keycloak and check it for a specific rolename?
I was not able to find something in Traefik Docs under https://doc.traefik.io/traefik/middlewares/forwardauth/ and https://github.com/thomseddon/traefik-forward-auth seems also not correct. I dont have traefik Enterprise so this https://doc.traefik.io/traefik-enterprise/middlewares/oidc/ is also not reachable for me :(.

I dont want to use Apache2 with its oidc mod since i move all my things to docker this issue is the only thing i could not managed to solve with Google.
Please Help me maybe with an easy docker-compose ? So i could learn?
Thank you very much :)

https://redd.it/njba3q
@r_devops

Anyone with experience with DevOps job assessments? (Graduate)

Hi all. I'm currently going through the application process for a DevOps graduate scheme at a big company in the UK, so competition will be tough. I've completed the first generic assessment, but the next one coming up will be a coding assessment involving shell scripting and YAML.

I'm a JavaScript graduate developer myself moving into DevOps (hopefully). I'm confident in my coding abilities, and can brush up on any algorithmic stuff from my studies.

I've started practicing shell scripting and looking at YAML as I've not had any experience with them before (it recommended learning the basics for this). Does anyone have experience going through a process like this? What kind of coding test can I expect? Any advice for a developer moving to DevOps?

Cheers all

https://redd.it/njb4gl
@r_devops

Shall we allow projects that don't want to do CI/CD?

We are operating an awesome CI/CD pipeline, and now a project that insists on using Gitflow (because they don't trust their external developers *cough*) has asked us to run their stuff through the pipeline. To be honest, my personal answer is a clear N.O., but my team members are more lenient. What do you think?

https://redd.it/nj2mck
@r_devops

How to pass Vercel's SSL certificate to Nginx backend?

I have a domain mydomain.com managed by Vercel serving a React frontend in **frontend.mydomain.com**, for both of which Vercel generates SSL certificates (including a wildcard *.mydomain.com). The only thing that Vercel shares with me is the certificate id, in the shape of cert_HasdfH234aDSFhasdfGAS1k5G

I have also set a DNS A record for the subdomain **backend.mydomain.com** pointing at the IP address of a DigitalOcean droplet which is running a server behind an Nginx Docker container. I had the same setup working with http in Namecheap, but since I've let Vercel manage my DNS I can't ping my server using http, and it seems that only https would work.

​

My question is:


How do I configure nginx.conf for the nginx container in my backend to accept that SSL certificate? Right now I have something like:

​

> events { worker_connections 1024; }
  http {
    server {
      listen 80;
      server_name backend.mydomain.com
      location / {
        proxy_pass https://backend:4000; #backend is the service name of the backend container.
      }
    }
    server {
      listen 443 ssl;
      server_name backend.mydomain.com
      location / {
        proxy_pass https://backend:4000;
      }
    }
  }


...which being incomplete, naturally throws the error:

>nginx: [emerg\] no "ssl_certificate" is defined for the "listen ... ssl" directive in /etc/nginx/nginx.conf:22

​

I have checked several guides for setting up HTTPS in nginx, but none seems to include adding certification through an ID such as the one provided by Vercel.

https://redd.it/njw4l5
@r_devops