Need some reinforcement regarding this career change decision:

I currently work in healthcare and this past year has made really jaded with how those of us have been treated. I started looking at the most in demand remote jobs with a view to improving my QoL (get a dog, travel more) and combined with consulting my cousin who is successful in tech sales, decided on cloud engineering/AWS.

Ive always had a passion for computers as a hobby, having built several pcs and dabbled in coding and so it was not difficult to get into the swing of learning via A Cloud Guru’a premium subscription.

Lately I’ve been looking at stack overflow and other sites to get an idea of what vacancies require so that I know when I have a reasonable amount of experience/certs to apply for jobs but I’ve been put off by the amount of vacancies asking for 3-5 years experience using these tools I’m learning. Some are less demanding in terms of aws experience but ask for some help desk experience instead but therein lies the problem - I can’t see many entry level help desk jobs in my area (Glasgow, UK) and so it’s starting to feel like I’m gatekept.

Would a strong independent portfolio showing my ability with python/AWS adequately replace the seemingly universal requirement of several years experience? I don’t want to grind away at this for 3 years on a wing and a prayer that I might land a job. It’s far too much to expect of someone as a time investment as I’m sure you’ll agree.

Any positive words of hope for this fledgling cloud engineer would be great!

https://redd.it/lgq4rt
@r_devops

GitLab pipeline fails because of composer

Hello r/devops!

We run GitLab CI to build our containers until yesterday everything was good and fancy.

But since then, our pipelines started to fail. We built our services in PHP and we are using the GitLab private composer repository to store them, so we passed down the CIJOBTOKEN to authenticate with the repository.
This is the point where we fail, composer can't authenticate with the repository so it can't download the packages.

Have anyone experienced something similar? Is there a breaking change in composer or in GitLab itself, or I did something bad or hacky way and a patch killed it?

One of our job:

image: docker:latest

stages:
    - build

services:
    - docker:dind

before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY

build_auth:
    stage: build
    script:
        - cd authentication
        - docker build --build-arg CI_JOB_TOKEN=${CI_JOB_TOKEN} --pull -t "$CI_REGISTRY_IMAGE/authentication" .
        - docker push "$CI_REGISTRY_IMAGE/authentication"
    only:
        refs:
            - master
        changes:
            - authentication/**/*
            - .gitlab-ci.yml

The Dockerfile that belongs to it:

FROM composer:2 AS autoloader
COPY . /app
WORKDIR /app
RUN if [[ -n ${CI_JOB_TOKEN+x} ]]; then git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/".insteadOf "[email protected]:"; fi \
    && composer install --quiet --no-dev --no-scripts --optimize-autoloader --ignore-platform-reqs

FROM php:7.4-apache
COPY ./apache2.conf /etc/apache2/apache2.conf
COPY --from=autoloader /app /var/www/html
RUN apt update \
 && apt install openssl libssl-dev libcurl4-openssl-dev libonig-dev $PHPIZE_DEPS -y \
 && docker-php-ext-install pdo pdo_mysql mbstring \
 && chown -R www-data:www-data /var/www/html \
 && a2enmod rewrite

https://redd.it/lgpeow
@r_devops

Tools to list and share subscriptions and their credentials

What sort of tools do small businesses use to manage and share a list of subscriptions and their credentials? Password managers are an option but that focuses more on credentials. Is there anything better?

We have Google Cloud, Rackspace, various shared hosting, Slack, Office365, Windows subscriptions/licenses. Need a good way to show and share consolidated data of what do we use, for what. As of now, I am just looking for something that IT and Product teams will just manually add when they pick a new thing or remove when it's stopped. It should have metadata to describe the purpose so it's easy for everybody to figure why it is being used, at a glance on a dashboard.

If it can allow me to securely store and share credentials, that'd be an added benefit.

It can either be hosted or self-hosted tool.

https://redd.it/li9gh3
@r_devops

Assistant app

Hi, I would like help with a project because I quck at programming.
I had the idea of a calendar app that would fill itself for you.
You give certain times of the day some attributes such as "lunch time", "work time",.. And tasks would have attributes too, as well as priority levels.
So when you want to had an event or task, the app would schedule them according to their attributes or priorities.

For instance, some basic rules I would implement are.
Grouping tasks as much as possible, filling the beginning of the week first,..

I hope I was clear enough

If you're interested in helping me it would be awesome.

https://redd.it/libubz
@r_devops

CircleCi Saas – on which public cloud is it running

Hi guys,

I can't find the info. Any of you know? Also, do you know where it is located?

thx!

https://redd.it/lieea5
@r_devops

Terraform cloud

We don't have a build process for tf currently, states in s3. we're just in the process of adding the builder and considering terraform cloud - the infrastructure is decently large with over 30 AWS accounts. the primary concern here is not having control of the build infrastructure and the security given a third party system has full access to your cloud.

if you use it in your enterprise, how did you get past these security concerns and build trust with tf cloud? also, did you face outages in the past?

tf cloud seems to be easier to work with than Atlantis and all the features available, so was giving preference to it.

https://redd.it/lidnmw
@r_devops

Embedding pictures into a work item with Flow

Hi guys. I'm doing an Internship in a company and I've been given a task.

It's my first time ever using Flow and Dev Ops, and it's only been a week.

Basically, to make it simple, the company wanted me to build a flow that will allow the creation of a work item once a mail is received and play with some filters. That was alright, I did that.

Then, they wanted to make it possible to pass the attachments in the work item too. I struggled a bit with that, but it's also done and I'm really happy although it does look easy to do for experienced users.

Now I'm trying to push it further: I'd really love to embed the pictures that I receive by mail in the Work Item description. I've tried looking around and it doesn't seem like a very common demand so I'm wondering if it's even possible to do it. Usually, on Outlook, people just drag and drop pictures and don't put them as attachments.

My goal is to have that same body with the picture embedded in DevOps and I'm struggling with this. Apparently, it's not a native feature. But I think there are ways to do it.

If so, could you guys maybe redirect me to an explanation or a link where someone does that?

That would be so helpful. Thanks a lot!

https://redd.it/lie410
@r_devops

What do you use for service management?

We are currently using Ambari to maintain all our custom services in a non-K8 environment. We have written custom service descriptors in Ambari for Prometheus, AlertManager, Thanos, Grafana, Kibana, Jaeger and many metric exporters and use them to start/stop/restart services.

I do like Ambari since it allows me to manage my services from a centralized UI but feel its not very generic and mainly useful for managing Hadoop/HDFS cluster (for which it does a good job).

I wanted to know what other tools would you guys recommend for this job. The only requirement is to be able to start/stop/restart/monitor my services from a centralized UI.

https://redd.it/liadg1
@r_devops

The role of sampling in distributed tracing

Distributed tracing is a technique that produces a high-fidelity observability signal: each data point (trace) represents a concrete execution of a code path. In an HTTP-based service, this typically means that each request would generate a trace containing data representing all the operations that were executed as a result of the request: database calls, message queue interactions, calls to downstream microservices, and so on.

As you can imagine, collecting this level of information for all requests received by a service can quickly generate a seemingly endless amount of data that is hard to manage. Making things even less appealing, the vast majority of this data will represent requests that are not that interesting, given that they’d represent successful operations. In the end, we might end up collecting, transferring, and storing data that will end up being deleted without being used at all.

The holy grail, the ultimate goal for distributed tracing is to collect only data that we’ll need in the future.

While this goal might be very hard to achieve, it’s certainly possible to get close to it by making use of a technique called sampling.

Continue reading here: https://blog.kroehling.de/the-role-of-sampling-in-distributed-tracing

https://redd.it/li7vii
@r_devops

Self Service Infrastructure?

So I have an idea, could you create an application like a webpage for drop down menu for provisioning infrastructure which then creates a terraform script for deployment for example. Like a self service app for developers, so instead of bugging operations teams they can manually say what they want, it automatically creates a terraform script and then it gets deployed and send the information back (e.g. Address, location and deployment stats). Is something like that possible? If so how would you do it? Thinking I could save some time!

https://redd.it/lil3oo
@r_devops

How should I explain my job to friends and family? Or what do I use as a simpler, easy-to-grasp job title to use for non-techies?

Sorry if this has been posted before, but I’d like to see your interesting/clever/funny responses to this.

I had a conversation like this two days ago with someone who is relatively tech-oriented (PC Master Race kind of guy):

Him: What do you do?

Me: Well I build and ship software.

Him: Oh cool, a software engineer then?

Me: Well... sorta... almost, but mainly automating the deployment of software and monitoring it and stuff, I don’t develop new features.

Him: Oh, so a system administrator? Can you fix my printer? It’s got this weird noi-

Me: NO! I mean, no, no, I don’t know anything about printers. Again, I build and ship software on a large scale for many different teams.

Him: Oh cool, like a software engineer?

Me: Yes. Sure. Yes.


Not that it matters because titles are mainly irrelevant and I know people get up in arms about what DevOps truly means... but I also don’t want to spend more than 10 seconds going into the land before time where developers and operations used to be on separate sides of a wall and then one day the fire nation attacked.... snooze

Again, just looking for quick one-liners like “tech pipeline builder” and funny stuff like that where there won’t be any follow up questions, but I also don’t give a false sense of what I know and can do.

https://redd.it/likpbt
@r_devops

Alternatives to Fluentd for an IoT Environment?

I am looking for a logging solution for: log enrichment, aggregation, buffering, routing, and forwarding to support an IoT system with constrained hardware resources that is often in disconnected/offline state. The log sources vary from OS and orchestration level to various custom services. Fluentd seems to be the most robust system I can find so far. Just wondering if anyone can recommend any similar systems/frameworks I should evaluate, both open source and commercial suggestions are fine. Thanks!

https://redd.it/li58yr
@r_devops

Health checking services without web server (queue only)

Hi all,

We have bunch of microservices build in NodeJS, running in docker that we need to health check.
That is easy thing to do with services that already have a REST API by simply adding endpoint for health, however we do have some services which only work with queues (RabbitMQ) and do not have a web server.

​

I talked with lead dev and he doesn't like the idea of adding web server to these services just for purpose of health check, though that should be quite easy thing to do.
Our current idea is to have a "health checking" service which will publish to health queues of individual services, and extend services to consume these messages and publish a response - basically a heart beat with some metadata.


I am not 100% sold on the idea for obvious reasons - increasing complexity, introducing single point of failure for health checks and i am not sure how it's easier to develop new service that needs to be built, deployed and monitored vs just adding web server and health endpoint to existing services.

​

Does anyone have a better alternative to this or have done similar in the past?

https://redd.it/li8qeh
@r_devops

Telegraph vs Metricbeat

Hi All,

Anyone experience working either of these ? I like to understand these tools push model , how to configure them to push metrics to different outputs ?

I am working on a use case, where i have to push metrics from the Kubernetes cluster to some storage service but not any time series database.

Interest to know your thoughts on this ?

Thanks

https://redd.it/li5n43
@r_devops

Can DevOps engineers handle some security or should AppSec handle 100% of it?

If we are to consider DevOps people the utility players that they are, they should be able to handle some security matters without specialist support. But some schools of software philosophy seem to push for dedicated AppSec teams in all situations. What kind of config would justify each type?

I'll get the ball rolling with my understanding.

DevOps can do security if:

1. Security can be ensured due to a smaller scope of software
2. App security is not mission-critical i.e. no financial or medical data
3. They have adequate planning ability to distribute security planning across their people

AppSec teams should be added on top if the converse of any of the above 3 exists.

https://redd.it/lixloa
@r_devops

How can I run my scripts like a cronjob without a server?

I have a virtual machine with some scripts. I would love to run these scripts outside of manually git pulling, configuring the passwords in the files, and using a cronjob to run the scripts.

I have much experience with Linux and virtual machines, but I'd love to have a new pipeline solution that just runs some code, fetches my secrets (passwords), and runs some Python code.

Simplicity is also important. I think AWS Lambda is already too complex. I'd prefer to see the output of the script in a CI/CD GUI (web page).

https://redd.it/lj9cmr
@r_devops

I'm new and need help, sorry

Sorry everyone, I'm a newbie.

I coded my own static site generator in python. I know how to use Github pages to host static content. My hope is to use webhooks to know when a Youtube video is uploaded so my script can scrape the video, and then publish the newly generated .html file to Github pages. I think Github actions can help me but I have no knowledge of devops I'm just so confused.

​

Is there a way for me to:

1) Make that webhook trigger a Github action that runs the python script.

2) Update and commit the files with the newly scraped content to a Github repository.

3) And then finally build/deploy/publish those newly generated .html files to Github pages.

​

I've heard Travis CI can help with steps 2 and 3.

I know my problem overall is very specific but any help would be greatly appreciated. Sorry for the noobness, I'm not a comp sci student, just really passionate about coding and building things.

https://redd.it/lj6rnd
@r_devops

Why is it so hard to find a long term part-time job?

I've been on various job listing sites looking for Dev/ops and have had trouble finding any positions that were 20 or less hours a week, remote, and potentially long-term. It's like I have to pick 2 of the 3 things I want out of a job, and that's not really worth it to me. Why is this type of work so hard to come by?

https://redd.it/lj66aw
@r_devops

Nginx Container Benchmarks

Hi All,

We have setup the Kubernetes cluster to execute performance benchmarks for a front-end and back-end (nginx and mariadb) pods for benchmarking ? I am using sysbench to perform benchmarks on the MySQL pod, however looking for recommended tool which is relatively close to real world load to benchmark nginx web server container ?

Thanks

https://redd.it/ljdo35
@r_devops

How can I minimize multiple layers COPY in Dockerfile?

Hello,
I have something came up in my mind. As I have “COPY --from=builder ...” for more than 3 lines, can I minimize it to be only one line to make it faster to build?


Ps. Why others try not to COPY them all to destination file?

https://redd.it/lj1m5s
@r_devops

Simple and easy CI to be triggered via Webhook

I'm looking for a lightweight, simple and easy CI solution to be used with Gitea.

Don't need to be with fancy gui and stuff.

The idea is to have a simple service that is triggered by a WebHook and then executing a build script (shell-script?). Reporting then via log file or mail.

Is such a tool available?

Otherwise i would consider to develop one.

Any thoughts?

https://redd.it/liyovy
@r_devops