GitLab Runner Infrastructure

What approach do you guys use for hosting GitLab runners?

I’m building out test automation infrastructure. I have the test runner in a docker image. Mostly web based GUI and API testing, with quite a bit of GUI testing that takes a while to run. (I know API>GUI testing but I work on large project with large distributed team, I can’t control everything)

I have a static EC2 image that handles jobs sent from the test pipeline. The tests run in parallel in a number of separate containers.

I’m fairly new to DevOps and not sure how to scale this up. My boss suggested a kubernetes cluster which from the sounds of it can do all that and a lot more. I’ve seen ECS and autoscaling EC2 groups in my research as other possible approaches. Any guidance or words of wisdom for a novice?

https://redd.it/qx49xs
@r_devops

Looking for a calendar web app prederably hosted in docker to add events and send reminders to some emails list before the events

Hey all. I'm looking for some simple web application to add some events (downtime and similar) with some date, time, details which will then send us some reminders X hours prior to the event. Preferably I would like to host this app in some docker container. Bonus part is to be able to subscribe to the calendar through Outlook for example to show the events with the normal calendar.

Any suggestions? Thanks.

https://redd.it/qx66st
@r_devops

Azure or AWS?

I am a system engineer with no cloud experience and when I browse job boards most all are looking for cloud experience. Which cloud infrastructure should I most focus on learning? I'm sure the concepts carry over but I want to focus on the one which will make me the most marketable.

https://redd.it/qxh397
@r_devops

Equivalent of Consul/Consul-Template in AWS for real-time configuration management?

New to cloud esp. to AWS. Unfortunately I could not find anything that I could use in AWS in-place of consul-templates that I am used to when working on-prem. Basically, right now we are having to manually change config files and update them again and again which is very annoying and cumbersome. What is the standard for this on AWS?

https://redd.it/qxctw9
@r_devops

Individual dev environment?

I'm currently working at a company where each developer has their own designated environment in AWS. Curious if anyone else out there uses a similar model. And, if so, very curious how much you all are spending per dev environment?

https://redd.it/qxkjax
@r_devops

History of SRE: Why Google Invented the SRE Role

We got curious and wrote about the origins of SRE and how they got started at Google here :)

https://redd.it/qxik9w
@r_devops

Why DevOps is paying so much?

Curious question, DevOps seems to be bombing and payscales seem to be at the top.

I am college dropout working as SRE and compared my salary to a data scientist with a master and I am earning a little above him when we discussed openly about salary.

I think like cheating because I believe the learning curve for being a data scientist is higher than someone who does SRE. Is this a bubble? Will it explode someday?

https://redd.it/qxop7b
@r_devops

Pulling/Pushing out any AWS ECR images from/to AWS ECR through AWS Route53 CNAME

The main idea of this article is to show how to use CNAME of Route53 to pull or/and push images from/to AWS ECR service. By default, Amazon doesn’t allow to do it (SSL handshake is not working. SSL has been signed by Amazon side).

I played around with Amazon API and Python and proxy and I have found several solutions:

* Use Python and develop wrapper to log in, pull & push AWS ECR images from/to ECR through AWS Route53 CNAME of AWS ECS service.
* Use Some proxy (Ex: Nginx, Traefik, etc) and make forwarding rules with needed headers. This implementation is \`TBD\` soon!
* Amazon ECR interface VPC endpoints (AWS PrivateLink).

The full article you can read here: [https://medium.com/@solo.metalisebastian/pulling-pushing-out-any-aws-ecr-images-from-to-aws-ecr-through-aws-route53-cname-7c92307f9c25](https://medium.com/@solo.metalisebastian/pulling-pushing-out-any-aws-ecr-images-from-to-aws-ecr-through-aws-route53-cname-7c92307f9c25)

The code: [https://github.com/SebastianUA/ecr-pull-push](https://github.com/SebastianUA/ecr-pull-push)

\#AWS #AWSRoute53 #Docker #AWSECR

https://redd.it/qxo7hr
@r_devops

Is docker swarm good enough for production?

I'm getting into docker swarm lately. Everything I've done with it I've done locally of in a test environment.

Especially looking for someone well versed in docker. I've heard swarm is great for dev environments but not for production. Is this true? And if it is why is it not good for production environments?

I haven't seen any issues so far but like I said I've never even considered using it for production until now.

https://redd.it/qxq1q9
@r_devops

Why is the Interview Process a pain in the ass?

Interviewing for my second title within the DevOps/SRE space.

Why the hell do so many companies have a multiple-hour interview or technical task as a part of the interview process?

I understand that they want to ensure that the person they're hiring understands what they're doing, but I simply cannot schedule a 2, 3, or even 4 hour block (all numbers that have been estimated to me) out of my free or working time to further myself along in the interview process. What happened to references, or simple 1-hour technical interviews?

So far, I've declined further candidacy for 3 jobs, and I'm sure that number will grow.

Anyone else commonly experience this mess? How do you handle it? Am I being ridiculous?

https://redd.it/qxqia2
@r_devops

How to learn high level devops concepts as a dev ?

Hi guys,

I'm a full stack dev but I must say my understanding of many devops concepts are very poor, I can get around by following some tutorials and using dev friendly solutions but it comes with limitation, sometimes vendor locking and I don't really like using GUIs as in my experience they cause more problems than they solve and I rarely use those since I mostly live in the shell anyway.

I believe even a higher level understanding can help troubleshoot some problems so I'm looking forward to better understand concepts like, reverse proxies, CDN, load balancing, understand docker better etc.

Thanks in advance.

https://redd.it/qxzecm
@r_devops

Can readfromhead work with posfile together?

Set posfile only will send data correctly. But if set with readfromhead will get error

<source>
  @type forward
  port 24224
  pos_file /a/pos_file/path
  read_from_head true
</source>

Because of readfromhead, fluentd can't start successfully.

If set readfromhead only, fluentd can start but can't send data.

Can't they work together?

https://redd.it/qxww8p
@r_devops

Has anyone considered moving from DevOps to just Dev?

A bit of background:

I switched careers 2 years from non-tech to DevOps and I’m now in my 3rd role and have been in it for 8 months. It’s a Senior role and pays pretty well but I’m not really involved in actual dev work, neither in my DevOps chapter nor the dev squad in which I sit.

The thing is, I really enjoy writing and contributing to application software but don’t have loads of experience in doing so. I’ve done some Python application work in a previous role and I’m currently learning Java in my spare time.

Anyone else been in a similar position? Would love to know how you approached it!

https://redd.it/qy36k5
@r_devops

Running AWS Services In A Laptop Using LocalStack

LocalStack is a fully functional mock of AWS services running locally on your computer. We can use it to develop and test cloud and serverless apps offline. It can run through the CLI, in a Docker container, or in a Kubernetes cluster. We can use it to create mocks of S3 buckets, Lambda functions, RDS databases, ECR repositories, and more.

https://www.youtube.com/watch?v=8hi9P1ffaQk

https://redd.it/qy3xqg
@r_devops

Following from a rant posted in this sub, what are some of the practical technical interview questions you have been tasked to do during a live coding interview?

Most interviews today require you to do some devops related tasks. Could be take home or live coding session. Im curious to know what these tasks are.

https://redd.it/qy5g9r
@r_devops

I got an opportunity to become a DevOps but I'm scared to go for it

Hope it's okay to discuss career direction here.

A little about me:

* 26 y/o
* 4 years in IT (3 as helpdesk, 1 in current role at a FAANG company)
* Almost no DevOps knowledge

**I know**: Python, PowerShell, some git, some Ansible, I'm OK at Linux, I know networking concepts and fundamentals, I'm comfortable at troubleshooting and I like documenting stuff.

**Weak points**: Storages, not fully comfortable with Ansible, don't fully understand a lot of Dev subjects like different APIs, webhooks and in general, anything I listed under "I know:", I'm still no expert at those things.

**Zero hands on knowledge (I know the concept)**: K8s, CI CD tools, IaC tools like Terraform, etc.

Another thing I wanted to mention is my learning process. Sometimes I feel like it takes me longer than average to get something. I'm actually lacking confidence in general at my field, but overall, I'd say I do a good work as a Sysadmin.

One of my former colleagues (from my current role) offered me to join the company he works at. My role would be DevOps even though he knows I have zero experience. He mentioned that he saw potential in my skills and that he'd help me develop whatever skills needed for this role.

Honestly I'm a bit confused as to whether I should go for it. Salary isn't a factor currently because I'm more focused on gaining experience. The location is the same as well, which is nice.

It's a startup, as opposed to my current FAANG company, which is a bureaucratic mess sometimes.

I'm afraid to disappoint my former colleague. We worked together for like 6 months and during that time he was somewhat of a mentor for me. I'm not sure if I'm ready for this role but I know it's a good place to take my career to at my age.

What do you guys think? What other factors should I consider before talking to him about whether I'm in or not. I'm only 11 months in to my current role and I'm worried my tenure is too short.

Thanks :)

https://redd.it/qy7o2j
@r_devops

Vault and 1Password for different cases

Does anybody uses vault and 1Password together? I feel like they solve different things. Vault is really for managing secrets for automation while 1Password is for a team or person to manage their passwords.

Sure, vault can be used for passwords but it’s a bit technical and not everyone on the team wants a cli based tool when they want to login somewhere.

https://redd.it/qyc3yw
@r_devops

Cloud Developer Environments

Hi, we are moving fast and were wondering about an option i.e. Cloud Developer Environments. Do you recommend cloud developer environments? What are the best practices around it? Is their a good resource to read about writing infra in a way, it can create cloud dev envs quickly?
Our current scenario- We have 3 envs as of now- dev, staging and prod and we deploy on GKE. Devs install the tools necessary to work on their services and for other services they just kubectl port forward it on their local.

https://redd.it/qy86d0
@r_devops

Deno for Infrastructure as Code

Hello folks, I am a long time lurker and first time poster.

I have put my thoughts about using Deno for writing infrastructure code in form of a blog post. Looking for critical feedback either here or in the comments section on the link below. Thank you !

https://adnaan.badr.in/blog/2021/11/20/deno-for-infrastructure-as-code/

https://redd.it/qyobcl
@r_devops

Writing sidecar tool for aws

I am trying to write a tool preferably in golang for web apps in aws that will read config from a source and update local app config file. Is there any pattern or base lib/framework i should be following? Suggestions are welcome.

https://redd.it/qyqeio
@r_devops

Introducing DevOps practices to a company with bad organization

So we have an R&D team that develops and maintains dozens of tools, some relatively big desktop software, some data processing server pipelines, some smaller command-line tools. Mostly for the use of internal teams, but increasingly also for external customers.

For desktop tools, the deployment "process" is: copy-paste the new code and its new dependencies on an NFS network share. The prod workstations download the new environment regularly from there. Tools aren't really reliable, and when I came in I had to rewrite a large part of the code for my new projects.

I'm the only one doing automated testing, I introduced Docker/docker-compose for server deployment (instead of scp'ing the code), and I'm trying to improve coding practices. Obviously, there's no CI/CD, and there won't be if I don't set it up myself. In the desktop Python environment, pip doesn't work correctly, so we can't be sure that a new deployment won't break another package's dependencies.

I'd like to ask for some advice on how to set up a CI/CD pipeline, for me and some colleagues who want better tooling. We don't need approval, and I think the others would be open to better practices if we showed the way. I have support from my manager for it.

The current plan is to set up Jenkins on some server, make it run the CMake build regularly, and run the few PyTest unit tests that already exist, all in Docker. That system would expand over time, as we train the rest of the team to use it.

Is Jenkins a good choice, considering that we'd be maybe two people maintaining it? Is there a good tutorial on how to set up such a platform from scratch? I looked at the Jenkins documentation, but it's not always clear. Any and all advice is appreciated.

https://redd.it/qyaj9v
@r_devops