How to Run GitlabCI jobs as a user other than root? (Docker as GLCI Runner executor)

I am a student attempting to learn about Gitlab by working on a hobby project so I apologize in advance if this is not the appropriate sub to ask this question. I will try to make this as short as possible.

I have a Digital Ocean droplet configured as a GitlabCI Runner using the Docker executor. I am still learning about Docker as well so I apologize if my problems turn out to be a misunderstanding of Docker rather than GitlabCI.

Essentially I am trying to execute a pipeline job in which the user within the Docker container for that job is NOT root, and I cannot figure out how to achieve this.

The reason running as root within a job's Docker container is a problem is that certain commands will not function correctly via the root user. Attempting to install/ configure CocoaPods is apparently one such case, yielding the following output.

I have spent hours trying different shenanigans including creating a user within the job and attempting to login as that user, but none of these methods have been successful and the container continues to be run in the context of the "root" user.

Is there a way to run my pipeline jobs as a user other than root, and what are the best practices (or even general practices) regarding this. Once again I apologize if I misunderstand something about Docker, please let me know if this is the case.

My problem is very similar to the question posed here, however, none of the solutions seemed to solve the problem:

https://stackoverflow.com/questions/48576412/running-gitlab-ci-pipeline-jobs-as-non-root-user


/root/.gem/gems/claide-1.0.3/lib/claide/command.rb:439:in \help!': \\e\[31m\
[!\] You cannot run CocoaPods as root.\\e\[39m (CLAide::Help)

\\e\[4mUsage:\\e\[24m

$ \\e\[32mpod\\e\[39m \\e\[32mCOMMAND\\e\[39m

CocoaPods, the Cocoa library package manager.

\\e\[4mCommands:\\e\[24m

\\e\[32m+ cache\\e\[39m Manipulate the CocoaPods cache

\\e\[32m+ env\\e\[39m Display pod environment

\\e\[32m+ init\\e\[39m Generate a Podfile for the current directory

\\e\[32m+ install\\e\[39m Install project dependencies according to
versions from a

Podfile.lock

\\e\[32m+ ipc\\e\[39m Inter-process communication

\\e\[32m+ lib\\e\[39m Develop pods

\\e\[32m+ list\\e\[39m List pods

\\e\[32m+ outdated\\e\[39m Show outdated project dependencies

\\e\[32m+ repo\\e\[39m Manage spec-repositories

\\e\[32m+ setup\\e\[39m Setup the CocoaPods environment

\\e\[32m+ spec\\e\[39m Manage pod specs

\\e\[32m+ update\\e\[39m Update outdated project dependencies and create
new Podfile.lock

\\e\[4mOptions:\\e\[24m

\\e\[34m--silent\\e\[39m Show nothing

\\e\[34m--version\\e\[39m Show the version of the tool

\\e\[34m--verbose\\e\[39m Show more debugging information

\\e\[34m--no-ansi\\e\[39m Show output without ANSI codes

\\e\[34m--help\\e\[39m Show help banner of specified command

from /root/.gem/gems/cocoapods-1.9.1/lib/cocoapods/command.rb:47:in \`run'

from /root/.gem/gems/cocoapods-1.9.1/bin/pod:55:in \`<top (required)>'

from /root/.gem/bin/pod:23:in \`load'

from /root/.gem/bin/pod:23:in \`<main>'

Running after script

00:01

Uploading artifacts for failed job

00:02

ERROR: Job failed: exit code 1



Thank you for your help!

https://redd.it/fnas1v
@r_devops

Team was reOrg'd this year, not getting any direction from Execs/Senior Management, Need advice/guidance/suggestions/etc.

Hi all, reaching out to the community as a last resort of sorts. This year has been a serious struggle for me and has literally sent me into a mental illness spiral of issues due to several things, this topic being a huge part of it, plus being overloaded and forcing me to stay in the weeds to meet/attain goals/tasks and not being able to focus on Managing. Anyway, any suggestions/advice/guidance/etc on how to handle my situation here would be much appreciated. Thanks in Advance!!

So at the end of December last year Execs decided to reOrg my team from Operations to the Development Team to attain a better model for feature delivery and improved availability/sustainability of the system. Some of my teams responsibilities are 24x7 sustain support of the core Business Applications (through a relentless Oncall Rotation), Develop Alarms for the Service, Automate things that weren't, Deploy and Coordinate updates that were delivered, Integrate/Architect solutions (Dev Teams don't have a big picture view of anything).

Previously, due to the different Orgs, there were various lines and processes established which enabled some separation of duties. This enabled Devs to throw things over the fence, in a way, which fell to my team to have to handle (some integrations/architecture/Networking/etc), but this forced some responsibility onto them for Apps that we weren't sustaining/supporting. From an outward prospective, things looked to be running properly internally and that there was a lot of order, but now in the new Org, I have noticed its more Wild West and some take advantage of this to their benefit. This has now lead to them trying to force various items onto an already taxed team.

Come to find out, Execs had no plan, no idea of how to integrate my team into existing Org. But they are asking that I come with a Charter with the only direction being they want us to adopt a more Site Reliability Engineering model. This was something I had attempted in the past but without much Buy-In at the time, so I am optimistic with them stating this. But I do not know where/how to start, what responsibilities should be on my team vs devs vs shared. Essentially how to break the cycle/spiral of crazy that is driving me down a path of worsening mental illness.

If you made it this far, Thanks for reading!!

https://redd.it/fo7rve
@r_devops

HashiTalks2020: Enterprise Deployment to Azure and AWS in Azure DevOps Talk Link

Here's my HashiTalks2020 presentation about my company's transition from single-cloud, single-devops to cloud agnostic Terraform and multicloud Azure DevOps, pipeline and infra-as-code environments. 25min presentation. #Terraform #DevOps

[Enterprise Deployment to Azure and AWS in Azure DevOps](https://www.hashicorp.com/resources/enterprise-deployment-to-azure-and-aws-in-azure-devops)

I think it's an interesting first-person take on our journey from CloudFormation-focused AWS-only DevOps organization to a multi-cloud, generalized provisioning tool environment, and the mistakes we made along the way.

And here's a [direct link to the YouTube recording](https://www.youtube.com/watch?v=Q3w37iACSFQ&feature=emb_title).

https://redd.it/fo72q4
@r_devops

What challenges would I face to migrate from AWS (eks) to a DIY Kubernetes cluster? Or is it worth to do so?

Hi,

Before anything else I think it's important to say that I started working with Kubernetes for about half a year and currently working alone in the devops team at the company I work for.

I recently finished the configuration of my cluster with rs, load balancing, monitoring, hpa and ca to run our product on EKS but one thing is getting in the way, cost to maintain the infrastructure required (2\~3 EC2 M5.large).

One suggestion from the board was to find other providers to DIY Kubernetes into them because the on-demand cost on them is much cheaper for machines with better specs (8gb on aws to 32gb in the new provider). But since I would be alone doing this it would take some time to get everything up and running and then more time to keep it up to date and running.

In which way migrate/replicate my already existing cluster which is managed by AWS to a DIY manner would be cost effective given my current situation and the costs of just hosting it?

Thanks.

https://redd.it/foae8r
@r_devops

Octopus Deploy Blog - Using DbUp and Octopus workers for database deployment automation

DbUp is a tool that is focused on running schema migrations for a range of different RDBMS.
This is an update of an older post that covers off some of the more recent changes to DbUp and Octopus Deploy.
[https://octopus.com/blog/dbup-database-deployment-automation](https://octopus.com/blog/dbup-database-deployment-automation)

https://redd.it/fofvdp
@r_devops

Video on how to take ANY AWS Certification Exam from Home or Office

Did you know that you can now take any AWS certification exam from your home or office? AWS has just announced additional support in extending certification expiration dates, exam retirements dates (SAA-C01 and BDS-C00) and voucher expiration dates.

Watch this video to learn more about the latest updates from AWS especially if you have been affected by exam center closures caused by the Coronavirus (COVID-19) pandemic.

[https://youtu.be/Wtp4GZHcq-8](https://youtu.be/Wtp4GZHcq-8)

https://redd.it/foj53e
@r_devops

What is the relation between message broker and message oriented middleware?

https://en.wikipedia.org/wiki/Message_broker says

> A message broker (also known as an integration broker or interface engine[1]) is an intermediary computer program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver. Message brokers are elements in telecommunication or computer networks where software applications communicate by exchanging formally-defined messages.[1] **Message brokers are a building block of message-oriented middleware (MOM) but are typically not a replacement for traditional middleware like MOM and remote procedure call (RPC)**

https://en.wikipedia.org/wiki/Message-oriented_middleware says

> Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems.

> The primary disadvantage of many message-oriented middleware systems is that **they require an extra component in the architecture, the message transfer agent (message broker)**. As with any system, adding another component can lead to reductions in performance and reliability, and can also make the system as a whole more difficult and expensive to maintain.

What is the relation between message broker and message oriented middleware?

1. What does it mean by "Message brokers are a building block of message-oriented middleware (MOM)" and "but are typically not a replacement for traditional middleware like MOM"? Does it mean that a component of something can't be replacement of that something?

2. Is message broker a component of message oriented middleware? What other components does message oriented middleware have?


3. Is it correct that message broker and message oriented middleware can be

- message queue,
- publish-subscribe, or
- both?

Thanks.

https://redd.it/foq3ok
@r_devops

Why are there such a lack of learning resources for azure devops as compared to jenkins?

I am a developer who wants to learn devops to be a better developer. MY organization uses azure devops. They do not use jenkins. But everywhere I looked pluralsight or udemy I could not find any reasonably good course on azure devops. However there are plenty when it comes to Jenkins. Should I just learn Jenkins instead? Would that help me grasp azure devops relatively easier in future?

https://redd.it/fov1m6
@r_devops

Python/Linux AWS Question

Just passed my CPP and going for my SA- Associate then Dev-Associate before I go for a junior engineer job within my company

Is it necessary to learn linux or can I get by with just being proficient in coding (python)?

https://redd.it/fp1rsw
@r_devops

Tell us the good and the bad of your company CI/CD set up ?

How did you guys set it up ? What tech stack did you use ? Did you automate it every step of the way or some part has to be manual. Was there a situation where you had to cut corner for a specific use case ? Would love to hear your story :)

https://redd.it/fp1iya
@r_devops

Glider ai Devops request, pre interview... Should I play along with this red flag or nope out?

Current company in VA may end up doing contractor layoffs due to the current situation. Along comes one of the big bank interested in interviewing me for a Cloud Eng position. I was expecting a calendar invite, but instead I get a invite to take a glider ai test. 26 questions in 60 minutes.

Usually I would nope right out of any company that participates in these canned coding tests. I, like many in the field, consider these tests big red flags and a lazy hiring practice. They also typically don't represent real world work situations. This company is already known for day+ long theme park interviews, so I should have seen this coming. 26 questions in only 60 minutes, I assume this must be multiple choice; not an actual code challenge.

Trying to decide if I want to go forward or politely nope out. I'm currently employed, but that may not last too much longer. Its a good job market and, I already have more leads. But, this is also probably the largest employer in the city, so I'm also tempted to just do it. The usual response is, "If you want the job or like the company, do it." Problem is, this company already has mixed employee reviews based on department, and a canned code test strikes me as a sign that this may be one of the negative departments people refer to.

What do you think? Take it or pass? Do you consider these tests a red flag? Is anyone familiar with glider ai's devops testing? Thanks in advance for the feedback.

https://redd.it/fozqf2
@r_devops

Exposing secrets on GitHub: What to do after leaking credentials and API keys



**Leaking secrets onto GitHub and then removing them, is just like accidentally posting an embarrassing tweet, deleting it and just hoping no one saw it or took a screenshot.**

For anyone that has had a oh crap! moment on git.

I wrote a blog on what to do if you leak a secret onto a public repository including;

* revoking the credential
* how to rewrite history to permanently delete it
* Best practices.

[https://blog.gitguardian.com/leaking-secrets-on-github-what-to-do/](https://blog.gitguardian.com/leaking-secrets-on-github-what-to-do/)

I hope it is helpful.

https://redd.it/forjzq
@r_devops

Is docker just bundling an OS with your app?

From a 10000 ft perspective, is Docker just bundling an OS/environment with your app? And is this really the state of the art right now?

https://redd.it/foxwq2
@r_devops

Avoiding cut and paste Makefiles, ci/cd scripts etc.

I generally try to avoid repetition everywhere, but it tends to creep in with similar Makefiles, and things like .gitlab-ci.yml / travis.yml / etc. in each git repo.

Some ways to address this - use git submodules, or perhaps go monorepo?

What do other people do?

https://redd.it/fp6lad
@r_devops

Designing apps for rolling updates

Are there any resources available for best practices when it comes to designing apps to support rolling updates? I'm having a hard time organizing all the required aspects in my mind. There needs to be some way to handle multiple versions of apis/protocols/schemas/etc. I'm interested in books/articles that provide the "mental framework" required when creating the application.

https://redd.it/fp4a18
@r_devops

Whats the best practices for Jenkins to use an AWS *pem file in an execute shell?

I have a terraform project that pulls an SSH key from \~/.ssh/mykey.pem which it uses for the machines to SSH to bootstrap. How would I offload this to Jenkins being that Jenkins does not have a normal user account and that we should not have Pem keys stored in source control?

https://redd.it/fotwi6
@r_devops

Battle of the Circuit Breakers: Resilience4J vs Istio

Check out this talk from GOTO Berlin 2019 by Nicolas Frankel, developer advocate at Hazelcast. You can find the talk link and full talk abstract pasted below:

[https://youtu.be/kR2sm1zelI4?list=PLEx5khR4g7PKMVeAqZdIHRdOwTM1yktD8](https://youtu.be/kR2sm1zelI4?list=PLEx5khR4g7PKMVeAqZdIHRdOwTM1yktD8)

Kubernetes in general, and Istio in particular, have changed a lot the way we look at Ops-related constraints: monitoring, load-balancing, health checks, etc. Before those products became available, there were already available solutions to handle those constraints.

Among them is Resilience4J, a Java library. From the site: "Resilience4j is a fault tolerance library designed for Java8 and functional programming." In particular, Resilience4J provides an implementation of the Circuit Breaker pattern, which prevents a network or service failure from cascading to other services. But now Istio also provides the same capability.

In this talk, we will have a look at how Istio and Resilience4J implement the Circuit Breaker pattern, and what pros/cons each of them has.

After this talk, you’ll be able to decide which one is the best fit in your context.

**What will the audience learn from this talk?**
The audience will learn about the semantics of the term "microservices", that one of the issue of webservices architecture is that it propagates failure, that the Circuit Breaker pattern can help cope with failure propagation, that both Istio and Resilience4J are both Circuit Breaker implementations, and about their pros and cons. Does it feature code examples and/or live coding?

**Does it feature code examples and/or live coding?**
No live coding, but demos. Repositories are available on Github.

https://redd.it/foqd9d
@r_devops

New Position, Inherited a Patchwork...

I'm at a loss at the moment. New to DevOps with programming skills from the early 2000s! I was promoted from software support. The brilliant person I replaced basically created all of our continuous development infrastructure, hundreds of scripts and linkage over a 12 year period. I received approximately 20 hours of loosely structured training and current documentation is rather swiss cheesy.

Beginning day 1: I had to newly map my workspace and couldn't use the substantial collection of workspaces my predecessor had created, builds began only partially succeeding, output isn't making it back to our Drops locations, our cloud backups fail nightly and have to be ran manually, our custom builds aren't happening. I'm told there's no way it's a permissions or auth issue or an individually assigned token or key, etc. I say there's no way it's all just a coincidence. Especially after finding a set of custom build scripts that call for login credentials that weren't Msft account info.

As I said, I'm at a loss and thinking maybe I'm not fit for the position.

Windows 10, TFS 2012, VS 2019, Azure cloud subscription. And yes, I've been through the MSDN knowledge base and even submitted a support ticket which was minimally helpful. Any suggestions appreciated, tia.

https://redd.it/foproj
@r_devops

What are you using for visualizing your techstack?

What do you use to visualize your tech stack in a good design.

I'm currently looking for a solution that visualizes our microservices in a code form like Mermaid. It could also be a vue app...

https://redd.it/fp9ovw
@r_devops

Keycloak Configuration as Code

In a recent project I had the task to automate the keycloak configuration through a build pipeline.

The mechanism of importing realms isn't sufficient for me because it's hard to maintain and parameterize the config.

So I wrote a tool to manage keycloak with migration files (like liquibase for databases): [https://github.com/klg71/keycloakmigration](https://github.com/klg71/keycloakmigration)

There is also a gradle plugin to embed in build pipelines: [https://github.com/klg71/keycloakmigrationplugin](https://github.com/klg71/keycloakmigrationplugin)

https://redd.it/fopqd1
@r_devops

DEVSECOPS POLICY DOCUMENT

I have been tasked with creating a DevSecOps policy document at work but I do not know where to begin. We use Gitlab and Artifactory for version control and deployment. It must include instructions for developers to submit code to be placed in Gitlab and later be store in Artifactory. Any suggestions on how to start would be great! thanks.

https://redd.it/fovh2r
@r_devops