The ConfigMap Pattern

[Config Maps](https://www.magalix.com/blog/the-configmap-pattern) and Secrets are storages to manage key-value pairs. Secrets are similar to [ConfigMaps](https://www.magalix.com/blog/the-configmap-pattern) but with the additional benefits of actual data encoding.

Here is This Week Article The ConfigMap Pattern.

your feedbacks are welcomed.

[https://www.magalix.com/blog/the-configmap-pattern](https://www.magalix.com/blog/the-configmap-pattern)

https://redd.it/e1fmpc
@r_devops

Setting up Gitlab CI runners on OpenStack

Hi there,

Question on some best practices: I need to set up some Gitlab CI runners for various projects in our org. Most runners need Docker, but I also need some for VMs (Virtualbox). Some require GPU, most won't.

So far I've set up one bare metal box that has GPUs with multiple runners using an Ansible script. I'd like to scale that up. We have a private OpenStack cloud I can use that includes flavors with GPUs. Some of our runners are a bit more needy with their hard drive space usage, so I probably need some simple flavors for most runners, some GPU flavors for the GPU runners, and some more complex runners with additional block storage for the HD-intensive runners (or just use more of them? Ideas?)

I was thinking about using Terraform and Ansible for this task. I think I can setup most of the runners as group runners, so I'll only need to account for a few different project registration tokens, but how do you guys keep that flexible? What if some other project needs a specific runner?

Any other pointers or ideas? Anything I'm missing? (Wouldn't surprise me).

Thanks!

Christoph

https://redd.it/e1b6n6
@r_devops

Best way or ways to give access to internal applications to employees?

I'm relatively new to setting up enterprise systems.

​

Right now we have a kubernetes cluster created using kops, on top of it we have deployed some public production applications accessed through ELB and at the same time we have some internal applications for monitoring (Prometheus, Grafana...), in order to access these internal applications, I've set up an ingress to these services and can be access using a public domain (and of course you need a username and password to access).

​

So far so good, the problem is, I don't like this approach of exposing services publicly accessible through a public domain for security reasons, I looked up for some approaches on how an enterprise usually secures their applications and came up with few approaches.

​

\- Enable bastion box option using kops, give access to the employees to this box, teach them how to use ssh with port forwarding options and at the same time use kubectl in order to forward these services back to their local machines and access the software from there, the problem is, it's very complex, and I'll have to teach some employees how to use such tools which is time-consuming, plus I'm not always available so if they're stuck, they'll have to wait, plus I'll have to figure out how to connect this bastion box to an external authentication system we have, so I won't have to manually add and remove public keys. (plus I'm not sure if kops will configure a new VPC for bastion or not)

​

\- Setup OpenVPN pod inside kubernetes, expose it through ingress, let other employees connect to it, and access applications using k8s services names, which means that the VPN network will be between the pods only, the problem here is that, there's no easy way to configure external authentication to OpenVPN in a pod, plus the cluster is still public, and if someone finds a vulnerability on kubernetes or publicly exposed internal systems (like Grafana) we're screwed.

​

\- Configure the VPC that contains kubernetes cluster private, create a public one, create an ec2 instance on it, connect the public VPC to the private VPC, install OpenVPN inside the public ec2 instance where employees connect to it, then use kubectl to port-forward applications and access them, I'm not sure if this is even a good idea because it still has some level of complexity.

​

What I'm looking for is a way where employees can access private internal applications, with as little headache as possible, I know that VPN is usually the go-to option here, but since we're using kubernetes for almost everything, this becomes a lot more complicated.

​

I've heard about "zero trust", I tried to look this up and got confused, it's like ditch the VPN, make everything public, but add a proxy/gate to everything and verify every single request, and setup roles properly.

​

To be completely honest I am lost over here, I not sure where to look anymore.

​

Should I invest more time understanding zero trust? Should I just go the bastion box route and teach everyone how to use some tools like ssh and kubectl?

​

What do you think is the best approach for such environment?

https://redd.it/e13w3m
@r_devops

To all those supporting e-commerce this week..

I wish you all good luck. May the scaling and load balancers be every in your favor.

https://redd.it/e30t3b
@r_devops

2019 DevOps Black Friday Deals

I was looking for a topic similar to [last year's](https://www.reddit.com/r/devops/comments/9zn95z/2018_devops_black_friday_deals/), but could't find any. Some of those deals might show up again; if you have anything of interest that is confirmed, feel free to post in the comments.

I'll start with:

[Bitdefender](https://www.bitdefender.com/media/html/consumer/new/Black-Friday-2019-opt), 60% off

[Linux Academy](https://linuxacademy.com), 299USD from 499USD

https://redd.it/e34dx3
@r_devops

Salary Range for DevOps in Austin?

Looking to switch from IT to DevOps in Austin, TX. What salary should be expected for a DevOps engineer with 0-2 years experience?

https://redd.it/e33ls9
@r_devops

Deploying an ECS service/task inside a natted subnet

I have an application which is running on EC2 instances right now inside a subnet which is attached to a nat gateway. I'd like to replace the EC2 instances with an ECS cluster/service. What gotchas am I likely to run into? Is there anything specific that I should be doing to make things easier or more secure?

https://redd.it/e32r9f
@r_devops

I think ill be resigning.. but.. i also dont want to.. (also request for resignation letter examples)
https://www.reddit.com/r/sysadmin/comments/e3cjag/i_think_ill_be_resigning_but_i_also_dont_want_to/

https://redd.it/e3d28s
@r_devops

Best Practices For Deployment on Test and Prod Environments.

So Im part of a startup (i am new to dev ops) and we are now trying to figure out the best practices for how we can deploy a CI/CD pipeline for both our test and prod envs. Test prod will be triggered automatically but Prod needs manual trigger.

Anyway, we have a local server which we have a docker swarm running with DAGs(this is used for our Prod Env). We push to bitbucket which triggers a Docker build job and publishes the Docker Image to aws ECR. Now, we have to always manually pull the image on our server then run the env, is there any way to automate this?

second thing, what is the best way to have a Test and Prod Env on the same machine. (we cannot shift to cloud due to security issues with data). Basically what I picture is having 2 docker swarms (1 test, 1 prod) that run side by side (each with their own set of containers). When we push to bitbucket, it runs the pipeline and creates a docker image on ECR which gets automatically pulled and run on our Test docker swarm. When we are happy, we manually trigger another pipeline to pull to our Production Docker Swarm.

Any other way, What do you guys think?

https://redd.it/e3d3ia
@r_devops

Oauth2 with spinnaker

Hi guys,

I'm trying to install spinnaker and i got it to run but I just cant work this oauth2 thing.

Somehow when the browser sends a OPTIONS http request to the gate (spinnaker) at /auth endpoint, it returns a 302 response.


Apparently this is an illegal behaviour and hence everything stops. Can anyone offer advice on how I can approach this?

https://redd.it/e3cnyt
@r_devops

Career path? Need suggestions.

To cut straight to the point, the problem I’m having is that I don’t know what technology or concepts to focus on, that will help push me in the right direction. I recently picked up coding, and absolutely love it. I also recently started using Linux as my main OS.

Although I’m not experienced with Linux yet, I really love working with it. Learning about cd, ls, small commands to move through a terminal, etc. I’m getting a bit off track, but I currently don’t have a job, and am extremely good at self-teaching.

So the question is: For someone who really loves coding, Linux, the stuff I hear about the DevOps space, what technology / concepts should I learn to be able to get me a decent paying job with six months of intense studying? Also where should I start heading for a career with great upwards trajectory? Linux Administration? VMWare Admin? Network Engineer?

Currently have my Security+ certification and am planning to get my RHCSA, followed up with my CCNA, and some AWS certs.

I’m happy to answer some more questions to help anyone zone down on a solid future path.

EDIT: I also understand I can’t focus on getting a “DevOps” job right now because I don’t have any experience.

EDIT 2: I am also aware that “DevOps” isn’t truly a title, but more so a cultural process.

https://redd.it/e30jxv
@r_devops

Career path. Need suggestions.

To cut straight to the point, the problem I’m currently facing is what to focus on (technology / concepts) to get a job currently. I absolutely love coding (Python), but at the same time, I love working with Linux.

From my knowledge, becoming a DevOps engineer isn’t something I can do currently to get into the job market. Reasons being, I don’t have enough experience. I currently also don’t have a job, but am extremely good at self-teaching.

Apologies if I went off track, but my question is: For someone who really loves programming, Linux, and the stuff I hear about the DevOps space, what is a solid career path to get into? What concepts / tech should I learn that could enable me to get a decent paying job within six months of intense studying, that has extremely great career trajectory?

I’m open to answering more questions if that could help you all figure out a solid path to start working towards as well.

https://redd.it/e308p6
@r_devops

Crypto Platform for DevOps

Hey Everyone!

My team and I are looking at repurposing a crypto platform so that it can be used by the DevOps community. Can you help us focus our efforts by answering a really quick survey?

Here is the link: [https://www.surveymonkey.com/r/DevOps-S](https://www.surveymonkey.com/r/DevOps-S)

Thanks!

https://redd.it/e34or6
@r_devops

Kops cluster 'rotting' over time

I have now created two KOPS cluster which seem to rot over time, what happens is that I create the cluster, work on it for a few hours or days, come back a week later and I'm unable to reach the servers. Everything looks fine on the AWS side. I have zero ssh access into the masters as it seems my key didn't get applied when creating, it shows some generated string in the key pair colum on ec2 console.

​

What options do I have for working with this and recovering or preventing it happening again? I want to move to Kubernetes in production but that isn't an option if the cluster blows up weekly with out warning.

​

Command I used to create the cluster

kops create cluster --name=k8s.<my domain>.com --master-zones us-east-1a,us-east-1b,us-east-1c --zones us-east-1a,us-east-1b,us-east-1c --node-count=2 --node-size=t3.medium --master-size=t2.micro --dns-zone=k8s.<my domain>.com --vpc vpc-<ID> --topology private --networking flannel --ssh-public-key "~/.ssh/id_rsa.pub

https://redd.it/e3fg0m
@r_devops

Anyone populating Jenkins Active Choice parameters from a Jenkinsfile via shared libraries?

I'm able to get the activeChoiceParam to run the shared library method but when I return the resulting List I get an `Expected named arguments` exception. Anyone have any experience with this?

https://redd.it/e3dwp0
@r_devops

Handling botocore.exceptions.ClientError: How to handle InvalidObjectState?

I just picked up python as my go-to scripting language and I am trying to figure how to do proper error handling with boto3.

I am trying to handle an exception that occurs when the object being called is on GLACIER. Instead of erroring out, I just want to print a message and make the loop keep going.
I can handle other botocore exceptions (like `if botocore.exceptions.ClientError.response['Error']['Code'] != "404"` for Object Not Found -- which works as expected.

I could not find an error code for InvalidObjectState. I tried these two approaches below without success:


```python
...
try:
storage_class = s3.Object(BUCKET_NAME, KEY).storage_class
except:
storage_class == "GLACIER"
print("STORAGE CLASS IS: ", storage_class)
pass
...
```

```python
...
try:
obj = s3.Object(BUCKET_NAME, KEY).load()
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == "InvalidObjectState":
pass
...
```

The problem is that it always errors out with the message:
*botocore.exceptions.ClientError: An error occurred (InvalidObjectState) when calling the GetObject operation: The operation is not valid for the object's storage class*

https://redd.it/e3kstl
@r_devops

Is a college degree needed for this type of career?



https://redd.it/e3lb5b
@r_devops

Workflow with Docker, Docker Compose, AWS ECR and Ansible

Trying to figure out the best workflow here. I inherited all that stuff a few months ago and I'm still learning.

For the past years until today:

1. 5 related services, each one has its own docker-compose.yml defining a single service
2. I deploy by: ssh on a server > "git pull" x 5 > "docker-compose up --build -d" x5 > repeat on the next server

Edit: the servers are bare metals, they are not on AWS.

---

Now I understand a bit more what's the right way to use Docker:

1. I should build the 5 images once and push them in a registry (I'll use AWS ECR)
2. I should have a single docker-compose.yml defining 5 services based on the images pushed in the registry
3. I could use Ansible (which I already have to provision/configure the servers) and the [docker-compose module](https://docs.ansible.com/ansible/latest/modules/docker_compose_module.html) to deploy on the servers

Am I right so far? Any suggestions?

(once this works, I guess the next step could be to add a CI/CD to build the images and deploy the containers automatically)

---

I still have some doubts tho:

1. should I merge the 5 git repositories in a monorepo to create the single docker-compose.yml in the same repo?
2. should I keep 5 repositories and define the docker-compose.yml in my Ansible project instead?
3. if 1, how do I access the docker-compose.yml from my Ansible project?
1. Do I copy/paste it in the Ansible project? (ew! not DRY, 2 sources of truth, risk to forget to update one)
2. Do I have to clone the monorepo on the Ansible control node to access its docker-compose.yml? (meh!)
4. if 2, how do I run the services in dev? Do I have to apply the Ansible playbook on my own machine? (meh!)
5. should I create a new repository only for the docker-compose.yml? (meh! introduce one more dependency…)

**TL;DR: where do I define the docker-compose.yml to make this Docker/Ansible workflow works?**

Thanks!

https://redd.it/e3j3ko
@r_devops

Outsourced company, do we need to replicate their setup

Hi,

Don’t wanna get into an A vs B argument, but could someone with actual experience advise and clarify please.

I work for a large company, we’re a team of 2 developers and 1 data guy. All our work is coded on Visual Studio and held on Azure DevOps Repos. We’ve a pipeline that takes the repos and pushes to an EC2 instance via CodeDeploy, all in Azure DevOps.

Works perfectly, never had a days problem.

We’ve had to outsource a large project to an external company, very well respected, and their setup is Jira with BitBucket and CircleCI to a serverless setup.

Ultimately, we’ll go serverless rather than EC2.

However, our Tech manager wants us to stop using Azure DevOps and go the same way as our outsourced “for about a year” as he feels working together and when the time comes for us to take over the project, we’ll have lots of problems.

Now I’m no expert on this, but surely if our outsource company is using BitBucket, we can jump into their repository quite easily, and any changes we/they make will be shared between both companies with ease.

Likewise, while we don’t use CircleCI ourselves, we could easily add their pipeline config in Azure DevOps (even add CircleCI itself) and again replicate the pipeline (via a 10 line YML config)

Am I wayyyyyy off or can it be done?

Like I said, really don’t want to get into a Jira+BitBucket+CircleCI vs Azure DevOps argument, just under a firm belief that both companies can easily stick with what they have.

Thanks.

https://redd.it/e3kjww
@r_devops

DevOps Career Question

Hello Team : I am been having challenges to find a job in IBM AS400 System Administration and would like to upgrade my skills to new technologies such as DevOps or Cloud Engineer.

(1) Would you please suggest what would be the best path for me specially I am experienced in System Administration and familiar with computer operations and SDLC.

(2) Any comments or suggestions on Edurekha or Udemey online course site that provides certification in DevOps or Cloud ?

Thank you.

https://redd.it/e3j4c6
@r_devops