Frustrated with trying to do configuration management on Jira

Is anyone else really frustrated with trying to do configuration management on Jira Server, especially while trying to do configuration as code? It just seems like trying to automate some things seems all over the place, sometimes you have configuration files, sometimes you need to do it via the REST API, sometimes you need to fiddle with some database tables it just seems that everything is so inconsistent.

The worst offender of this seems to be plugins, we tried to implement SAML SSO via a plugin but automating that setup seems outright impossible.

Is there anything obvious I'm overlooking here or do other people have troubles like this too?

https://redd.it/g6ob66
@r_devops

Difficuly of upgrading hashicorp vault oss to enterprise

I am looking at setting up hashicorp vault. We are thinking of starting with the oss version and then later moving to enterprise. Is the upgrade procedure straight forward?

https://redd.it/g6mvlx
@r_devops

Tools for Terraform and Azure testing

Hey All,

What tools are you using to test your Terraform meets expectations in an Azure environment? I see tflint has a plugin but its just started development - anything else you guys use?

Cheers

https://redd.it/g6jqj6
@r_devops

Jenkins integration with Jira

Hi team,

Did anyone tried integrating jenkins with Jira. I saw couple of old video on youtube. However its not working for me. Can some please point to right link or how to video.

https://redd.it/g6o6y9
@r_devops

Communication between private AWS resources: Route53 and Security groups

My network-fu is not the hottest and I'm running into some problems with my AWS stack, hoping the community here can point me in the right direction. My problem is my security groups appear to be blocking traffic that uses domain names for addressing instead of raw IPs. For context, I am using ansible to provision and manage my AWS resources.

Basic picture is that I have a publicly available load balancer which proxies traffic to private app servers, which talk to a private DB instance. I have security groups roughly set up so that the load balancer is in a SG called `sg::load_balancer`, the app servers are in `sg::app_server` and the database in `sg::db`, so that `sg::db` instances only accept incoming traffic from `sg::app_server` instance which only accept traffic from `sg::load_balancer` instances. Now this works fine if all the network calls actually use the IPs of the relevant instances, i.e. if the app servers make networks calls directly to the DB IP, and the load balancer's nginx config has the app server IPs listed as upstream servers. However, I would much rather have semantically meaningful domain names in my configs (`app-server1.mydomain.com`) instead of templates with IPs injected into them at provision time. It kicks the can to having to update Route53 with the new IP for a given domain when provisioning new resources but it feels cleaner to me.

The problem is that when addressing between instances using domain names instead of IPs, the security groups block the traffic. I don't fully understand the situation, but I guess when using domain names, the traffic is proxying through the Route53 servers and so the incoming traffic to private instances is no longer coming from the machines in the specified security groups. So, is there a way for Route53 to play nice with my security groups or is there a better strategy for this altogether?

https://redd.it/g6o531
@r_devops

any recommendations for managing storage in a private cloud environment?

since adopting a more "devops-y" culture with engineering & sysadmin, our infrastructure team is struggling to keep up with storage demands now that we've adopted a higher velocity through devops collaboration, particularly around managing the storage to be used for clustered databases. unfortunately the collaboration is being done between engineering & sysadmins, but the infrastructure team still wishes to operate autonomously, so i'm trying to help enable them to keep up by solving some of their pain points and routine tasks.

we primarily use nimble & netapp storage platforms. i'm looking at their native tools and APIs, but i'm wondering if there's a better way to approach this.

initial search for terraform seems to be lacking providers for the on-prem offerings from what i can see. ansible has some good stuff for netapp, nimble seems to all be community created from what i can see.

https://redd.it/g6ng6e
@r_devops

Overriding Prometheus alerts

Hey, we've been struggling with overriding alerting rules in our Prometheus monitoring setup.

We have a set of generic rules that applies to all cluster resources across multiple environments and wanted to alter or disable alerts for a few subsets of resources.

Take a look at how we've solved it [https://medium.com/p/how-we-solved-our-need-to-override-prometheus-alerts-b9faf9a4558c?source=email-fe7d0a36f5f9--writer.postDistributed&sk=7eb8d3b8d2c0c6fd710a5f19cc649399](https://medium.com/p/how-we-solved-our-need-to-override-prometheus-alerts-b9faf9a4558c?source=email-fe7d0a36f5f9--writer.postDistributed&sk=7eb8d3b8d2c0c6fd710a5f19cc649399)


Feel free to post other ideas and suggestions.

https://redd.it/g6jz30
@r_devops

Tools for writing CI tests

Hi everyone.

I know this is a very broad topic but. Can people please like list tools or frameworks that they use or know of to make tests for CI pipelines. And maybe list what languages they are commonly used to test for. I know for some like selenium and behave. But would like to see what others there are. So I can make a better decision.

Thanks, Jpac14

https://redd.it/g6m5jk
@r_devops

[Article] I just finished writing my second medium blog post on Writing a Custom Concourse Resource - The check.

I just finished writing my second medium blog post on Writing a Custom Concourse Resource - The check. Please check it out! The more people read it the more motivated I will be to write the next posts ("The in", and "The out")!

https://redd.it/g6lr99
@r_devops

How We Build a Release in One Minute

Any team working on a growing software project eventually has to adopt more internal infrastructure to manage the code’s complexity and the development team’s size. I'd like to show you our article about how our team arrived at our current production build toolchain that helps us to build a release in a mere minute - [https://flussonic.com/blog/news/how\_we\_build\_a\_release\_in\_one\_minute/](https://flussonic.com/blog/news/how_we_build_a_release_in_one_minute/)

https://redd.it/g6kqb4
@r_devops

Any up-to-date books about Jenkins and Docker?

Hello,

I've been studying Docker and Jenkins for a week and a half now and I'm pretty comfortable with Docker in general. However, the book by which I am studying is about 3 years old and is made for Jenskins 2.36 (I think?) and some features are either missing in the latest Jenkins version or work in a completely different way, forcing me to troubleshoot or blindly guess which options I should use or what I should do.


Any book recommendations would be very appreciated!

https://redd.it/g6jm7b
@r_devops

What’s the cheapest way to get dashboard metrics on lambda functions?

I built a grafan instance to monitor lambda functions across accounts then attached it to cloud watch metrics and then realised it would be expensive.
Is there anything I could try that might be a cheeper solution?

https://redd.it/g6ivqn
@r_devops

On-prem: How would you clone a k8s cluster

Situation: I want my prod cluster to exist on another site, the hypervisor might change, but the cluster services must still be the same.

The cloned cluster will process the same data schema, its just different sites should have different clusters (so that means absolutely not extending the prod cluster to that site)

What I already did:
1. Packer to generate machine images - can export to vmdk/vdi/others
2. **Ansible** - specifically, **Kubespray**, when I need to deploy to a site I just need to get 3 IP addresses (plus another 3 for metallb)
3. use the clusterinfodump from the main cluster to deploy services to replicate the state of the prod machine


I know there are better ways to do step 3 like templating using helm charts, am in the process of cleaning it up so that I wont use the clusterinfodump anymore.

have you solved this in your environment?

**How would you assure the fidelity of site clusters to your core cluster state?**

^this is not an interview question, I have actually done steps 1 and 2 (in a way 3, when I borked the cluster), and after passing tests I'll need to have some idea on 3, I'm basically priming myself for information now on how to do it properly.

https://redd.it/g6fm9n
@r_devops

New to DevOps and looking for mentor/buddy

I am new to devops..I am working in Kubernetes,AWS,EKS,GitOps.. I am looking for mentor or buddy to whom I can ask my queries or guidance..Please let me know..

https://redd.it/g7dwkf
@r_devops

Junior devops job hunting

Hi everybody, I have been submitting applications for an entry level devops job for a while now but they all revolve around your profile is good but we won't go forward with you.

I haven't even passed the first phase yet and i think it's because my cv is heavily revolved around web projects.

I studied a technical diploma which gives us a lot of projects and labs to make. most of them are about backend web development and a few with the devops tools like terraform jenkins docker etc and i am always applying the devops cycle on each web project i make. And naturally i put those web projects in my cv.

Any idea what am i doing wrong or have advice on what to do. Some ps: i am from egypt and i apply both in the country and in europe. I always make sure the company accepts foreigners and clearly say that they will help with visa etc. Even in egypt i can't seem to pass the first phase.

[The cv if u have any advice](https://drive.google.com/open?id=1wfGdRtk89b1tS2lHziKQgGvfvL6YPDtq)

https://redd.it/g7dtvw
@r_devops

How to implement ephemeral dev environments

Hello!

Just wanted to have some feedback from the field about how to implement ephemeral dev environment.

The purpose of this, is to provide a temporary environment for a developer in order to perform certain tasks. IE: deploy backend servers, run some jobs...

The difficulty resides in trying to detected when the environment is not used so it can be safely shut down in order to avoid extra costs. It also needs to alert the developer in order to ask them if they need to extend the time.

There are a lot of automated way to scale up and down already, auto-scaling groups, auto-scale in AKS already, but I cannot find an easy-ish approach to this.

My target systems are on kubernetes and aws (ec2/rds...).

There is a kubernetes project [https://github.com/hjacobs/kube-janitor](https://github.com/hjacobs/kube-janitor) but it does not quite fit what I want to do.

So I'm actually wondering if anybody implemented something similar!

https://redd.it/g795wy
@r_devops

PoP: Special Episode: Considerations For Moving An Engineering Organization Fully Remote

The next episode of the [Practical Operations Podcast](https://operations.fm) is now available. We interview Ashi Sheth, the Senior Manager of Global End User Services at LinkedIn, to talk about the logistical and personal considerations of moving a 20,000 user organization to full-time remote work with very little notice.

Find this episode [here](https://link.chtbl.com/QYAOBFMc), or wherever you find your podcasts!

https://redd.it/g78zgf
@r_devops

CI/CD Best Practices? Am I on the right track?

Theres a couple design options I've been considering when thinking about the CI/CD pipeline, we run a kubernetes cluster and pipeline will be in Gitlab. Some of the design options I've been debating are the following (by I like the best to least). If anyone has any tips, constructive critism, or good documents to point me in the right direction it'd be a lot of help!


1. CI/CD Pipeline in each service repo, new tag built will be built and deployed (if meeting correct tag & branch criteria). This will only update/deploy the new deployment type on k8s with the new tag (using ansible, taking env vars that are set in the repo for each file). All other K8s resources (secrets, ingress, service) that are seldom changed will be managed in a central repo (for fast disaster recovery).
2. Same as above, but have all resources within each repo (each service repository will deploy the ingress, service, secrets, and deployment types).
3. One centralized repo for all configuration management (service, deployment, secrets, ingress in k8s) changes, including deployments, the tag of the build/env vars will be updated in a configuration file for deployment.

Let me know what you guys think.

https://redd.it/g7atks
@r_devops

How to access services exposed on internal load balancer with nginx ingress

I have an EKS cluster with only private subnets, So i tried to expose a service on internal load balancer by adding this annotation to the nginx ingress controller service :

**service.beta.kubernetes.io/aws-load-balancer-internal: "true"**

the porblem is that i cannot access the hostname defined in the ingress resource e.g [my-hostname.com](https://my-hostname.com) ) with vpn. but when i expose my service directly on internal load balancer with the above annotation and without nginx ingress i can access it through the randomly generated url ([internal-xxxxxxxxx.eu-west-1.elb.amazonaws.com](https://internal-a1199d09d80a511ea8ede0ad0e990f58-432110363.eu-west-1.elb.amazonaws.com)).

https://redd.it/g7awx0
@r_devops

Implementing CI/CD pipeline/s in a microservices architecture

Hello everyone,

I'd like to use your experience with using different CI/CD tools and different CI/CD methodologies to try and figure out what's the best approach and technology at tackling a web application with numerous microservices using different technologies as opposed to a monolithic application/package.

​

My main worry is that once I'll pick a tool or methodology to implement that would be more suited for a monolithic/small application that it would require a lot of maintenance and be a burden and a hustle to replicate across multiple services instead of actually helping the development process and making things more efficient.

​

Thanks in advance!

https://redd.it/g7gw3q
@r_devops

Failover Conf Wrapup

Hi,

I wrote a blog post about Failover Conf, which happened on Tuesday, It was a really fun event with great talks. You can read it here:

[https://www.firehydrant.io/blog/failover-conf-wrapup/](https://www.firehydrant.io/blog/failover-conf-wrapup/)

https://redd.it/g7hf03
@r_devops