cache problem with jfrog Artifactory HA on AWS

Hello devops,

does anyone implemented jfrog artifactory HA on aws environment ? if so how did you implement the chaching of the efs server on both instances ?

https://redd.it/f258sv
@r_devops

Logging service for QA environment (simple, fast)

We are looking to making our QA's job little bit easier by having our application(s) log into some kind of logging service with web UI that shows a log tail of currently running services.

I've looked at Azure's Applicaction Insights and while it does have some features that would be useful for our QA, I find that it isn't tailored for use case when QA wants to immediatelly see logs and metrics from last request she made against service she is currently testing.

Any suggestions for logging service that is :

* Simple to deploy
* Simple to use
* Shows immediately all requests made, with all log entries under that request
* Doesn't need to scale much, as there will be little of actual throughput

https://redd.it/f24etw
@r_devops

What to expect in a DevSecOps Role

Received an entry level SWE job offer from a big defense contractor about a month ago and accepted. Today I found out i was put on the DevSecOps team. Does anyone have any experience working in such a team and what the responsibilities might include?

https://redd.it/f2tyv1
@r_devops

How do you started using IaC and conf mgmt tools at your org?

We need to start from zero. The difference between the desired and actual state is super big right now and I don't want to reinvent the wheel. Currently our cloud footprint is quite small and I'm converting everything already created into TF config files, just that; next I would like to create modules from that config.

Some personal experience, workflow or reference would be great. Also...it becomes easier while you advance or just change the kind of problems you have?

https://redd.it/f2txdl
@r_devops

CI/CD for Terraform environments

Hi all,

I was wondering if anyone has encountered the same problem as we have in my team - we have fairly large environments on both Azure and AWS (scaling up to 100s of resources, lots of configuration). Deployment is done 100% via Terraform, including workspaces split for multiple environments (Dev, Integration, PreProd, Prod) and some custom modules we have. All Terraform code is stored in a Git repository with full version control, where about 5 engineers are working on the same environment simultaneously on average.

Issue is, when we started talking about CI (that was the first step) for our infrastructure, we couldn't find any workaround to deployment times - some resources take up to 40 minutes to deploy (for a single resource), which makes it practically impossible to run any kind of testing on the environment (the most basic smoke tests).

Has anyone implemented some CI method for large infrastructures? How did you manage to run a large number of tests (which take time as well, in addition to deployment time) on the environment? Any kind of help would be greatly appreciated.

Cheers.

https://redd.it/f2t3zn
@r_devops

Products to help manage python dependencies?

Is there a product that can act as an internal central package repo for pip? I'd like to control what libraries are available to which systems instead of just opening everything up to the public repo.

https://redd.it/f2rncv
@r_devops

Jenkins pipeline shared library vs parameterized builds

A little (oof... this turned into way more than a little) background on my situation is definitely warranted. I work on a team who is tasked with "modernizing" a lot of the deployment practices in an attempt to move to a Push On Green model for a decently large number of teams in my business unit (~70 teams and service deployables, I think).

We're a fairly large tech org, and have a fair amount of internal infrastructure that orchestrates the rollout to production:

* large "kitchen sink" jenkins shared library
* in-house artifact repository
* in-house deployment manager

Every project defines several jenkinsfiles for various tasks (updating libraries, handling branch deploys, build->QA->Prod workflows, etc)

Due to the combination of "kitchen sink"-ness of the shared library, as well as a considerable amount of organic growth, very few of the deployables are configured the same. You can tell what era a given project was written in by the rough skeleton of the jenkinsfiles it has. There's quite a lot of copy/paste code as teams have incorporated HOTFIX logic in some places, canary deploys in other, etc. And a slew of deprecated library calls that nobody has bothered to go in and update. It also requires a fairly intimate knowledge of the shared-library to do any non-trivial work.

One other drawback of the current system is that there is a fairly tight coupling between the build step and some post-deploy actions. In particular, stuff like JIRA ticket ids get set in the build step, and then comments/links are updated after the deploy.

This all makes iterating on modernization an extremely painful process, since quite a lot of time is spent waiting for a build to complete even though re-using a pre-built artifact would have sufficed.

Additionally, the team that manages the shared library seems to have realized that the current situation is pretty bad and is building a new version to Fix It For Real This Time. But really, it already does look quite a lot better. It's just not ready yet.

Ultimately, it seems like there's quite a lot of opportunity to attempt to standardize on some deployment models.

One thing I'm considering is setting up a handful of "Push On Green Deployment Jobs" that would accept as parameters: a job to build and publish the artifact (manged by the dev teams), a job that runs whatever tests the team wants, post-deploy monitors, maybe some canary logic, etc. To me, this has the advantage of hiding how the sausage gets made a bit and centralizes deploy logic into one place. Using parameters enforces a looser coupling of the build steps, so it should be easier to convert to the new library once it's ready. These jobs would be triggered by the deploy.jenkinsfile of the deployable project.

The other approach, and one that some other business units at my company have used with mixed results, would be to roll our own deployment library that exposes these same tasks, but lets them do whatever customization they want.

Ultimately, I feel like some teams will always want to manage their own deploys, and that's certainly up to them. But I'd like to do as much as possible to let them focus on the things they should care about (building, testing and monitoring) and not concern themselves as much with the deploying/rollback/hotfix details.

Anyway, I'm mostly just interested in whether anyone has had luck leveraging paramaterized builds as a way to manage and centralize this sort of logic.

https://redd.it/f2l4la
@r_devops

Who are you other than your job title ?

Do you have a side gig / What do you do for fun in your free time to break monotonicity of DevOps role and be purposeful ( earn some bucks maybe ) ?

https://redd.it/f2l052
@r_devops

Graduated with CIS degree. IT Help Desk -> Junior DevOps or Entry level cloud positions?

Help Desk to Junior DevOps?

I graduated with a Bachelors Degree in Computer Information Systems and spent the past 3 years in IT Support / Help Desk roles. I gained a great bit of knowledge and experience working in those roles as an intern (at a Fortune 500, at a Wealth/Finanace Company, and at a MSP).

I was lucky enough to accept a great job out of college doing IT & Software Support. About 5 months in, I realize that my heart really isn't in doing Tier 1 type Help Desk stuff (resetting passwords, program installs, recovering emails). I am starting to work with the software support team and realize all of these guys have a lot of coding knowledge in work in DevOps type roles. I figured I worked too hard in college and spent most of that time busting my butt interning to just be complacent with a job doing low level work. I am thankful for where I ended up... but I want to continue learning and climbing the ladder rather than be complacent.

My goal was to start studying for the AWS Certified Solutions Architect - Associate certification. After that, I wanted to grab a better understanding and study up on Linux (I have minor knowledge in Linux atm). After I got those 2 down - I wanted to dedicate my time learning Python and automation.

Is this a good learning track towards a Junior role? Has anyone moved out of a Help Desk role into a Junior DevOps role?

All suggestions help!! Thanks!

Resume: https://m.imgur.com/gallery/u51h9wa

https://redd.it/f2j64q
@r_devops

Scale testing

Hello everyone, I am working on a side project and I was curious on what people do to test large scale infrastructures/deployments. What do people do to do this?

https://redd.it/f317n4
@r_devops

Monitoring tools: Graylog

Heey guys!

I'm using Graylog as monitoring tool, so I'm collecting some logs, graylog can alert if the query used in the event definition matches with some incoming log.
So here is my doubt: Can Graylog alert if there is no incoming messages in a while in a input???

https://redd.it/f2zxlq
@r_devops

Feedback on open source monitoring project

I've been working on an open source monitoring project along the lines of Prometheus, but takes a different tack by statistically modeling your metrics so that you don't need to define complicated alerting rules to detect when something has changed. It works for latencies, error rates, distributed traces, etc.

The idea is to take advantage of statistical modeling to detect when things start to go wrong before it would be obvious from looking at Grafana. This lets you intervene earlier before the whole thing falls over. A side benefit is that the tests can be tuned to give only a small probability of a false alarm, without you needing to handcraft a bespoke alerting rule.

Here are my goals:

**No configuration** \- No YAML config files, no need to pre-register your metrics, no alerting rules to define. It scans your structured log output to find your metrics, monitors them, and sends an alert when something changes (and also figures out when something returns back to normal on its own)

**Stats are better than graphs** \- We know how things like latency and error rates can be modeled statistically. Monny models your metrics statistically to detect when a change in these metrics is significant, without the need to look at graphs or figure out the alerting rule yourself. It can detect small changes that would lead to lots of false alarms if done in something like Prometheus.

**Simple deployment** \- Single binary client and server that reads your application logs and finds your metrics automatically. It can monitor things like latency, distributed traces, memory consumption, CPU utilization, and error rates. Works with Kubernetes, bare metal, Docker, and whatever comes next. No external database required, making it easy to run yourself.

**Advanced alerting** \- Send alerts to email, text, Slack, and many more. Get alerts only when something needs human intervention. Only want an alert when less than 2 of 5 processes are functioning normally? No problem. Want to silence an alert, snooze it, or send it to someone else? It has an email or slack based workflow to deal with alerts right where you get them.

**Only the context you need** \- Alerts aren't just metrics, but come with log context so you can see what led up to the alert. You don't need to run ELK plus Prometheus, it's all combined together in an intuitive UI so you can figure out what's wrong, fix it, and get back to what you were doing.

It's on [github here](https://github.com/BTBurke/monny), and I'm looking for beta testers, but any feedback would be appreciated.

https://redd.it/f2y9yp
@r_devops

In which language would you create your cloud native application in 2020

I plan to work on a new open-source project designed for a kubernetes cluster.

There will be a lot of interaction with the kubernetes api and third party applications mostly databases.

I thought about go cause kubernetes, terraform and all that stuff is written in go.

My colleague goes for python.

How to convince him (or me)?

https://redd.it/f36r05
@r_devops

how to remove the extra line in the formula bar ?

excel has extra line break on the formula bar, how to remove it? when i click the cell, on the formula bar i see white space but when i click dropdown button i see the data at the next line. how can i remove this extra line. i have three cells, only two cells has extra white space line but the third one does not. i am using ansible and lineinfile module to get the data and exporting to excel. i dont think it is ansible because the rest of the data is fine.

https://redd.it/f2wrd7
@r_devops

Alert Firing times for APM tools

One thing I can't ever seem to find is how long it takes from an event (anomaly, threshold exceedance or something else to be detected and an alert triggered. I'm not concerned with the propagation (I know email is slow, etc). So how long from does AppDynamics take to detect and trigger an alert (versus, say, Dynatrace or Instana).

https://redd.it/f2w5xm
@r_devops

An Awesome Terraform List

So browsing around looking for some resources I stumbled across this pretty amazing site. [https://project-awesome.org/shuaibiyy/awesome-terraform](https://project-awesome.org/shuaibiyy/awesome-terraform).

It does have a lot more other resources if you spend more time on it. But This does have a pretty good collection of Tools, Videos, Documentation and other things that would help with people's Terraform Journey.

https://redd.it/f3m4rt
@r_devops

How to document your CI/CD chain?

Hello, I have a simple question, I have Three environments, with a Kubernetes cluster and bench of AWS resource for each env, deployed using Terraform, the CI/CD chain has Many Pipelines, we also use Jfrog Artifactory and Github, I need to document the work, and don't know how to summarize and document all this infrastructure and ci/cd chain, I have searched online, but I haven't found anything useful.

So my question is how do you guys document your CI/CD work for future colleague? and how would you structure it?
Thanks in the advance

https://redd.it/f3bmj8
@r_devops

From Docker to OCI: What is a container?

Let's review the basics: What are containers suitable for, and how do they work? I get a lot of questions about containers and Docker, so I wrote [an article perfect for anyone new to DevOps](https://www.padok.fr/en/blog/container-docker-oci). Check it out and tell me if it seems more accessible to you.

https://redd.it/f3b9na
@r_devops

Jenkins Reporting to Bitbucket

Currently I have an automated Bitbucket webhook process that triggers a Jenkins suite of unit tests.

I would like to have my Jenkins job send back to Bitbucket if the job build passed or failed.

Suggestions?

https://redd.it/f3e5w9
@r_devops

Few thoughts on how container technology has influenced software development

[https://medium.com/@rvprasad/why-not-containers-4597e266aa1f](https://medium.com/@rvprasad/why-not-containers-4597e266aa1f)

https://redd.it/f3ckfk
@r_devops