Where to learn the CD in CI/CD

Hello I've been working in the CS industry for just over a year now and I've been working a bit with docker and docker-compose, I'm still relatively new the technologies but I think I've got the hang of things for basic stuff.

I can create automated pipelines to schedule jobs that build / test my code, however I have zero experience in deploying these applications to our production servers.

In order to learn I'm currently I'm trying to develop a personal project which runs on my Raspberry PI on my LAN. Ideally I would like to have a fully automated CI/CD pipeline that once I merge my changes to master it will run build/test stages and finally deploy the code to my Raspberry PI.

Right now, the CI part I'm fine with, but when I read up on how to deploy my docker app I see a lot of posts about Kubernetes and Docker Swarm. I'm not familiar with either but when reading about them, these two technologies seem super overkill for what I want to achieve, I currently only have a single docker container that I want to deploy to a single server.

Is there somewhere I can read to learn about how to go about doing this?

Thanks for reading

https://redd.it/e4lnnd
@r_devops

how does testing change when moving to canary deployments

How does application testing during the deployment phase change when going to canary deployments?

​

I'm referring mostly around things like integration testing, performance testing, config testing, smoke tests etc

https://redd.it/e4peim
@r_devops

So, do you think you're ready to SRE?
https://www.reddit.com/r/sre/comments/e4jf56/so_do_you_think_youre_ready_to_sre/

https://redd.it/e4jv6n
@r_devops

SSH: RSA keys, and ssh-agent for SSH keys and their passwords management

During keyring configuration for the Nextcloud client (see the Linux: the Nextcloud client, qtkeychain and the "The name org.freedesktop.secrets was not provided by any .service files" error post) - I decided to clean up the mess in my SSH keys, as I have a lot of them and sometimes authentication became just pain.

[https://medium.com/@setevoy4/ssh-rsa-keys-and-ssh-agent-for-ssh-keys-and-their-passwords-management-61c68001f54b](https://medium.com/@setevoy4/ssh-rsa-keys-and-ssh-agent-for-ssh-keys-and-their-passwords-management-61c68001f54b)

https://redd.it/e4hzyz
@r_devops

What do you guys think about this curriculum for somebody who wants to get into cloud engineering?

Created it 100% from Linux Academy.
Disclaimer: This is combined with side projects, with the goal of landing a cloud engineering job to get professional experience.

Linux Academy Curriculum:

Junior Linux Administrator - Entry Level

Junior DevOps Engineer - Entry Level

Junior AWS Cloud Engineer - Entry Level

AWS Cloud Engineer - Mid Level

During this progression, the certifications to get are the RHCSA and CCNA for fundamental Linux operating system knowledge, and fundamental networking knowledge.

Also, programming (Python) on the side.

https://redd.it/e4u6ao
@r_devops

I was just hired as a vRealize/vOrchestrator automation engineer. How “in demand” are jobs like these?

I was laid off a month ago from a small company where I was learning automation with Ansible and semi-digging into DevOps tools while also building off of my systems engineering experience.

I scrambled to find a job and I landed this one where they are looking to have me automate their infrastructure on a private virtualized cloud built on the back of vRealize and vOrchestrator. I have only been researching these applications recently and while I am anxious to dig into something new, I am hard pressed to find any DevOps coorelation online. DevOps seems all about public cloud, AWS, Azure, etc. but I want to know if I can build and further a DevOps career with this experience or if I am going to be spinning my wheels with this particular work.

Anyone have any familiarity and can maybe help me see how this can benefit my DevOps career in the long run?

https://redd.it/e49irz
@r_devops

Poll: CI/CD tools in reality

I know that there are quite a few tools which state they are CI/CD, or at least enable you to do CI/CD with them. But I'm curious to know what the community is leveraging and what the reality of that solutions looks like (setup, maintenance, administration, scalability, etc.).

Let me know!

https://redd.it/e50vue
@r_devops

What's your favorite feature flagging tool?



https://redd.it/e50rpa
@r_devops

Cleaning up and organizing a huuuuge email inbox. Anybody done this? Success stories?

Hello,
My email inbox is out of proportions and I am looking for suggestions. One of my ideas is to export all email headers to a database to run some queries to know top X senders sorted by number of received emails and the stuff like that. After I know a big picture I believe it's possible to make automatic rules to transfer emails to a different folders based on conditions, such as sender, year and the like.


Do tools to export email headers to a database exist? Or can you suggest some desktop/self-hosted web client that stores retrieved emails in an DB with accessible format to query tables directly?


Thanks.

https://redd.it/e4z222
@r_devops

suspicious-pods: easy debugging for big k8s clusters

Hi /r/devops! I recently wrote a tool that is making my life much easier when it comes to debugging Kubernetes clusters, and I thought I'd share it here in can make someone's life easier too.

Suspicious pods is a very simple tool, which does a very simple task: print a list of pods in your Kubernetes cluster that might not be working correctly, along with a reason on why that pod is considered suspicious. I find it is useful in big clusters, where you need to hunt down problems with several iterations of \`kubectl get po\` and \`kubectl describe po\`.

This tool reduces the huntdown process to a single step. Enjoy!

[https://github.com/edrevo/suspicious-pods](https://github.com/edrevo/suspicious-pods)

https://redd.it/e4uj67
@r_devops

UPDATE: I want to teach someone DevOps principles and tools

Update to [my previous thread](https://www.reddit.com/r/devops/comments/e45hft/i_want_to_teach_someone_devops_principles_and/)

​

After the barrage of requests, I have accepted to work with 2 respondents personally. However, as many have asked, I have created a Discord server for this initiative so I can personally help as many people as possible.

Please join the server here: [https://discord.gg/A5Ts88](https://discord.gg/A5Ts88)

The link will be open for 24 hours and limited to 50 sign ups. This is because I would like to be involved with this personally and I don't think I can handle any more than 50 people (if that).

The proposed format is:

* general questions and answers will be open nearly full time
* I intend to run a small workshop every week or so (depends on availability). The topic will be elected democratically by the community. If I feel I am unable to competently speak about the topic in question, I will try to find someone who is.
* I also intend to have 2 hours of office hours every week. At this time, I will be available either via text chat or voice chat (TBD) where you can AMA.

​

To reiterate:

* This is not a training course or a bootcamp of any kind. This is just one person, me, trying to help as much a possible
* If you would like to follow a structured course on DevOps, I suggest [**u/brooksa321**](https://www.reddit.com/user/brooksa321/)'s course [here](https://www.reddit.com/r/devops/comments/e2ags6/ran_a_free_12_week_devops_bootcamp_on_twitch_here/)

​

Hope this helps!

https://redd.it/e52o7p
@r_devops

devopsdays nyc 2020 tickets are now on sale (earliest bird pricing!)

Check out our March 3-4, 2020 event in NYC!

[https://www.eventbrite.com/e/devopsdays-nyc-2020-tickets-83067223301?ref=estw](https://www.eventbrite.com/e/devopsdays-nyc-2020-tickets-83067223301?ref=estw)

We have tickets priced at our lowest, $150

https://redd.it/e57g7q
@r_devops

Unique Christmas ideas for my team?

I manage a team of DevOpsy folks who've done really well this year and would like to get them all a nice holiday gift. Thinking something personalized (perhaps laser engraved?) and tech-related. Any ideas? Looking to spend about $50-100 each.

One thought was a nice USB power pack with our team/company logo, their name, and their start date engraved on it. Maybe that's a bit cliche, but looking for other ideas that fall into that type of realm.

https://redd.it/e58ign
@r_devops

Netdata release v1.19

Hey all,

Introducing a major rewrite of our web log collector, [cmocka](https://www.facebook.com/hashtag/cmocka?source=feed_text&epa=HASHTAG&__xts__%5B0%5D=68.ARDKJpuwODv-hPc236C7fM2r9vCYx9vTJeH93a18odECvej08YhKL8yikqtkH5YC0qIPbakDgFmmf15tnogjIj3cf_Wya85PQYEzfw9Y2OUmmX9y_0GfH_ikrxDZS109LzcjMTWo8zoUXSy893FBfR0_AxyBw6IFCcIkBObQiTOtCIjrxws3vt14_Jg5s_H61VJwB_KoiG9hPRXpuA6vV5Vm9rUSZUwM0KpucD6_6XrlRI2xwd7Sz5M_3Arnkqr4DckHC5NXNGGTVMUYJgz2H3mq1KcdwguQvlsigL2Pz4k6QW1ZCCYCvG21xk1UzcDAEEr15Yqxx-AEGa1AlNoxRqzZc8MM&__tn__=%2ANK-R) unit testing, improvements on our [Unbound](https://www.facebook.com/hashtag/unbound?source=feed_text&epa=HASHTAG&__xts__%5B0%5D=68.ARDKJpuwODv-hPc236C7fM2r9vCYx9vTJeH93a18odECvej08YhKL8yikqtkH5YC0qIPbakDgFmmf15tnogjIj3cf_Wya85PQYEzfw9Y2OUmmX9y_0GfH_ikrxDZS109LzcjMTWo8zoUXSy893FBfR0_AxyBw6IFCcIkBObQiTOtCIjrxws3vt14_Jg5s_H61VJwB_KoiG9hPRXpuA6vV5Vm9rUSZUwM0KpucD6_6XrlRI2xwd7Sz5M_3Arnkqr4DckHC5NXNGGTVMUYJgz2H3mq1KcdwguQvlsigL2Pz4k6QW1ZCCYCvG21xk1UzcDAEEr15Yqxx-AEGa1AlNoxRqzZc8MM&__tn__=%2ANK-R) collector and even more! Check out our blog post for full details or read our release notes below.

Release v1.19.0 contains 2 new collectors, 19 bug fixes, 17 improvements, and 19 documentation updates. Full release notes can be found [here](https://github.com/netdata/netdata/releases/tag/v1.19.0).

**At a glance**

We completed a major rewrite of our web log collector to dramatically improve its flexibility and performance. The [new collector](https://github.com/netdata/go.d.plugin/pull/141), written entirely in Go, can parse and chart logs from Nginx and Apache servers, and combines numerous improvements. Netdata now supports the LTSV log format, creates charts for TLS and cipher usage, and is amazingly fast. In a test using SSD storage, the collector parsed the logs for 200,000 requests in about 200ms, using 30% of a single core.

This Go-based collector also has powerful custom log parsing capabilities, which means we're one step closer to a generic application log parser for Netdata. We're continuing to work on this parser to support more application log formatting in the future.

We have a new tutorial on [enabling the Go web log collector](https://docs.netdata.cloud/docs/tutorials/collect-apache-nginx-web-logs/) and using it with Nginx and/or Apache access logs with minimal configuration. Thanks to [Wing924](https://github.com/Wing924) for starting the Go rewrite!

We introduced more **cmocka unit testing** to Netdata. In this release, we're testing how Netdata's internal web server processes HTTP requests—the first step to improve the quality of code throughout, reduce bugs, and make refactoring easier. We wanted to validate the web server's behavior but needed to build a layer of parametric testing on top of the CMocka test runner. Read all about our process of testing and selecting cmocka on our blog post: [Building an agile team's 'safety harness' with cmocka and FOSS](https://blog.netdata.cloud/agile-team-cmocka-foss/).

Netdata's **Unbound collector** was also [completely rewritten in Go](https://github.com/netdata/go.d.plugin/pull/287) to improve how it collects and displays metrics. This new version can get dozens of metrics, including details on queries, cache, uptime, and even show per-thread metrics. See our [tutorial](https://docs.netdata.cloud/docs/tutorials/collect-unbound-metrics/) on enabling the new collector via Netdata's amazing auto-detection feature.

We [fixed an error](https://github.com/netdata/netdata/pull/7220) where **invalid spikes** appeared on certain charts by improving the incremental counter reset/wraparound detection algorithm.

Netdata can now send [**health alarm notifications to IRC channels**](https://docs.netdata.cloud/health/notifications/irc/) thanks to [Strykar](https://github.com/Strykar)!

And, Netdata can now monitor [**AM2320 sensors**](https://docs.netdata.cloud/collectors/python.d.plugin/am2320/), thanks to hard work from [Tom Buck](https://github.com/tommybuck).

**Improvements**

* **New Collectors**
* AM2320 sensor collector plugin

* Added parsing of /proc/pagetypeinfo to provide metrics on fragmentation of free memory pages
* The unbound collector module was completely rewritten, in Go [go.d.plugin/#287](https://github.com/netdata/go.d.plugin/pull/287)

**Collector improvements**

* **We rewrote our web log parser in Go, drastically improving its flexibility and performance**
* The [Kubernetes kubelet collector](https://docs.netdata.cloud/collectors/go.d.plugin/modules/k8s_kubelet/) now reads the service account token and uses it for authorization. We also added a new default job to collect metrics from https://localhost:10250/metrics
* Added a new default job to the [Kubernetes coredns](https://docs.netdata.cloud/collectors/go.d.plugin/modules/coredns/) collector to collect metrics from [https://kube-dns.kube-system.svc.cluster.local:9153/metrics](https://kube-dns.kube-system.svc.cluster.local:9153/metrics)
* apps.plugin: Synced FRRouting daemons configuration with the frr 7.2 release
* apps.plugin: Added process group for git-related processes-apps.plugin: Added balena to the container-engines application group
* web\_log: Treat 401 Unauthorized requests as successful
* xenstat.plugin: Prepare for xen 4.13 by checking for check xenstat\_vbd\_errorpresence
* mysql: Added galera cluster\_statusalarm
* **Metrics Database**
* Netdata generates alarms if the disk cannot keep up with data collection
* **Health**
* Fine tune various default alarm configurations
* Update SYN cookie alarm to be less aggressive
* Added support for IRC alarm notifications
* **Installation/Packages**
* Corrected the Makefile.am files indentation, to prevent unexpected errors
* Rationalized ownership and permissions of /etc/netdata
* Made various improvements to the installer script netdata-installer.sh
* Include go.d.plugin version v0.11.0
* **Other**
* Improve Travis build warnings
* cmocka testing for http requests
* CI/CD: Prevented nightly jobs from timing out

https://redd.it/e54dql
@r_devops

Best way to automatically update many containers when new image version is available in my repository?

So as the title says, i am wondering what would be the best way to store my docker images and update my running containers on, lets say, a lot of different machines. I was thinking about using gitlab and watchtower. Is there a better way?

https://redd.it/e56wtr
@r_devops

How Do You Get Out of Your Rut?

You've been there - just going through the motions at a job you don’t love anymore.

You’re feeling stuck - but you don’t know why.

You probably think a new job will fix it.

And it might, for a while.

But when that doesn’t last, what will you do?

Getting to the underlying reason that you’re stuck will help.

**How do you do that?**

https://redd.it/e5c2ec
@r_devops

How can I logically link two branches in two separated git repos for the creation of a temporary frontend app that should query its backend?

Our devs wants to have a temporary web app running whenever they push a feature branch to the repo. I got this part going already, I'm using Heroku for the infra and GitLab for CI/CD.

On the API repo they do the same thing, they push a branch and they get a temporary version of said push.

The problem is that I don't know how to logically link both of these automatically. The temporary frontend should talk to the temporary backend.

I know that I have to set and environment variable to point the front to the back, but I'm stuck in the part of how to link both of these independent branches together programmatically in the pipeline, if that makes sense.

I thought about using the same git tags on both branches but didn't get around it yet. I also thought about using Terraform in the pipeline to spin both apps and then make it somehow query its state to link both up.

Any help?

https://redd.it/e55t9z
@r_devops

Tech Professionals - Challenges with Fitness

I know that many Tech Professionals are struggling with losing weight, getting in shape and building healthy eating habits. For all those that are struggling with getting in good shape, what are some of your biggest challenges in this area of your life ?

What keeps you from reaching your fitness goals & What bothers you the most about ?

https://redd.it/e4t7pc
@r_devops

How to upgrade Ruby Bundle version in AWS Opsworks

I have a project running on AWS Opsworks, which has current bundler version of ***1.5.3*** on ***Ruby 2.2.2***.

i want to upgrade the bundle version to ***1.17.3***

but when i try to do it i am getting this error

Compiled Resource:
------------------
# Declared in /var/lib/aws/opsworks/cache.stage2/cookbooks/opsworks_bundler/recipes/default.rb:2:in `from_file'

gem_package("Installing Bundler 1.17.3") do
provider Chef::Provider::Package::Rubygems
action [:install]
retries 0
retry_delay 2
package_name "bundler"
version "1.17.3"
cookbook_name "opsworks_bundler"
recipe_name "default"
gem_binary "/usr/local/bin/gem"
end



[2019-12-03T07:24:08+00:00] INFO: Running queued delayed notifications before re-raising exception
[2019-12-03T07:24:08+00:00] ERROR: Running exception handlers
[2019-12-03T07:24:08+00:00] ERROR: Exception handlers complete
[2019-12-03T07:24:08+00:00] FATAL: Stacktrace dumped to /var/lib/aws/opsworks/cache.stage2/chef-stacktrace.out
[2019-12-03T07:24:08+00:00] ERROR: gem_package[Installing Bundler 1.17.3] (opsworks_bundler::default line 2) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /usr/local/bin/gem install bundler -q --no-rdoc --no-ri -v "1.17.3" ----
STDOUT:
STDERR: ERROR: Error installing bundler:
"bundle" from bundler conflicts with /usr/local/bin/bundle
---- End output of /usr/local/bin/gem install bundler -q --no-rdoc --no-ri -v "1.17.3" ----
Ran /usr/local/bin/gem install bundler -q --no-rdoc --no-ri -v "1.17.3" returned 1
[2019-12-03T07:24:08+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)


[Opswork configuration](https://i.stack.imgur.com/eUaxe.png) the bundler version was previously 1.5.3 i changed it to 1.17.3 and thats when the errors started to come

when i run bundle -von the server i am getting bundler version as **1.5.3.** what ever i do i cannot change the bundle version from that.

how to add configuration in either the chef recipe or anywhere else to update the bundle version ?

any suggestion will help me a lot. i have no experience working with chef or opsworks i have good working experience in working with EC2s and linux bash.

Our devops team doesn't know how to work with either. i am trying to solve an issue on a client's application which was launched on 2014 and hasn't ever been touched.

https://redd.it/e5g3jf
@r_devops

Apache Archiva in Front of Maven

I thought this would be no-brainer, but it's a little more difficult than I was led to believe by the [quick start documentation](https://archiva.apache.org/docs/2.1.1/quick-start.html).

I added the URL to the local Archiva server as a [mirror](https://archiva.apache.org/docs/2.1.1/userguide/using-repository.html) in settings.xml, and I know it has an effect because the maven build now breaks. The first thing was that the missing version stamps on the helper plugins went from a warning to an error:

[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.github.jitpack:maven-simple:jar:0.2-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-source-plugin is missing. @ line 35, column 15
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-javadoc-plugin is missing. @ line 49, column 15
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------< com.github.jitpack:maven-simple >-------------------
[INFO] Building Simple Maven example 0.2-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.512 s
[INFO] Finished at: 2019-12-01T20:57:26-06:00
[INFO] ------------------------------------------------------------------------
[ERROR] Error resolving version for plugin 'org.apache.maven.plugins:maven-source-plugin' from the repositories [local (/home/tad/.m2/repository), archiva.default (https://localhost:8080/archiva/repository/internal/)]: Plugin not found in any plugin repository -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] https://cwiki.apache.org/confluence/display/MAVEN/PluginVersionResolutionException


But I fixed that by adding current version numbers to each plugin mentioned in the pom.xml.

Now it is complaining about missing pom.xml files for each of those plugins:

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< com.github.jitpack:maven-simple >-------------------
[INFO] Building Simple Maven example 0.2-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.apache.maven.plugins:maven-source-plugin:jar:3.2.0 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.127 s
[INFO] Finished at: 2019-12-01T20:59:09-06:00
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-source-plugin:3.2.0 or one of its dependencies could not be resolved: Failure to find org.apache.maven.plugins:maven-source-plugin:jar:3.2.0 in https://localhost:8080/archiva/repository/internal/ was cached in the local repository, resolution will not be reattempted until the update interval of archiva.default has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the erro