Looking for insight into this DevOps books

Hello everyone one i was looking for reviews and searching these books and came here looking for more thoughts on this

https://www.fanatical.com/en/bundle/dev-ops-bundle-3-rd-edition

That's all thank you!

https://redd.it/zuzzug
@r_devops

Beanstalk Blue/Green Deployments - Inactive Environment Receives Traffic Up to 48 Hours After Deploy

We've implemented B/G deployments with 2 Beanstalk environments using CNAME swap as described in AWS docs https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.CNAMESwap.html

I see in the end of this article they have an explanation for what we experience.

> After Elastic Beanstalk completes the swap operation, verify that the new environment responds when you try to connect to the old environment URL. However, do not terminate your old environment until the DNS changes are propagated and your old DNS records expire. DNS servers don't always clear old records from their cache based on the time to live (TTL) that you set on your DNS records.

We observe in load balancer logs that inactive environment continues receiving traffic in the next 2 days after deploy. At first, it's something like up to 10% of all traffic and then it falls down to zero as time goes on. We have TTL of few minutes set on our end in route53, so it looks like it's a client side DNS cache that doesn't refresh DNS records.

I'm worried about this as it means that if we deploy some hotfix for example, these 10% of clients with stale DNS records won't see it immediately. Before we implemented B/G deployments, we haven't had this class of issues at all.

Did anyone have similar concerns about client side DNS caching in regards to B/G deployments and are there some techniques that may be useful to address these issues?

https://redd.it/zv4zjy
@r_devops

Storing Grafana Loki log data in a Google Cloud Storage bucket & possible alternatives

Hi all. I've been using Grafana Loki for a while to collect logs in Kubernetes and search them using Grafana.

I like that it's a lightweight solution but I had a problem where it wasn't easy to predict the amount of storage required depending on the growing amount of logs and the retention we configure. Someone on Slack suggested that I switch to object storage so I don't have to worry about the size of the volume and I can store an unlimited amount of logs, which is nice.

So today I tried setting up the Loki distributed Helm chart using a Google Cloud Storage bucket (since we use GKE and the Google Cloud Platform for everything); I got it working (I also took notes in a blog post if anyone is interested at https://vitobotta.com/2022/12/25/storing-grafana-loki-log-data-in-a-google-cloud-storage-bucket/).

To confirm that it reads the logs from the bucket I tried uninstalling Loki completely and in fact I was able to search through old logs after reinstalling it once again.

So it seems to work for the goal I was trying to achieve.

However before I commit to this I was wondering, would there be any advantage by switching to something else like ElasticSearch or other? All I want to do is be able to collect an indefinite amount of logs and search those logs easily and quickly. Is Loki good enough for this or should I try alternatives as well? if yes which one(s) do you recommend? Thanks!

https://redd.it/zv3sv7
@r_devops

Which stack to use for long term metrics storage?

Which storage to use with Prometheus?

1. Cortex (development stopped?)
2. Thanos
3. Uber M3
4. Grafana Mimir

Which one are you using? Which is one more future proof?

https://redd.it/zut835
@r_devops

Devops mentoring

I am looking to build out a mentorship program. I was wondering what people would either like to see out of mentorship program or what they have done in the past as part of their program.

https://redd.it/zvcpe2
@r_devops

CI : Run tests in parallel

Question about CI/CD best practices: Let's suppose I have an application/service that has unit tests and security tests and we have also SonarQube. Currently, I am new in my team and I want to suggest some improvements as we are migrating our pipelines from Jenkins to Gitlab CI/CD.

I think to save us milliseconds, maybe it would be better to run security tests and unit tests in parallel ( we also have some projects that use e2e and integration tests ).

Any advice or guidance on this ?

https://redd.it/zujltc
@r_devops

Check out rbac-police for k8s rbac testing

This isn't my tool, but I watched the talk on it from blackhat US this year (it's on YouTube).

It's quite interesting. It looks at the cluster roles in kubernetes and then let's you know which pods you can "escape" from and takeover the kubernetes cluster.

I ran it this week against 2 nginx ingress deployments. One was deployed from the nginx helm chart, the other from the kubernetes nginx helm chart. The official one has two ways of taking over the cluster, but the kubernetes one has none. Obviously I've switched to using that one.


https://github.com/PaloAltoNetworks/rbac-police

https://redd.it/zstcwa
@r_devops

Advice/Tips needed for new hire

I’m starting a new position (first job in swe) as a devops engineer. I’m looking for advice that will help me become productive and efficient in this role. I would also like to know what types of questions I should be asking and things to do to go “above and beyond”. What are some things that you would have done if you were a new hire or things that you would want your employees to do? Any feedback is appreciated and thank you for your time!

https://redd.it/zvi7i5
@r_devops

Deploy on Azure via Github Actions and Docker-compose



I am learning CI/CD and I did a tutorial on Github Actions

So far I want to build a Spring Boot Project with MongoDB connection. I use the standard image for MongoDB and MongoExpress but a custom image for my Spring Boot Application.

So I have a workflow where I test, build and can use docker-compose with three containers that communicate with each other.

My next step would be to set up the containers (via docker-compose) on Azure.

I have no idea about how to use azure. I signed up on student plan and created a container registry. I have NO idea what to do from here on out. How do I get the containers from my github action workflow on Azure based on my docker compose.

What are the general steps? I am completely at a loss right now and can't find a tutorial on this.

https://redd.it/zvkatx
@r_devops

1 month as a Jr DevOps Engineer

Hi Everyone! Im a month old Jr DevOps engineer and I just wanted to go through the things I’ve learnt so far/tasks and duties, I feel like people understand what devops means but not actually what you do, whether it be day-day or by project basis.

Background:
I have been a Jr devops engineer for a month but an apprentice devops engineer for around 13/14 months before, so this is from my understanding. I worked for a big tech company now I’m working for a start-up (growing up phase they like to use).

At the fintech:
Things were split between BAU and just general devops ig?

But BAU essentially was automation driven business as usual. This included:

- Automations using Jenkins
- Using lambdas (AWS)
- Essentially being ‘front line’ support, when issues happened in instances we went in to debug, if for examples components weren’t starting etc. if the Jenkins jobs were failing.
- packaging and uploading patches/mainline releases
- creating and testing images etc
- maintaining dashboards

The other side:

- creating infra setting up the new cloud stuff
- setting up new tools
- setting up new repositories etc

Not really too sure as I mainly worked BAU for this company.

At the startup:

Essentially the devops team was started up about Feb this year and was one person. When I joined I was the third member of the devops team. I like the work culture and the job so far, and I’m enjoying everything and getting stuck in, so here is an overview:

- setting up monitoring and alerting using Prometheus and Grafana, this was essentially a really critical part of my job because we needed this in place to ensure smooth delivery while everyone is on holiday during the Christmas period (company wide holiday)

- setting up Kubernetes (I haven’t been super involved in this project but I’m hoping to get more involved in this hands on in the new year) essentially moving from using one production cluster to 3 clusters testing, staging and production. I’m hoping to get involved with setting one up for pull requests.

- moving to GitHub, currently we are moving stuff to GitHub from bitbucket so we can make use of GitHub actions.

- We are currently setting up a new office and I’m looking forward to helping setup infrastructure, and move network stuff in house rather than a 3rd party IT company. This would be an amazing learning experience and I’ve spoken to my manager about allowing me to get involved with doing this (as it’s technically a ‘different’ team but is only one person)

Thank you everyone for being apart of my journey so far!

https://redd.it/zvkpx1
@r_devops

What's new in the Prometheus ecosystem?

Missed the KubeCon and PromCon updates? Here’s a quick rundown of updates on Prometheus:
https://horovits.medium.com/prometheus-roadmap-and-latest-updates-fbede9b5cc9

https://redd.it/zvkkfd
@r_devops

What about cloud cost optimization?

I'm curious to find out if as a DevOps, part of your role is to optimize cloud cost?


I've worked with several big organizations in the past, we were spending tons of money on the cloud, in some cases, it came to 50 million dollars a year, and I could literally do a quick scan of our servers and reduce the monthly invoice by 250k$ by simply eliminating idle machines.

I used to come to the DevOps team with these lists and numbers and was usually shooed away and dismissed.

So I'm seriously curious if you as a DevOps care about cloud cost? Do you work on optimizing it? who in your organization cares?

I've also written a short article about cloud cost optimization. Might be relevant to the topic:

https://zukeep.com/what-is-cloud-cost-optimization-3-actions-you-can-take-today-to-reduce-your-cloud-cost/

https://redd.it/zvnfix
@r_devops

Is anyone there who have changed their career after 10 years from test automation engineer (selenium + java) to Devops engineer?

How was the learning curve and how tough is it?

https://redd.it/zssoq5
@r_devops

Moving to gitlab/GitHub from ADO

I'm trying to write a proposal for my org to consider switching to a better platform like Gitlab or GitHub. ADO is lacking most basic features, i for a fact knew that both the gitlab and GitHub has shit Ton of neat features and also considering that MSFT is only focusing on GitHub now, i thought why not?

If anyone has gone down this road , please do share your inputs.

https://redd.it/zvq7d1
@r_devops

What is key usage of artifactory (nexus or Jfrong)?

3 year experience level question about artifactory?

https://redd.it/zsrw89
@r_devops

How to mark Airflow DAG failed if a function return false value inside the DAG?

GOAL: I am building a DAG that if it returns a false value at the end I want to make that DAG run a failure if it returns true than it can finish as a succesful run.

https://redd.it/zsqyci
@r_devops

My Terraform Bootcamp Udemy Course is Free!

Hey everyone, hope you all had a good holiday.

I'm an avid user of Terraform, and for the past 3 years working in 4 different companies, my career has been largely revolved around managing and scaling infra with Terraform.

I've created my first Udemy course with around 10+ hours of content, which focuses on ramping people up from a beginner Terraform user, to an intermediate one.

I absolutely love what I do and teaching others is just pretty fun.

The course is completely free. There's currently 95 coupons left until the promotion runs out (just how Udemy works).

https://www.udemy.com/course/terraform-iac-bootcamp/?couponCode=8F3602ECE527CA598D99

I'm really hoping this course helps someone understand Terraform and actually use Terraform at their workplace.

If this post goes against the subreddit rules, please let me know and I will take this down asap.

Cheers!


EDIT:

Looks like the coupon has ran out. Unfortunately I've used up all my free promotion coupons and not able to send out any more :/.

If this is your first time using Udemy:

NEVER BUY A COURSE FULL PRICE! Udemy continuously & frequently discounts the courses to $9.99. Please wait for people to review my course, to see if it's up to standards, then wait for a discount before thinking about purchasing the course.

​

​

https://redd.it/zvyzqh
@r_devops

Is GKE Autopilot suitable for running CI pipelines?

My company uses CircleCI right now and I'm looking to bring costs and build times down. Circle has a new "container runner" for self hosting that works by spinning up individual pods in a k8s cluster -- like, one pod per CI job, so a pipeline might execute over a whole bunch of potentially heterogeneous pods.

I'm pretty new to k8s, so I was considering a GKE Autopilot cluster, which seemed easier to manage. Looking into it more closely, though, it seems like the scaling characteristics might be mismatched: as far as I can tell, Autopilot only spins up resources when there is already demand for them in scheduled work, with no concept of time to live. It seems like the way to scale smoothly with Autopilot is to use horizontal pod autoscaling with resource utilization triggers, so that more pods get allocated before the cluster is overwhelmed.

CI pipelines are obviously burst-y work, and it's worth overallocating a bit to prevent startup delays. So my intuition is that I'm better off managing the node pools myself for this application. Is that right, or is there an elegant way to do this with Autopilot I'm not seeing?

https://redd.it/zvzm63
@r_devops

Do you use a Helm chart repository?

Do you push the helm chart first into a repository before deploying it or do you install the helm chart directly from your git repository?

Would you please give me the reasoning why you prefer one over the other?

https://redd.it/zw19w3
@r_devops

Can't SSH after installing Letsencrypt cert

I am using an EC2 where I have installed Letsencrypt ssl cert, after that I can't ssh to my server. What should I do now?
I have followed this tutorial

https://redd.it/zshpyl
@r_devops

Dumb Question: Boost the capacity of my phone to detect a wifi ?

Hey guys , it might be a dumb question but is it possible to boost the capacity of my phone to detect a wifi by using an app ?
I'm trying to convince my friend that it is not possible.

https://redd.it/zshodg
@r_devops