Confusion with unit and integration testing in CI pipeline

Trying to get a better understanding of running unit and integration tests in a CI pipeline. I feel like I understand it, start working on it, and a bunch more questions come up, confusing me. Hoping this set of questions will be the last and it will all finally click.

# Unit Tests

I've been using this Dockerfile as a template of sorts because it pretty clearly delineates the various stages and concerns in a multi-stage Dockerfile.

The test and linting stages makes sense and is pretty straight forward to me: in the CI pipeline, target these stages and if passing target the production stage. Using RUN for these stages makes sense to me because you are just building and testing this code, not how it integrates with other services, and not deploying images of these stages, and just trying to determine as quickly as possible if the build is passing. If not, the build will fail. It seems somewhat unnecessary to add steps of building and then deploying just for this purpose.

Q1: Should these testing and linting stages be deployed as a container if they are just running unit tests, therefore converting the RUN to a CMD?

# Integration Tests

I'm struggling with these the most.

My understanding is that the flow should be:

PR ->
Build Code ->
Unit Tests (test and linting stages) ->
If passing, Build Production images (production stage) ->
Push to Container Registry ->
Pull from Container Registry ->
Deploy to Test Kubernetes Cluster ->
Integration Tests

This seems to necessitate deploying integration tests into separate containers for a couple reasons:

1. The production images have no development dependencies so you shouldn't be able to run tests in them.
2. RUN wouldn't work in this setup since no images are being built.

So my questions are:

Q2: Is this correct that integration test containers should be deployed?

Q3: Should there be a stage for integration tests in the Dockerfile that uses a CMD to be run when the image is deployed to a container?

Q4: I'm struggling to understand what this image would have on it: just tests that target the microservice end-points (e.g.., /api, /client, etc.) or is it a copy of the production build that still has testing dependencies?

Q5: If it is the latter, why deploy the production image since you aren't really testing it but a copy of it with the testing dependencies on it?

After typing this all out, I feel like the "correct" answer is having a unit-test stage that is RUN in the process of building production. Then having test-runner containers that just run integration tests with CMD against the running production images.

Q6: Or is what I just described more E2E than integration testing?

Thanks in advance for any feedback.

https://redd.it/pyw0ii
@r_devops

Nutanix Calm

Anybody have experience using or considered using Nutanix Calm for enterprise IaC deployments? Want to know if it’s worth paying for over just using terraform

https://redd.it/pyulns
@r_devops

Script getting started with Terraform in an Azure tenant

If you've ever wanted to get any Azure tenant setup and don't want to have to reference this article: https://learn.hashicorp.com/collections/terraform/azure-get-started or that article: https://docs.microsoft.com/en-us/azure/developer/terraform/overview

How about just trying my script? https://seehad.tech/2021/08/30/use-powershell-to-setup-any-azure-environment-for-terraform/

​

Check out my site for other good scripts for Azure! https://seehad.tech

https://redd.it/pyymcx
@r_devops

Good resources for learning/Testing Ansible automation?

I am kind of hoping for something like Tryhackme but realistically if I can't find that ill just spin some stuff up in my lab. but if anyone has any resources/advice it would be really helpful. I am not great on dev ops stuff and my role is more or less security dev-ops patching Linux servers and applying CIS baselines with Ansible. It would be nice to get more hands-on experience.

https://redd.it/pym6vs
@r_devops

AZDO & Variables

Shot in the dark here but I think I’m really close.

We’re utilizing Azure DevOps for our CI. We’re a legacy Windows app.

The goal is to tokenize a variety of configs (some in and out of the root) so that we can then change those tokens per environment. We would also like to manage our variables in our Library in variable groups, labeled after envs.

Using the default msbuild.exe task, we’re able to transform the root web.config using the web.release.config.

Ok great - but how can we transform configs out of the root? We can’t get the damn file transform task to do anything and there is one line of error it throws which is useless (v1 and v2).

Lastly, assuming I even can tokenize all of my configs, how can I push variables from AZDO libraries into these tokens?

<3

https://redd.it/pyzs7j
@r_devops

quay.io Down again!

Any idea when quay.io back in service? I am tired of these outages with quay.io.

https://redd.it/pye0va
@r_devops

Gcloud oauth credentials automate

So in my company, we are using gcloud as service provider and we create new projects for each of the clients.

For internal access to application, we have to manually configure the oauth from the credentials screen from the Console. I don't think google has made oauth API public, has anybody tried to automate the oauth configuration. Need to create 2 oauth groups.

https://redd.it/pydkm0
@r_devops

Question, how to prevent -- Internet goes down for millions, tech companies scramble as key encryption service expires

Per title, how could (or should) have DevOps identified and resolved this issue months ago?

Better test tools?
Better search/static analysis?
Improved (technical) risk identification and mitigation?
Better culture?
Better organization (people) to research lower level dependencies?
Better leadership from outside organizations (e.g. IEEE or similar)

Internet goes down for millions, tech companies scramble as key encryption service expires

> The expiration of a key digital encryption service on Thursday Sept 30, 2021 sent major tech companies nationwide scrambling to deal with internet outages that affected millions of online users.

> Tech giants — such as Amazon, Google, Microsoft, and Cisco, as well as many smaller tech companies — were still battling with an endless array of issues by the end of the night. The problems were caused by the forced expiration of a popular digital certificate that encrypts and protects the connection between devices and websites on the internet. The certificate is issued by Let's Encrypt , the largest issuer of such certificates in the world.

> At least 2 million people have seen an error message on their phones, computers, or smart gadgets in the past 24 hours detailing some internet connectivity problems due to the certificate issue, according to Scott Helme, an internet security researcher and well-known cybersecurity expert.

> “So many people have been affected, even if it's only the inconvenience of not being able to visit certain websites or some of their apps not working,” Helme said.

> “This issue has been going on for many hours, and some companies are only just getting around to fixing it, even big companies with a lot of resources. It's clearly not going smoothly,” he added.

> There was an expectation before the certificate expired, Helme said, that the problem would be limited to gadgets and devices bought before 2017 that use the Let’s Encrypt digital certificate and haven't updated their software. However, many users faced issues on Thursday despite having the most cutting-edge devices and software on hand.

> Dozens of major tech products and services have been significantly affected by the certificate expiration, such as cloud computing services for Amazon, Google, and Microsoft; IT and cloud security services for Cisco; sellers unable to log in on Shopify; games on RocketLeague; and workflows on Monday.com.

> This problem has flown under the radar of many major tech manufacturers, including Big Tech companies such as Apple, Google, Sony, and Microsoft — none of which have made announcements to customers about the issues, Helme told the Washington Examiner on Wednesday before the certificate expired.

https://www.washingtonexaminer.com/news/tech-companies-struggle-millions-suffer-digital-certificate-expiry

https://redd.it/pz4e55
@r_devops

How to start learning devops?

I have been doing development (as a full stack developer) for a while now, and currently working on various personal projects which requires devops stuff, rn I have a Django-nginix-graphql-nestjs dockerized full stack webapp and thought of using azure for all the deployments stuff but don't know where to start.

Any help would be great!

https://redd.it/pyc4jq
@r_devops

Kubernetes Visualizer Ideas

Hey guys! I'm currently brainstorming ideas for a visualizer OSP for Kubernetes, and was wondering if developers could tell me what metrics you want more interaction with?

https://redd.it/pybskw
@r_devops

Making Kubernetes Operations Easy with kubectl Plugins

Hi /r/DevOps,

All of us who use Kubernetes have to perform a lot of tasks with kubectl, some of which can be quite clunky and time-consuming. That however, can be avoided with use of kubectl plugins which I explain and showcase in my latest article, which also includes list of some super useful plugins that I use on daily basis. If you're interested in the topic, then here's link:

https://towardsdatascience.com/making-kubernetes-operations-easy-with-kubectl-plugins-206493c1f41f

Feedback is very much appreciated!

https://redd.it/pz7wki
@r_devops

Hashicorp Vault for Developers?

We are implementing Vault (HCP) across our infrastructure and services, we run them all in K8S and we need to provide better security (certificates, passwords, etc). We are confident on the DevOps side we can implement it effectively.

The challenge is that if we are going to implement this we want our applications to leverage the platform for authentication, certificates, and other security use cases. We've been asking Hashicorp and our partners for Developer enablement we can use for both existing and new hires for the future. I checked the usual suspects for courses and such (Udemy, etc) and the courses are too in-depth or long and more focused on Ops versus Dev.

Any advice from you all?

https://redd.it/pz7ppj
@r_devops

Monthly 'Shameless Self Promotion' thread - 2021/10

Feel free to post your personal projects here. Just keep it to one project per comment thread.

https://redd.it/pza4a0
@r_devops

Monthly 'Getting into DevOps' thread - 2021/010

What is DevOps?

[AWS has a great article](https://aws.amazon.com/devops/what-is-devops/) that outlines DevOps as a work environment where development and operations teams are no longer "siloed", but instead work together across the entire application lifecycle -- from development and test to deployment to operations -- and automate processes that historically have been manual and slow.

Books to Read

The Phoenix Project - one of the original books to delve into DevOps culture, explained through the story of a fictional company on the brink of failure.
[The DevOps Handbook](https://www.amazon.com/dp/1942788002) - a practical "sequel" to The Phoenix Project.
Google's Site Reliability Engineering - Google engineers explain how they build, deploy, monitor, and maintain their systems.
[The Site Reliability Workbook](https://landing.google.com/sre/workbook/toc/) - The practical companion to the Google's Site Reliability Engineering Book
The Unicorn Project - the "sequel" to The Phoenix Project.
[DevOps for Dummies](https://www.amazon.com/DevOps-Dummies-Computer-Tech-ebook/dp/B07VXMLK3J/) - don't let the name fool you.

What Should I Learn?

Emily Wood's essay - why infrastructure as code is so important into today's world.
[2019 DevOps Roadmap](https://github.com/kamranahmedse/developer-roadmap#devops-roadmap) - one developer's ideas for which skills are needed in the DevOps world. This roadmap is controversial, as it may be too use-case specific, but serves as a good starting point for what tools are currently in use by companies.
This comment by /u/mdaffin - just remember, DevOps is a mindset to solving problems. It's less about the specific tools you know or the certificates you have, as it is the way you approach problem solving.
[This comment by /u/jpswade](https://gist.github.com/jpswade/4135841363e72ece8086146bd7bb5d91) - what is DevOps and associated terminology.
Roadmap.sh - Step by step guide for DevOps or any other Operations Role

Remember: DevOps as a term and as a practice is still in flux, and is more about culture change than it is specific tooling. As such, specific skills and tool-sets are not universal, and recommendations for them should be taken only as suggestions.

Previous Threads
https://www.reddit.com/r/devops/comments/pfwn3g/monthlygettingintodevopsthread202109/

https://www.reddit.com/r/devops/comments/ow45jd/monthlygettingintodevopsthread202108/

https://www.reddit.com/r/devops/comments/obssx3/monthlygettingintodevopsthread202107/

https://www.reddit.com/r/devops/comments/npua0y/monthlygettingintodevopsthread202106/

https://www.reddit.com/r/devops/comments/n2n1jk/monthlygettingintodevopsthread202105/

https://www.reddit.com/r/devops/comments/mhx15t/monthlygettingintodevopsthread202104/

https://www.reddit.com/r/devops/comments/lvet1r/monthlygettingintodevopsthread202103/

https://www.reddit.com/r/devops/comments/la7j8w/monthlygettingintodevopsthread202102/

https://www.reddit.com/r/devops/comments/koijyu/monthlygettingintodevopsthread202101/

https://www.reddit.com/r/devops/comments/k4v7s0/monthlygettingintodevopsthread202012/

https://www.reddit.com/r/devops/comments/jmdce9/monthlygettingintodevopsthread202011/

https://www.reddit.com/r/devops/comments/j3i2p5/monthlygettingintodevopsthread202010/


Please keep this on topic (as a reference for those new to devops).

https://redd.it/pza4yc
@r_devops

A newbie question about EKS and EBS

Hi everyone.

So if I have three EKS nodes with three EBS volumes attached to each of them for persistent storage, and one of the nodes goes down, that nodes EBS volume gets detached and stays detached while a new node goes up automatically.

Is there a way to store data for EKS nodes so it persists after a node going down and gets associated with the new one? I'm talking about an ECK / ELK cluster and data nodes going down.

I probably sound like a dumbass but that's the question lol. Thank you!

https://redd.it/pzbcbu
@r_devops

How do you guys go about database backups in terms of multi-cloud?

The title is self explanatory, but to give more context I'm trying to find a solution for multi-cloud database backups in order to be have a more proactive approach to ransomware attacks.

In the past I used 2ndquadrant's Barman with BarmanS3 to save the data to a S3 bucket on another cloud account and worked with PITR without a problem but now I'm working with GCP's Cloud SQL instead and can't find a good alternative to achieve a similar result.

So, how do you guys go about database backups in terms of multi-cloud?

Any opinions and suggestions about other approaches to protect the data is also welcome.

Thanks.

https://redd.it/pzcqbr
@r_devops

Have you read Google's 2021 State of DevOps report?

In particular I enjoyed how it highlighted ways SRE's can build risk-tolerant cultures and the importance of it. Curious if you disagree with anything mentioned in the report: https://cloud.google.com/blog/products/devops-sre/announcing-dora-2021-accelerate-state-of-devops-report

I also wrote a summary of it here with 4 takeaways.

https://redd.it/pzfa2o
@r_devops

Replacing Dummy Fields to fix MD5 Checksum Error

Remove if not allowed, but I need help with some code (stack overflow post)

I need to replace "dummy fields" in this code, specifically data-timestamp and data-signature. data-signature is the most important one, if the timestamp is wrong i don't really care...

the signature variable is giving me a checksum error no matter what i do and that is the biggest issue.

this is for an acuity scheduling custom referral tracking and referralcandy integration.

&#x200B;

><div
>
> id="refcandy-mint"
>
> data-app-id="--------"
>
> data-fname="%first%"
>
> data-lname="%last%"
>
> data-email="%email%"
>
> data-amount="%price%"
>
> data-currency="USD"
>
> data-timestamp="NEED VARIABLE"
>
> data-external-reference-id="%id%"
>
> data-signature="NEED VARIABLE"
>
>\></div>
>
>
>
><script>
>
> (function(e){
>
>var t,n,r,i,s,o,u,a,f,l,c,h,p,d,v;
>
>z="script";
>
>l="refcandy-purchase-js";
>
>c="refcandy-mint";
>
>p="go.referralcandy.com/purchase/";
>
>t="data-app-id";
>
>r={
>
>email:"a",
>
>fname:"b",
>
>lname:"c",
>
>amount:"d",
>
>currency:"e",
>
>accepts-marketing:"f",
>
>timestamp:"g",
>
>referral-code:"h",
>
>locale:"i",
>
>external-reference-id:"k",
>
>signature:"ab"
>
>};
>
>i=e.getElementsByTagName(z)[0\];
>
>s=function(e,t){
>
>if(t){
>
>return "" + e + "=" + encodeURIComponent(t)
>
>}
>
>else{
>
>return ""
>
>}
>
>};
>
>d=function(e){
>
>return "" + p + h.getAttribute(t) + ".js?aa=75&"
>
>};
>
>if (!e.getElementById(l)) {
>
>h=e.getElementById(c);
>
>if (h) {
>
>o=e.createElement(z);
>
>o.id=l;
>
>a=function(){
>
>var e;
>
>e=[\];
>
>for(n in r){
>
>u=r[n\];
>
>v=h.getAttribute("data-"+n);
>
>e.push(s(u,v))
>
>}
>
>return e
>
>}();
>
>o.src="//"+d(h.getAttribute(t))+a.join("&");
>
>return i.parentNode.insertBefore(o,i)
>
>}
>
>}
>
>})(document);
>
></script>

https://redd.it/pzh8g6
@r_devops

Where is the line between dev and devops?

I came into devops more from the ops side than from the dev side and haven't been working in the field for that long. That said, I have a solid grasp of typical DevOps technologies like automation, containerization, scalability, etc. and was recently hired as the sole devops engineer at a small-ish company that just landed some huge contracts. Each project, for each client, has a team of dedicated developers, and then me, who's on every project.

Long story short, there's no standardization between projects, and they all want me to be able to deploy their code to various environments. The code itself is a mess, and as a result, the deployments are a mess, too, with 40-ish manual steps *after* their integration and delivery. The best part, though, is that due to lack of standardization, no one can tell me how to deploy their product for the individual projects.

So where exactly is the division of responsibility here? Should I be able to look at their code base and be able to deduce how to deploy it, or should I expect that at least one dev on each project should be able to tell me how to deploy the code, what its dependencies are, etc.?

Basically, how much should the devs give me, versus how much should I be able to do on my own? Because I'm driving myself up the wall trying to figure all of this out on my own when deployment deadlines are coming up in less than two weeks. For multiple projects.

Sanity check, anyone?

https://redd.it/pzht2o
@r_devops

Multiple Domain Mapping in AWS Opensearch (ELK)

I have added the Custom URL to AWS Opensearch (ELK) and having HTTPS access (SSL Certificate attached).

Now i want to add two more domain to it. So when i point that two custom domain to Opensearch whether it is AWS provided Opensearch URL or My own custom URL it shows me SSL Error even though my two custom domain/ website are working fine .

I am getting only SSl Error.

Any Soln??

Thsnk you

https://redd.it/pzg6ye
@r_devops

Modern Build/deploy strategy should always be artifact based.

Opinionated poll.
I'd like to posit that any modern build/deploy strategy should be artifact based.

e.g. given a branch we want to test (in any number of "environments") and get to production, we build once, create artifacts and deploy those I'm a repeatable way to each environment (automated preferably) including prod.

View Poll

https://redd.it/pzkkax
@r_devops