Examples how to use Pulumi fo AWS

I am interested in using Pulumi (Python interface) but can't find good docs how to use it . E.g. how to create network (VPC, subnets, NAT gateways etc.) in AWS with it?

Could somebody share some usable docs? Any AWS related and Python Pulumi.

Posted on Pulumi site looks pretty basic and all of them uses default VPC.

https://redd.it/g3uopo
@r_devops

This Week in DevOps: AWS EFS supports ECS Fargate

Does anybody know if EFS works with EKS Fargate also? This seems to be a container level mount so it seems like it should but EKS is not mentioned in the announcement.

[https://thisweekindevops.com/2020/04/17/weekly-roundup-april-17th-2020/](https://thisweekindevops.com/2020/04/17/weekly-roundup-april-17th-2020/)

https://redd.it/g3qvvg
@r_devops

Stuck with no way back

Hello everyone, trying to create multi LAMP stack with vagrant and basically , I am stuck and nothing is working :D Any tutorials you could suggest for the multi machine ansible setup?

​

Project here:

It's messy because my brain is fried [https://github.com/RausisM/tryingtosurvive](https://github.com/RausisM/tryingtosurvive)

https://redd.it/g4spjm
@r_devops

Cisco Prime, pythons Nornir, or Ansible for network config mgmt

Hi guys, I am a network engineer who has been using python libraries for simple network config changes or show commands. The libraries I have used in production are netmiko and paramiko both with multi-threading when I need to run a script on multiple devices faster.

I have used ansible In a virtual environment but not production. I’m not to strong in Linux at the moment so I use ansible in a virtual Ubuntu environment so I don’t break my production.

I am currently learning Nornir and liking it more than ansible.

My job wants me to automate network configs back ups/ schedule them. They don’t want to buy solarwinds NCM and want me to try using our current Cisco Prime software. I don’t really like it and prefer NCM. Is it worth using Cisco Prime and learning it. Because I would rather go the either the Ansible or Nornir route and I feels it has more flexibility and it’s open source.

I plan to make a web api flask or dgango so that others in my Dept can use the python scripts without having to know the backend.

What do you guys think? Learn Prime and do the config mgmt/scheduling with it or continue with the python based scripting. Also which is more recommended right now Nornir or ansible. I am biased on which one so would like opinions from those that have been down this road already.

https://redd.it/g4vnz4
@r_devops

Slack actions to a private server

Hi I am new to devops and want to do simple tasks via slack, problem is my server is in a private subnet.

​

I am able to send notifications to slack via an internet gateway, but can't do actions towards a notification i.e service restart, logrotate, approve and decline jenkins deployment

​

My idea is to poll messages via crontab with a curl that sends keeps monitoring a channel for new chat messages, but implementation would be messy and I know would cause some type of overhead

​

I slightly understand that all polling actions have overhead, but hope anyone has a suggestion on how to do this more 'gracefully'

https://redd.it/g4urdi
@r_devops

General experience with datadog log agent performance?

Curious if anyone could speak to the how performance the datadog agent is for gathering system logs. The agent appears to be written in python. Looking to move .5 terabytes of logs/host.

https://redd.it/g4zcuj
@r_devops

How-to build a Docker image(using Github Actions) and run it using Azure DevOps [Article]

Hello.
Wrote two articles about using CI/CD for working with Docker:
* [Github part](https://github.com/groovy-sky/azure/tree/master/iac-02#introduction)
* [Azure DevOps part](https://github.com/groovy-sky/azure/tree/master/iac-03#introduction)

https://redd.it/g4y5nw
@r_devops

s3cr3t: a supercharged S3 inverse proxy. Feedback is highly appreciated!

Link to Github: [https://github.com/axl89/s3cr3t](https://github.com/axl89/s3cr3t)

​

After the good people of NGINX released a lot of free resources because of the current situation, I ran into the **ngx\_http\_secure\_link\_module** module ([link](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html)), which existence was unknown to me.

After playing with it a little bit, an idea clicked inside my head to develop an easy "Download server" with an S3 bucket as the backend with 3 basic restrictions:

* Client IP address
* URI
* Expiration time

I found this development super interesting and fun, and I'd love to hear from you. Let me know your thoughts and feel free to contribute! :)

https://redd.it/g4ziaz
@r_devops

Places to find DevOps work, preferably in management or leadership?

Looking for a new opportunity. Besides Linkedin, and the common boards (indeed, ZipRecruiter, Dice, etc) is there another site I’m missing to find good opportunities?

https://redd.it/g4wvjn
@r_devops

Mixing declarative infrastructure plus deploy scripts... suggestions or alternatives?

I have an application that has 1 image but several (over 20) different ECS services per environment. The services mostly share a common set of environment variables and secrets, with some small differences due to each service starting up a different process, using a different DB, etc.

I tag the image with the commit SHA. I do not want to use \`latest\`.

I currently manage the infrastructure (ALBs, Task Definitions, Services, Roles, etc.) in Terraform, and deploy by updating the Task Definition and Service with the API. However this means that even though I use a script to tell Terraform what the current deploy is, the state is old and it requests an update.

I'm looking for a better way to do this. It seems wrong to be managing the task definition state in two places (Terraform and deploy script). I like having something like Terraform state to keep track of my resources so I can be declarative vs. writing standup and teardown scripts. I want to, for example, have a resource removed when I remove it from code, not by running a cleanup script.

As mentioned, I'm using ECS and Terraform right now, but I see this issue when I'm looking at tools like CDK, Pulumi, and Kubernetes as well. How are people managing this difference between declaring infrastructure but deploying via scripts?

https://redd.it/g52xvc
@r_devops

Direction to move to improve on Docker based personal cloud VPS

Hi everyone,

I'm running a little VPS with Docker to host, among others, nextcloud, gitea, caddy webserver, bitwarden password container, syncthing service, ... all proxied through a traefik container. For most of the containers I'm using docker-compose files with docker swarm in a single-node setup. It's all working nicely so far, but I feel that I'm not doing it optimally yet. For some services, that I would like to try out, a docker image is not readily available and building a proper docker image using docker files can be a hassle because one has to worry about how the image is layered etc, instead of just booting up a VM and install it like on a bare-metal server. Other services, like bigbluebutton, don't like being deployed in containers at all.

I'm wondering therefore if I should look into microvms like firecracker or other means of separating my services from each other (which is from my point of view the biggest advantage of using containers / vms at all). At the same time I also don't want to loose handy integration of a container runtime with my reverse proxy service, that automatically generates rules for newly started containers. So given that docker is getting less relevant as orchestration tool due to kubernetes, perhaps I should look into single-node kubernetes setups instead?

I hope you somehow got a feel of my predicament and I welcome any (justified) recommendation in which direction I should go next.

https://redd.it/g52lyc
@r_devops

Considering creating an automation tool - looking for volunteers to interview for market research

I'm a software developer and I have an idea for a product targeted at development teams. I want to do my due diligence and validate that there is indeed demand for the product. Would anyone be willing to take part in a \~30 minute interview to help me out?

Specifically, I would need people who are familiar with the automated steps that occur on their company's repositories (eg. build pipeline, dependabot)

If you fit the above description, please reply or message me. I would absolutely be willing to return the favor in any way I can help (eg. offer the product to you for free if/when it launches, participate usability interviews, etc.)

https://redd.it/g52a3v
@r_devops

Gitlab and Chef CICD Pipeline

I have to create a CICD pipeline within Gitlab that calls a recipe from an external Chef Node server. So basically the recipe on the Chef Node server is updated and once that takes place I will then use Gitlab to call on that recipe and run it within the pipeline. Anyone know if it is possible to do or a better way to go about it?

Thanks!

https://redd.it/g4yrk8
@r_devops

gitlab ci flow newbie question

I am newbie to DevOps..I have some queries in the gitlab ci flow.Please help me clarify the queries below.

Hw gitlab ci works?

1. Developer creates a new module in his branch.He tests and everything is fine. So he pushes the code to gitlab remote branch

2. He creates an MR so he can get review comments and then the code can merged to main code in master branch

3. The other team members review the code and finally the code is good to be merged to master. -->Will this be merged with master ?When do the gitlab ci pipeline will run?

4. The conflicts with the master branch are resolved and merged —>if its already merged to master, then other developers may pull this code even before it was run by pipeline.? And what is the role of release tags here?

5. This triggers ci [ ](https://pipeline.So)pipeline and the ci runner will do whatever instructions given in gitlabci file.It will build, test and deploy the code

6. Some of these steps could even be manual. Like deploy can be manual so it will wait for approval from someone before deploying.

https://redd.it/g4u5xt
@r_devops

Verifying plugin integrity in Jenkins

Hi all,

I noticed Jenkins fetches its plugins using `http` as defined in [update-center.json](https://updates.jenkins.io/current/update-center.json). Not an issue if the integrity is checked. However, I tried to find the checksum without success.According to [JENKINS-46428](https://issues.jenkins-ci.org/browse/JENKINS-46428) they verify the integrity using a sha1 checksum in base64 format, but I cannot find it. I checked several plugins without success.

If someone could point to documentation or something related which can be used to verify that statement would awesome.

https://redd.it/g4tog8
@r_devops

Newbie to Chef

Hello,

I currently work on an devops infrastructure team (primarily with Gitlab and Artifactory) and I would like to learn CHEF in order to help with my pipeline build. What would be the best way to learn chef "quickly" and can you please provide resource links or suggestions.

Thank you!

https://redd.it/g50z3u
@r_devops

Using Azure Function Apps to process Blob Storage event triggers (CLI + javascript)

I recently found myself needing to processes files uploaded to Azure Blob Storage. Initially I was a bit frustrated by the fact that all the blogs and tutorials out there seem to be written for Visual Studio users coding in C#. This tutorial provides a step by step explanation for hooking up a serverless Azure Function App written in Javascript to Azure Blob Storage using only CLI tools. The Function App framework provides a mechanism for reading and writing to Blob Storage storage without needing to make SDK API calls using the function input and output bindings. This is a very handy piece of innovation that I hadn't seen before.

[https://www.filemage.io/blog/azure-functions-blob-trigger-javascript/](https://www.filemage.io/blog/azure-functions-blob-trigger-javascript/)

https://redd.it/g4wg91
@r_devops

Staging bloated with broken microservices

Hi everyone !

In my company (400 devs), we have around 100 microservices that are communicating through gRPC and brokers like Kafka and RabbitMQ, on top of Kubernetes. We have 4 staging environments, but everything is always broken because everyone is testing on them.

​

* Data in the databases is always weird
* Microservices are sometimes broken and in crashloop mode and we can't test our services which have dependencies on them

​

As Head of Platform, I would like to implement 10 to 20 staging environments (one for every tribe or squad), but would like to know if you have any insights on this topic.

​

Thanks!

https://redd.it/g4owzc
@r_devops

What is default deploy placement in docker swarm?

I noted one thing about some of the compose files.

Some of the services constrainted the deploy placement to node manager, but other services has it unspecified.

Does it mean that these services can go anywhere?

https://redd.it/g4uiqr
@r_devops

kapacitor alerting

I am using influxdb, telegraf and grafana. I would like to use kapictor to generate an alert if no data is coming for a host. Is this a good use case for kapictor?

https://redd.it/g4soh9
@r_devops

comparing tsddb

Among Metrictank , biggraphite and m3db which tsdb is the best in terms of availability, scalability and handling more metrics per second.

https://redd.it/g4q8lk
@r_devops