Writing sidecar tool for aws

I am trying to write a tool preferably in golang for web apps in aws that will read config from a source and update local app config file. Is there any pattern or base lib/framework i should be following? Suggestions are welcome.

https://redd.it/qyqeio
@r_devops

Introducing DevOps practices to a company with bad organization

So we have an R&D team that develops and maintains dozens of tools, some relatively big desktop software, some data processing server pipelines, some smaller command-line tools. Mostly for the use of internal teams, but increasingly also for external customers.

For desktop tools, the deployment "process" is: copy-paste the new code and its new dependencies on an NFS network share. The prod workstations download the new environment regularly from there. Tools aren't really reliable, and when I came in I had to rewrite a large part of the code for my new projects.

I'm the only one doing automated testing, I introduced Docker/docker-compose for server deployment (instead of scp'ing the code), and I'm trying to improve coding practices. Obviously, there's no CI/CD, and there won't be if I don't set it up myself. In the desktop Python environment, pip doesn't work correctly, so we can't be sure that a new deployment won't break another package's dependencies.

I'd like to ask for some advice on how to set up a CI/CD pipeline, for me and some colleagues who want better tooling. We don't need approval, and I think the others would be open to better practices if we showed the way. I have support from my manager for it.

The current plan is to set up Jenkins on some server, make it run the CMake build regularly, and run the few PyTest unit tests that already exist, all in Docker. That system would expand over time, as we train the rest of the team to use it.

Is Jenkins a good choice, considering that we'd be maybe two people maintaining it? Is there a good tutorial on how to set up such a platform from scratch? I looked at the Jenkins documentation, but it's not always clear. Any and all advice is appreciated.

https://redd.it/qyaj9v
@r_devops

Vault in production?

Does anyone have any tips, tricks, configurations that they can share? We've been mandated by management to move to Vault (community version of course 🙄). Just curious how everyone else is doing it!

https://redd.it/qycufs
@r_devops

Are DevOps jobs better suited for Digital Nomad lifestyle?

I am currently working as a Data Engineer, and I plan to become a digital nomad in 5 years time. I have some DevOps experience, and I feel that this role is better suited for a digital nomad lifestyle where I can travel all over the world and work remotely.

Is my thinking correct? If so, then how can I better orient myself towards this goal?

https://redd.it/qy8ru8
@r_devops

Application Insights apim vs app level monitoring

I have a question regarding optics in Azure, I now have logging set at my web app level and the apim level. This is not currently instrumented in code so apim seems to give me more information regarding dependency info. Is this redundant and can/should I clean up my web app Insights resources and just instrument at apim level?

https://redd.it/qyxesb
@r_devops

Future Job Growth - Management or Technical track

Hey Guys,

As the subject states, I am getting to a point in my career where the next step forks and I will have to decide if I want to stay and get deeper technically (architect level) or if I want to go the management path (manager level). Currently, I am a Principal (Senior) Engineer on my team and I do a lot of 'team lead / technical lead' tasks, but I still get my hands dirty with the engineering and technical details often.

However, I have been told and I know that my 'soft skills' have always been a big strength of mine (communication, empathy, people stuff, etc.) and several of my old managers always commented that they thought I would make a good manager. I know being a manager goes deeper than the people skills though, such as could I stomach firing someone and preventing them from bringing food home to their families, etc. if they are dead weight for example.

So, I guess what I am trying to ask here is for those that have made the decision to make the jump to a manager path, why did you do it? What are the best parts of your job now that you are a manager and not the individual contributor / engineer that is doing the technical work? What are the worst parts? Do you regret your decision, and wish you stayed as an individual contributor and still did a lot of the engineering / devOps work?

Any feedback / insight / comments would be super appreciated! Maybe I should look if there is a manager subreddit to also post this question to.

Thanks everyone!

https://redd.it/qz28ls
@r_devops

Tooling for dependency management

Hi,

I have a tooling question. In my organization, we are running IBM App Connect Enterprise (a low-code platform for developing integrations) applications, and we are currently building a CI/CD pipeline for handling everything from build and test, to deployment.

Some of our applications have dependencies. Mostly other IBM ACE apps, but also some Java library dependencies from time to time. All of our dependencies are stored in our organization’s git repo.

To build our applications, we run a build command which requires all application sources to be in the same workspace. So before we can run the build command, we need to git clone all of our application sources. The challenge is that all of our dependencies may also have dependencies, and their dependencies also have dependencies etc. You get the picture. I guess this is what's known as dependency hell, and what management tools like Maven etc. are for.

Since this isn't a popular, well-known language, there aren't any dedicated tools for the job. My first thought was to write a Python or bash script with a recursive function or something like that. I have also seen that both Maven and Gradle plugins exist, but they don't seem to be maintained.

So my question is - how does one handle this when dealing with "generic" systems which doesn't have a dedicated build management system? I basically need something to do the following for me:

1. Git clone app X into workspace A
2. Check app X's dependencies
1. For each of app X's dependencies, repeat stage 1 and 2
3. Build application from workspace A

Will something like Maven or Gradle handle this for me? Or is it easier to just script this?

https://redd.it/qyup2q
@r_devops

Git publisher on Jenkins does not work properly

Hi to everyone,
I have a problem with Jenkins freestyle project, and in particular I set the whole project with just a shell scripting that make some commands, and after that it delete a file. I want to update my repository (main branch) with the change. I’m using Git Publisher plugin, but even if in the log Jenkins print the push command that seems to work well, I do not see the changes on my GitHub repository.

I already set my credentials (it uses it well, just because when I want to create tags, it will create them without problems); so I don’t know what is the problem…

https://redd.it/qz4tv6
@r_devops

Switch Jobs to take a new offer focused in Kubernetes?

I have recently being offered a new role, I told my current job, and they instantly matched the offer. So my thoughts are, continue to finish out the projects for my current role (AWS, Containers, EC2 terraform, puppet, powershell jenkins, java) and stay and not use Kubernetes, or take this new role where they work solely with Kubernetes on Azure. I am not sure what else they do entirely, but I am expected to build out their cloud K8 infrastructure.

I would be very keen to learn, and they know I am no expert in k8s, as I told them in no uncertain terms however I was recommended to them by old colleague (project manager). So my issue is do I just take dive off the deep end here and go learn Kubernetes. Or do I stay and really strengthen out my skills in Terraform puppet etc.

I bring this up as I am only a little under 2 years into my DevOps career and am not super confident in my skill set holding water.

I am basically asking here, do ya'll here think it is worth the risk to get a good hands-on experience with Kubernetes? I am basically asking for opinions.

https://redd.it/qz72kh
@r_devops

what does Re-enabling F5 for load balancer mean?

does it simply mean load balancers, start doing your job and balancing requests to live servers?

also when we enable tasks on a host, does that mean just start the app on the app server, start accepting traffic on a webserver, etc?

https://redd.it/qzah4s
@r_devops

Example of CKA exam exercise

Short video walkthrough of one of the possible and trickier tasks you might have to do during Certified Kubernetes Administrator exam: https://youtu.be/-wepQpDAGuI

https://redd.it/qzje4w
@r_devops

Select VPN and Subnet as parameters with troposphere.

I am creating a CloudFormation template using Troposphere, but when I create from CloudFormation the stack, AWS fails because do not know which VPN to use. So, I am trying to figure out how to specify as parameters the VPN and subnet. If someone has more experience with this, I am so glad about your comments.

https://redd.it/qzoggd
@r_devops

How to pass the new Kubernetes and Cloud Native Associate (KCNA) Exam

I have just finished my latest blog to help people study and pass the new KCNA exam, hope it helps!
https://blog.bradmccoy.io/how-to-pass-your-kcna-exam-cf98cfa7d70f

https://redd.it/qzr44i
@r_devops

What are the biggest issues you guys face on a day to day?

Hey guys, as the title says I'm just curious towards what are the biggest issues you guys face on a day to day or consistent basis working in devops? And how are you solving these issues if they are solvable?

https://redd.it/qzxvfc
@r_devops

Best way to overcome my weaknesses?

I'm in a "senior sre" position (seems like just a title and not reflective of the truth) where I mainly do k8s, cicd, platform and iac work. I'd like to start moving towards architecture and providing solutions for developers, like helping them to choose cloud technologies for their projects. The problem is that while I'm strong in the infra and cicd and networking world I have never been a developer. I write IAC with typescript and do some golang for tests but that's about it.

I'd say my weakest points that I will need to improve to reach my goal is probably writing code and having a deeper understanding of technology requirements (message queues, db tech, step functions, etc), as well as understanding database technologies (differences between them, what to use and when, writing queries, their architecture, etc).

Should I go to college for these things? Ie computer science. Or should I start taking online courses? Does anyone have experience going from becoming a platform architect (providing a platform for devs) to a true SRE? would the aws architect certs help me with this?

Thank you!

https://redd.it/qzw4k1
@r_devops

Production Support

Where does production support fall when it comes to development on servicenow or RPA platforms for example?

Does production support land on developers with support of the platform's engineer/admin? Does it land on the engineers of those platforms? Do you have a point person or team that does triage?

Curious how this works elsewhere for production support. Our company's product is not an app or software itself. Our development is all improvements that enhance internal functions.

https://redd.it/r010al
@r_devops

I deploy using AWS beanstalk, should I move to cloudformation or something else?

Hi,


I work on a startup and I have a small experience on deploying software to the cloud. I spent quite some time setting up the deployment of our solution (python-django API Back) to AWS servers using Docker + AWS elasticbeanstalk. Bitbucket push triggers circleCI jobs that test and then deploy to beanstalk.

In the near future, we will need a higher availability of our API as well as complex deployment strategies (ex: Canary Deployments). We will also want (in a little further future) for our application to run on client's cloud environment that could be google or azure.

I wonder if I should stick to this and spend my time learning more about elasticbeanstalk or if I should use that time learning something lower level like cloudformation.

https://redd.it/r0744n
@r_devops

What we learnt by migrating from CircleCI to Buildkite

We recently rolled out a new CI system at Hasura and we are writing this post to take you on a journey about why and how we did it. This is a collection of ideas and implementation details which we believe to have helped us arrive at an optimal solution that worked out for us.


To control costs and for a better CI experience, we moved from CircleCI to Buildkite. We wrote about the various choices we explored, how costing differs in these solutions and why we chose Buildkite.


Read it from here - https://hasura.io/blog/what-we-learnt-by-migrating-from-circleci-to-buildkite/

https://redd.it/r0al0u
@r_devops

Ops guy transitioning to SRE or devops

Hi people,


Just popping in hoping to get some feedback from people in the devops/SRE field. I'm considering transitioning into SRE or devops.


I'm a 42 years old guy, been into IT since my mid 20's and almost all of my career is in ops (virtualization, networking, linux).


Can't say I have done any extensive development but I'm quite comfortable scripting automation with bash and python. I have pieced together automation and CLI tools (integrations with APIs and so on) that made my life easier. Got a few projects on my github account.


Fast forward until 2-3 years ago. I work for a pretty small company (50 employees). I started, on my own initiative, getting into Azure as I didn't see how we would be able to keep hosting our own hardware and continue to scale. Slowly but surely I adopted IaC using Terraform. I started implementing config management using Ansible, for some 200+ linux hosts. I have set up and learned Kubernetes for running micro services loads. I have developed a couple of micro services myself for executing automation tasks (kind of like my own Ansible Tower) and simple stuff like Slack integration for alerts. I'm running Prometheus for gathering metrics on everything we run in K8s. The CI/CD pipeline I have set up is based on Bitbucket pipelines to automate tests/validation and trigger image build. FluxCD running in our K8s cluster to automatically deploy new versions. I love git and I kind of like the concept of gitops.


Now, I feel I have outgrown my company. Actually I have felt that way for several years but stayed because of loyalty (which I realize now was pretty stupid considering the management have changed, and now they don't really care about tech). The concepts of devops just doesn't make that much sense within the company, and the other ops people don't seem too keen on learning new skills. The only driving force to learn has come from myself.

​

Anyway, I see there are a tonne of jobs out there, mainly devops and SRE type roles. I feel what I'm lacking to move on is experience working in a team, working within that discipline framework, and considering I'm self taught I might not adhere to best practices. And finally I'll admit that I lack some confidence. Like, I know a bunch of these tools, but perhaps not on the level I need to.


What do you guys think? Would I be a reasonable fit for devops or SRE? Sorry for the rambling, I'm just looking for some feedback! Many thanks!

https://redd.it/r0f40b
@r_devops

Netmaker v0.9 release updates

# Hi all,

Just wanted to keep you updated on the progress with Netmaker, a WireGuard VPN automation and management platform.

We've just released v0.9, which comes with a couple cool updates. The first is a totally refactored UI. It doesn't change the functionality much, but we think it looks and works a lot better.

The second is support for routers. We've now built in support for OpenWRT and FreeBSD systems. This means you should be able to run the netclient on a lot of open source router OS's and mesh everything behind the router into your network.

For those of you who are just now learning about Netmaker, here's a recap on the features:

Centralized WireGuard config management
Automatic Peer discovery and configuration.
UDP hole punching to establish connections behind NAT, firewall, etc.
Add in phones to your networks
Create gateways into networks (e.g. home / office network)
Configure peers as relay servers
self-hosted with easy 1-click installation
Works on Linux, Mac, Windows, and FreeBSD devices.

https://redd.it/r0d3ws
@r_devops

Continuous Deployment with Github Actions: An Example

Wrote a blog that takes a deeper dive into setting up CD with Github Actions https://www.dolthub.com/blog/2020-11-23-continous-deployment-with-github-actions/

https://redd.it/r0i16n
@r_devops