How to deal with node_modules in a Dockerized Node application?

Posted on — Leave a comment

There are tens of thousands of blogs, articles and forum threads out there on containerizing a Node.js application. We have everything from outright bad advices to informed and tested opinions. This short post is about a very specific aspect of Dockerinzing a Node app, something that is usually not addressed or given as an after-thought in these articles.

Once you have a Node app up and running in Docker, things are well until you – like all good developers – start adding or removing NPM packages. Nasty things happen, such as:

  • packages get installed on your local (host) filesystem but are not available inside the container
  • packages get installed on your container’s filesystem but are not available on your local
  • there’s a mismatch between versions of the same package in host and container
  • permission issues while installing or removing packages

All these can be solved by correctly using Docker volumes. A volume is basically a way to make parts of container’s filesystem available to host and vice-versa. Not understanding volumes properly may also lead to one of the issues listed above.

I had a similar experience in one of my projects. See the fix:
https://github.com/universalnative/un-website/commit/0eaaca98adac46bf9b1553e66d6acd9d731db43d

As noted in the stackoverflow answer cited in the commit:

When docker builds the image, the node_modules directory is created within the worker app directory, and all the dependencies are installed there. Then on runtime the worker app directory from outside docker is mounted into the docker instance (which does not have the installed node_modules), hiding the node_modules you just installed. You can verify this by removing the mounted volume from your docker-compose.yml.

https://stackoverflow.com/a/32785014/1775160

This is a powerful thing to remember. Once you know how volumes work, you’ll be able to better troubleshoot your Node app when things are wrong. For example, when I recently installed a new NPM package in my local I knew I had to do something like this to make it available inside the container as well:

https://github.com/universalnative/un-website/commit/3fa9c9a79c35a3ddbf39079d16b7dbca12105c0d#diff-4e5e90c6228fd48698d074241c2ba760

Why didn’t I just mount my host node_modules to container’s node_modules? Here’s the explanation:

An alternative approach is to mount host node_modules as a volume in container, but that will override container’s own node_modules folder with host’s. Keeping things independent allows for cleaner and easier troubleshooting of installed/missing packages.

This the approach I prefer, which of course is not certified gold. I works for me well. Besides, yarn installing packages each time container comes up does so incrementally (only new packages are fetched and installed). A win-win 🙂

Hello, Deno

Posted on — Leave a comment

Deno v1 is here, and so is my experiment. Unlike my usual first programs, this one is a bit better than a pure “Hello, world”. So not only it is a meaningful program (does not literally print out “hello world”), it also has data models, unit tests and CI integration (Travis).

From its documentation:

A super-simple Deno app that pulls (from an API) national COVID-19 stats for India, and displays the total count of people who have recovered from the disease till yesterday.

What else did you expect? Let’s spread some positivity in tough times!

Check it out here and let me know your thoughts (if you exist):

https://github.com/anuragbhd/hello-deno

Btw, this is the best damn geeky one-liner code to explain what is Deno. Totally cool.

Express API Refkit

Posted on — Leave a comment

https://github.com/anuragbhd/express-api-refkit

I frequently create APIs for muse/professional apps, and every time I find myself scrambling to pick the best pieces from my previously built APIs or online repositories.

To streamline this, I recently created this reference kit (mostly for myself) to help me in writing a production-grade Express-based API from scratch. Sharing with this group in hopes that my fellow XTians will benefit from it when writing their own RESTful APIs.

As noted in the docs:

This is NOT a starter template. The purpose of this repository is to provide reference code to anyone looking to create a beautiful, secure and well-tested Node/Express-based API from scratch.

Creating from scratch has the invaluable benefit of learning while doing. Most likely, there will be struggle which will make you better. You will know by heart all parts under the hood and exactly how they interact with each other. Basing your project off of a starter kit takes that away from you. Hence the need for a reference kit (or refkit, as I like to call it).

If you are experienced in Express, feel free to open pull requests to add improvements.

Contributions in any form are welcome 💁‍♂️

Featured image credit Nick Youngson CC BY-SA 3.0 Alpha Stock Images

Book Review: The 4-Hour Work Week by Tim Ferriss

Posted on — Leave a comment

Cross-posted from Goodreads: https://www.goodreads.com/review/show/3196927726

This one is bold! May not be for the faint of heart. 4WW is a book about the realization that living your dreams is as important as doing your 9-5 job. Ferriss’ casual style of writing is motivating and quirky. I personally found his thought process and “lifestyle design” tips to be extremely helpful. Even though at times his (or one of his interviewee’s) story feels larger than life, I would recommend this book to anyone looking to once and for all stop deferring their “grand” vacation/retirement plans and start living them today. Ferriss provides practical advice to make time to do all that you want by somehow managing to complete your routine job work in exponentially less time.

Having read Deep Work: Rules for Focused Success in a Distracted World before this one, I found a lot of ideas and techniques very-very similar described by Newport. It’s important to realize, though, that 4WW came a decade before!