Skip to main content

Posts

Showing posts from 2018

JS constructor names make it easier to analyze memory dumps

As they say, "There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors". Out of these 2/3 things, naming is the most difficult. We all know that being lazy when naming things will hurt you down the road when you'll need to make sense of the code that you've written half a year ago. However, one more interesting  aspect of good naming paying off I didn't previously think about is memory dump analysis.
Let's imagine, you are facing what looks like a memory leak or some strange behavior of the system and memory dump may help you understand what's happening inside the brain of your system in a particular environment. So you dump your heap, Google Chrome does a great job of grouping your object by constructor name, but all you see is this:

Since your service classes where isolated in your modules/files and you default-exported them, it seemed unnecessary to think of fancy names. The path to the file identifies the…

AWS EFS and ENOTEMPTY (not reading your own writes)

We often perform non-empty directory removal in our Node.js application. Obviously, one can not do this directly, directory contents first need to be cleared recursively. Of course, we could have used an NPM module like rimraf. However, we use our own simple, Promise-based recursive function with the only special error handling being ignoring ENOENT.

It's been working for years without problems until we switched to Elastic File System (Amazon EFS). We started receiving ENOTEMPTY error when trying to remove a directory immediately after clearing it. It wasn't immediately clear what was happening. However, in retrospect, it was so obvious - we were observing a classic distributed system inconsistency - we were not "reading our own writes". The only solution is to retry a few times with an increasing delay. That's actually exactly what rimraf is doing. This is how it looks in our case: