Skip to main content

Posts

Showing posts from June, 2018

AWS EFS and ENOTEMPTY (not reading your own writes)

We often perform non-empty directory removal in our Node.js application. Obviously, one can not do this directly, directory contents first need to be cleared recursively. Of course, we could have used an NPM module like rimraf. However, we use our own simple, Promise-based recursive function with the only special error handling being ignoring ENOENT.

It's been working for years without problems until we switched to Elastic File System (Amazon EFS). We started receiving ENOTEMPTY error when trying to remove a directory immediately after clearing it. It wasn't immediately clear what was happening. However, in retrospect, it was so obvious - we were observing a classic distributed system inconsistency - we were not "reading our own writes". The only solution is to retry a few times with an increasing delay. That's actually exactly what rimraf is doing. This is how it looks in our case: