421 private links
Note: "The key insight is that loading massive files into memory all at once is rarely a good idea, even when technically possible."
- Read the whole file and call it a day with
fs.readFile - Read by byte chunks with file handles and buffers
- (preferred) Use stream
The streams handle the bytes and UTF-8 encoding automatically. Each state can be defined with a callback: error, data, end.
Use appropriate buffer sizes for streams. Default is 64kB. The highWaterMark can adjust it though.
When to use streams: large files, real time data processing, memory is limited, composability, unknown file sizes.
When to use file handles: precise control.
When to use fs/promises: known small files. readFileSync() and writeFileSync() are acceptable for cli tools and scripts, but forbidden for web servers and concurrent apps.
Prefer Promise.all or Promise.allSettled to leverage concurrency.
Handle specific errors. Always. await handle.close() or stream.on('error', () => stream.destroy()). Another solution is await pipeline(source, transform, destinationl)
