Add convenience method to read JSON file

[sindresorhus]

Is your feature request related to a problem? Please describe.

Reading a JSON file is a very common need. It would be nice to have a convenience method for it.

In addition to making it easier to do this common task, it could also:

• Strip UTF-8 BOM
• Make syntax errors more human-friendly

For example, I have a package for this which is quite popular and has 700 dependents, so it's clearly a need for it: https://www.npmjs.com/package/load-json-file

It is also quite a popular question on Stack Overflow: https://stackoverflow.com/questions/10011011/using-node-js-how-do-i-read-a-json-file-into-server-memory?rq=1

Having it in Node.js core would also help reduce dependency trees, which would in turn reduce npm install times and node_modules bloat. It would also improve DX by having better syntax errors.

Describe the solution you'd like

I propose adding a fs.readJson() and fs.readJsonSync() method.

Describe alternatives you've considered

Continue using https://www.npmjs.com/package/load-json-file

---

I didn't want this to dilute the proposal, but one future enhancement to this could be to make the JSON parsing fully async by doing it in a worker.

[targos]

I tried to do an initial implementation in #37944 but nobody approved it.

[aduh95]

If we land web methods like .text() and .arrayBuffer() to FileHandle like in #40021, I think we may also consider adding .json().
There's also the promise of being able to do await import('./pathToFile.json') if/when #37375 lands.

[jimmywarting]

with that said ☝️ then I don't think we will need any special fs.readJson function

[sindresorhus]

There's also the promise of being able to do await import('./pathToFile.json') if/when #37375 lands.

Importing something is semantically different from reading something directly from disk. I assume import caches the import. And bundlers will treat it differently, often inlining imports, but ignoring fs reading. In addition, fs methods read relative to CWD, while dynamic imports read relative to the file.

[sindresorhus]

with that said ☝️ then I don't think we will need any special fs.readJson function

FileHandle is not a replacement for fs methods.

[jasnell]

In the stream/consumers module that currently has not yet gone out in a release because it hasn't yet been backported to 16.x or 14.x (there's a small smever-major that it currently depends on)... there is a json() method that takes either a ReadableStream, stream.Readable or AsyncIterator as input...

so... you can do something like

await json(fs.createReadStream('...'));

And it should just work.

As a reminder, stream/consumers exposes:

• await arrayBuffer(input)
• await blob(input)
• await buffer(input)
• await text(input)
• await json(input)

Where input can be ReadableStream, stream.Readable, or AsyncIterator

This has already landed, just waiting to go out in a release.

[jimmywarting]

@jasnell, i experimented with stream consumer a bit... i think it could be useful if blob accepted a optional mime type as well (just a suggestion)
await blob(input, [type])

[jasnell]

@sindresorhus

I didn't want this to dilute the proposal, but one future enhancement to this could be to make the JSON parsing fully async by doing it in a worker.

You could do this fairly easily with FileHandle and the new stream/consumers... e.g. open the file as a FileHandle, transfer that to a Worker, use await json(fh.createWebStream()) and postMessage the results back. It would be fairly simple.

[jasnell]

@jimmywarting:

i experimented with stream consumer a bit... i think it could be useful if blob accepted a optional mime type as well (just a suggestion)

Yeah, let's take that to a separate issue. I think adding an options argument to each of the convenience consumers makes sense.

[benjamingr]

Can anyone explain what's wrong with requireing the JSON file? (or importing it in ESM land with json modules active)?

Like, why is fs.readJson('./foo.json better than require('./foo.json') (old) or await import('./foo.json')? Is the "moduleness" of it (caching behaviour etc) the problematic bit?

[sindresorhus]

@benjamingr #40089 (comment)

[benjamingr]

In addition, fs methods read relative to CWD, while dynamic imports read relative to the file.

Oh yeah that makes sense. Especially if this lets us (maybe?) parse the JSON on a libuv thread and not the main thread though I suspect the main use case for this is probably config files in which the pathing is the important bit.

[kaizhu256]

Can anyone explain what's wrong with requireing the JSON file? (or importing it in ESM land with json modules active)?

require reads the file once and caches it. I've come across use-cases where json-file is intentionally modified and needs to be physically re-read.

That said, typing JSON.parse(await fs.promises.readFile(...)) is not that much of a chore, nor does readability take that much of a hit.

[bmeck]

Probably the most important thing here is that files with a BOM do not parse with JSON.parse: JSON.parse('\uFFEF""')