Consider something clever for extension-less executables

[haraldrudell]

It would be nice if ECMAScript shebang files with no extension
a. worked :)
b. were assumed to be modules for Node.js current+

Especially for server people that have lots of command-line scripts

And I think we should plan for a future without require

exhibit:

node --print '"#!/usr/bin/env node\n// node --experimental-modules e\nimport {inspect} from QutilQ\nconsole.log(`${inspect(QabcQ)}`)".replace(/Q/g, String.fromCharCode(39))' >e && chmod +x e
./e
(node:38910) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
/opt/foxyboy/sw/pri/subtree/ecmascript2049/packages/e:3
import {inspect} from 'util'
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1063:16)
    at Module._compile (internal/modules/cjs/loader.js:1111:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
    at Module.load (internal/modules/cjs/loader.js:996:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47
$ cat e
#!/usr/bin/env node
// node --experimental-modules e
import {inspect} from 'util'
console.log(`${inspect('abc')}`)

This shebang no work either:
#!/usr/local/bin/node --experimental-modules

The solutions at present is to:

1. use .mjs extension
2. symlink to a file that has .mjs extension. Not portable, I suppose, on certain file systems from up north and Applish implementations of WebDAV
3. have a package.json in the same or superior directory. May cause unforeseen troubles
4. symlink to a directory with a package.json. Possibly not portable

node -v && uname -a
v13.11.0
Darwin c87m1.local 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64

A related trouble is if you happen to have a native-code dependency. requires node_modules and such

[devsnek]

Duplicate of #23868

[devsnek]

We can't pull data out of the hashbang (see the issue I linked), so I'm not sure as this issue is phrased there's anything actionable. In general, extensionless mjs is a place where improvement could be made though.

[guybedford]

This use case was explicitly enabled in early implementations, but disabled by the PRs by @bmeck and @GeoffreyBooth in #31021 and #31415.

I disagreed with both of these PRs, but let them pass as I did not want to block new work unnecessarily.

I still think the original approach with both of those PRs reverted would be better (31415 followed by 31021), but will not work on it personally.

If anyone wants to put something together along those lines though it would have my full support and argument in the group, although many disagree with the idea that the process main can be used as resolution information. Personally I have no issue with that since it is environment-static, and any environment-static state is fine to assume in the resolver (the definition of what is allowed!).

[bmeck]

I've stated in the past my concerns with being clever about these things.

Per #31021 , it ended up supporting files without extensions and delegated them to the .js extension handling. I would never want to revert the actual change of errors on unknown extensions it contains. Things like a new .wasi file format changing from a .js handler to its own is very concerning to me.

For context, per #31415 , that was done because of #31388 which was made to support loading WASM entry points for WASI being complicated by extension-less mains. Reverting out any extension-less support was seen as preferred for now as some people didn't want to expand the "type" field in package.json.

@haraldrudell an alternative is to use symlinks that point to a file with an extension or have your package.json contain the mapping like: "bin": {"foo": "bin/foo.js"}.

[guybedford]

Per #31021 , it ended up supporting files without extensions and delegated them to the .js extension handling. I would never want to revert the actual change of errors on unknown extensions it contains. Things like a new .wasi file format changing from a .js handler to its own is very concerning to me

Bin users writing bin files ending in .wasi is a very obscure use case to cause the overall feature to be blocked on. While in theory breaking, such a change would have low enough usage to be possible from a practical standpoint.

[bmeck]

@guybedford I don't understand the last comment, the concern is about .wasi files changing interpretation not necessarily about them being the main entry point. These discussions become more complex as things like application runners do wrap the main entry point and would need to account for the variance.

[GeoffreyBooth]

@haraldrudell If you’re looking for something clever, this gist by @WebReflection certainly fits that bill and might fulfill your need:
https://gist.github.com/WebReflection/8840ec29d296f2fa98d8be0102f08590

[WebReflection]

@GeoffreyBooth thanks for pointing at my good old gist! It looks like time is about right to improve that gist and place a temporary {"type":"module"} package.json file as the dirname "$0" path and/or the user $HOME while executing, but I feel like the elephant in the room is that there's no flag to handle also all imports as ESM when a script is being executed as ESM, which is the expected behavior, IMHO.

[WebReflection]

Hint: --import-type=module sounds about right to me 👋

edit: --default-type=module maybe is a good one too

[guybedford]

I don't understand the last comment, the concern is about .wasi files changing interpretation not necessarily about them being the main entry point

@bmeck the previous code only affected files as the main entry point. So node x.wasi would run it as JS, and yes that would be breaking to change it to wasi in future, but only if it were realistically used by people as a JS entry point with a .wasi extension. The previous approach did not support non-extensions for anything other than the "node main".

[haraldrudell]

This must be done and it is actionable

require is over; it should not be used anymore

note: why I bring it up now, is that I came up with a toolchain where the source, config files, packages and output executable all use import rollup/rollup#3443

I know about that hack suggested in comment. I don't do hacks

the question is what's the best way and who are we breaking. To me personally, I will just retranspile any failing executable using my fancy toolchain: I am good

note that a shebang other than #!/usr/bin/env node is not portable. In particular other shebang, if done for Linux, will not work on macOS and Android

erase the past

[bmeck]

@haraldrudell I have no idea what most of the last comment is trying to say except that you want to break things, but not how nor any response to the other comments in the thread except that gist. Is there are reason none of the alternatives work clearly / what is the clear use case that we are not fulfilling besides a specific solution to the use case?