fix(nextjs,clerk-js,types): Stop Clerk component flickering when used in App Router

[nikosdouvlis]

A few things going on in this PR.

Navigation metadata

When using router.push from next/navigation (app router) the components in the current page will unmount before the navigation. In our case, if a component uses path-routing all internal navigations (for example /user > /user/security use router.push but that means that the component will unmount and immediately mount again in the new URL.

However, our components are client components and they can support client-side-only navigations. This type of navigation is usually called shallow navigation. App router apps can use the native window.history APIs in order to trigger shallow navigations.

We need to differentiate between internal navigations (navigations within the component) and external navigations (navigation outside the scope of the component) as we want to use shallow navigations for the former and normal router.push navigations for the latter.

To achieve that, the clerk-js router now sends a metadata object with each navigation that can be used by the consuming framework wrapper, enabling more fine-grained control over the type of navigation.

Shallow navigations and nextjs

We want to be as little intrusive as possible, so when trigger shallow navigations, we need to make sure to preserve the default next router behavior. When calling the window.history APIs, we should respect how windowHistorySupport and let any changes to the URL update the next hooks as normal.

• windowHistorySupport was enabled by default in next@14.1.0.
• For apps >= 14.1.0, we need to call push/replaceState with null as the first argument for WHS to work
• For older apps, we need to call the APIs with the window.history.state

Refs:

• Add experimental support for history.pushState and history.replaceState vercel/next.js#58335
• Shallow routing on Next.js 13's next/navigation vercel/next.js#48110
• Ability to push changes to url without re-rendering vercel/next.js#18072
• https://github.com/47ng/nuqs/pull/482/files#diff-713238c17ffa37b63ad76fc5a5d9a2c6e923cc009e6593f86adf8fb32feef493R183

See inline comments for more details.

usePathnameWithoutCatchAll

This hook let us automatically determine the path a component is mounted on, while taking into account that components can be mounted in catch-all routes, so simply using usePathname() is not enough.
For example, imagine we are using path-based routing and we have a UserProfile component mounted at /user/[[...rest]]/page.tsx. The possible paths the component can use would be /user and /user/security, but the base path needs to be set as /user always for our internal router to work.

We're now saving the initial value in a ref so it never gets updated after the initial (browser) navigation to the page. For details on what this hook does, please refer to the inline comments.

Description

Checklist

• npm test runs as expected.
• npm run build runs as expected.
• (If applicable) JSDoc comments have been added or updated for any package exports
• (If applicable) Documentation has been updated

Type of change

• 🐛 Bug fix
• 🌟 New feature
• 🔨 Breaking change
• 📖 Refactoring / dependency upgrade / documentation
• other:

Packages affected

• @clerk/backend
• @clerk/chrome-extension
• @clerk/clerk-js
• @clerk/clerk-expo
• @clerk/fastify
• gatsby-plugin-clerk
• @clerk/localizations
• @clerk/nextjs
• @clerk/clerk-react
• @clerk/remix
• @clerk/clerk-sdk-node
• @clerk/shared
• @clerk/themes
• @clerk/types
• build/tooling/chore

[changeset-bot]

🦋 Changeset detected

Latest commit: bc350f8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages

Name	Type
@clerk/clerk-js	Patch
@clerk/nextjs	Patch
@clerk/types	Patch
@clerk/chrome-extension	Patch
@clerk/clerk-expo	Patch
@clerk/backend	Patch
@clerk/fastify	Patch
gatsby-plugin-clerk	Patch
@clerk/clerk-react	Patch
@clerk/remix	Patch
@clerk/clerk-sdk-node	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[panteliselef]

💯 I enjoyed reading that PR description