From b1cefe16c2bcfa5575e14e483775849e1a7c71bc Mon Sep 17 00:00:00 2001 From: Benni Mack Date: Sun, 2 Aug 2026 12:38:53 +0200 Subject: [PATCH] [FEATURE] Provision functional test instances in composer mode TYPO3 ships two installation modes and only one of them was covered by functional tests: every test instance was a classic mode installation, even though composer mode is what most projects run. Set TYPO3_TESTING_INSTANCE_MODE=composer to provision composer mode instances instead. Classic mode stays the default, and the mode is part of the instance identifier, so a test case's classic and composer instances never collide. A composer mode instance is not its own composer installation. One shared installation containing every system extension and every fixture extension is built once per run by Build/Scripts/setupFunctionalComposerSuperset.php, and each instance borrows its vendor tree and its package artifact by symlink: /vendor -> /vendor /public/_assets -> /public/_assets /config/system own configuration /public own document root, fileadmin, typo3temp/assets /var own caches and logs Per test case only the *active* package set differs, which is expressed by narrowing the artifact in memory. Building a real installation per test configuration was measured at about five seconds each, which for the roughly two hundred configurations in this repository would cost far more than the provisioning this saves. Classes are deliberately not loaded from the shared installation. The root autoloader already maps every system extension and every fixture extension, so no second autoloader is registered and no second copy of any third party package is ever loaded. One consequence: class alias maps that the shared installation registered at install time are not active, because the root installation never processed those packages. Notable details, each of which was a bug on the way and none of which fails loudly: * TYPO3 decides between the classic layout (typo3conf, typo3temp/var) and the composer layout (config, var) by whether the project root and the document root differ - not by composer mode. Test instances therefore set TYPO3_PATH_ROOT to the public directory in composer mode. * Package::getPackagePath() resolves the artifact's relative path against Environment::getProjectPath() once and writes the result back. Caching one PackageCacheEntry for the process therefore binds every package to whichever instance touched it first, and every later instance looks for its packages inside that instance. Only the raw artifact data is cached; the entry is rebuilt whenever the instance changes. * On the command line in composer mode TYPO3 reports the console binary as the entry script, because in a real installation it sits outside the document root. Functional tests are command line processes simulating web requests, and NormalizedParams derives the site URL by subtracting the entry script's directory from the request - so it subtracted typo3/sysext/core/bin/ from every request, the site path collapsed to an empty string, and every generated link silently lost its leading slash. The front end entry script is reported instead, which is what a web request has. * A console command run in a sub process cannot use EXT:core/bin/typo3: that script derives its autoloader from its own directory, and since the shared installation's packages are symlinks into the checkout, it resolves back to the root autoloader, never defines TYPO3_COMPOSER_MODE and runs in classic mode. Composer mode instances get a generated entry point instead, which getCliEntryPoint() hands to the test. * Applications bootstrapped by a test - the install tool, notably - must boot through bootTestInstance(), because Bootstrap resolves the artifact from the root installation, and the mono repo has none. * fileadmin and typo3temp/assets are web accessible and belong below public/. Getting that wrong makes the default file storage point at nothing. * Frontend sub requests bootstrap a second time and need the same narrowed package set. Their server parameters also have to name the real document root, or the site path is derived from a script outside the public path. * The sites configuration cache removal in tearDown() has to use the layout's variable path, or it silently does nothing and the stale configuration leaks into the next test. * Paths declared by test cases are given relative to the document root, which only carries the extension tree in classic mode; they now fall back to the original root. A "PKG:typo3/app:" resource identifier carries a path relative to the project root, which is the document root only in classic mode, so getAppRelativePublicPath() is provided for tests naming such resources. TestingBootstrapRunner overrides Bootstrap::createPackageCache() through late static binding, so no change to typo3/cms is required. --- .../ComposerMode/ComposerModeInstance.php | 317 ++++++++++++++++++ .../ComposerMode/InMemoryPackageCache.php | 56 ++++ .../Core/Functional/FunctionalTestCase.php | 246 ++++++++++++-- Classes/Core/SystemEnvironmentBuilder.php | 27 +- Classes/Core/Testbase.php | 47 ++- Classes/Core/TestingBootstrapPackageCache.php | 33 ++ Classes/Core/TestingBootstrapRunner.php | 42 +++ 7 files changed, 724 insertions(+), 44 deletions(-) create mode 100644 Classes/Core/Functional/Framework/ComposerMode/ComposerModeInstance.php create mode 100644 Classes/Core/Functional/Framework/ComposerMode/InMemoryPackageCache.php create mode 100644 Classes/Core/TestingBootstrapPackageCache.php create mode 100644 Classes/Core/TestingBootstrapRunner.php diff --git a/Classes/Core/Functional/Framework/ComposerMode/ComposerModeInstance.php b/Classes/Core/Functional/Framework/ComposerMode/ComposerModeInstance.php new file mode 100644 index 00000000..5c6e1fed --- /dev/null +++ b/Classes/Core/Functional/Framework/ComposerMode/ComposerModeInstance.php @@ -0,0 +1,317 @@ +/vendor -> /vendor + * /public/_assets -> /public/_assets + * /config/system/… own configuration + * /var, fileadmin, … own state + * + * Package paths inside the artifact are relative to the project path, so the vendor + * symlink is what makes them resolve. Classes are *not* loaded from the superset: + * the root autoloader already maps every system extension and every fixture extension, + * so registering a second autoloader - and with it a second copy of every third party + * package - is unnecessary and avoided. + * + * Per test case, only the active package set differs. That is expressed by narrowing the + * artifact's configuration, which the package manager treats as "these are active" while + * still knowing about every installed package. + * + * @internal + */ +final class ComposerModeInstance +{ + /** + * Raw artifact data, read from disk once per process. + * + * Deliberately the raw array and not the PackageCacheEntry built from it: see + * getSupersetArtifact(). + */ + private static ?array $supersetArtifactData = null; + + /** + * The artifact currently handed out, and the instance it was built for. + */ + private static ?PackageCacheEntry $supersetArtifact = null; + private static string $supersetArtifactInstancePath = ''; + + /** + * Absolute path of the shared superset installation. + */ + public static function getSupersetPath(): string + { + $configured = (string)getenv('TYPO3_TESTING_SUPERSET_PATH'); + if ($configured !== '') { + return rtrim($configured, '/'); + } + if (!defined('ORIGINAL_ROOT')) { + throw new Exception('ORIGINAL_ROOT is not defined, cannot locate the composer superset.', 1754092800); + } + return rtrim(ORIGINAL_ROOT, '/') . '/typo3temp/var/tests/composer-superset'; + } + + /** + * Fails with an actionable message rather than a confusing bootstrap error when the + * superset has not been built. + */ + public static function assertSupersetIsBuilt(): void + { + $artifact = self::getSupersetPath() . '/vendor/typo3/PackageArtifact.php'; + if (!is_file($artifact)) { + throw new Exception( + sprintf( + 'Composer mode functional tests need the shared composer installation, which was' + . ' not found at "%s". Build it with' + . ' Build/Scripts/setupFunctionalComposerSuperset.php, or run the suite through' + . ' runTests.sh, which builds it for you.', + self::getSupersetPath() + ), + 1754092801 + ); + } + } + + /** + * Creates the instance directory. Everything shared with other instances is a symlink, + * everything a test may write to is the instance's own. + * + * @param non-empty-string $instancePath + * @param non-empty-string[] $additionalFoldersToCreate + */ + public static function provisionInstanceDirectory(string $instancePath, array $additionalFoldersToCreate = []): void + { + $superset = self::getSupersetPath(); + // Plain mkdir rather than GeneralUtility::mkdir_deep: this runs before the + // instance is bootstrapped, so the global configuration the latter reads its + // permission masks from does not exist yet. + // A composer installation splits the instance in two: everything web accessible + // lives below public/, everything else beside it. fileadmin and typo3temp/assets + // are web accessible, var/ and config/ are not - getting that wrong makes the + // default file storage point at a directory that does not exist. + foreach ([ + '/config/system', + '/var/transient', + '/var/log', + '/public', + '/public/fileadmin', + '/public/typo3temp/assets', + '/public/typo3temp/var/transient', + ] as $directory) { + self::createDirectory($instancePath . $directory); + } + foreach ($additionalFoldersToCreate as $directory) { + self::createDirectory($instancePath . '/public/' . ltrim($directory, '/')); + } + self::symlink($superset . '/vendor', $instancePath . '/vendor'); + if (is_dir($superset . '/public/_assets')) { + self::symlink($superset . '/public/_assets', $instancePath . '/public/_assets'); + } + } + + /** + * Name of the command line entry point written into a composer mode instance. + */ + public const CLI_ENTRY_POINT = 'typo3-testing-cli.php'; + + /** + * Writes the command line entry point a composer mode instance is driven by. + * + * Tests that run a console command in a sub process cannot use EXT:core/bin/typo3 in + * composer mode. That script derives its autoloader from its own __DIR__, and since + * the instance borrows the superset's vendor tree - whose packages are themselves + * symlinks into the core checkout - __DIR__ resolves all the way back to the core + * checkout and the sub process boots with the root autoloader. TYPO3_COMPOSER_MODE is + * then undefined, so the sub process runs in classic mode and scans for packages in a + * directory the instance does not have. + * + * The generated script does in a sub process exactly what setUp() does in the test + * process: force composer mode and boot with this instance's narrowed package set. + * + * @param non-empty-string $instancePath + * @param non-empty-string[] $activeExtensionKeys + */ + public static function writeCliEntryPoint(string $instancePath, array $activeExtensionKeys): void + { + $script = sprintf( + <<<'PHP' + get(\TYPO3\CMS\Core\Console\CommandApplication::class) + ->run() + ); + PHP, + var_export(self::getRootAutoloadPath(), true), + var_export($instancePath, true), + var_export($instancePath . '/public', true), + var_export(self::getSupersetPath(), true), + var_export($instancePath, true), + var_export($instancePath, true), + var_export($activeExtensionKeys, true) + ); + file_put_contents($instancePath . '/' . self::CLI_ENTRY_POINT, $script . PHP_EOL); + } + + /** + * The autoloader the test process itself uses - the one that maps every system + * extension and every fixture extension. + */ + private static function getRootAutoloadPath(): string + { + return (new ComposerPackageManager())->getVendorPath() . '/autoload.php'; + } + + /** + * The package cache a test instance boots from: every package of the superset stays + * known, only the given ones are active. + * + * @param non-empty-string $instancePath + * @param non-empty-string[] $activeExtensionKeys + */ + public static function createPackageCache(string $instancePath, array $activeExtensionKeys): PackageCacheInterface + { + $full = self::getSupersetArtifact($instancePath); + $configuration = $full->getConfiguration(); + $narrowed = ['version' => $configuration['version'] ?? 5, 'packages' => []]; + $missing = []; + foreach ($activeExtensionKeys as $key) { + if (!isset($configuration['packages'][$key])) { + $missing[] = $key; + continue; + } + $narrowed['packages'][$key] = $configuration['packages'][$key]; + } + if ($missing !== []) { + throw new Exception( + sprintf( + 'Extension(s) "%s" are not part of the composer superset installation. Every system' + . ' extension and every fixture extension below Tests/ is installed into it, so this' + . ' usually means the superset is stale - rebuild it with' + . ' Build/Scripts/setupFunctionalComposerSuperset.php.', + implode('", "', $missing) + ), + 1754092802 + ); + } + // The root package of the superset is registered as a package too, and TYPO3 + // expects the app package to be active. + foreach (array_keys($configuration['packages']) as $key) { + if (str_contains((string)$key, '/')) { + $narrowed['packages'][$key] = $configuration['packages'][$key]; + } + } + + return new InMemoryPackageCache( + PackageCacheEntry::fromPackageData( + $narrowed, + $full->getAliasMap(), + $full->getComposerNameMap(), + $full->getPackages() + ) + ); + } + + /** + * The artifact stores package paths relative to the project path, and Package + * resolves them against Environment::getProjectPath() on first access - once, + * writing the absolute path back into the object. Package objects are therefore + * bound to the instance that first touched them, and reusing one artifact across + * instances makes every later instance look for its packages inside the first + * instance's directory. Rebuild the entry whenever the instance changes; only the + * raw artifact data is kept, and reading it from disk happens once per process. + * + * @param non-empty-string $instancePath + */ + private static function getSupersetArtifact(string $instancePath): PackageCacheEntry + { + if (self::$supersetArtifact !== null && self::$supersetArtifactInstancePath === $instancePath) { + return self::$supersetArtifact; + } + if (self::$supersetArtifactData === null) { + self::assertSupersetIsBuilt(); + $data = require self::getSupersetPath() . '/vendor/typo3/PackageArtifact.php'; + if (!is_array($data)) { + throw new Exception('The composer superset package artifact could not be read.', 1754092803); + } + self::$supersetArtifactData = $data; + } + self::$supersetArtifactInstancePath = $instancePath; + + return self::$supersetArtifact = PackageCacheEntry::fromCache(self::$supersetArtifactData); + } + + private static function createDirectory(string $path): void + { + if (is_dir($path)) { + return; + } + if (!@mkdir($path, 0o777, true) && !is_dir($path)) { + throw new Exception(sprintf('Directory "%s" could not be created.', $path), 1754092805); + } + } + + private static function symlink(string $from, string $to): void + { + if (is_link($to) || file_exists($to)) { + return; + } + if (!@symlink($from, $to)) { + throw new Exception(sprintf('Creating link failed: from %s to %s', $from, $to), 1754092804); + } + } +} diff --git a/Classes/Core/Functional/Framework/ComposerMode/InMemoryPackageCache.php b/Classes/Core/Functional/Framework/ComposerMode/InMemoryPackageCache.php new file mode 100644 index 00000000..056df2bf --- /dev/null +++ b/Classes/Core/Functional/Framework/ComposerMode/InMemoryPackageCache.php @@ -0,0 +1,56 @@ +entry; + } + + public function store(PackageCacheEntry $cacheEntry): void + { + // Nothing to store: the entry is derived from the superset artifact on every boot. + } + + public function invalidate(): void + { + // Nothing to invalidate for the same reason. + } + + public function getIdentifier(): string + { + return $this->entry->getIdentifier() ?? 'testing-framework-composer-mode'; + } +} diff --git a/Classes/Core/Functional/FunctionalTestCase.php b/Classes/Core/Functional/FunctionalTestCase.php index 1dbec1f4..4992e801 100644 --- a/Classes/Core/Functional/FunctionalTestCase.php +++ b/Classes/Core/Functional/FunctionalTestCase.php @@ -35,8 +35,10 @@ use TYPO3\CMS\Core\Utility\GeneralUtility; use TYPO3\CMS\Core\Utility\HttpUtility; use TYPO3\CMS\Frontend\Http\Application; +use TYPO3\TestingFramework\Composer\ComposerPackageManager; use TYPO3\TestingFramework\Core\BaseTestCase; use TYPO3\TestingFramework\Core\Exception; +use TYPO3\TestingFramework\Core\Functional\Framework\ComposerMode\ComposerModeInstance; use TYPO3\TestingFramework\Core\Functional\Framework\DataHandling\DataSet; use TYPO3\TestingFramework\Core\Functional\Framework\DataHandling\Snapshot\DatabaseAccessor; use TYPO3\TestingFramework\Core\Functional\Framework\DataHandling\Snapshot\DatabaseSnapshot; @@ -44,6 +46,8 @@ use TYPO3\TestingFramework\Core\Functional\Framework\Frontend\InternalRequest; use TYPO3\TestingFramework\Core\Functional\Framework\Frontend\InternalRequestContext; use TYPO3\TestingFramework\Core\Testbase; +use TYPO3\TestingFramework\Core\TestingBootstrapPackageCache; +use TYPO3\TestingFramework\Core\TestingBootstrapRunner; /** * Base test case class for functional tests, all TYPO3 CMS @@ -77,6 +81,13 @@ */ abstract class FunctionalTestCase extends BaseTestCase implements ContainerInterface { + /** + * Where the instance configuration is written. A classic mode installation keeps it + * below the document root, a composer mode installation beside it. + */ + private const SETTINGS_DIR_CLASSIC = 'typo3conf/system'; + private const SETTINGS_DIR_COMPOSER = 'config/system'; + /** * Unique identifier for this test case. Location of the test * instance and database name depend on this. Calculated early in setUp() @@ -264,15 +275,16 @@ abstract class FunctionalTestCase extends BaseTestCase implements ContainerInter private ContainerInterface $container; /** - * These two internal variable track if the given test is the first test of - * that test case. This variable is set to current calling test case class. - * Consecutive tests then optimize and do not create a full - * database structure again but instead just truncate all tables which - * is much quicker. + * True when this test had to provision the instance, i.e. when it is the + * first test in this process using this instance configuration. */ - private static string $currentTestCaseClass = ''; private bool $isFirstTest = true; + /** + * Test case class currently being executed, used to detect the first test of a class. + */ + private static string $currentTestCaseClass = ''; + /** * Set up creates a test instance and database. * @@ -286,8 +298,13 @@ protected function setUp(): void $this->identifier = static::getInstanceIdentifier(); $this->instancePath = static::getInstancePath(); - putenv('TYPO3_PATH_ROOT=' . $this->instancePath); + // TYPO3 derives the layout of an installation from whether the project root and + // the public path differ: same directory means the classic layout with typo3conf + // and typo3temp/var, different directories mean the composer layout with config + // and var next to a public/ document root. putenv('TYPO3_PATH_APP=' . $this->instancePath); + putenv('TYPO3_PATH_ROOT=' . $this->instancePath + . ($this->getInstanceMode() === 'composer' ? '/public' : '')); $testbase = new Testbase(); $testbase->setTypo3TestingContext(); @@ -308,11 +325,20 @@ protected function setUp(): void // Reusing an existing instance. This typically happens for the second, third, ... test // in a test case, so environment is set up only once per test case. GeneralUtility::purgeInstances(); - $this->container = $testbase->setUpBasicTypo3Bootstrap($this->instancePath); + if ($this->getInstanceMode() === 'composer') { + TestingBootstrapPackageCache::$packageCache = ComposerModeInstance::createPackageCache( + $this->instancePath, + $this->getActiveExtensionKeys() + ); + } + $this->container = $testbase->setUpBasicTypo3Bootstrap( + $this->instancePath, + $this->getInstanceMode() === 'composer' + ); $this->initializeTestDatabaseAndTruncateTables($testbase, $this->initializeDatabase, $dbPathSqlite, $dbPathSqliteEmpty); $testbase->loadExtensionTables(); } else { - DatabaseSnapshot::initialize(dirname($this->getInstancePath()) . '/functional-sqlite-dbs/', $this->identifier); + DatabaseSnapshot::initialize(dirname(static::getInstancePath()) . '/functional-sqlite-dbs/', $this->identifier); $testbase->removeOldInstanceIfExists($this->instancePath); // Basic instance directory structure $testbase->createDirectory($this->instancePath . '/fileadmin'); @@ -334,11 +360,22 @@ protected function setUp(): void 'Resources/Core/Functional/Extensions/json_response', 'Resources/Core/Functional/Extensions/private_container', ]; - $testbase->setUpInstanceCoreLinks($this->instancePath, $defaultCoreExtensionsToLoad, $this->coreExtensionsToLoad); - $testbase->linkTestExtensionsToInstance($this->instancePath, $this->testExtensionsToLoad); - $testbase->linkFrameworkExtensionsToInstance($this->instancePath, $frameworkExtension); - $testbase->linkPathsInTestInstance($this->instancePath, $this->pathsToLinkInTestInstance); - $testbase->providePathsInTestInstance($this->instancePath, $this->pathsToProvideInTestInstance); + if ($this->getInstanceMode() === 'composer') { + // Composer mode instances borrow the shared installation's vendor tree + // instead of linking extensions into a classic directory structure. + ComposerModeInstance::provisionInstanceDirectory($this->instancePath, $this->additionalFoldersToCreate); + ComposerModeInstance::writeCliEntryPoint($this->instancePath, $this->getActiveExtensionKeys()); + } else { + $testbase->setUpInstanceCoreLinks($this->instancePath, $defaultCoreExtensionsToLoad, $this->coreExtensionsToLoad); + $testbase->linkTestExtensionsToInstance($this->instancePath, $this->testExtensionsToLoad); + $testbase->linkFrameworkExtensionsToInstance($this->instancePath, $frameworkExtension); + } + // Destinations of linked and provided paths are given relative to the document + // root. In classic mode that is the instance directory itself, in composer mode + // it is the public/ directory inside it. + $documentRoot = $this->instancePath . ($this->getInstanceMode() === 'composer' ? '/public' : ''); + $testbase->linkPathsInTestInstance($documentRoot, $this->pathsToLinkInTestInstance); + $testbase->providePathsInTestInstance($documentRoot, $this->pathsToProvideInTestInstance); $localConfiguration = []; $localConfiguration['DB'] = $testbase->getOriginalDatabaseSettingsFromEnvironmentOrLocalConfiguration(); @@ -418,15 +455,32 @@ protected function setUp(): void $localConfiguration['SYS']['caching']['cacheConfigurations']['imagesizes']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\NullBackend'; $localConfiguration['SYS']['caching']['cacheConfigurations']['pages']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\NullBackend'; $localConfiguration['SYS']['caching']['cacheConfigurations']['rootline']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\NullBackend'; - $testbase->setUpLocalConfiguration($this->instancePath, $localConfiguration, $this->configurationToUseInTestInstance); - $testbase->setUpPackageStates( + $testbase->setUpLocalConfiguration( $this->instancePath, - $defaultCoreExtensionsToLoad, - $this->coreExtensionsToLoad, - $this->testExtensionsToLoad, - $frameworkExtension + $localConfiguration, + $this->configurationToUseInTestInstance, + $this->getInstanceMode() === 'composer' ? self::SETTINGS_DIR_COMPOSER : self::SETTINGS_DIR_CLASSIC + ); + if ($this->getInstanceMode() === 'composer') { + // No PackageStates in composer mode: the package set comes from the + // artifact of the shared installation, narrowed to this test case. + TestingBootstrapPackageCache::$packageCache = ComposerModeInstance::createPackageCache( + $this->instancePath, + $this->getActiveExtensionKeys() + ); + } else { + $testbase->setUpPackageStates( + $this->instancePath, + $defaultCoreExtensionsToLoad, + $this->coreExtensionsToLoad, + $this->testExtensionsToLoad, + $frameworkExtension + ); + } + $this->container = $testbase->setUpBasicTypo3Bootstrap( + $this->instancePath, + $this->getInstanceMode() === 'composer' ); - $this->container = $testbase->setUpBasicTypo3Bootstrap($this->instancePath); $this->initializeTestDatabase( $this->container, $testbase, @@ -450,14 +504,18 @@ protected function tearDown(): void // Remove any site configuration, and it's cache files, most likely created by SiteBasedTestTrait if (!in_array('typo3conf/sites', $this->pathsToLinkInTestInstance, true) && !in_array('typo3conf/sites/', $this->pathsToLinkInTestInstance, true) - && is_dir($this->instancePath . '/typo3conf/sites') + && is_dir($this->getSiteConfigurationPath()) ) { - GeneralUtility::rmdir($this->instancePath . '/typo3conf/sites', true); + GeneralUtility::rmdir($this->getSiteConfigurationPath(), true); } - if (file_exists($this->instancePath . '/typo3temp/var/cache/code/core/sites-configuration.php') - && is_file($this->instancePath . '/typo3temp/var/cache/code/core/sites-configuration.php') - ) { - @unlink($this->instancePath . '/typo3temp/var/cache/code/core/sites-configuration.php'); + // The variable path differs by installation layout. Using the classic one in a + // composer mode instance leaves the cache in place, and the next test case sees + // the site configurations of the previous one - or, worse, its emptiness. + $sitesConfigurationCache = $this->instancePath + . ($this->getInstanceMode() === 'composer' ? '/var' : '/typo3temp/var') + . '/cache/code/core/sites-configuration.php'; + if (is_file($sitesConfigurationCache)) { + @unlink($sitesConfigurationCache); } parent::tearDown(); } @@ -582,7 +640,11 @@ protected function getBackendUserRecordFromDatabase(int $userId): ?array private function createServerRequest(string $url, string $method = 'GET'): ServerRequestInterface { $requestUrlParts = parse_url($url); - $docRoot = $this->instancePath; + // The document root is the instance directory only in classic mode. Getting this + // wrong does not fail outright: the site path is then derived from a script that + // is not below the public path, comes out empty, and every generated link silently + // loses its leading slash. + $docRoot = $this->instancePath . ($this->getInstanceMode() === 'composer' ? '/public' : ''); $serverParams = [ 'DOCUMENT_ROOT' => $docRoot, @@ -1006,7 +1068,12 @@ private function retrieveFrontendSubRequestResult( $_SERVER['HTTP_HOST'] = $_SERVER['SERVER_NAME'] = $request->getUri()->getHost() ?: 'localhost'; $outputBufferingLevel = ob_get_level(); - $container = Bootstrap::init(ClassLoadingInformation::getClassLoader()); + // Frontend sub requests bootstrap a second time. In composer mode that bootstrap + // must use the same narrowed package set as the test instance, otherwise it looks + // for a package artifact next to the root installation, where there is none. + $container = $this->getInstanceMode() === 'composer' + ? TestingBootstrapRunner::init(ClassLoadingInformation::getClassLoader()) + : Bootstrap::init(ClassLoadingInformation::getClassLoader()); /** @var InternalRequest $serverRequest */ $serverRequest = $request->withAttribute('typo3.testing.context', $context); @@ -1103,14 +1170,126 @@ protected function withDatabaseSnapshot(?callable $createCallback = null, ?calla } /** - * Create a 7 char long hash of class name as identifier. + * Installation mode the test instance is provisioned in. + * + * TYPO3 ships two installation modes and both need functional test coverage, so the + * mode is chosen per run rather than baked in. Classic mode stays the default; set + * TYPO3_TESTING_INSTANCE_MODE=composer to provision composer mode instances. + * + * @return 'classic'|'composer' + */ + final protected static function getInstanceMode(): string + { + $mode = (string)getenv('TYPO3_TESTING_INSTANCE_MODE'); + if ($mode === '' || $mode === 'classic') { + return 'classic'; + } + if ($mode === 'composer') { + return 'composer'; + } + throw new \RuntimeException( + sprintf('TYPO3_TESTING_INSTANCE_MODE must be "classic" or "composer", "%s" given.', $mode), + 1754092805 + ); + } + + /** + * Extension keys that make up this test case's instance, as the package manager knows + * them. + * + * Test cases may name extensions as an extension key, a composer package name or a + * classic mode path, and all three have to end up as the key the artifact is indexed + * by. The default set and the framework's own extensions are always active, matching + * what classic mode writes into PackageStates. + * + * @return non-empty-string[] + */ + private function getActiveExtensionKeys(): array + { + $composerPackageManager = new ComposerPackageManager(); + $keys = ['core', 'backend', 'frontend', 'extbase', 'fluid', 'json_response', 'private_container']; + foreach ([...$this->coreExtensionsToLoad, ...$this->testExtensionsToLoad] as $extension) { + $packageInfo = $composerPackageManager->getPackageInfoWithFallback($extension); + $keys[] = $packageInfo?->getExtensionKey() ?? basename($extension); + } + + return array_values(array_unique($keys)); + } + + /** + * Boots this test instance again, the way an entry script would. + * + * Tests that exercise an application (the install tool, most notably) bootstrap a + * second container themselves. In composer mode that bootstrap must use this + * instance's narrowed package set: Bootstrap resolves the package artifact from the + * vendor directory of the *root* installation, and the mono repo has none, so calling + * it directly fails with "Package artifact not found". + */ + final protected function bootTestInstance(bool $failsafe = false): ContainerInterface + { + return $this->getInstanceMode() === 'composer' + ? TestingBootstrapRunner::init(ClassLoadingInformation::getClassLoader(), $failsafe) + : Bootstrap::init(ClassLoadingInformation::getClassLoader(), $failsafe); + } + + /** + * Path of the document root relative to the project root, with a trailing slash, or + * an empty string when the two are the same directory. * - * @internal + * A "PKG:typo3/app:…" resource identifier carries a path relative to the *project + * root*, which is the document root only in classic mode - in composer mode the same + * file lives one level deeper, below the web directory. Tests naming such resources + * have to prefix them with this rather than assuming either layout. + */ + final protected function getAppRelativePublicPath(): string + { + if (Environment::getPublicPath() === Environment::getProjectPath()) { + return ''; + } + + return substr(Environment::getPublicPath(), strlen(Environment::getProjectPath()) + 1) . '/'; + } + + /** + * Absolute path of the PHP script a test uses to run a console command in a sub + * process against this test instance. + * + * Classic mode instances use the core binary directly. Composer mode instances + * cannot: see ComposerModeInstance::writeCliEntryPoint(). + * + * @return non-empty-string + */ + final protected function getCliEntryPoint(): string + { + if ($this->getInstanceMode() === 'composer') { + return $this->instancePath . '/' . ComposerModeInstance::CLI_ENTRY_POINT; + } + + return $this->instancePath . '/typo3/sysext/core/bin/typo3'; + } + + /** + * Where site configurations live in this instance. Classic mode keeps them below + * typo3conf, composer mode below config. + */ + private function getSiteConfigurationPath(): string + { + return $this->instancePath . '/' + . ($this->getInstanceMode() === 'composer' ? 'config' : 'typo3conf') + . '/sites'; + } + + /** + * Identifier of the test instance. The instance directory and the database name + * are derived from it. Composer mode and classic mode instances of the same test + * case must not collide, so the mode is part of it. + * + * @internal Extensions functional tests should usually not fiddle with this. This may break anytime. * @return non-empty-string */ protected static function getInstanceIdentifier(): string { - return substr(sha1(static::class), 0, 7); + return substr(sha1(static::class . '|' . static::getInstanceMode()), 0, 7); } /** @@ -1121,7 +1300,6 @@ protected static function getInstanceIdentifier(): string */ protected static function getInstancePath(): string { - $identifier = static::getInstanceIdentifier(); - return ORIGINAL_ROOT . 'typo3temp/var/tests/functional-' . $identifier; + return ORIGINAL_ROOT . 'typo3temp/var/tests/functional-' . static::getInstanceIdentifier(); } } diff --git a/Classes/Core/SystemEnvironmentBuilder.php b/Classes/Core/SystemEnvironmentBuilder.php index 65199fda..e699fa69 100644 --- a/Classes/Core/SystemEnvironmentBuilder.php +++ b/Classes/Core/SystemEnvironmentBuilder.php @@ -54,11 +54,36 @@ public static function run(int $entryPointLevel = 0, int $requestType = 0, ?bool Environment::getPublicPath(), Environment::getVarPath(), Environment::getConfigPath(), - Environment::getCurrentScript(), + static::determineCurrentScript(), Environment::isWindows() ? 'WINDOWS' : 'UNIX' ); } + /** + * Entry script the instance should be treated as running. + * + * On CLI in composer mode, TYPO3 deliberately reports the console binary + * (typo3/sysext/core/bin/typo3) as the current script, because in a real composer + * installation that binary sits outside the document root and relative path + * calculations would otherwise break. + * + * Functional tests are CLI processes that simulate *web* requests, and the entry + * script is what the site URL is derived from: NormalizedParams subtracts the entry + * script's directory, relative to the public path, from the request URL. Leaving the + * console binary in place therefore subtracts "typo3/sysext/core/bin/" from every + * request and the site path collapses to an empty string - every generated link + * silently loses its leading slash. Point at the front end entry script instead, + * which is what a web request would have and what classic mode already gets. + */ + protected static function determineCurrentScript(): string + { + if (!Environment::isCli() || !static::usesComposerClassLoading()) { + return Environment::getCurrentScript(); + } + + return Environment::getPublicPath() . '/index.php'; + } + /** * Manage composer mode separated from TYPO3_COMPOSER_MODE define set by typo3/cms-composer-installers. * diff --git a/Classes/Core/Testbase.php b/Classes/Core/Testbase.php index 6316e94e..df080334 100644 --- a/Classes/Core/Testbase.php +++ b/Classes/Core/Testbase.php @@ -401,9 +401,15 @@ public function linkPathsInTestInstance(string $instancePath, array $pathsToLink { foreach ($pathsToLinkInTestInstance as $sourcePathToLinkInTestInstance => $destinationPathToLinkInTestInstance) { $sourcePath = $instancePath . '/' . ltrim($sourcePathToLinkInTestInstance, '/'); + if (!file_exists($sourcePath)) { + // See providePathsInTestInstance(): sources are given relative to the + // document root, which only carries the extension tree in classic mode. + $sourcePath = rtrim(ORIGINAL_ROOT, '/') . '/' . ltrim($sourcePathToLinkInTestInstance, '/'); + } if (!file_exists($sourcePath)) { throw new Exception( - 'Path ' . $sourcePath . ' not found', + 'Path ' . $sourcePathToLinkInTestInstance . ' not found, neither in the test instance' + . ' nor below ' . ORIGINAL_ROOT, 1476109221 ); } @@ -434,9 +440,16 @@ public function providePathsInTestInstance(string $instancePath, array $pathsToP { foreach ($pathsToProvideInTestInstance as $sourceIdentifier => $designationIdentifier) { $sourcePath = $instancePath . '/' . ltrim($sourceIdentifier, '/'); + if (!file_exists($sourcePath)) { + // Sources are given relative to the instance, which works in classic mode + // because the extensions are linked into it. A composer mode instance has + // no such tree, so fall back to the same path below the original root. + $sourcePath = rtrim(ORIGINAL_ROOT, '/') . '/' . ltrim($sourceIdentifier, '/'); + } if (!file_exists($sourcePath)) { throw new Exception( - 'Path ' . $sourcePath . ' not found', + 'Path ' . $sourceIdentifier . ' not found, neither in the test instance nor below ' + . ORIGINAL_ROOT, 1511956084 ); } @@ -559,16 +572,25 @@ public function testDatabaseNameIsNotTooLong(string $originalDatabaseName, array * @param array $overruleConfiguration Overrule factory and base configuration * @throws Exception */ - public function setUpLocalConfiguration(string $instancePath, array $configuration, array $overruleConfiguration): void - { + /** + * @param non-empty-string $relativeConfigurationPath Where TYPO3 expects its settings file, + * relative to the instance root. Classic mode instances keep it below typo3conf, + * composer mode instances below config. + */ + public function setUpLocalConfiguration( + string $instancePath, + array $configuration, + array $overruleConfiguration, + string $relativeConfigurationPath = 'typo3conf/system' + ): void { // Base of final LocalConfiguration is core factory configuration $coreExtensionPath = $this->composerPackageManager->getPackageInfo('typo3/cms-core')?->getRealPath() ?? ''; $finalConfigurationArray = require $coreExtensionPath . '/Configuration/FactoryConfiguration.php'; $finalConfigurationArray = array_replace_recursive($finalConfigurationArray, $configuration); $finalConfigurationArray = array_replace_recursive($finalConfigurationArray, $overruleConfiguration); - $this->createDirectory($instancePath . '/typo3conf/system'); + $this->createDirectory($instancePath . '/' . trim($relativeConfigurationPath, '/')); $result = @file_put_contents( - $instancePath . '/typo3conf/system/settings.php', + $instancePath . '/' . trim($relativeConfigurationPath, '/') . '/settings.php', 'getPackagesPath() . '/autoload.php'; - SystemEnvironmentBuilder::run(0, SystemEnvironmentBuilder::REQUESTTYPE_CLI, false); + SystemEnvironmentBuilder::run(0, SystemEnvironmentBuilder::REQUESTTYPE_CLI, $composerMode); $outputBufferingLevel = ob_get_level(); - $container = Bootstrap::init($classLoader); + $container = $composerMode + ? TestingBootstrapRunner::init($classLoader) + : Bootstrap::init($classLoader); // Make sure output is not buffered, so command-line output can take place and // phpunit does not whine about changed output bufferings in tests. TYPO3 v14 // opens an implicit output buffer in Bootstrap::init(), TYPO3 v15 does not. diff --git a/Classes/Core/TestingBootstrapPackageCache.php b/Classes/Core/TestingBootstrapPackageCache.php new file mode 100644 index 00000000..759847e3 --- /dev/null +++ b/Classes/Core/TestingBootstrapPackageCache.php @@ -0,0 +1,33 @@ +