Karma Unit tests are failing to run on Angular 15

[mivanyna]

Which @angular/* package(s) are the source of the bug?

Don't known / other

Is this a regression?

Yes

Description

I'm working on an old angular project and the folder structure is a bit different from what's in angular documentation.

After upgrading to Angular 15 karma/jasmine unit tests are failing to run.
Project structure:

Karma config:

module.exports = function (config) {
  config.set({
    basePath: '..',
    frameworks: ['jasmine', '@angular-devkit/build-angular'],
    plugins: [
      require('karma-jasmine'),
      require('karma-phantomjs-launcher'),
      require('karma-chrome-launcher'),
      require('karma-jasmine-html-reporter'),
      require('karma-coverage-istanbul-reporter'),
      require('karma-spec-reporter'),
      require('@angular-devkit/build-angular/plugins/karma')
    ],
    client: {
      clearContext: false // leave Jasmine Spec Runner output visible in browser
    },
    coverageIstanbulReporter: {
      dir: require('path').join(__dirname, './coverage/testapp'),
      reports: ['html', 'lcovonly', 'text-summary'],
      fixWebpackSourcePaths: true
    },
    reporters: ['spec'],
    port: 9876,
    colors: true,
    logLevel: config.LOG_INFO,
    autoWatch: true,
    browsers: ['ChromeHeadless'],
    singleRun: true,
    restartOnFileChange: true
  });
};

When I run UTs I get an error:

When running it with debug logging found:

Please provide a link to a minimal reproduction of the bug

No response

Please provide the exception or error you saw

Error: Specified patterns: "**/*.spec.ts" did not match any spec files.

Please provide the environment you discovered this bug in (run ng version)

Angular: 15.0.4
Works on angular 14.2.4

Anything else?

No response

[clydin]

From the debug logging shown above it appears that the projectSourceRoot contains an additional www directory segment. This may indicate that the project as defined within the angular.json file has a misconfigured root and/or sourceRoot field. Can you provide the values assigned to each for the project encountering the issue? Based on the directory structure shown, both the root and sourceRoot fields should probably be "" (or the equivalent, ".").

[mivanyna]

It is and was www for sourceRoot:

And this used to work.

[clydin]

In v15, the test files are discovered based on the include option which defaults to ["**/*.spec.ts"] and uses the sourceRoot as the base directory to search. This change removed the need to use Webpack specific features within the test.ts file and provides the capability to more easily control test execution from the command line. The reason this no longer works in this project is that the sourceRoot directory of www does not appear to exist. All paths within the angular.json file are relative to the location of the angular.json file. Changing sourceRoot to "" should alleviate the issue.

[mivanyna]

I'll check tomorrow, will it impact project building as it seems to depend on those values too?

[mivanyna]

It works except exclude in Karma config(and files seems to be ignored):

 exclude: [
      "**/*.client.spec.ts",
],

Previously I had a logic to exclude integration tests (which ends with 'client.spec.ts') in test.ts:

const context = require.context('./app/', true, /\.spec\.ts$/)
context.keys()
   .filter((fileName: string) => fileName.indexOf('.client.spec.ts') < 0)
   .map(context)

Now, there is an issue that those tests are included and exclude filter doesn't work to filter them out.

If I'm adding that exclude to tsconfig.spec.json getting errors:
Error: Module build failed (from ./node_modules/@ngtools/webpack/src/ivy/index.js):
.client.spec.ts is missing from the TypeScript compilation. Please make sure it is in your tsconfig via the 'files' or 'include' property.

[mivanyna]

@clydin could you please help me with this?

[johncrim]

@mivanyna - try moving the exclude to your tsconfig.spec.json (or whatever it may be called).

@clydin - thank you for the pointer. We were also bitten by the fact that we had incorrect sourceRoot paths in the angular.json - eg "sourceRoot": "libs/util/src" when it should have been "sourceRoot": "libs/util"; this previously wasn't causing any problems, but after upgrading to ng 15, and after the fix you pointed us to in #24471, it was causing Error: Specified patterns: "**/*.karma-spec.ts" did not match any spec files. It took me a while to find that cause, I wish I had read your comments above when I was trying to solve that problem.

A general approach I use for solving difficult-to-debug project configuration issues is to generate a new workspace and projects using the latest version of angular, customize it to approximate the broken project and get it working, and then diff that project with the broken upgraded project and iterate to find the issue. Then you'll either find the solution or find a repro case that can be shared.

[mivanyna]

If I'm adding that exclude to tsconfig.spec.json getting errors:
Error: Module build failed (from ./node_modules/@ngtools/webpack/src/ivy/index.js):
.client.spec.ts is missing from the TypeScript compilation. Please make sure it is in your tsconfig via the 'files' or 'include' property.

@johncrim I have specified already that getting another error