New very slow build time with webpack 5.96.0

[NathZ1]

Bug report

Hi team, I just bumped my webpack from 5.95.0 to the newly released 5.96.0 and this has resulted in increased build times from 2-3minutes to ~15minutes for my React SPA. I first noticed this in my automated deployment pipelines in Bitbucket, but the same thing is noticed on my local dev machine.

I downgraded back to 5.95.0 and the builds returned to the usual times, so definitely something to do with 5.96.0.

Happy to help with debugging this - please let me know how to setup my environment to provide you with required logs.

webpack.config.ts:

import webpack from 'webpack'
import { relative, resolve } from 'path'
import HtmlWebpackPlugin from 'html-webpack-plugin'
import MiniCssExtractPlugin from 'mini-css-extract-plugin'
import CssMinimizerPlugin from 'css-minimizer-webpack-plugin'
import TerserPlugin from 'terser-webpack-plugin'
import CopyWebpackPlugin from 'copy-webpack-plugin'
import ESLintPlugin from 'eslint-webpack-plugin'
import ReactRefreshWebpackPlugin from '@pmmmwh/react-refresh-webpack-plugin'
import ReactRefreshTypeScript from 'react-refresh-typescript'
import ForkTsCheckerWebpackPlugin from 'fork-ts-checker-webpack-plugin'
import { InjectManifest } from 'workbox-webpack-plugin'
import { getPublicUrlOrPath } from './src/utilities/getPublicUrlOrPath'
import { WebpackManifestPlugin } from 'webpack-manifest-plugin'

module.exports = (env: { APP_BUILD?: string; FUNCTION_APP?: string }) => {
  //get PUBLIC_URL, which is needed for production builds where process (which is a Node server var), doesn't exist
  const publicUrlOrPath = getPublicUrlOrPath(process.env.NODE_ENV === 'development', undefined, process.env.PUBLIC_URL)

  //set whether are creating source maps with prod builds
  // eslint-disable-next-line @typescript-eslint/no-inferrable-types
  const genSourceMaps: boolean = true

  const prod = process.env.NODE_ENV === 'production'

  return {
    mode: prod ? 'production' : 'development',
    cache: { type: 'filesystem' },
    infrastructureLogging: { level: 'info' },
    stats: 'normal',
    // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition
    devtool: !prod ? (genSourceMaps ? 'source-map' : false) : false, //https://webpack.js.org/configuration/devtool/
    bail: false, //this needs to be set to false for dev builds otherwise dev-server will break every time an import can't be resolved
    resolve: {
      modules: [resolve(__dirname, 'node_modules'), resolve(__dirname, './src')]
    },
    entry: './src/index.tsx',
    output: {
      path: resolve(__dirname, 'build'),
      publicPath: publicUrlOrPath,
      filename: prod ? '[name].[contenthash:8].js' : undefined,
      chunkFilename: prod ? '[name].[contenthash:8].chunk.js' : undefined,
      assetModuleFilename: 'media/[name].[hash][ext]',
      clean: true, //clears the output dir prior to building

      // Point sourcemap entries to original disk location (format as URL on Windows)
      // eslint-disable-next-line @typescript-eslint/no-explicit-any
      devtoolModuleFilenameTemplate: (info: any) => {
        // eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
        if (prod) return relative('./src', info.absoluteResourcePath).replace(/\\/g, '/')
        // eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
        else return resolve(info.absoluteResourcePath).replace(/\\/g, '/')
      }
    },
    devServer: {
      port: '3000',
      static: ['./public'],
      open: true,
      hot: true,
      historyApiFallback: true,
      client: {
        overlay: {
          errors: true,
          warnings: false,
          runtimeErrors: true
        }
      },
      devMiddleware: {
        writeToDisk: true //massively speeds up loading of dev server
      }
    },
    module: {
      rules: [
        //Typescript loader
        //NOTE: uses fork-ts-checker-webpack-plugin for TS checking in dev on a separate thread
        {
          test: /\.([cm]?ts|tsx)$/,
          exclude: /node_modules/,
          resolve: {
            extensions: ['.ts', '.tsx', '.js'] // --> JSON and HTML gets parsed by webpacks internal loaders
          },

          use: [
            {
              loader: require.resolve('ts-loader'),
              options: {
                getCustomTransformers: () => ({
                  before: [!prod && ReactRefreshTypeScript()].filter(Boolean)
                }),
                transpileOnly: !prod
              }
            }
          ]
        },

        //CSS loaders
        {
          test: /\.css$/,
          exclude: /node_modules/,
          use: [
            //In prod, MiniCSSExtractPlugin extract CSS to file(s), but in development "style" loader enables hot editing of CSS.
            prod && MiniCssExtractPlugin.loader,

            //style loader turns CSS into JS modules that inject <style> tags
            !prod && require.resolve('style-loader'),

            //css-loader resolves paths in CSS and adds assets as dependencies
            {
              loader: require.resolve('css-loader'),
              options: { sourceMap: genSourceMaps }
            },

            //postcss loader applies autoprefixer to CSS
            {
              loader: require.resolve('postcss-loader'),
              options: { sourceMap: genSourceMaps }
            }
          ].filter(Boolean),
          sideEffects: true //See https://github.com/webpack/webpack/issues/6571
        },

        //File loader (including images)
        {
          exclude: /node_modules/,
          test: /\.(png|jp(e*)g|svg|gif|ttf)$/,
          type: 'asset/resource'
        }
      ].filter(Boolean)
    },
    plugins: [
      //Generates an `index.html` file with the <script> injected.
      new HtmlWebpackPlugin({ filename: 'index.html', template: 'index.html' }),

      //Copies the public directory into the root of build directory
      new CopyWebpackPlugin({ patterns: [{ from: 'public' }] }),

      //Makes environment variables available to the build code
      new webpack.DefinePlugin({
        'process.env': {
          NODE_ENV: JSON.stringify(process.env.NODE_ENV),
          PUBLIC_URL: JSON.stringify(publicUrlOrPath.slice(0, -1)),
          APP_BUILD: JSON.stringify(env.APP_BUILD),
          FUNCTION_APP: JSON.stringify(env.FUNCTION_APP)
        }
      }),

      ...(prod
        ? [
            //Extracts CSS into separate files
            new MiniCssExtractPlugin({
              filename: '[name].[contenthash:8].css',
              chunkFilename: '[name].[contenthash:8].chunk.css'
            }),

            //Generate a service worker script that will precache, and keep up to date, the HTML & assets that are part of the webpack build
            new InjectManifest({
              swSrc: './src/service-worker.ts',
              dontCacheBustURLsMatching: /\.[0-9a-f]{8}\./,
              exclude: [/\.map$/, /asset-manifest\.json$/, /LICENSE/],
              maximumFileSizeToCacheInBytes: 5 * 1024 * 1024 //5MB currently to cache large builds, but should split and lazy-load
            }),

            // Generate an asset manifest file
            new WebpackManifestPlugin({
              fileName: 'asset-manifest.json',
              publicPath: publicUrlOrPath,
              generate: (seed, files, entrypoints) => {
                const manifestFiles = files.reduce((manifest, file) => {
                  manifest[file.name] = file.path
                  return manifest
                }, seed)

                const entrypointFiles = entrypoints.main.filter(fileName => !fileName.endsWith('.map'))

                return {
                  files: manifestFiles,
                  entrypoints: entrypointFiles
                }
              }
            })
          ]
        : []),

      ...(!prod
        ? [
            //Checks code for issues with eslint
            new ESLintPlugin({ extensions: ['js', 'mjs', 'jsx', 'ts', 'tsx'] }),

            //Enables new React hot reloading in dev builds
            new ReactRefreshWebpackPlugin(),

            //Used for TS checking - does the checks on a separate process to the build
            //NOTE: This plugin uses TypeScript's, not Webpack's modules resolution (i.e. uses tsconfig.json)
            new ForkTsCheckerWebpackPlugin()
          ]
        : [])
    ],
    optimization: {
      nodeEnv: false, //do not modify/set the value of process.env.NODE_ENV as this is done by DefinePlugin
      minimizer: [
        //minimise JS
        new TerserPlugin({
          extractComments: false,
          terserOptions: {
            format: {
              comments: false
            }
          }
        }),

        //minimise CSS
        new CssMinimizerPlugin()
      ]
    }
  }
}

Other relevant information:
webpack version: 5.96.0
Node.js version: 20.11.1
Operating System: Windows 10 pro
Additional tools: N/A

[alexander-akait]

@NathZ1 Please provide reproducible code or profiling file - https://webpack.js.org/plugins/profiling-plugin/, or better - https://developer.chrome.com/docs/devtools/performance/nodejs, without it I can't do anything.

Yes, regressions can happen. And I'm happy to fix it. Move to discussion right now because the reproducible part does not allow to understand what the problem is, as soon as I get more information we will return to issues

[absidue]

For the project I work on the build time increase only happens in production mode. I also managed to capture an events.json file with the ProfilingPlugin but the file won't load in the Performance tab in the chromium developer tools instead producing the error:

Status: Recording failed
Description: Cannot read properties of undefined (reading `detail`)

I was able to open the event.json file with https://ui.perfetto.dev/ though, which showed a big gap where nothing was happening, which seems to correlate with the increase in build time. So the cause seems to be "nothing".

As the file is 177MB but GitHub only allows up to 25MB I can't upload it in this comment. Please let me know if there is somewhere else you would like me to upload the file.

[alexander-akait]

@absidue

Description: Cannot read properties of undefined (reading detail)

I saw this problem somewhere from other users, maybe you can create a reproducible repository so that I can study this error, and then we will see where the time is spent.

You can try to disable that option and compare the time - https://webpack.js.org/configuration/optimization/#optimizationavoidentryiife

[absidue]

Disabling that option fixes the problem, so it could be related to #18845.

[absidue]

The project is open source, but I'll try to create a slightly more stripped down repo, so that you don't have to install all the eslint, stylelint and electron-builder dependencies. But if you already want to take a look:

Repo: https://github.com/FreeTubeApp/FreeTube
webpack command: yarn pack:renderer
webpack config: _scripts/webpack.renderer.config.js

[alexander-akait]

@absidue we can try to cache this step for better perf... Feel free to open an issue

[tfaller]

I've found the issue and opened PR #19331 to fix it. Changes in #18845 introduced the performance regression.

[tfaller]

@NathZ1 @absidue PR #19331 was merged and is available since 1bd06e3/v5.99.0. Maybe your issues are fixed too.