Enhancement #6: As a QA engineer, I want the plugin versions to be automatically handled so that I can easily run tests on a selected version

[Miraeld]

Description

Fixes #6
Eliminates manual plugin version management for E2E tests. QA engineers can now configure versions in plugin.config.ts and the framework automatically downloads/builds required files. Supports version numbers, GitHub branches/tags, and direct URLs. Tests auto-check and download missing plugins before running. Fully backward compatible, existing workflows continue to work unchanged.

Type of change

• Enhancement (non-breaking change which improves an existing functionality).

Detailed scenario

What was tested

All new npm commands

How to test

1. Copy config/plugin.config.sample.ts to config/plugin.config.ts
2. Run npm run plugin:setup to download/build configured plugin versions
3. Verify with npm run plugin:list - should show 3 plugin files
4. Run npm run test:e2e to verify automatic plugin setup in tests
5. Test force rebuild: npm run plugin:setup -- --force

Technical description

Documentation

This pull request introduces automated management of WP Rocket plugin versions for E2E testing, eliminating the need for manual setup of plugin zip files. It adds a configuration-driven plugin manager that can automatically download, build, validate, and clean required plugin versions, along with CLI commands and documentation updates to support this workflow.

Automated Plugin Version Management

• Added a new plugin manager (utils/plugin-manager.ts) that automatically downloads or builds required WP Rocket plugin versions based on a configuration file, replacing the previous manual process. The manager supports fetching by version, GitHub branch/tag, or direct URL, and provides functions for setup, validation, listing, and cleaning plugin files.
• Introduced a CLI tool (plugin-manager-cli.ts) with commands to set up, list, clean, and validate plugin versions, integrated into npm scripts for easy use (plugin:setup, plugin:list, plugin:clean, plugin:validate). [1] [2]

Configuration and Documentation

• Added a sample configuration file (config/plugin.config.sample.ts) for specifying which plugin versions to use, with support for version numbers, GitHub branches/tags, or URLs. Users copy and edit this file to control test versions.
• Updated README.md to document the new automated plugin management workflow, configuration steps, and available CLI commands, replacing the old manual instructions. [1] [2]

Integration with Test Lifecycle

• Updated test hooks (src/support/hooks.ts) to automatically validate and set up plugin versions before running tests, ensuring all required plugin files are present and up-to-date. [1] [2]

Mandatory Checklist

Code validation

• I validated all the Acceptance Criteria. If possible, provide screenshots or videos.
• I triggered all changed lines of code at least once without new errors/warnings/notices.
• I implemented built-in tests to cover the new/changed code.

Code style

• I wrote a self-explanatory code about what it does.
• I protected entry points against unexpected inputs.
• I did not introduce unnecessary complexity.
• Output messages (errors, notices, logs) are explicit enough for users to understand the issue and are actionnable.

Unticked items justification

No test for this

[Copilot]

Pull request overview

This PR introduces automated plugin version management for E2E testing, eliminating manual setup of WP Rocket plugin zip files. The implementation adds a configuration-driven plugin manager that can download from releases, build from GitHub branches/tags, or fetch from direct URLs. The system integrates into the test lifecycle to automatically validate and setup required plugin versions before running tests.

Key Changes

• New plugin manager utility that automates downloading/building plugin versions based on configuration
• CLI commands for managing plugin versions (plugin:setup, plugin:list, plugin:clean, plugin:validate)
• Automatic plugin validation and setup integrated into test hooks to ensure required files are present before tests run

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
utils/plugin-manager.ts	Core plugin manager implementing download, GitHub build, validation, and cleanup functionality
plugin-manager-cli.ts	CLI interface providing commands to manage plugin versions with help documentation
config/plugin.config.sample.ts	Sample configuration file with interface definitions and usage examples for version configuration
src/support/hooks.ts	Integration of automatic plugin validation and setup into BeforeAll test hook
package.json	Addition of npm scripts for plugin management commands
README.md	Documentation updates describing the new automated plugin management workflow
.gitignore	Added entries to ignore plugin config file and temporary build directory

Comments suppressed due to low confidence (1)

utils/plugin-manager.ts:171

• Unused variable pluginDirName.

        const pluginDirName = name;

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Miraeld]

All feedback has been addressed:

Security fixes in 7dedcb8:

• ✅ Command injection vulnerability (version validation) - Added input validation for downloadFromReleases
• ✅ Path traversal vulnerability - Added destination path validation in downloadFile
• ✅ GitHub token exposure - Now using git credential helper instead of embedding token in URL
• ✅ Command injection in buildFromGitHub - Added sanitizeShellInput function with pattern validation
• ✅ Race condition in error handler - Using try-catch instead of existsSync check
• ✅ Missing redirect location validation - Added null check before using redirect URL
• ✅ Missing config file handling - Added graceful error with helpful message

Code quality fixes:

• ✅ Force parameter usage in 7dedcb8 - Now properly uses parameter via internal flag
• ✅ Inconsistent flag documentation in 7dedcb8 - Updated to --force throughout
• ✅ Unused variable in 7dedcb8 - Removed pluginDirName
• ✅ Error handling in 69e5ee8 and 7dedcb8 - Using proper TypeScript type checking
• ✅ Beta version note in 68151aa - Added documentation about pre-release versions

All ESLint errors are now resolved as well.

[jeawhanlee]

Great work on this Sir 🙌
Would it also be possible to enhance this to accept arguments of plugin versions directly from the CLI that way we might not need to modify the config file all the time for each test run or maybe this doesn't make sense.

CC @wp-media/qa-team Do you see any usefulness in this suggestion?

[jeawhanlee]

To make things a bit more automated, Would it make sense to have an endpoint we can connect to that would always return the latest version and the previous stable version of WPR, that way, we'll eliminate configs

CC @wp-media/qa-team @wordpressfan

[jeawhanlee]

Approving this PR with need for some enhancement mentioned here

[Miraeld]

CLI Version Override

I've implemented the feedback from @jeawhanlee about accepting plugin version arguments directly from the CLI without needing to modify the config file each time.

What's New

CLI Override Support - You can now override plugin versions at runtime using multiple methods:

1. NPM config arguments:

   npm run test:e2e --wpr=3.16.4.1 --previous=branch:release/3.16.4

2. Environment variables:

   WPR_VERSION=tag:3.16.4.1 PREVIOUS_VERSION=3.16.3 npm run test:e2e

3. Command line arguments:

   npm run test:e2e -- --wpr=https://example.com/custom-build.zip

Supported Formats

All the same formats from plugin.config.ts are supported:

• Version numbers: 3.16.4.1
• GitHub branches: branch:develop, branch:release/3.16.4
• GitHub tags: tag:3.16.4.1
• Direct URLs: https://example.com/plugin.zip

Implementation Details

New Files:

• utils/version-override.ts - Handles parsing and applying CLI overrides

Modified Files:

• utils/plugin-manager.ts - Integrated override system with getActivePluginConfig() and updated setupPluginVersions() to accept overrides
• src/support/hooks.ts - Added parseVersionOverrides() call in BeforeAll hook to apply CLI arguments before plugin setup
• README.md - Added comprehensive documentation with examples and usage scenarios

Example Use Cases

Test a specific version without config changes:

npm run test:e2e --wpr=3.16.4.1

Test a branch against previous stable:

npm run test:e2e --wpr=branch:feature/new-optimization

Override both versions for comparison testing:

npm run test:e2e --wpr=branch:develop --previous=3.16.3

The config file still serves as the default, but CLI arguments take precedence when provided. This makes it easy to run quick tests without editing configuration files!

[Mai-Saad]

• Run test with versions set in config files (no zips)
• Run test with versions set in config file and have already zips in plugin directory
• Run test with latest set in CLI and zips in plugin directory
• Run test with both latest and previous set in CLI
• Run test with both latest and previous set in CLI and report send to shared folder

[Miraeld]

Priority-Based Plugin Version Management

I've implemented a more flexible and QA-friendly approach to plugin version management based on team feedback. The system now uses a three-tier priority system that makes it easier to work with different testing scenarios.

How It Works

The plugin manager now follows a clear priority order:

1️⃣ CLI Overrides (Highest Priority)
When you specify versions via command line, they take precedence over everything else:

• ✅ Plugin folder is cleaned
• ✅ Plugins are re-downloaded with your specified versions
• Overrides any config file settings

# Quick test with specific versions
npm run test:e2e -- --new-release=3.16.4 --previous-stable=3.16.3

# Test a branch
npm run test:e2e -- --new-release=branch:develop

# Mix different formats
npm run test:smoke -- --new-release=tag:3.16.4-beta5 --previous-stable=3.16.3

2️⃣ Config File (Medium Priority)
When NO CLI overrides are provided, but config/plugin.config.ts exists:

• ✅ Plugin folder is cleaned
• ✅ Plugins are re-downloaded from config
• Ensures fresh downloads every time

# Uses versions from config/plugin.config.ts
npm run test:e2e

3️⃣ Existing Plugins (Lowest Priority - NEW!)
When NO CLI overrides AND NO config file exist:

• ❌ Plugin folder is NOT cleaned
• ✅ Uses existing plugin files in plugin/ folder
• Just validates that required files exist

# Manually place plugin files in plugin/ folder
ls plugin/
# previous_stable.zip  new_release.zip  wp-rocket_3.10.9.zip

# Run tests without re-downloading
npm run test:e2e

Why This Is Better

For QA:

• ✅ Can use existing plugin files when testing (no re-download needed)
• ✅ Quick version switching via CLI without editing config
• ✅ Clear error messages when files are missing

For CI/CD:

• ✅ Predictable behavior with priority system
• ✅ Always get fresh downloads when using config
• ✅ Can use pre-downloaded plugins for faster runs

Changes Made

Modified files:

• src/support/hooks.ts - Implements priority-based logic in BeforeAll hook
• utils/plugin-manager.ts - Made config optional, added hasPluginConfigFile()

Commit: e429aa6

Testing Examples

# Scenario 1: Use existing files (no config)
rm config/plugin.config.ts  # Remove config
npm run test:e2e            # Uses existing plugin files

# Scenario 2: Use config file
cp config/plugin.config.sample.ts config/plugin.config.ts
npm run test:e2e            # Cleans and re-downloads from config

# Scenario 3: Override via CLI
npm run test:e2e -- --new-release=branch:feature/new-optimization
# Cleans and downloads the branch version

[Miraeld]

Update: Removed specificVersion Configuration

Commit: 87ace3a

The specificVersion field has been removed from the plugin configuration as it was added by mistake and is not used by any test scenarios.

Changes:

• ❌ Removed specificVersion from PluginVersionConfig interface
• ❌ Removed specificVersion from VersionOverrides interface
• ❌ Removed specificVersion file mapping from PLUGIN_FILES
• ❌ Removed specificVersion processing from all plugin manager functions
• ✅ Updated sample configuration file to reflect simplified structure

Why?

After investigation, no test features actually require the specific version file (wp-rocket_3.10.9.zip):

• The roll-back test doesn't use it
• The upgrading-plugin tests only use previous_stable and new_release
• The field was optional and appears to have been legacy/unused configuration

The plugin manager now only handles the two versions that are actually used in tests: previousStable and newRelease.

[Miraeld]

Update: Plugin Downloads Now Use GitHub Releases

Commit: 7b66bc4

Changes

The plugin version management now downloads plugin versions from GitHub releases instead of wp-rocket.me, keeping downloads private to the organization.

What Changed:

1. Download Source: Version numbers (e.g., 3.20.1) now download from https://github.com/wp-media/wp-rocket/archive/refs/tags/v{version}.zip
2. Authentication: Requires GITHUB_TOKEN to access private repository releases
3. Automatic Archives: Uses GitHub's automatically generated source archives (always available for any tag)
4. Tag Format: Automatically adds v prefix to version numbers (e.g., 3.20.1 → v3.20.1)

Configuration Required:

export const pluginConfig: PluginVersionConfig = {
    previousStable: '3.20.1',
    newRelease: '3.20.2',
    repository: {
        owner: 'wp-media',
        name: 'wp-rocket',
        token: process.env.GITHUB_TOKEN  // Required for downloads
    }
};

Why This Change?

• ✅ Security: Downloads remain private within wp-media organization
• ✅ Reliability: GitHub source archives are automatically generated for every tag
• ✅ Consistency: Uses same authentication mechanism as branch/tag builds
• ✅ No Public Access: Prevents unauthorized downloads of WP Rocket

Supported Version Formats:

• 3.20.1 - Downloads from GitHub releases (requires GITHUB_TOKEN)
• branch:develop - Clones and builds from branch (requires GITHUB_TOKEN)
• tag:v3.20.1 - Clones and builds from tag (requires GITHUB_TOKEN)
• https://example.com/file.zip - Direct URL download

All automated methods now require proper GitHub authentication, ensuring plugin files remain secure.

[Miraeld]

Update: Automatic Build Process for Version Downloads

Commit: c03ebad

Issue

GitHub's source archives contain only raw source code without built dependencies:

• No vendor/ directory (Composer dependencies)
• No node_modules/ (npm dependencies)
• No built assets (CSS/JS)

Solution

Version downloads now automatically build the plugin after downloading:

1. Download source archive from GitHub releases
2. Extract to temporary directory
3. Install Composer dependencies: composer install --no-dev --no-scripts
   • Skips post-install scripts that require dev dependencies (like Mozart)
   • Manually runs composer dump-autoload for optimized autoloading
4. Install npm dependencies: npm install (if package.json exists)
5. Build assets: Runs build scripts if available (e.g., build:css, build:js)
6. Create plugin zip with all built dependencies included

Result

• Downloaded plugins now include vendor/ directory (~600KB of dependencies)
• Plugins are ready for use in E2E tests without missing dependencies
• No manual build steps required

[jeawhanlee]

To be honest, I don't know how I feel about compiling the plugin on e2e, we might need to brainstorm on this.

[Miraeld]

@jeawhanlee I kinda agree, but we can't download it from wp-rocket.me as it's subject to authentification.