Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ jobs:
- codspeedhq-arm64-ubuntu-22.04
- codspeedhq-arm64-ubuntu-24.04
mode:
- simulation
- instrumentation
- walltime

Expand Down Expand Up @@ -69,8 +70,8 @@ jobs:
matrix:
version:
- "latest"
- "4.2.0"
- "v4.2.0"
- "4.4.0"
- "v4.4.0"

runs-on: ubuntu-latest
env:
Expand All @@ -81,5 +82,5 @@ jobs:
uses: ./
with:
runner-version: ${{ matrix.version }}
mode: instrumentation
mode: simulation
run: echo "Testing version format ${{ matrix.version }}!"
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ GitHub Actions for running [CodSpeed](https://codspeed.io) in your CI.
run: "<YOUR_COMMAND>"

# [REQUIRED]
# The measurement mode to use, either: "instrumentation" or "walltime".
# The measurement mode to use: "simulation" (recommended), or "walltime".
# More details on the instruments at https://docs.codspeed.io/instruments/
mode: "instrumentation"
mode: "simulation"

# [OPTIONAL]
# CodSpeed recommends using OpenID Connect (OIDC) for authentication.
Expand Down Expand Up @@ -105,7 +105,7 @@ jobs:
- name: Run benchmarks
uses: CodSpeedHQ/action@v4
with:
mode: instrumentation
mode: simulation
run: pytest tests/ --codspeed
```

Expand Down Expand Up @@ -150,7 +150,7 @@ jobs:
- name: Run the benchmarks
uses: CodSpeedHQ/action@v4
with:
mode: instrumentation
mode: simulation
run: cargo codspeed run
```

Expand Down Expand Up @@ -190,6 +190,6 @@ jobs:
- name: Run benchmarks
uses: CodSpeedHQ/action@v4
with:
mode: instrumentation
mode: simulation
run: npx vitest bench
```
13 changes: 7 additions & 6 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,12 @@ inputs:

mode:
description: |
The mode to to run the benchmarks in. The following modes are available:
- `instrumentation`: Run the benchmarks with CPU instrumentation measurements.
The mode to run the benchmarks in. The following modes are available:
- `simulation`: Run the benchmarks with CPU simulation measurements.
- `walltime`: Run the benchmarks with walltime measurement.
- `instrumentation`: (Deprecated) Legacy name for `simulation`. Please use `simulation` instead.

We strongly recommend starting with the `instrumentation` mode.
We strongly recommend starting with the `simulation` mode.

Using the `walltime` mode on traditional VMs/Hosted Runners might lead to inconsistent data. For the best results, we recommend using CodSpeed Hosted Macro Runners, which are fine-tuned for performance measurement consistency.
Check out the [Walltime Instrument Documentation](https://docs.codspeed.io/instruments/walltime/) for more details.
Expand Down Expand Up @@ -99,7 +100,7 @@ runs:
# Validate required inputs
# (custom message for smoother v4 migration)
if [ -z "${{ inputs.mode }}" ]; then
echo "::error title=Missing required input 'mode'::The 'mode' input is required as of CodSpeed Action v4. Please explicitly set 'mode' to 'instrumentation' or 'walltime'. Before, this variable was automatically set to instrumentation on every runner except for CodSpeed macro runners where it was set to walltime by default. See https://codspeed.io/docs/instruments for details."
echo "::error title=Missing required input 'mode'::The 'mode' input is required as of CodSpeed Action v4. Please explicitly set 'mode' to 'simulation' or 'walltime'. Before, this variable was automatically set to instrumentation on every runner except for CodSpeed macro runners where it was set to walltime by default. See https://codspeed.io/docs/instruments for details."
exit 1
fi

Expand Down Expand Up @@ -132,11 +133,11 @@ runs:
elif [ "$VERSION_TYPE" = "branch" ]; then
# Install from specific branch using cargo
source $HOME/.cargo/env
cargo install --git https://github.com/CodSpeedHQ/runner --branch "$RUNNER_VERSION"
cargo install --locked --git https://github.com/CodSpeedHQ/runner --branch "$RUNNER_VERSION"
elif [ "$VERSION_TYPE" = "rev" ]; then
# Install from specific commit/rev using cargo
source $HOME/.cargo/env
cargo install --git https://github.com/CodSpeedHQ/runner --rev "$RUNNER_VERSION"
cargo install --locked --git https://github.com/CodSpeedHQ/runner --rev "$RUNNER_VERSION"
else
# Release version
head_status=$(curl -I -fsSL -w "%{http_code}" -o /dev/null https://github.com/CodSpeedHQ/runner/releases/download/v$RUNNER_VERSION/codspeed-runner-installer.sh)
Expand Down
2 changes: 1 addition & 1 deletion examples/nodejs-typescript-codspeed.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@ jobs:
- name: Run benchmarks
uses: CodSpeedHQ/action@v4
with:
mode: instrumentation
mode: simulation
run: node -r esbuild-register benches/bench.ts
2 changes: 1 addition & 1 deletion examples/python-pytest-codspeed.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,5 @@ jobs:
- name: Run benchmarks
uses: CodSpeedHQ/action@v4
with:
mode: instrumentation
mode: simulation
run: pytest tests/ --codspeed
2 changes: 1 addition & 1 deletion examples/rust-cargo-codspeed.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,5 @@ jobs:
- name: Run the benchmarks
uses: CodSpeedHQ/action@v4
with:
mode: instrumentation
mode: simulation
run: cargo codspeed run
Loading