Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2024 Matthew Thomas
Copyright (c) 2024 Matthew Poulton-White

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
310 changes: 222 additions & 88 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,129 +1,263 @@
# CTRF Common JavaScript Library
# CTRF Reference Implementation TypeScript

Common JavaScript library for working with CTRF reports, including type definitions and utility functions.
The reference implementation in TypeScript for the [Common Test Report Format (CTRF)](https://github.com/ctrf-io/ctrf) specification.

<div align="center">
<div style="padding: 1.5rem; border-radius: 8px; margin: 1rem 0; border: 1px solid #30363d;">
<span style="font-size: 23px;">💚</span>
<h3 style="margin: 1rem 0;">CTRF tooling is open source and free to use</h3>
<p style="font-size: 16px;">You can support the project with a follow and a star</p>
## Open Standard

<div style="margin-top: 1.5rem;">
<a href="https://github.com/ctrf-io/ctrf-core-js">
<img src="https://img.shields.io/github/stars/ctrf-io/ctrf-core-js?style=for-the-badge&color=2ea043" alt="GitHub stars">
</a>
<a href="https://github.com/ctrf-io">
<img src="https://img.shields.io/github/followers/ctrf-io?style=for-the-badge&color=2ea043" alt="GitHub followers">
</a>
</div>
</div>
[CTRF](https://github.com/ctrf-io/ctrf) is an open standard built and shaped by community contributions.

<p style="font-size: 14px; margin: 1rem 0;">
Your feedback and contributions are essential to the project's success:

Contributions are very welcome! <br/>
Explore more <a href="https://www.ctrf.io/integrations">integrations</a> <br/>
We’d love your feedback, <a href="https://app.formbricks.com/s/cmefs524mhlh1tl01gkpvefrb">share it anonymously</a>.
- [Contribute](CONTRIBUTING.md)
- [Discuss](https://github.com/orgs/ctrf-io/discussions)

</p>
</div>
## Support

You can support the project by giving this repository a star ⭐

## Installation

```sh
npm install ctrf
npm install [email protected]
```

## Quick Start

```typescript
import { ReportBuilder, TestBuilder, validateStrict } from 'ctrf'

// Build a report using the fluent API
const report = new ReportBuilder()
.tool({ name: 'jest', version: '29.0.0' })
.addTest(
new TestBuilder()
.name('should add numbers')
.status('passed')
.duration(150)
.build()
)
.addTest(
new TestBuilder()
.name('should handle errors')
.status('failed')
.duration(200)
.message('Expected 5 but got 4')
.build()
)
.build()

// Validate the report
validateStrict(report)
```

## API Reference

### Schema Types
> 📚 **Full API Documentation:** [API Reference](../docs/README.md)

### Types

- [Attachment](docs/interfaces/Attachment.md)
- [Baseline](docs/interfaces/Baseline.md)
- [Environment](docs/interfaces/Environment.md)
- [InsightsMetric](docs/interfaces/InsightsMetric.md)
- [Report](docs/interfaces/Report.md)
- [Results](docs/interfaces/Results.md)
- [RetryAttempt](docs/interfaces/RetryAttempt.md)
- [RootInsights](docs/interfaces/RootInsights.md)
- [Step](docs/interfaces/Step.md)
- [Summary](docs/interfaces/Summary.md)
- [Test](docs/interfaces/Test.md)
- [TestInsights](docs/interfaces/TestInsights.md)
- [Tool](docs/interfaces/Tool.md)
- [TestStatus](docs/type-aliases/TestStatus.md)
Full TypeScript types are provided for all CTRF entities.

### File Operations Methods
```typescript
import type { CTRFReport, Test } from 'ctrf'

- [readReportFromFile](docs/functions/readReportFromFile.md)
- [readReportsFromDirectory](docs/functions/readReportsFromDirectory.md)
- [readReportsFromGlobPattern](docs/functions/readReportsFromGlobPattern.md)
const report: CTRFReport = { /* ... */ }
const test: Test = { /* ... */ }
```

### Validation

```typescript
import { isValid, validate, validateStrict, isCTRFReport, ValidationError } from 'ctrf'

// Quick validation (returns boolean)
if (isValid(report)) {
console.log('Report is valid')
}

### Report Processing Methods
// Detailed validation (returns ValidationResult)
const result = validate(report)
if (!result.valid) {
result.errors?.forEach(err => console.error(err.message))
}

- [enrichReportWithInsights](docs/functions/enrichReportWithInsights.md)
- [mergeReports](docs/functions/mergeReports.md)
- [sortReportsByTimestamp](docs/functions/sortReportsByTimestamp.md)
- [storePreviousResults](docs/functions/storePreviousResults.md)
// Strict validation (throws on invalid)
try {
validateStrict(report)
} catch (error) {
if (error instanceof ValidationError) {
console.error('Invalid report:', error.errors)
}
}

### Validation Methods
// Type guards
if (isCTRFReport(data)) {
// data is typed as CTRFReport
}
```

- [isValidCtrfReport](docs/functions/isValidCtrfReport.md)
- [validateReport](docs/functions/validateReport.md)
- [validateReportStrict](docs/functions/validateReportStrict.md)
### Building Reports

### Tree Operations Methods
```typescript
import { ReportBuilder, TestBuilder } from 'ctrf'

// ReportBuilder - fluent API for constructing reports
const report = new ReportBuilder()
.tool({ name: 'vitest', version: '1.0.0' })
.environment({ os: 'linux', arch: 'x64' })
.addTest(/* ... */)
.build()

// TestBuilder - fluent API for constructing tests
const test = new TestBuilder()
.name('User login test')
.status('passed')
.duration(1500)
.suite(['Authentication', 'Login'])
.tags(['smoke', 'critical'])
.filePath('tests/auth/login.test.ts')
.browser('chrome')
.build()
```

- [findSuiteByName](docs/functions/findSuiteByName.md)
- [findTestByName](docs/functions/findTestByName.md)
- [flattenTree](docs/functions/flattenTree.md)
- [getAllTests](docs/functions/getAllTests.md)
- [getSuiteStats](docs/functions/getSuiteStats.md)
- [organizeTestsBySuite](docs/functions/organizeTestsBySuite.md)
- [traverseTree](docs/functions/traverseTree.md)
### Parsing Reports

### Test Operations Methods
```typescript
import { parse, stringify } from 'ctrf'

- [findTestById](docs/functions/findTestById.md)
- [generateTestIdFromProperties](docs/functions/generateTestIdFromProperties.md)
- [getTestId](docs/functions/getTestId.md)
- [setTestId](docs/functions/setTestId.md)
- [setTestIdsForReport](docs/functions/setTestIdsForReport.md)
// Parse from string
const parsed = parse(jsonString)

### Utility Types
// Stringify with formatting
const json = stringify(report, { pretty: true, indent: 2 })
```

- [SortOrder](docs/enumerations/SortOrder.md) (enumeration)
- [TestTree](docs/interfaces/TestTree.md)
- [TreeNode](docs/interfaces/TreeNode.md)
- [TreeOptions](docs/interfaces/TreeOptions.md)
- [ValidationResult](docs/interfaces/ValidationResult.md)
- [TreeTest](docs/type-aliases/TreeTest.md)
- [CTRF\_NAMESPACE](docs/variables/CTRF_NAMESPACE.md)
### Filtering & Querying

## TypeScript Types
```typescript
import { filterTests, findTest } from 'ctrf'

// Filter by criteria
const filtered = filterTests(report, {
status: 'failed',
suite: 'Authentication',
tags: ['smoke'],
})

// Find specific test
const test = findTest(report, { name: 'login test' })
const testById = findTest(report, { id: 'test-uuid' })
```

The library exports comprehensive TypeScript types for working with CTRF reports:
### Merging Reports

```typescript
import type { Report, Test, Insights } from "ctrf";
import { mergeReports } from 'ctrf'

function analyzeReport(report: Report): void {
const flakyTests = report.results.tests.filter((test: Test) => test.flaky);
const insights = report.insights as Insights;
// Merge multiple reports into one
const merged = mergeReports([report1, report2, report3], {
deduplicateTests: true, // Remove duplicate tests by ID
})
```

console.log(`Flaky rate: ${insights?.flakyRate.current}`);
}
### ID Generation

```typescript
import { generateTestId, generateReportId } from 'ctrf'

// Generate deterministic test ID from properties
const testId = generateTestId({
name: 'should add numbers',
suite: ['Math', 'Addition'],
filePath: 'tests/math.test.ts',
})

// Generate random report ID
const reportId = generateReportId()
```

### Insights & Analytics

```typescript
import { addInsights, isTestFlaky } from 'ctrf'

// Enrich a report with insights from historical data
const enriched = addInsights(
currentReport,
historicalReports,
{ baseline: baselineReport }
)

// Access insights
console.log(enriched.insights?.passRate) // { current: 0.95, baseline: 0.90, change: 0.05 }
console.log(enriched.insights?.flakyRate) // { current: 0.02, baseline: 0.05, change: -0.03 }

// Check if a test is flaky
const isFlaky = isTestFlaky(test)
```

### Summary Calculation

```typescript
import { calculateSummary } from 'ctrf'

// Calculate summary from tests
const summary = calculateSummary(tests)
// { tests: 10, passed: 8, failed: 1, skipped: 1, pending: 0, other: 0, ... }
```

## What is CTRF?
### Constants

CTRF is a universal JSON test report schema that addresses the lack of a standardized format for JSON test reports.
```typescript
import {
REPORT_FORMAT,
CURRENT_SPEC_VERSION,
SUPPORTED_SPEC_VERSIONS,
TEST_STATUSES,
CTRF_NAMESPACE,
} from 'ctrf'

REPORT_FORMAT // 'CTRF'
CURRENT_SPEC_VERSION // '0.0.0'
SUPPORTED_SPEC_VERSIONS // ['0.0.0']
TEST_STATUSES // ['passed', 'failed', 'skipped', 'pending', 'other']
CTRF_NAMESPACE // UUID namespace for deterministic IDs
```

**Consistency Across Tools:** Different testing tools and frameworks often produce reports in varied formats. CTRF ensures a uniform structure, making it easier to understand and compare reports, regardless of the testing tool used.
### Error Handling

**Language and Framework Agnostic:** It provides a universal reporting schema that works seamlessly with any programming language and testing framework.
```typescript
import { validateStrict, ValidationError, ParseError } from 'ctrf'

try {
validateStrict(report)
} catch (error) {
if (error instanceof ValidationError) {
console.error('Schema validation failed:', error.errors)
}
}

**Facilitates Better Analysis:** With a standardized format, programatically analyzing test outcomes across multiple platforms becomes more straightforward.
try {
const parsed = parse(jsonString)
} catch (error) {
if (error instanceof ParseError) {
console.error('Invalid JSON:', error.message)
}
}
```

### Schema Access

```typescript
import { schema, getSchema, getCurrentSpecVersion, getSupportedSpecVersions } from 'ctrf'

// Get the current JSON Schema
console.log(schema)

## Support Us
// Get schema for specific version
const v0_0Schema = getSchema('0.0.0')

If you find this project useful, consider giving it a GitHub star ⭐ It means a lot to us.
// Get version info
const version = getCurrentSpecVersion() // '0.0.0'
const supported = getSupportedSpecVersions() // ['0.0.0']
```
Loading