Skip to content

[pixels-cpp] Add Test & Feature Support: Double/Single Buffer Switch, Performance Scripts, and Filter Pushdown #1230

@whzruc

Description

@whzruc

Overview

This task focuses on enhancing the pixels-cpp testing framework and improving its compatibility with the latest Parquet specifications, specifically addressing buffer management flexibility and fixing filter pushdown implementation.

Detailed Work Items

1. Implement Double/Single Buffer Switching

Introduce a runtime configuration to toggle between Double Buffer and Single Buffer modes, allowing for flexible memory management testing and comparison.

Action: Modify configuration in pixels-cpp.properties and update logic in PixelsScanFunction.cpp to support the buffer switch.

2. Integrate Performance Test Suite

Add necessary performance testing scripts and configurations to facilitate convenient benchmarking and testing.

Action: Add performance test scripts (Perf Scripts), introduce benchmark.json for test definitions, and create a testcase directory for input data or structured test cases.

3. Fix Filter Pushdown for New Parquet Version

Update the filter pushdown implementation to ensure correct functionality and compatibility with the behavior of the newer Parquet version.

Action: Revise and update the filter pushdown logic within the PixelsFilter.cpp module to adapt to the new Parquet specification.
Image

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingenhancementNew feature or request

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions