[Feature Request] Require partition filter

[arturowczarek]

Feature request

Which Delta project/connector is this regarding?

• Spark
• Standalone
• Flink
• Kernel
• Other (fill in here)

Overview

We should have a table property requirePartitionFilter which would allow to read the table only if a predicate on partition is provided in the query. This functionality is present in BigQuery: https://docs.cloud.google.com/bigquery/docs/querying-partitioned-tables#require_a_partition_filter_in_queries

Motivation

Analysts often query huge tables with long history. Usually they are interested in recent data and they have to wait long before the query completes if they incidentally query the entire table.

Further details

I've submitted a proposal how this could be implemented: #5630

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

• Yes. I can contribute this feature independently.
• Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
• No. I cannot contribute this feature at this time.

It think I can contribute this myself but would welcome any support to have it merged. I'd welcome any information if you think it may not work or confirmation that it should work and the submitted solution proposal is the way it should be implemented.