Support native NV12 textures for improved performance

[FreddyFunk]

Summary

NV12 is a common video format (4:2:0 chroma subsampling, 12 bits per pixel) that many cameras output natively. Currently, the application recognizes NV12 but always converts it to RGBA (32 bits per pixel) through GStreamer's videoconvert. This results in ~2.67x memory expansion and unnecessary CPU/GPU work.

Current Implementation

• NV12 recognized: Codec::NV12 enum variant exists in src/media/formats/codec.rs
• Always converted to RGBA: Pipeline enforces video/x-raw,format=RGBA output
• Single-plane textures: GPU filter uses only RGBA textures, no multi-plane support
• Memory overhead: NV12 (12 bpp) → RGBA (32 bpp) for all processing

Key locations:

• src/media/formats/codec.rs - Format definitions (NV12 has preference rank 3)
• src/backends/virtual_camera/gpu_filter.rs - GPU filter renderer (RGBA only)
• src/shaders/filter_compute.wgsl - Compute shader (RGBA input/output)

Proposed Changes

1. Update frame structure for multi-plane support

   pub enum PixelFormat {
       RGBA,
       NV12,
   }
   
   pub struct CameraFrame {
       pub format: PixelFormat,
       pub planes: Vec<Arc<[u8]>>,    // Y plane + UV plane for NV12
       pub strides: Vec<u32>,          // Stride per plane
   }

2. GPU texture changes

   • Create separate Y and UV textures for NV12
   • Update compute shaders to sample from both planes
   • Handle 4:2:0 chroma sampling in shader

3. Pipeline pass-through

   • Skip videoconvert when NV12 can be used directly
   • Only convert to RGBA when filters require it

Upstream Dependency: WGPU/libcosmic/iced

Native NV12 texture support requires enabling additional WGPU features during initialization. Currently, the application uses cosmic::iced_wgpu::wgpu from libcosmic, which means:

• WGPU feature flags: Need to enable multi-planar texture formats (NV12/P010/etc.)
• libcosmic/iced changes: The WGPU instance initialization would need to request these features
• Potential upstream PR: May require changes to libcosmic or iced_wgpu to expose these capabilities

This feature request may need to be coordinated with or depend on upstream libcosmic changes.

Benefits

• ~2.67x memory savings: NV12 (12 bpp) vs RGBA (32 bpp)
• Reduced conversion overhead: No CPU/GPU time spent on format conversion
• Better compatibility: Many cameras output NV12 natively
• Lower bandwidth: Less data to transfer between components

Considerations

• Need fallback path for systems without multi-planar texture support
• Shaders need to handle YUV→RGB conversion when displaying
• Virtual camera output may need format negotiation with consumers