Implement a rendering pipeline with middleware (adds support for Brotli and Gzip, cleans up minification)

[EvanCarroll]

I've updated this architecture to support a rendering pipeline which everything goes through.

• Minification is now a middleware.
• Livereload is now a middleware.

At the end of every pipeline we call write(). It's no longer called as IO call at the end of a function (which only know it was writing an index.html, and thus would minify .txt). This works and passes all tests,

• livereload knows if it's in a pipeline generated from an HTML
• minification knows if its in a pipeline generating HTML

With this infrastructure, this patch

• adds compression closing Support gzip compression in zola serve #1684
• and encryption Is there any way to create private post? #2993

AI summary of patch..

Core Architecture

  ┌─────────────┐
  │  Renderable │ (trait)
  └──────┬──────┘
         │
         ├─ Page
         ├─ Section
         ├─ Taxonomy
         ├─ Pager
         ├─ Feed
         ├─ Sitemap
         └─ ... (11 types)
         │
         ▼
  ┌──────────────────┐
  │    Pipeline      │
  ├──────────────────┤
  │ 1. Render        │◄── Tera templates
  │ 2. Metadata      │
  │ 3. Middleware    │◄── Pluggable chain
  │ 4. Process       │
  │ 5. Write         │
  └──────────────────┘
         │
         ├─ LiveReloadMiddleware
         ├─ MinifyMiddleware
         └─ [future: compression, encryption]
         │
         ▼
  ┌──────────────────┐
  │  ProcessedContent│
  └──────────────────┘
         │
         ▼
  ┌──────────────────┐
  │  ContentWriter   │
  ├──────────────────┤
  │ BuildMode::Disk  │─── File system
  │ BuildMode::Memory│─── In-memory cache
  │ BuildMode::Both  │─── Both
  └──────────────────┘

Key Components

1. Renderable Trait (renderable.rs)

• Purpose: Unified interface for all content types
• Methods:
  • render() - produces HTML/XML/JSON from templates
  • metadata() - provides routing & content type info
• Implementations: 11 types (Page, Section, Taxonomy, Feed, Sitemap, robots.txt, 404, Pager, Redirect, etc.)

2. Pipeline (pipeline.rs)

• Purpose: Orchestrates the 5-stage rendering process
• Stages:
  a. Render: Template → String (via Tera)
  b. Metadata: Extract path, content type, permalink
  c. Context: Wrap in MiddlewareContext
  d. Middleware Chain: Sequential transformations
  e. Return: ProcessedContent ready to write

3. Middleware Trait (middleware.rs)

• Purpose: Extensible content transformation
• Interface: process(&self, ctx: &mut MiddlewareContext)
• Context includes:
  • content: String - the HTML/XML/JSON
  • metadata: ContentMetadata - path, template, content type
  • config: Arc - site configuration

4. Concrete Middleware

• LiveReloadMiddleware (middleware/livereload.rs):
  • Injects <script src="/livereload.js"> before
  • Only processes ContentType::Html
• MinifyMiddleware (middleware/minify.rs):
  • Removes whitespace from HTML
  • Conditional on config.minify_html
  • Only processes ContentType::Html

5. ContentWriter (writer.rs)

• Purpose: Handles output destination
• BuildMode:
  • Disk: Write to filesystem (public/ directory)
  • Memory: Store in SITE_CONTENT cache (for live reload)
  • Both: Do both operations

Design Benefits

1. Separation of Concerns: Rendering, processing, and writing are isolated
2. Type Safety: ContentType enum ensures correct middleware application
3. Extensibility: Add new middleware without touching core logic
4. Testability: Each component has unit tests
5. Future-Ready: Designed for compression & encryption (per commit message)
6. Unified Flow: All 11 content types go through the same pipeline

Before vs After

Before: Rendering logic scattered, minification/livereload handled per-type

After: Single pipeline with pluggable middleware - every rendered item flows through the same chain

This architecture mirrors common web frameworks (Express.js middleware, ASP.NET pipeline) and provides a clean foundation for adding post-processing features like
compression or encryption.

---

• write_content isn't smart enough to do what we need.
• this is a cleaner architecture.
• Now pages with txt templates are not minified.
• Add test for text template whitespace preservation with minification
  • Test verifies that non-HTML templates (e.g., .txt) preserve whitespace
  • Created text_content.txt template with multiple spaces, tabs, and newlines
  • Created test page using the text template
  • Test passes with the minification fix (ed36c1e6) and fails without it
  • Updated page count assertions due to new test page

Addresses #3006

[EvanCarroll]

@Keats if you determine this architecture isn't something you want to move towards, please tell me so I can make adjustments or quit closing out issues.

[ccjmne]

Seems really cool, but the current MR is a lot for a human to digest.
Do you think you could keep the encryption middleware in another PR for the time being? Though maybe @Keats can review it like that.

[EvanCarroll]

Seems really cool, but the current MR is a lot for a human to digest. Do you think you could keep the encryption middleware in another PR for the time being? Though maybe @Keats can review it like that.

Fair and certainly doable. Because the architecture isn't cleanly in one patch it makes this patch even more complex. I tweak it a few times. If this is needed. what I can do is rewrite the the patches entirely to sink the commits separately. It's a waste of time though if that's not needed.

Actually if the middlewares themselves had their own config parser, they would be entirely self-contained and this would be trivial..

The next patch that I want to do pulls out markdown rendering putting that into the middleware too. And closes out #2930, and #2420 and #2790. This would make adding first-class Tyrst the same way trivial.

[Keats]

It is indeed way too big to review. A few other things:

• The issue for secret content is closed so for now it shouldn't be implemented
• Same-ish for pre-compression, I don't think it's super important
• I think it's too complex overall? All we need to have is a way to differentiate HTML and other content and do different operations when wrtiting the content.

[EvanCarroll]

I'll get back this and rebase the patch to make it easier to process. I got one more project I have to get off my plate. The architectural changes, should make all the middle wares one file (or relatively small) so you can pick whatever middleware you want.