Squirreling SQL Engine

Squirreling is a streaming async SQL engine built for the web. It is designed to query over various data sources and provide efficient streaming of results. 100% JavaScript with zero dependencies.

Features

• Lightweight and fast
• Easy to integrate with frontend applications
• Lets you move query execution closer to your users
• Supports standard SQL queries
• Async streaming for large datasets
• Native javascript Promises, AsyncGenerators, AbortSignals
• Async user-defined functions (UDFs)
• Constant memory usage for simple queries with LIMIT
• Robust error handling and validation designed for LLM tool use
• In-memory data option for simple use cases
• Late materialization for efficiency
• Select only

Usage

Squirreling returns an AsyncGenerator of AsyncRows, allowing you to process rows one at a time without loading everything into memory. AsyncRows are made up of AsyncCells, allowing for late materialization of values.

import { executeSql } from 'squirreling'

// Input table (in-memory for this example)
const users = [
  { id: 1, name: 'Alice', active: true },
  { id: 2, name: 'Bob', active: false },
  { id: 3, name: 'Charlie', active: true },
  // ...more rows
]

// Squirreling return types
interface AsyncRow {
  columns: string[]
  cells: Record<string, AsyncCell>
}
type AsyncCell = () => Promise<SqlPrimitive>

// Returns an AsyncIterable of rows with async cell loading
const asyncRows: AsyncIterable<AsyncRow> = executeSql({
  tables: { users },
  query: 'SELECT * FROM users',
})

// Process rows as they arrive (streaming)
for await (const { id, name } of asyncRows) {
  console.log(`User id=${await id()}, name=${await name()}`)
}

Squirreling exports a helper function collect to gather all rows into an array:

import { collect, executeSql } from 'squirreling'

// Collect all rows and cells into a materialized array
const rows: Record<string, SqlPrimitive>[] = await collect(executeSql({
  tables: { users },
  query: 'SELECT active, count(*) as cnt FROM users GROUP BY active',
}))
console.log(`Collected rows:`, rows)
// Collected rows: [ { active: true, cnt: 2 }, { active: false, cnt: 1 } ]

Supported SQL Features

• SELECT statements with WHERE, ORDER BY, LIMIT, OFFSET
• WITH clause for Common Table Expressions (CTEs)
• Subqueries in SELECT, FROM, and WHERE clauses
• JOIN operations: INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN, POSITIONAL JOIN
• GROUP BY and HAVING clauses

Functions

• Aggregate: COUNT, SUM, AVG, MIN, MAX, JSON_ARRAYAGG
• String: CONCAT, SUBSTRING, REPLACE, LENGTH, UPPER, LOWER, TRIM, LEFT, RIGHT, INSTR
• Math: ABS, CEIL, FLOOR, ROUND, MOD, RAND, RANDOM, LN, LOG10, EXP, POWER, SQRT
• Trig: SIN, COS, TAN, COT, ASIN, ACOS, ATAN, ATAN2, DEGREES, RADIANS, PI
• Date: CURRENT_DATE, CURRENT_TIME, CURRENT_TIMESTAMP, INTERVAL
• Json: JSON_VALUE, JSON_QUERY, JSON_OBJECT
• Regex: REGEXP_SUBSTR, REGEXP_REPLACE
• User-defined functions (UDFs)