skip to content
Kostas Pardalis

Batch Inference, Type Systems, and Why Cortex AISQL Got Me Excited

AI Snowflake SQL inference

Snowflake’s Cortex AISQL announcement got me excited, and I want to explain why. It represents a paradigm shift in integrating large language models into data systems as structured, composable functions rather than opaque tools.

Prompt Operators as Primitives

Distilling LLM capabilities into five well-defined operators that address 80% of use cases signals meaningful progress. This approach prioritizes reproducibility and composability over raw prompt flexibility.

Batch Inference Gets First-Class Support

Unlike current infrastructure optimized for low-latency interactive applications, AISQL emphasizes high throughput, high latency workloads. This reflects genuine demand for large-scale data processing rather than conversational interfaces.

Structured Data Integration

The system enables combining structured databases with unstructured content (PDFs, audio, images) within unified query plans—a powerful capability for multi-modal workflows.

Type Systems Evolution

The introduction of a File type represents necessary evolution as computational paradigms shift, making multi-modal processing more ergonomic.

Open Questions

Critical concerns about composability in AISQL:

  • Cost efficiency: Re-running multiple LLM-invoking CTEs for testing creates expense problems
  • Determinism: Non-deterministic outputs complicate isolating prompt changes from upstream variation

I’m interested in community perspectives on LLM inference within analytical workflows.