null

Summary

Allow users to specify JavaScript user-defined functions (expressions) and accumulators in Agg pipelines. This functionality was delivered as part of “MapReduce in Agg” (PM-776) and will be exposed through this project.

Motivation

We want to empower users to write custom expressions in order to implement behaviors not supported by MQL. This will bring us closer to parity with many databases, in particular DocumentDB, that support UDFs in JS and several other languages. It was also a critical feature for ETL pipelines in Tableau Prep.

The most common use case is projecting computed columns and filtering, however, the concept can also be used to parameterize operations like sorting with a custom comparator or clustering with a distance function, as well as custom $group keys.
Last but not least, this effort is a prerequisite to deprecate the MR command (PM-1470).

Documentation

Scope Document
Technical Design Document