databricks SQL dialectsrc/dialects/databricks/index.ts).DatabricksDialect extends MySQLDialect purely to reuse| Property | Value |
|---|---|
| Dialect name | databricks |
| Extends | MySQLDialect (mysql) |
| Driver | mysql2 |
| Cloud SDK | none |
| Default port | 443 |
| Local runtime | none — hosted service only |
export class DatabricksDialect extends MySQLDialect {
readonly name = 'databricks';
constructor(config: MySQLDialectOptions) {
// Databricks SQL Warehouses are reached over HTTPS on port 443.
super({ port: 443, ...config });
}
// ...
}build*Sql/build*Query methods@databricks/sql) rather than mysql2; the inherited connection code isMySQLDialect:escapeId(identifier: unknown): string {
const id = String(identifier ?? '');
return `\`${id.replace(/`/g, '``')}\``;
}dialect.escapeId('my_table'); // => `my_table`$ npm install ts-prorm-orm mysql2$ pnpm add ts-prorm-orm mysql2$ yarn add ts-prorm-orm mysql2$ bun add ts-prorm-orm mysql2mysql2 is the same driver the mysql dialect uses.[!CAUTION] A SQL-generation layer
The driver the catalog lists,mysql2, is the one the parent dialect loads; it does not speak Databricks's own wire protocol, so connecting to a live server is unverified. The builders — everything under Generated SQL — are what this dialect is for. See Caveats.
import { Prorm } from 'ts-prorm-orm';
export const db = new Prorm({
dialect: 'databricks',
host: 'dbc-a1b2c3d4-e5f6.cloud.databricks.com',
dialectOptions: {
catalog: 'main',
schema: 'default',
},
});import { DataTypes } from 'ts-prorm-orm';
const User = db.define('User', {
id: { type: DataTypes.INTEGER(), primaryKey: true, autoIncrement: true },
email: { type: DataTypes.STRING(255), allowNull: false, unique: true },
name: { type: DataTypes.STRING(100) },
});
await db.sync(); // CREATE TABLE — see Generated SQL belowimport { Op } from 'ts-prorm-orm';
await User.create({ email: 'ada@example.com', name: 'Ada' });
const page = await User.findAll({
where: { name: { [Op.like]: 'A%' } },
order: [['id', 'ASC']],
limit: 20,
offset: 40,
});query(), captured with the driver stubbed out. The model layer inlines the values.db.sync() for the model in the Quick start:CREATE TABLEawait User.create({ email: 'ada@example.com', name: 'Ada' });
await User.update({ name: 'Grace' }, { where: { email: 'ada@example.com' } });
await User.destroy({ where: { id: 1 } });await User.upsert({ id: 1, email: 'ada@example.com', name: 'Ada Lovelace' });MERGE statementupsert() also reads the row by primary key before and after the write, to return the instance.await User.findAll({
where: { name: { [Op.like]: 'A%' } },
order: [['id', 'ASC']],
limit: 20,
offset: 40,
});await db.transaction(async (t) => {
await User.create({ email: 'grace@example.com', name: 'Grace' }, { transaction: t });
// A nested transaction is a SAVEPOINT; its failure undoes only its own work.
await db
.transaction(async (sp) => {
await User.destroy({ where: { id: 1 }, transaction: sp });
throw new Error('keep the row');
}, { transaction: t })
.catch(() => {});
});STRING → STRING, auto-increment INTEGER → INT, BOOLEAN → BOOLEAN, DATE → TIMESTAMP, UUID → STRING, JSON → STRING. The type comparison lines the major engines up side by side.import { Prorm } from 'ts-prorm-orm';
const orm = new Prorm({
dialect: 'databricks',
host: 'dbc-xxxxxxxx-xxxx.cloud.databricks.com',
// port defaults to 443 (HTTPS SQL Warehouse); no need to set it
});import {
DatabricksDialect,
createDatabricksDialect,
} from 'prorm/dialects/databricks';
const dialect = new DatabricksDialect({});
// or the factory that mirrors the other dialects:
const dialect2 = createDatabricksDialect();DatabricksDialect extends MySQLDialect (mysql) andObject.getOwnPropertyNames(DatabricksDialect.prototype),| Area | Overridden |
|---|---|
| DML builders | buildDeleteQuery, buildInsertQuery, buildLimitOffset, buildMergeSql, buildSetColumnCommentSql, buildSetColumnNullabilitySql, buildSetTablePropertiesSql, buildUpdateQuery, buildUpsertQuery, quoteTable |
| Types | buildAddColumnSql, buildColumnDefinitionSql, buildDropColumnSql, buildRenameColumnSql, getDataTypeSql, legacySparkTypeSql, mapNativeType, renderSparkType |
| DDL and schema | addColumn, addIndex, buildAddColumnSpecSql, buildCreateTableAsSelect, buildCreateTableSql, buildDropTableSql, changeColumn, createFulltextIndex, createIndex, createSpatialIndex, createTable, dropIndex, dropTable, optimize, removeColumn, removeIndex, renameColumn, vacuum |
| Other | buildCopyIntoSql, buildOptimizeSql, buildRestoreSql, buildTimeTravelSelectSql, buildUnsetTablePropertiesSql, buildVacuumSql, copyInto, mergeInto, qualifiedTable, restore, runtimeAtLeast, selectAsOf, setColumnComment, setColumnNullability, setTableProperties, structFieldName, unsetTableProperties |
buildCreateTableSql(tableName, columns, options?) emits Delta/SparkCREATE TABLE grammar. The Databricks-specific options are:| Option | Emits |
|---|---|
using (DatabricksTableFormat) | USING <format> — defaults to DELTA. Values: DELTA, PARQUET, CSV, JSON, ORC, AVRO |
partitionedBy: string[] | PARTITIONED BY (cols) |
clusteredBy: { columns, buckets } | CLUSTERED BY (cols) INTO n BUCKETS |
location: string | LOCATION '...' (external/unmanaged table) |
comment: string | trailing COMMENT '...' |
tblproperties: Record<string,string> | TBLPROPERTIES ('k'=v, ...) |
ifNotExists: boolean | CREATE TABLE IF NOT EXISTS |
ENGINE= / ROW_FORMAT= / charset concept.dialect.buildCreateTableSql(
'events',
{
id: { type: 'BIGINT' },
country: { type: 'STRING' },
created_at: { type: 'DATE' },
},
{ using: 'DELTA', partitionedBy: ['country'] },
);using defaults to DELTA when omitted:dialect.buildCreateTableSql('t', { id: { type: 'BIGINT' } });
// => CREATE TABLE `t` (`id` BIGINT) USING DELTAdialect.buildCreateTableSql(
'sales',
{ id: { type: 'BIGINT' }, region: { type: 'STRING' } },
{
clusteredBy: { columns: ['id'], buckets: 8 },
location: 's3://bucket/sales',
tblproperties: { 'delta.appendOnly': 'true' },
},
);buildCreateTableSql is:USING → PARTITIONED BY → CLUSTERED BY ... INTO n BUCKETS → LOCATIONCOMMENT → TBLPROPERTIES.AUTO_INCREMENT. A primaryKey column is emitted as an inline (informational)PRIMARY KEY — see Caveats.dialect.buildCreateTableSql('users', {
id: { type: 'BIGINT', primaryKey: true, autoIncrement: true },
name: { type: 'STRING' },
});buildColumnDefinitionSql): NOT NULL is emittedallowNull === false; GENERATED ALWAYS AS IDENTITY when autoIncrement;DEFAULT <value> when defaultValue is set and the column is not anPRIMARY KEY when primaryKey; and a trailing COMMENT.buildCreateTableAsSelect(tableName, selectQuery, options?) emitsCREATE TABLE ... USING <format> AS SELECT .... Only format,ifNotExists, and tblproperties apply —dialect.buildCreateTableAsSelect(
'summary',
'SELECT country, COUNT(*) AS n FROM events GROUP BY country',
{ using: 'DELTA' },
);buildUpsertQuery)ON DUPLICATE KEY UPDATE. Upserts are expressed as aMERGE INTO statement where the row to upsert is supplied as ansource relation (SELECT ? AS col, ...) and matched on the conflictconst { sql, values } = dialect.buildUpsertQuery(
'users',
{ id: 1, name: 'Ada', email: 'ada@example.com' },
{ conflictFields: ['id'] },
);values is [1, 'Ada', 'ada@example.com'] (parameterized in column order).conflictFields picks the ON/match key columns; it defaults to allupdateOnDuplicate restricts which columns the WHEN MATCHED branchconflictFields). If there are no non-key columns to update, theWHEN MATCHED branch is omitted entirely.Literal value is inlined into the source SELECT (<val> AS col)buildUpsertQuery throws.buildInsertQuery routes to this MERGE builder whenever options.upsert isdialect.buildInsertQuery('users', { id: 1, name: 'Ada' });
// => INSERT INTO `users` (`id`, `name`) VALUES (?, ?)
dialect.buildInsertQuery(
'users',
{ id: 1, name: 'Ada' },
{ upsert: true, conflictFields: ['id'] },
);
// => MERGE INTO `users` ... (never ON DUPLICATE KEY)getDataTypeSql(dataType) maps ORM types to Spark SQL / Delta types. TheSTRINGVARCHAR(n)/TEXT), and the complex typesARRAY<>/MAP<>/STRUCT<> are supported.ORM type (key) | Spark SQL type |
|---|---|
STRING, CHAR, TEXT, ENUM, SET, UUID, JSON, JSONB | STRING |
INTEGER (default) | INT |
INTEGER length: 1 | TINYINT |
INTEGER length: 2 | SMALLINT |
INTEGER length: 8 | BIGINT |
BIGINT | BIGINT |
FLOAT | FLOAT |
DOUBLE | DOUBLE |
DECIMAL | DECIMAL(precision,scale) (defaults 10,0) |
BOOLEAN | BOOLEAN |
DATE | TIMESTAMP (date + time) |
DATEONLY | DATE |
TIME | STRING (Spark has no standalone TIME) |
BLOB | BINARY |
ARRAY | ARRAY<elem> |
MAP | MAP<keyType, valueType> |
STRUCT | STRUCT<field: type, ...> |
| anything else | STRING |
ARRAY/MAP, an unspecified element/key/value type defaults to STRING.VARCHAR, CHARACTER VARYING, NVARCHAR, CHAR, TEXT/NTEXT,TINYTEXT, MEDIUMTEXT, LONGTEXT, or CLOB returns STRING; any otherdialect.getDataTypeSql('VARCHAR(255)'); // 'STRING'
dialect.getDataTypeSql({ key: 'DECIMAL', precision: 12, scale: 4 }); // 'DECIMAL(12,4)'
dialect.getDataTypeSql({ key: 'BLOB' }); // 'BINARY'
dialect.getDataTypeSql({ key: 'DATE' }); // 'TIMESTAMP'
dialect.getDataTypeSql({ key: 'ARRAY', type: { key: 'STRING' } }); // 'ARRAY<STRING>'
dialect.getDataTypeSql({
key: 'MAP',
keyType: { key: 'STRING' },
valueType: { key: 'BIGINT' },
}); // 'MAP<STRING, BIGINT>'
dialect.getDataTypeSql({
key: 'STRUCT',
fields: { a: { key: 'STRING' }, b: { key: 'BIGINT' } },
}); // 'STRUCT<a: STRING, b: BIGINT>'PRIMARY KEY / FOREIGN KEY constraints are informationalNOT ENFORCED, used by the optimizer for query planning);PRIMARY KEY when a column requests it (see the identity example above)RETURNING clause. The dialect throws rather thanbuildUpsertQuery(..., { returning: true }) throwsDatabricks (Delta MERGE) does not support the RETURNING clause. ...buildUpdateQuery(..., { returning: true }) throwsbuildDeleteQuery(..., { returning: true }) throwsUPDATE and DELETE support WHERE but not a trailing LIMIT. Anylimit option is stripped before delegating to the inherited builder:dialect.buildUpdateQuery('users', { name: 'Ada' }, { id: 1 }, { limit: 5 });
// => UPDATE `users` SET `name` = ? WHERE ... (no LIMIT), values ['Ada', 1]
dialect.buildDeleteQuery('users', { id: 1 }, { limit: 5 });
// => DELETE FROM `users` WHERE ... (no LIMIT)OFFSET, so buildLimitOffsetLIMIT 18446744073709551615 OFFSET n sentinel hack:dialect.buildLimitOffset(undefined, 20); // ' OFFSET 20'
dialect.buildLimitOffset(5); // ' LIMIT 5'
dialect.buildLimitOffset(5, 10); // ' LIMIT 5 OFFSET 10'OFFSET is only valid on the outermost query and requires a recentWHERE predicate for deep pagination.buildAddColumnSql emits Spark's pluralALTER TABLE t ADD COLUMNS (col type [COMMENT '...']) — not MySQL'sADD COLUMN. NOT NULL, DEFAULT, PRIMARY KEY, and identitydialect.buildAddColumnSql('users', 'age', { type: 'INT' });
// => ALTER TABLE `users` ADD COLUMNS (`age` INT)
dialect.buildAddColumnSql('users', 'bio', {
type: 'STRING', allowNull: false, defaultValue: 'x', comment: 'biography',
});
// => ALTER TABLE `users` ADD COLUMNS (`bio` STRING COMMENT 'biography')buildRenameColumnSql uses RENAME COLUMN old TO new (not MySQL CHANGE).buildDropColumnSql → ALTER TABLE t DROP COLUMN col (requires DeltachangeColumn() throws — MySQL-style CHANGE col <full-def> is notrenameColumn(), setColumnComment()ALTER COLUMN col COMMENT '...'), or setColumnNullability()ALTER COLUMN col SET|DROP NOT NULL) for the supported in-place edits.buildDropTableSql never emits CASCADE (Spark has no DROP TABLE ...
CASCADE); a passed cascade option is ignored.buildSetTablePropertiesSql →ALTER TABLE t SET TBLPROPERTIES (...), buildUnsetTablePropertiesSql →ALTER TABLE t UNSET TBLPROPERTIES [IF EXISTS] (...).CREATE INDEX. addIndex, createIndex,removeIndex, dropIndex, createFulltextIndex, and createSpatialIndexOPTIMIZE ... ZORDER BY), partitioning, or a Bloom-filter index instead.tests/dialects/databricks.test.ts,escapeId.CREATE TABLE with USING DELTA, PARTITIONED BY,CLUSTERED BY ... INTO n BUCKETS, LOCATION, TBLPROPERTIES, defaultDELTA, and GENERATED ALWAYS AS IDENTITY + informationalPRIMARY KEY.buildCreateTableAsSelect).buildUpsertQuery) SQL shape and parameter order, plus theRETURNING rejection.INSERT routing, and bulkInsert multi-row VALUES.SELECT clause assembly and standalone OFFSET (no BIGINT sentinel).UPDATE/DELETE without LIMIT, increment, and RETURNING rejection.ADD COLUMNS, DROP COLUMN, RENAME COLUMN,ALTER COLUMN comment/nullability, SET/UNSET TBLPROPERTIES,DROP TABLE without CASCADE, changeColumn rejection).databricks-crud@databricks/sqlOFFSET on DBR 11+, DeltaDROP COLUMN, identity-column support). Treat the SQLcapabilitiesOf() (src/transaction-support.ts),whereSupportOf() (src/query-builders/expression-render.ts),sqlObjectCapabilitiesOf() (src/sql-objects/capabilities.ts) and the| Capability | databricks |
|---|---|
| Transactions atomic | yes |
| Savepoints | yes |
| Isolation | up to SERIALIZABLE |
| Transaction modes | read-only yes; deferrable no; constraint deferral no |
| RETURNING | not emitted; returning is accepted and ignored |
| Upsert | MERGE INTO … WHEN MATCHED / WHEN NOT MATCHED |
| JSON column | STRING (JSONB → STRING) |
| Vector column | ARRAY<FLOAT> |
| Array column | ARRAY<INT> |
| WHERE builder | 28 operator keys plus the $json: prefix; $and/$or/$not; $exists: no; $in subquery: no; expression operands: no |
| SQL objects | family unknown: triggers no; procedures no; functions no; materialized views no; sequences no; enums check; RLS no |
| Streaming | queryStream() present; LISTEN absent; createExtension() present |
FindOptions becomes SQL