Migrate — the data transfer subsystem

Migrate moves a dataset from one place to another: a database to another
database, a database to object storage, one environment to another. It is the
concrete implementation of the compliance checklist's cross-border transfer
control — every move is over a modern, authenticated, integrity-checked, logged
transport — and it is built to carry anything from a few megabytes to petabytes.

It has two halves. The code library (ts-prorm-orm Migrate) plans,
prices, gates and certifies a job. The platform (migrate.prod-orm.io) runs
it: a long transfer is the wrong shape for a request to the control plane, so a
job is created, started, and runs as a serverless function that notifies on
completion with a status.

import { Migrate } from 'ts-prorm-orm';

const job = Migrate.TransferJob.create({
  strategyId: 'sftp',                 // how it moves
  tier: 'premium',                    // the account's tier gates the strategy
  source: { kind: 'database', dialect: 'postgres', location: 'db://staging', region: 'eu' },
  dest:   { kind: 'database', dialect: 'postgres', location: 'db://prod',    region: 'eu' },
  bytes: 42 * 1024 ** 3,
  safe: true,                         // keep a restorable backup
  requiredControls: ['ssh-ciphers', 'key-auth', 'integrity-manifest'],
});

job.scope;        // 'unscoped' — same engine, environment management
job.cost;         // { totalCents, transferCents, safetyCents, … }

Strategies, by tier and risk

Every way to move data is a strategy, grouped the way it is used, and gated two
ways: the account tier, and the security risk it carries.

Group	Strategies	Tier
Standard encrypted	HTTPS multipart, WebDAV	pro
	SFTP, SCP/rsync over SSH, FTPS	premium
Accelerated	QUIC/HTTP-3	pro
	Globus/GridFTP, Aspera, Signiant, UDT	premium (high-risk)
Managed File Transfer	GoAnywhere/MOVEit/Axway/…	premium
EDI	PGP payload	pro
	AS2/AS4	premium
Network	VPN / private interconnect	premium
DB replication	native replication / CDC over TLS	premium
Offline	Snowball / Data Box / Transfer Appliance	premium (high-risk)

SSH and SFTP are the premium baseline prod-orm runs directly; the higher-risk
accelerated, MFT, EDI and offline strategies are premium and, where they
enlarge the surface, orchestrated around a customer- or vendor-supplied
endpoint rather than run by us. Migrate.strategiesForTier(tier) lists what an
account may use; Migrate.availability(id, tier) explains a refusal.

Migrate.strategiesForTier('pro').map((s) => s.id);
// [ 'https', 'webdav', 'quic', 'pgp' ]

Migrate.availability('aspera', 'pro');
// { allowed: false, reason: 'Aspera FASP (IBM) needs a premium account; this account is pro' }

Whatever the transport, the same checklist controls apply and are recorded on
each strategy: modern TLS or SSH ciphers, key/certificate auth (never
passwords), a SHA-256 integrity manifest, resumability, and retained transfer
logs. A job can require the controls its regime demands, and a strategy that
cannot satisfy them is refused at creation.

Scoped vs unscoped

Unscoped — a structured, homogeneous move: one database to another of the
same engine and shape. This is environment management (staging →
production). No mapping, and it needs no scope beyond the two endpoints.

Scoped — the destination has different requirements: a different engine,
a different kind of store, or different data categories. It needs a mapping,
and the caller must hold scope for what it reaches. classifyScope decides,
and says why.

Migrate.classifyScope(
  { kind: 'database', dialect: 'postgres', location: 'db://staging' },
  { kind: 'database', dialect: 'sqlite', location: 'file:./snapshot.db' },
);
// { scope: 'scoped', reasons: [ 'different engines: postgres → sqlite' ] }

Safe migrations

A safe job keeps a restorable backup before it runs, and if the transfer
fails it puts the source back from that backup — so a failed migration never
leaves the data half-moved. Safety costs more (the backup is held for a
retention window), which is part of the price.

Pricing

A job is priced by size, because the data is held ephemerally while it runs (in
case it has to be retried or restored): 5 MB genuinely costs less than 100 GB.
The price is a flat base, a per-gigabyte transfer charge on progressive
brackets (the marginal rate falls as the job grows), a safety surcharge for a
safe job's backup, and a multiplier for a commercial/accelerated strategy.
Migrate.estimateCost({ bytes, safe, strategyId }) quotes it before it runs.

Band	Rate
every job	$1.00 base
first 1 GB	50¢/GB
1–100 GB	20¢/GB
100 GB–1 TB	8¢/GB
beyond 1 TB	3¢/GB
safe job	+2¢/GB per day the backup is kept (7 days by default)
Aspera, Signiant ×2.5 · offline ×3 · UDT ×1.8 · Globus, MFT ×1.5	multiplies the transfer charge

import { Migrate } from 'ts-prorm-orm';

const GB = 1024 ** 3;
for (const [label, input] of [
  ['5 MB over SFTP', { bytes: 5 * 1024 ** 2, strategyId: 'sftp' }],
  ['100 GB over SFTP', { bytes: 100 * GB, strategyId: 'sftp' }],
  ['100 GB over SFTP, safe', { bytes: 100 * GB, strategyId: 'sftp', safe: true }],
  ['100 GB over Aspera', { bytes: 100 * GB, strategyId: 'aspera' }],
] as const) {
  const cost = Migrate.estimateCost(input);
  console.log(label.padEnd(24), Migrate.dollars(cost.totalCents));
}

5 MB over SFTP           $1.00
100 GB over SFTP         $21.30
100 GB over SFTP, safe   $35.30
100 GB over Aspera       $51.75

The safe job's extra

14.00 i s 100 GB \times 2¢ \times 7 d a yso f ba c k u p; A s p er a^{'} s i s i t s 2.5 \times m u lt i pl i ero n t h e

20.30 transfer charge.

Scheduled syncs, and running on your own device

A transfer can repeat — give it a cron schedule and it becomes a standing
sync rather than a one-off move:

A transfer can also run entirely on your own device. The cloud runner
cannot reach a database on your laptop, and it should not hold the credentials
for one — so a job with a local endpoint runs on the device, the credentials
never leave it, and only the outcome (a manifest digest and a signed
certificate) is reported back:

Create the job

Mark the endpoint local=true. The response carries the job's klock, once:

$ prorm transfer create -s db-replication \
    --source kind=database,dialect=postgres,location=db://localhost/shop,local=true \
    --dest   kind=database,dialect=postgres,location=db://prod
! Local job: keep its klock to run it on-device (shown once):
…
✓ Job tr_ab12cd — created, unscoped, projected $1.00
Start it: prorm transfer start tr_ab12cd

Confirm the price

$ prorm transfer start tr_ab12cd
Projected price: $1.00 (charged on completion)
Run this transfer and be charged $1.00? [y/N] y
✓ Job tr_ab12cd — queued

Run it here

Your local connection URLs are used on this machine and never sent anywhere:

$ prorm transfer run tr_ab12cd \
    --source-url postgres://app:pw@localhost/shop \
    --dest-url   "$DEST_URL" \
    --tables users,orders,invoices
  users: +1000 rows (1000 total)
  …
✓ Ran tr_ab12cd on-device — 3 table(s), … bytes. Certified server-side.

The credentials in --source-url and --dest-url appear only on the left of
that diagram.

This covers local → cloud (upload a database only you can reach), local →
local, and cloud → cloud. The Migrate library does the moving —
databaseSource/databaseSink copy a table at a time over the ORM, so
postgres→sqlite, sqlite→postgres and localhost→cloud all work the same — and the
job is certified server-side with its transfer klock, the same passwordless
credential the rest of prod-orm uses. A headless agent can run due scheduled
jobs on a device and report them the same way.

The transfer klock

Every completed transfer is certified. A transfer klock — a secret kept in
a .klock — signs a certificate over the manifest digest and the job's
metadata, so a holder of the same secret can prove that what was moved was what
was intended, and that it arrived intact. job.certificate carries it;
job.klock.verify(cert) checks it.

For a local job, the same secret signs the on-device agent's status reports
(Authorization: Klock <sig>), so the agent needs no session token. It is
returned once by prorm transfer create and kept in the agent config, not in a
.klock file. How it relates to the passwordless .klock is in
klock.

Billing

A transfer is billed, and the shape is one the customer agrees to before
anything runs:

A card on file is required. No card, no transfer.

The projected price is shown, and confirmed, before the job starts. A
price the holder did not agree to is never charged.

The job runs, and its actual price is computed from what was moved — which
can differ from the projection if the size did.

On completion the card is charged. Or, for a team that elected it, the
charge is a line on the team's end-of-term bill instead.

prorm transfer create --start and prorm transfer start show the projected
price and ask to confirm (--yes / -y to skip the prompt), and pass that
confirmed number to the API; the server refuses to start a job whose projected
price was not confirmed, or was confirmed at a stale number. A team sets its
mode with the API's PUT /teams/:id/transfer-billing (immediate or
deferred); a personal job is billed immediately.

[!IMPORTANT]
Without a terminal there is no one to confirm the price, so prorm transfer start tr_ab12cd (and transfer create --start) exits 2 with No terminal to confirm the projected price on. Re-run with --yes to accept it (e.g. in a script or CI). The job stays created, and nothing is charged. Pass
--yes when a pipeline is meant to start (and pay for) the transfer. Before
4.9.2 it printed Not started. and exited 0, so a pipeline did not notice.

Local jobs are priced at a flat rate at least

A local job runs on your device, and its bytes never pass through the
platform (a local database copy does not touch the klock relay either), so
the platform cannot measure what it moved. A local job is therefore priced at
least as if it moved 10 GiB (TRANSFER_LOCAL_FLAT_BYTES on the API),
whatever size you declare, report or leave out:

3.30 o v er SFTP, a g ain s t

1.50 for 1 GB. Above that floor it is billed the largest of the price you
confirmed at start, the price of the bytes your side reports, and the price
of the bytes the klock relay metered for the run.

The quote's and the job's cost carries basis: local-flat when the flat
volume set the price (with pricedBytes and flatBytes), or declared.
Job views also carry meteredBytes.

The on-device agent can send the relay's signed receipt with its events
(POST /transfer/jobs/:id/events). The API checks it against the verified
klock keyset and counts it once: 422 if it is invalid, not completed or
started too early, 403 if it is not for your job, 409 if it was already
counted, 503 if the API has no verified keyset.

A started local job that is reported failed or restored, or is cancelled, is
still billed its confirmed price, at least the flat rate.

Certificates say how the volume was established: platform (a cloud job),
relay-metered (relay receipts), or self-attested (your own report).
Treat self-attested as your claim, not a platform measurement.

[!NOTE]
This is new in the control plane on feat/klock-capabilities (API
6b18fcc, PRORM-SA-2026-026).
Unfinished local jobs priced below the floor are re-priced by migration 0032
and again when they start, settle or re-arm, so confirming the old price
answers 409 with "re-quote": quote the job again and confirm the new
price.

From the command line

prorm transfer drives the whole thing without writing code:

An endpoint is a compact key=value spec — kind (database, object-store,
sftp, https, file, stream) plus dialect, location, region and types
(data categories joined with +) as they apply. --bytes takes a human size
(20mb, 1.5gb, 2tb). --controls lists the controls the transport must
satisfy, and a strategy that cannot is refused at creation.

Running a job

The transport is an Executor the platform supplies — a serverless function
behind migrate.prod-orm.io, one per strategy. job.start(executor) walks the
status machine:

created → queued → [backing-up] → running → verifying →
completed

, or, for a safe job that fails, failed → restoring → restored.
job.toJSON() is the snapshot the status API returns, and onStatus is the
hook a caller is notified through. Illegal jumps are refused, so a job's status
is always a true account of where it is.

(backing_up is backing-up in the API; Mermaid state names cannot carry a hyphen.)

A complete job, on your own machine

The library runs the same status machine anywhere an Executor is available.
databaseCopyExecutor is the one a database-to-database job should use: it
copies each table in keyset-paged batches and builds the manifest from what was
written. This copies a table between two SQLite databases:

import { Migrate, Prorm } from 'ts-prorm-orm';

const source = new Prorm({ dialect: 'sqlite', storage: './local.db' });
const dest = new Prorm({ dialect: 'sqlite', storage: './copy.db' });
await source.connect();
await dest.connect();

const job = Migrate.TransferJob.create({
  strategyId: 'db-replication',
  tier: 'premium',
  source: { kind: 'database', dialect: 'sqlite', location: 'file:./local.db', local: true },
  dest: { kind: 'database', dialect: 'sqlite', location: 'file:./copy.db', local: true },
  bytes: 0, // unknown up front: verify against what was actually moved
  onStatus: (status) => console.log('status:', status),
});

await job.start(
  Migrate.databaseCopyExecutor(source, dest, ['users'], {
    batchSize: 2,
    onBatch: (b) => console.log('batch', b),
  }),
);

console.log(job.status, job.manifest?.digest);
console.log('certificate valid:', job.klock.verify(job.certificate!));

With three rows in users:

status: created
status: queued
status: running
batch { table: 'users', rows: 2, lastKey: 2, totalRows: 2 }
batch { table: 'users', rows: 1, lastKey: 3, totalRows: 3 }
status: verifying
status: completed
completed 117b1a88f3bc4b23032cf4275811a08faacbc53ff92e985e7a198efc15c3b6b8
certificate valid: true

batch.lastKey is the resume token: pass it back as resumeAfter and an
interrupted copy starts after the last row it wrote rather than from the top.

[!WARNING]
bytes is checked, not just displayed. A job created with bytes: 4096 whose
manifest totals 86 bytes goes verifying → failed with moved 86 bytes, expected 4096, and no certificate. Pass the real size, or 0 when you do not
know it, which means "as measured".

Migrate — the data transfer subsystem

Strategies, by tier and risk#

Scoped vs unscoped#

Safe migrations#

Pricing#

Scheduled syncs, and running on your own device#

Create the job#

Confirm the price#

Run it here#

The transfer klock#

Billing#

Local jobs are priced at a flat rate at least#

From the command line#

Running a job#

A complete job, on your own machine#