Skip to content

feat: background file imports and SaaS connections - #19

Merged
1pxone merged 1 commit into
mainfrom
feat/imports-and-connections
Nov 12, 2026
Merged

1pxone merged 1 commit into
mainfrom
feat/imports-and-connections

Conversation

@1pxone

@1pxone 1pxone commented Nov 11, 2026

Copy link
Copy Markdown
Member

Client support for the platform's background file imports and SaaS connections (rush-db/platform#36). Both are RushDB Cloud only: in direct mode they throw CloudOnlyError before any request is sent.

Before publishing

  • Platform first. rush-db/platform#36 must be merged and deployed to RushDB Cloud: these methods call /api/v1/imports, /connections and /connectors/specs.
  • CI on this PR. Typecheck, tests and build should pass.
  • Try against Cloud once deployed.
    • importFiles with a CSV and a gzipped NDJSON file.
    • An S3 multipart upload of a file larger than one part.
    • connections.create → test → sync → wait on a Stripe test key.
  • After release: bump the SDK range in rush-db/mcp-server, whose connection and import tools need db.connections and db.imports.

What

  • db.imports:
    • create, addFiles, removeFile;
    • upload: an S3 multipart upload that sends parts to presigned URLs in parallel (default 4) with retries, or a direct upload of the bytes through the API;
    • preview: column types, suggested key and links, and personal-data findings with include | skip | hash suggestions;
    • start, get, list, wait, cancel, retry, delete;
    • importFiles, which does all of it in one call with mappings, links between files and progress.
    • Upload data can be a Blob/File, bytes, a web ReadableStream or a Node Readable.
  • db.connections:
    • catalog and pushTemplates (Zapier, Make, n8n);
    • list, get, create, update, delete({ deleteData });
    • test, preview, sync, runs, pause, resume, rotateWebhookToken;
    • field review: fields profiles a stream with personal-data findings, and reviewFields saves the choices as an allowlist;
    • agent connections (connectorId: 'agent') return an agentToken once.
  • Exports: the DTO types are exported, and both namespaces are listed in CLOUD_ONLY_METHODS.
  • HTTP clients: both clients accept a raw request body for uploads.
  • Docs: the README has a section on file imports and connections, and docs-reference is regenerated.
  • Changeset: minor (imports-and-connections).

Testing

  • pnpm types:check and pnpm build pass.
  • pnpm test: 140 passing, 16 skipped (the live-engine suites). That includes imports.spec.ts (13 tests, against a mocked API and S3) and connections.spec.ts (4), and direct-engine.spec.ts checks that every new method refuses direct mode.

Notes

db.imports creates an import, uploads its files (S3 multipart parts in
parallel with retries, or through the API), previews the mapping with
personal-data findings, starts and follows the run; importFiles does it
in one call. db.connections lists the connector catalog and push
templates and manages connections: create, update, delete, test,
preview, sync, runs, pause, resume, field review and webhook token
rotation. Both are RushDB Cloud only and listed in CLOUD_ONLY_METHODS.
The HTTP clients accept a raw body for uploads.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant