Mujeeb Muhideen
All products

Database tooling · Open source

PGStream

Move large MySQL databases to PostgreSQL with control.

View source (opens in a new tab)
Status
Open source
Platform
CLI · web UI · self-hosted
Role
Creator & engineer
Source
GPL-3.0 license

Product preview

Swipe to explore

What it does

Focused on the work that matters.

  • COPY-protocol bulk loading with idempotent resume from per-batch checkpoints
  • Consistent snapshots, including lock-free alignment for parallel table workers
  • Dry-run planning that scans data quality: zero dates, NUL bytes, invalid enums, orphaned foreign keys
  • Data-driven type decisions for UUID and unsigned-bigint columns, kept consistent across foreign keys
  • pgloader-inspired cast rules, schema-only/data-only phases, and comment migration
  • Fail-closed conversions with actionable errors and a manual-work DDL file
  • Live progress in a web UI via server-sent events, resumable from CLI or browser
Built with

A stack chosen for the product.

GoMySQLPostgreSQLSQLReactTypeScript

About

PGStream migrates large MySQL databases to PostgreSQL in bounded memory. It loads data through the COPY protocol from consistent snapshots, resumes from per-batch checkpoints, decides tricky type conversions by scanning the actual data, and plans every migration with a dry run that inspects data quality — not just schema. A local server with an embedded web UI streams live progress over any run, and the whole tool was hardened against a real 83-table production database, which it migrates end to end — 8.5 million rows — in about ten minutes.

Hardened by real data

PGStream was tested against a real 83-table production backup, and every failure became a feature: zero dates, invalid enum markers, UUID-shaped keys that are not UUIDs, orphaned foreign-key rows, NUL bytes, and unsigned bigint keys are all detected in the data and handled with explicit, lossless-first rules.

Fail closed, hand over the fix

Unsupported conversions stop the run rather than silently changing the schema, and every stop ships its own remediation — the query to find the bad rows, the exact ALTER to reconcile a drifted target, or the cast flag that opts into a different mapping. The dry run surfaces all of it before a single write.

Measured, not assumed

Every completed table logs a time breakdown showing whether source reads, target writes, or transformation dominated. Wire compression and read/write pipelining took a two-million-row table from 23 minutes to under 10 over the same link, five parallel workers now move the complete 8.5-million-row database end to end in about the same time, and the published benchmarks include the configurations that did not help.

Next products3ase