All products

Database tooling

PGStream

Move large MySQL databases to PostgreSQL with control.

My role
Creator & engineer
Status
Open source
Platform
CLI · web UI · self-hosted
Source
GPL-3.0 license

The idea behind it.

PGStream migrates large MySQL databases to PostgreSQL in bounded memory. It loads data through the COPY protocol from consistent snapshots, resumes from per-batch checkpoints, decides tricky type conversions by scanning the actual data, and plans every migration with a dry run that inspects data quality — not just schema. A local server with an embedded web UI streams live progress. In one documented production-backup run, five workers moved 8.5 million rows across 83 tables in about ten minutes; timing depends on the data, network, and configuration.

System overviewA consistent MySQL snapshot moves through a planned, resumable migration to PostgreSQL.
MySQL
  • Consistent snapshot
Migration
  • Dry-run planning
  • COPY loading
  • Per-batch checkpoints
PostgreSQL
  • Target database

What it supports.

  • COPY-protocol bulk loading with idempotent resume from per-batch checkpoints
  • Consistent snapshots, including lock-free alignment for parallel table workers
  • Dry-run planning that scans data quality: zero dates, NUL bytes, invalid enums, orphaned foreign keys
  • Data-driven type decisions for UUID and unsigned-bigint columns, kept consistent across foreign keys
  • pgloader-inspired cast rules, schema-only/data-only phases, and comment migration
  • Fail-closed conversions with actionable errors and a manual-work DDL file
  • Live progress in a web UI via server-sent events, resumable from CLI or browser

Engineering decisions.

Hardened by real data

PGStream was tested against a real 83-table production backup, and every failure became a feature: zero dates, invalid enum markers, UUID-shaped keys that are not UUIDs, orphaned foreign-key rows, NUL bytes, and unsigned bigint keys are all detected in the data and handled with explicit, lossless-first rules.

Fail closed, hand over the fix

Unsupported conversions stop the run rather than silently changing the schema, and every stop ships its own remediation — the query to find the bad rows, the exact ALTER to reconcile a drifted target, or the cast flag that opts into a different mapping. The dry run surfaces all of it before a single write.

Measured, not assumed

Every completed table logs a time breakdown showing whether source reads, target writes, or transformation dominated. Wire compression and read/write pipelining took a two-million-row table from 23 minutes to under 10 over the same link, five parallel workers now move the complete 8.5-million-row database end to end in about the same time, and the published benchmarks include the configurations that did not help.

From the build 5 min read

What a production database taught pgstream

Notes from hardening a MySQL-to-PostgreSQL migration tool against a real 83-table production backup — the edge cases, the philosophy, and the performance lessons.

Read the essay

Keep exploring

All products

Cloud tooling

s3ase

A calmer control plane for S3-compatible storage.

Say hello Back to contact