Storage Version 4: Object Storage and Incremental Reprocessing for MongoDB (Beta)

PowerSync Service v1.26.0 introduces storage version 4, a new version of the format the PowerSync Service uses to store synced data. It removes the main bottleneck when many clients sync at once or sync high volumes of data. If your instance uses a MongoDB source database and Sync Streams, you also get incremental reprocessing when you deploy Sync Streams changes, so far less data needs to re-sync to clients in most cases.

Storage version 4 is in beta and will become the default for all instances once the beta ends. We're rolling it out to all Free plan instances automatically, and instances on other plans can opt in ahead of general availability. The rest of this post covers the new features and what to expect.

S3 object storage

Your instance keeps the data it syncs to clients in its own storage database, alongside everything else it needs to run. With storage version 4, larger batches of that data move to S3 object storage instead, which is built for bulk, read-heavy data, and the Service syncs it to clients directly from there. This reduces the load on the storage database cluster:

• Faster initial sync, most noticeably for clients that sync large amounts of data and when many clients connect at the same time.

• A more stable instance overall, because the storage database no longer does the heavy lifting.

• PowerSync instances can scale to handle far more concurrent clients.

The new storage format also improves sync and reprocessing performance in general, independent of object storage.

Incremental reprocessing

If your source database is MongoDB and you use Sync Streams, you also get incremental reprocessing:

• When you update your sync config, only the streams you added or changed are reprocessed, instead of all streams.

• Clients only re-download the data for those affected streams. They no longer re-sync all their data after every Sync Streams deploy.

Some changes still trigger a full reprocess, such as defragmenting or changing your top-level sync config.

Incremental reprocessing isn't available yet for Postgres and other source databases, but is planned. See the original proposal for background.

Incremental reprocessing won't be supported for Sync Rules, so if you still use Sync Rules, now is a good time to migrate. The migration tool in the PowerSync Dashboard or CLI converts your Sync Rules to Sync Streams syntax with the same behavior: clients keep syncing everything upfront, and no client code changes are needed. See the migration guide.

Free plan instances are upgraded automatically

We're rolling out storage version 4, including S3 object storage, to all Free plan instances automatically as part of this beta. No action is needed on your end. Here's what to expect:

• The upgrade runs like a Sync Streams or Sync Rules deploy. We re-replicate your data in the background while your current version keeps serving clients, then switch over. No downtime.

• After the switch, clients do a one-time full re-sync, as after a deploy.

• You see a new deploy event in the PowerSync Dashboard.

• This also enables incremental reprocessing if your source database is MongoDB and you use Sync Streams.

Opting in on other plans

Opt in by setting config.storage_version: 4 in your sync config. This triggers reprocessing, and your data then uses storage version 4. It behaves like the Free plan upgrade described above, so expect one full re-sync for clients and a new deploy event.

You get the general performance improvements immediately, plus incremental reprocessing if your source database is MongoDB and you use Sync Streams.

S3 object storage is a separate step:

• PowerSync Cloud: we enable it per instance during the beta. Contact us if you'd like it on your instance ahead of the wider rollout.

• Self-hosted: you configure your own bucket storage, so you can point your instance at S3 yourself. See the self-hosted S3 setup docs.

Questions and help

Reach us on Discord with any questions, or if you notice any issues after a migration.