Sample script to sapwn pgedge on microVMs
pgEdge Multi-Master Setup on fcctl (devbox:1 / Ubuntu 24.04)
1. Infrastructure
# Create private network
fcctl net create pgedge-net
# Spin up 2x 4GB VMs on the same network
fcctl create --shape s-2vcpu-4gb --rootfs devbox:1 --name pgedge-n1 --network pgedge-net --disk-mib 30720
fcctl create --shape s-2vcpu-4gb --rootfs devbox:1 --name pgedge-n2 --network pgedge-net --disk-mib 30720
2. OS prep (on each VM, as root)
# Create non-root user (pgedge CLI refuses to run as root)
useradd -m -s /bin/bash pgedge
echo "pgedge ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
# Fix Python access: devbox:1 uses asdf, new users can't resolve python3 via shim
chmod 755 /root && chmod -R o+rX /root/.asdf
ln -sf /root/.asdf/installs/python/3.12.7/bin/python3.12 /usr/local/bin/python3
# Give pgedge user asdf context so shims work
cat > /home/pgedge/.bash_profile <<'EOF'
export ASDF_DIR=/root/.asdf
export ASDF_DATA_DIR=/root/.asdf
export PATH=/root/.asdf/shims:/usr/local/bin:/usr/bin:/bin
EOF
echo "python 3.12.7" > /home/pgedge/.tool-versions
chown pgedge:pgedge /home/pgedge/.bash_profile /home/pgedge/.tool-versions
3. Install pgEdge (as pgedge user)
su - pgedge -c 'python3 -c "$(curl -fsSL https://pgedge-download.s3.amazonaws.com/REPO/install.py)"'
# Install missing Python deps (ctlibs doesn't support Ubuntu 24.04)
su - pgedge -c 'pip3 install -q "psycopg[binary]" fire'
4. Setup PostgreSQL 16 + Spock (as pgedge user)
# NOTE: -U cannot be 'pgedge' (reserved), use something like 'admin'
su - pgedge -c 'cd $HOME/pgedge && ./pgedge setup -U admin -P Passw0rd -d demo --pg_ver 16 --port 5432'
This installs pg16, snowflake extension, spock 5.0.8, sets wal_level=logical, and runs CREATE EXTENSION spock.
5. Open PostgreSQL for remote access
PGDATA=/home/pgedge/pgedge/data/pg16
sed -i "s/^#*listen_addresses.*/listen_addresses = '*'/" $PGDATA/postgresql.conf
cat >> $PGDATA/pg_hba.conf <<'EOF'
host all admin 0.0.0.0/0 scram-sha-256
host replication admin 0.0.0.0/0 scram-sha-256
EOF
su - pgedge -c 'cd $HOME/pgedge && ./pgedge restart pg16'
6. Grant REPLICATION to admin
# pgedge CLI checks for REPLICATION privilege; admin doesn't have it by default
PSQL=/home/pgedge/pgedge/pg16/bin/psql
su - pgedge -c "$PSQL -p 5432 -d demo -c 'ALTER ROLE admin REPLICATION'"
7. Create Spock nodes (bypass pgedge CLI, use psql directly)
PSQL=/home/pgedge/pgedge/pg16/bin/psql
# On n1:
su - pgedge -c "$PSQL -p 5432 -d demo -c \"SELECT spock.node_create('n1', 'host=pgedge-n1.pgedge-net.fc.local port=5432 dbname=demo user=admin password=Passw0rd')\""
# On n2:
su - pgedge -c "$PSQL -p 5432 -d demo -c \"SELECT spock.node_create('n2', 'host=pgedge-n2.pgedge-net.fc.local port=5432 dbname=demo user=admin password=Passw0rd')\""
8. Create bidirectional subscriptions
# On n1 — subscribe to n2:
su - pgedge -c "$PSQL -p 5432 -d demo -c \"SELECT spock.sub_create('sub_n2', 'host=pgedge-n2.pgedge-net.fc.local port=5432 dbname=demo user=admin password=Passw0rd')\""
# On n2 — subscribe to n1:
su - pgedge -c "$PSQL -p 5432 -d demo -c \"SELECT spock.sub_create('sub_n1', 'host=pgedge-n1.pgedge-net.fc.local port=5432 dbname=demo user=admin password=Passw0rd')\""
9. Test replication
# On both nodes, create table and add to replication set:
$PSQL -p 5432 -d demo -c 'CREATE TABLE test_mm (id BIGINT PRIMARY KEY, msg TEXT, node TEXT)'
$PSQL -p 5432 -d demo -c "SELECT spock.repset_add_table('default', 'test_mm')"
# Write to n1, write to n2, read from both — rows appear everywhere within ~1s
---
Key gotchas on devbox:1:
- pgEdge refuses to run as root → need a non-root user
- ctlibs doesn't support Ubuntu 24.04 → manually pip install psycopg[binary] fire
- asdf shims break for new users → fix symlinks + set ASDF_DATA_DIR
- admin needs REPLICATION granted manually
- pgedge CLI spock node-create rejects admin even after grant → use psql directly
❯ how can i stop replication temporary n pgedge so i can do migration
⏺ In Spock there are two levels you can pause at:
Option 1: Disable a subscription (cleanest for migration)
-- On the node you want to isolate (e.g. n2 stops receiving from n1)
SELECT spock.sub_disable('sub_n1');
-- Do your migration work...
ALTER TABLE ..., INSERT ..., etc.
-- Re-enable when done
SELECT spock.sub_enable('sub_n1');
To fully isolate a node (stop both directions):
-- On n1: stop receiving from n2
SELECT spock.sub_disable('sub_n2');
-- On n2: stop receiving from n1
SELECT spock.sub_disable('sub_n1');
Spock buffers WAL while the subscription is disabled, so changes made during the migration will catch up when you re-enable. Just make sure you don't let it buffer too long
or you'll run out of replication slots space.
---
Option 2: Pause replication worker (lighter touch)
-- Suspend the apply worker without dropping the subscription
SELECT spock.sub_alter_interface('sub_n1', 'paused'); -- if your spock version supports it
-- Or use pg's built-in to pause the slot
SELECT pg_replication_slot_advance('spk_demo_n1_sub_n1', pg_current_wal_lsn());
---
Option 3: For schema migrations specifically — use spock.replicate_ddl
If the migration is DDL (schema changes), Spock doesn't replicate DDL automatically. You need to run it on both nodes anyway:
-- Run on both n1 and n2 manually:
ALTER TABLE your_table ADD COLUMN new_col TEXT;
Or use pgEdge's DDL replication wrapper if available:
SELECT spock.replicate_ddl('ALTER TABLE your_table ADD COLUMN new_col TEXT', ARRAY['default']);
---
Recommended flow for a migration
-- 1. Disable subscriptions on both sides
-- On n1:
SELECT spock.sub_disable('sub_n2');
-- On n2:
SELECT spock.sub_disable('sub_n1');
-- 2. Run your migration on both nodes (DDL must be applied to both)
ALTER TABLE ...;
-- data migrations, index builds, etc.
-- 3. Re-enable
-- On n1:
SELECT spock.sub_enable('sub_n2');
-- On n2:
SELECT spock.sub_enable('sub_n1');
-- 4. Verify subscription caught up
SELECT sub_name, sub_enabled FROM spock.subscription;
SELECT * FROM spock.sub_show_status(); -- shows lag/state