How to duplicate schemas in PostgreSQL
Asked Answered
S

8

39

I have a database with schema public and schema_A. I need to create a new schema schema_b with the same structure than schema_a. I found the function below, the problem is that it does not copy the foreign key constraints.

CREATE OR REPLACE FUNCTION clone_schema(source_schema text, dest_schema text)
  RETURNS void AS
$BODY$
DECLARE
  object text;
  buffer text;
  default_ text;
  column_ text;
BEGIN
  EXECUTE 'CREATE SCHEMA ' || dest_schema ;

  -- TODO: Find a way to make this sequence's owner is the correct table.
  FOR object IN
    SELECT sequence_name::text FROM information_schema.SEQUENCES WHERE sequence_schema = source_schema
  LOOP
    EXECUTE 'CREATE SEQUENCE ' || dest_schema || '.' || object;
  END LOOP;

  FOR object IN
    SELECT table_name::text FROM information_schema.TABLES WHERE table_schema = source_schema
  LOOP
    buffer := dest_schema || '.' || object;
    EXECUTE 'CREATE TABLE ' || buffer || ' (LIKE ' || source_schema || '.' || object || ' INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING DEFAULTS)';

    FOR column_, default_ IN
      SELECT column_name::text, REPLACE(column_default::text, source_schema, dest_schema) FROM information_schema.COLUMNS WHERE table_schema = dest_schema AND table_name = object AND column_default LIKE 'nextval(%' || source_schema || '%::regclass)'
    LOOP
      EXECUTE 'ALTER TABLE ' || buffer || ' ALTER COLUMN ' || column_ || ' SET DEFAULT ' || default_;
    END LOOP;
  END LOOP;

END;
$BODY$  LANGUAGE plpgsql

How can I clone/copy schema_A with the foreign key constraints?

Sleepyhead answered 13/8, 2013 at 18:25 Comment(1)
If you want to clone the schema with SQL query, check out this answer.Longspur
O
54

You can probably do it from the command line without using files:

pg_dump -U user --schema='fromschema' database | sed 's/fromschmea/toschema/g' | psql -U user -d database

Note that this searches and replaces all occurrences of the string that is your schema name, so it may affect your data.

Ohara answered 15/1, 2014 at 17:26 Comment(2)
How can I avoid to affect the data?Superheat
@Superheat You can dump to a file and then use grep -E "^[[:digit:]]+.*fromschema.*" dumpfile | wc -l to count the occurrences of the schema name in the data rows. If the result is greater than 0, you have data that contains the 'fromschema' string.Knar
P
21

I would use pg_dump to dump the schema without data:

-s
--schema-only

Dump only the object definitions (schema), not data.

This option is the inverse of --data-only. It is similar to, but for historical reasons not identical to, specifying --section=pre-data --section=post-data.

(Do not confuse this with the --schema option, which uses the word "schema" in a different meaning.)

To exclude table data for only a subset of tables in the database, see --exclude-table-data.

pg_dump $DB -p $PORT -n $SCHEMA -s -f filename.pgsql

Then rename the schema in the dump (search & replace) and restore it with psql.

psql $DB -f filename.pgsql

Foreign key constraints referencing tables in other schemas are copied to point to the same schema.
References to tables within the same schema point to the respective tables within the copied schema.

Privatdocent answered 13/8, 2013 at 19:22 Comment(1)
Yes, thank you! but I wanted to avoid working with files. I'm looking for a quick way to do it.Sleepyhead
D
10

I will share a solution for my problem which was the same with a small addition. I needed to clone a schema, create a new database user and assign ownership of all objects in the new schema to that user.

For the following example let's assume that the reference schema is called ref_schema and the target schema new_schema. The reference schema and all the objects within are owned by a user called ref_user.

1. dump the reference schema with pg_dump:

pg_dump -n ref_schema -f dump.sql database_name

2. create a new database user with the name new_user:

CREATE USER new_user

3. rename the schema ref_schema to new_schema:

ALTER SCHEMA ref_schema RENAME TO new_schema

4. change ownership of all objects in the renamed schema to the new user

REASSIGN OWNED BY ref_user TO new_user

5. restore the original reference schema from the dump

psql -f dump.sql database_name

I hope someone finds this helpful.

Dipterous answered 13/9, 2018 at 18:19 Comment(1)
i need to copy without data. What is the solution. Need only structuresOldest
F
6

A bit late to the party but, some sql here could help you along your way:

get schema oid:

namespace_id = SELECT oid 
                  FROM pg_namespace 
                 WHERE nspname = '<schema name>';

get table's oid:

table_id = SELECT relfilenode 
                FROM pg_class 
               WHERE relnamespace = '<namespace_id>' AND relname = '<table_name>'

get foreign key constraints:

SELECT con.conname, pg_catalog.pg_get_constraintdef(con.oid) AS condef 
  FROM pg_catalog.pg_constraint AS con 
  JOIN pg_class AS cl ON cl.relnamespace = con.connamespace AND cl.relfilenode = con.conrelid 
 WHERE con.conrelid = '<table_relid>'::pg_catalog.oid AND con.contype = 'f';

A good resource for PostgreSQL system tables can be found here. Additionally, you can learn more about the internal queries pg_dump makes to gather dump information by viewing it's source code.

Probably the easiest way to see how pg_dump gathers all your data would be to use strace on it, like so:

$ strace -f -e sendto -s8192 -o pg_dump.trace pg_dump -s -n <schema>
$ grep -oP '(SET|SELECT)\s.+(?=\\0)' pg_dump.trace

You'll still have to sort through the morass of statements but, it should help you piece together a cloning tool programmatically and avoid having to drop to a shell to invoke pg_dump.

Frey answered 14/1, 2015 at 1:35 Comment(1)
Here's a python class I just wrote up to perform a schema clone (including data copy) using all SQL and no shelling out to pg_dump: https://gist.github.com/rabbitt/97f2c048d9e38c16ce62Frey
M
2
CREATE DATABASE [Database to create]
WITH TEMPLATE [Database to copy]
OWNER [Your username];
Myrlmyrle answered 27/10, 2023 at 7:40 Comment(1)
Thank you for your interest in contributing to the Stack Overflow community. This question already has quite a few answers—including one that has been extensively validated by the community. Are you certain your approach hasn’t been given previously? If so, it would be useful to explain how your approach is different, under what circumstances your approach might be preferred, and/or why you think the previous answers aren’t sufficient. Can you kindly edit your answer to offer an explanation?Guarded
R
0

Just ran into same. Sometimes I am missing remap_schema :)
The problem - neither from above addresses the Fc - standard format which is crucial for large schemas.
So I came up with something which uses it :
Pseudo code below - should work.
Requires rename of source for duration of pg_dump which, of course, might not be an option :(
Source :

pg_dump --pre-data in sql format
psql rename sosurce to target
pg_dump -Fc --data-only
psql rename back
pg_dump --post-data in sql format

Target :

sed source_schema->target_schema pre-data sql |psql
pg_restore Fc dump
sed source_schema->target_schema post-data sql |psql

sed above usually will include any other manipulations ( say different user names between source and target ) But it will be way much faster as data will not be part of the file

Retinoscopy answered 13/2, 2019 at 8:41 Comment(0)
L
0

You could modify pg_dump not to emit schema names. To build a Docker image with these changes, create a Dockerfile with the following content:

FROM alpine:latest

WORKDIR /usr/src/postgresql

RUN apk add --no-cache \
    alpine-sdk \
    perl \
    readline-dev \
    zlib-dev \
    bison \
    flex \
    git \
    coreutils \
    linux-headers \
    util-linux-dev

RUN git clone \
    --depth 1 \
    --branch feat/schema-less-pg-dump-11 \
    https://github.com/tindzk/postgres.git .

RUN ./configure && make

RUN cp src/bin/pg_dump/pg_dump /usr/bin/ && \
    cp ./src/interfaces/libpq/libpq.so* /usr/lib/

RUN apk del alpine-sdk \
    && rm -rf /var/cache/apk/* /usr/src/postgresql

CMD ["pg_dump"]

Then, run:

docker build -t pgdump-11 .

Now you can copy a database from the public schema into another schema in two steps:

docker run \
  -e PGPASSWORD=$db_password \
  -v "$(pwd)":/data/ \
  pgdump-11 \
  pg_dump \
  --verbose \
  --schema default \
  "postgres://$db_user@$db_host:$db_port/$db_name" \
  --file /data/dump.sql

(echo "drop schema $db_target_schema cascade; create schema $db_target_schema; set search_path = '$db_target_schema';"; cat dump.sql) | \
  psql "postgres://$db_user@$db_host:$b_port/$db_name"

The benefit of this approach is that the source schema does not need to be renamed which would otherwise require downtime.

Lindesnes answered 12/12, 2023 at 8:48 Comment(0)
C
0

Just wanted to cross-link "fast and simplest" solution from dba.stackexchange - https://dba.stackexchange.com/a/10475 which works if you have ownership on source schema and may allow some downtime for it.

In general need to make a dump or source schema, rename schema to destination name and next restore dump of the source schema (which will create source). This flow excludes renames in a dump file which is either long manual work or dangerous automatic "find and replace" action.

Chequer answered 8/5 at 5:48 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.