MOLT Fetch Commands and Flags

On this page Carat arrow pointing down

Commands

Command Usage
fetch Start the fetch task. This loads data from a source database to a target CockroachDB database.

Subcommands

Command Usage
tokens list List active continuation tokens.

Flags

Global flags

Flag Description
--source (Required) Connection string used to connect to the Oracle PDB (in a CDB/PDB architecture) or to a standalone database (non‑CDB). For details, refer to Source and target databases.
--source-cdb Connection string for the Oracle container database (CDB) when using a multitenant (CDB/PDB) architecture. Omit this flag on a non‑multitenant Oracle database. For details, refer to Source and target databases.
--target (Required) Connection string for the target database. For details, refer to Source and target databases.
--allow-tls-mode-disable Allow insecure connections to databases. Secure SSL/TLS connections should be used by default. This should be enabled only if secure SSL/TLS connections to the source or target database are not possible.
--assume-role Service account to use for assume role authentication. --use-implicit-auth must be included. For example, --assume-role='user-test@cluster-ephemeral.iam.gserviceaccount.com' --use-implicit-auth. For details, refer to Cloud Storage Authentication.
--bucket-path The path within the cloud storage bucket where intermediate files are written (e.g., 's3://bucket/path' or 'gs://bucket/path'). Only the URL path is used; query parameters (e.g., credentials) are ignored. To pass in query parameters, use the appropriate flags: --assume-role, --import-region, --use-implicit-auth.
--case-sensitive Toggle case sensitivity when comparing table and column names on the source and target. To disable case sensitivity, set --case-sensitive=false. If = is not included (e.g., --case-sensitive false), the flag is interpreted as --case-sensitive (i.e., --case-sensitive=true).

Default: false
--cleanup Whether to delete intermediate files after moving data using cloud or local storage. Note: Cleanup does not occur on continuation.
--compression Compression method for data when using IMPORT INTO (gzip/none).

Default: gzip
--continuation-file-name Restart fetch at the specified filename if the task encounters an error. --fetch-id must be specified. For details, see Fetch continuation.
--continuation-token Restart fetch at a specific table, using the specified continuation token, if the task encounters an error. --fetch-id must be specified. For details, see Fetch continuation.
--crdb-pts-duration The duration for which each timestamp used in data export from a CockroachDB source is protected from garbage collection. This ensures that the data snapshot remains consistent. For example, if set to 24h, each timestamp is protected for 24 hours from the initiation of the export job. This duration is extended at regular intervals specified in --crdb-pts-refresh-interval.

Default: 24h0m0s
--crdb-pts-refresh-interval The frequency at which the protected timestamp's validity is extended. This interval maintains protection of the data snapshot until data export from a CockroachDB source is completed. For example, if set to 10m, the protected timestamp's expiration will be extended by the duration specified in --crdb-pts-duration (e.g., 24h) every 10 minutes while export is not complete.

Default: 10m0s
--direct-copy Enables direct copy, which copies data directly from source to target without using an intermediate store.
--export-concurrency Number of shards to export at a time per table, each on a dedicated thread. This controls how many shards are created for each individual table during the data export phase and is distinct from --table-concurrency, which controls how many tables are processed simultaneously. The total number of concurrent threads is the product of --export-concurrency and --table-concurrency. Tables can be sharded with a range-based or stats-based mechanism. For details, refer to Table sharding.

Default: 4
--export-retry-max-attempts Maximum number of retry attempts for source export queries when connection failures occur. Only supported for PostgreSQL and CockroachDB sources.

Default: 3
--export-retry-max-duration Maximum total duration for retrying source export queries. If 0, no time limit is enforced. Only supported for PostgreSQL and CockroachDB sources.

Default: 5m0s
--filter-path Path to a JSON file defining row-level filters for the data import phase. Refer to Selective data movement.
--fetch-id Restart fetch task corresponding to the specified ID. If --continuation-file-name or --continuation-token are not specified, fetch restarts for all failed tables.
--flush-rows Number of rows before the source data is flushed to intermediate files. Note: If --flush-size is also specified, the fetch behavior is based on the flag whose criterion is met first.
--flush-size Size (in bytes) before the source data is flushed to intermediate files. Note: If --flush-rows is also specified, the fetch behavior is based on the flag whose criterion is met first.
--ignore-replication-check Skip querying for replication checkpoints such as pg_current_wal_insert_lsn() on PostgreSQL, gtid_executed on MySQL, and CURRENT_SCN on Oracle. This option is intended for use during bulk load migrations or when doing a one-time data export from a read replica.
--import-batch-size The number of files to be imported at a time to the target database during the data import phase. This applies only when using IMPORT INTO for data movement. Note: Increasing this value can improve the performance of full-scan queries on the target database shortly after fetch completes, but very high values are not recommended. If any individual file in the import batch fails, you must retry the entire batch.

Default: 1000
--import-region The region of the cloud storage bucket. This applies only to Amazon S3 buckets. Set this flag only if you need to specify an AWS_REGION explicitly when using IMPORT INTO for data movement. For example, --import-region=ap-south-1.
--local-path The path within the local file server where intermediate files are written (e.g., data/migration/cockroach). --local-path-listen-addr must be specified.
--local-path-crdb-access-addr Address of a local file server that is publicly accessible. This flag is only necessary if CockroachDB cannot reach the local address specified with --local-path-listen-addr (e.g., when moving data to a CockroachDB Cloud deployment). --local-path and --local-path-listen-addr must be specified.

Default: Value of --local-path-listen-addr.
--local-path-listen-addr Write intermediate files to a local file server at the specified address (e.g., 'localhost:3000'). --local-path must be specified.
--log-file Write messages to the specified log filename. If no filename is provided, messages write to fetch-{datetime}.log. If "stdout" is provided, messages write to stdout.
--logging Level at which to log messages (trace/debug/info/warn/error/fatal/panic).

Default: info
--metrics-listen-addr Address of the Prometheus metrics endpoint, which has the path {address}/metrics. For details on important metrics to monitor, refer to Monitoring.

Default: '127.0.0.1:3030'
--mode Configure the MOLT Fetch behavior: data-load, export-only, or import-only. For details, refer to Fetch mode.

Default: data-load
--non-interactive Run the fetch task without interactive prompts. This is recommended only when running molt fetch in an automated process (i.e., a job or continuous integration).
--pprof-listen-addr Address of the pprof endpoint.

Default: '127.0.0.1:3031'
--row-batch-size Number of rows per shard to export at a time. For details on sharding, refer to Table sharding. See also Best practices.

Default: 100000
--schema-filter Move schemas that match a specified regular expression.

Default: '.*'
--skip-pk-check Skip primary-key matching to allow data load when source or target tables have missing or mismatched primary keys. Disables sharding and bypasses --export-concurrency and --row-batch-size settings. Refer to Skip primary key matching.

Default: false
--table-concurrency Number of tables to export at a time. The number of concurrent threads is the product of --export-concurrency and --table-concurrency.

Default: 4
--table-exclusion-filter Exclude tables that match a specified POSIX regular expression.

This value cannot be set to '.*', which would cause every table to be excluded.

Default: Empty string
--table-filter Move tables that match a specified POSIX regular expression.

Default: '.*'
--table-handling How tables are initialized on the target database (none/drop-on-target-and-recreate/truncate-if-exists). For details, see Target table handling.

Default: none
--transformations-file Path to a JSON file that defines transformations to be performed on the target schema during the fetch task. Refer to Transformations.
--type-map-file Path to a JSON file that contains explicit type mappings for automatic schema creation, when enabled with --table-handling drop-on-target-and-recreate. For details on the JSON format and valid type mappings, see type mapping.
--use-console-writer Use the console writer, which has cleaner log output but introduces more latency.

Default: false (log as structured JSON)
--use-copy Use COPY FROM to move data. This makes tables queryable during data load, but is slower than using IMPORT INTO. For details, refer to Data movement.
--use-implicit-auth Use implicit authentication for cloud storage URIs.
--use-stats-based-sharding Enable statistics-based sharding for PostgreSQL sources. This allows sharding of tables with primary keys of any data type and can create more evenly distributed shards compared to the default numerical range sharding. Requires PostgreSQL 11+ and access to pg_stats. For details, refer to Table sharding.

tokens list flags

Flag Description
--conn-string (Required) Connection string for the target database. For details, see List active continuation tokens.
-n, --num-results Number of results to return.

See also

×