Home / Repstance User Guide / DATA COMPARISON

DATA COMPARISON

Data comparison checks that the Target Database holds the same data as the Source Database while the replication keeps running. It does not copy data to the Target Database: the Repstance server reads the rows of the Source Database and of the Target Database, calculates a hash of every row and compares the hashes. Only the keys and the hashes are saved. The rows that differ, or are missing in the Target Database, are reported and can be repaired with one command. The feature has been available since version 5.2.0170 and works in the Web UI, in repcli and in the REST API.

When to use it

After an initial load, to prove that the Target Database is complete. Compare with Data Validation % 100.
Regularly, to find the rows that the replication has missed or that somebody has changed in the Target Database. A comparison of a part of the rows takes less time, and an incremental comparison looks only at the rows that have changed since the previous run.
After an outage, a restart or a reset of a process, or a change of the replication rules.
Before a switch-over to the Target Database.

How it works in short

A comparison task is a list of tables with the settings for each of them. When the task runs, a compare process of the Repstance server reads the rows of the Source Database in the order of the key, in portions called chunks, and calculates a hash of every row. It saves the keys and the hashes of a chunk in a file and notes the chunk in the Source Database. The Capture Process carries this note in the stream of the replicated changes. When the Apply Process meets the note it reads the same rows from the Target Database, calculates their hashes in the same way and compares them. The report of the task shows, for every table, the rows that match, the rows that differ (failed rows) and the rows that are missing in the Target Database (missed rows). The details are in How Data Comparison Works; what a comparison needs and what it cannot compare is in Requirements and Limits.

Quick start

Create a task that compares 5 % of the rows of the tables of the schema dbo:

repcli compare add capture=1 taskname=TASK1 tables=dbo.% deep=5

Run it. A task that has not been run before starts from the beginning, a stopped task continues where it stopped (the other run modes are in Start/Stop Task):

repcli compare run capture=1 taskname=TASK1

Follow the progress and read the result:

repcli compare status capture=1 taskname=TASK1

The Capture Process and the Apply Process have to be running. The same steps in the Web UI are described in Compare Data.

One feature, three interfaces

The Web UI, repcli and the REST API do the same things under different names. The commands:

You want to	Web UI	repcli	REST API
Create or change a task	“+ Add”, “Edit”	compare add	/compare/add
Show the settings of a task	“Edit”	compare show	/compare/show
List the tasks	the tabs of the Compare window	compare list	/compare/list
Delete a task	“Delete” in the Edit window	compare remove	/compare/delete
Run a task	“Run”	compare run	/compare/run with run 1
Stop a task	“Stop”	compare stop	/compare/run with run 0
Read the report	the report of the task	compare status	/compare/status

The settings of a task:

Setting	Web UI	repcli	REST API
The name of the task	Taskname	taskname	name
The tables	the schema and the table with Any, Like, Contains and =	tables	tables
The share of the rows that is compared in every chunk, 10 by default	Data Validation %	deep	deep
The number of the rows that are read at a time, 10000 by default	Rows per Chunk	chunk	chunk
The columns that are not compared	“Exclude” on a column	excludecols	excludecols
The data types that are not compared	Exclude Types	excludetypes	excludetypes
A function for a column	“Add f(x)” on a column	mapcols	mapcols
A function for a data type	Map Types	maptypes	maptypes
The key of a table	“Add key”, “Remove key”	keys	keys
A condition for the rows	not in the Web UI	where	where
List the tables without saving the task	“Show results”	preview	preview
The details of the columns of the tables	the arrow in front of a table	not shown	showcols

The run modes (the parameter mode of compare run and the element mode of the REST body):

Run mode	Web UI	mode	What it does
Resume	“Resume”	0, the default	Continues a stopped task. A task that has not been run before starts from the beginning.
Full	“Full Validation”	1	Compares the tables from the start and replaces the previous result.
Incremental	“Incremental”	2	Compares only the rows that have changed since the previous run. It needs the trail files of the Capture Process, see Start/Stop Task.
Repair	“Repair”	3	Reloads the failed and the missed rows and compares them once more.

The tables that are ticked in the report of the Web UI are the parameter tables of compare run and the element tablelist of the REST body.

The pages of this section

Page	What you find there
How Data Comparison Works	The parts and the steps of a comparison, chunks and sampling, how a row is hashed, where the files and the tables are.
Requirements and Limits	The supported databases, what has to be running and prepared, keys, how the data types are compared, what is not reported, rows that change during a comparison.
Create Task	The parameters of the command that creates a task, with examples.
Start/Stop Task	The run modes, how to stop a task, the trail files of the incremental mode, the repair.
Show Task, List of the Tasks, Remove Task	The commands that show, list and delete tasks.
Task Status	The report of a task and the values of the status.
Compare Data, Task Configuration, Tasks Control	The same in the Web UI, step by step, with pictures.

Report Process

How Data Comparison Works