I’ve been playing around with the built-in ! operator in find mode, which finds all records with non-unique values in a field.

If I create a calculation field to concatenate multiple fields, I can use the ! operator to find non-unique values in multiple fields.

And since my databases (and I suspect those of many of you) are so complex, I don’t want to let a script automatically delete first, last or any other duplicate value. I need to make that decision myself.

So my deduplicate solution is to use the ! operator, sort the results, then click a check box field to identify records I want to delete. This works great for me.

Later!

Following script using a tecnique introdced to me by Agnès Barouh of Tic Tac:

[code]Sort Records [ Speci?ed Sort Order: Contacts::FistName; ascending
Contacts::LastName; ascending
Contacts::ZipCode; ascending ]
[ Restore; No dialog ]
Replace Field Contents [ Contacts::gStrainedList; Replace with calculation: Case(
Get( RecordNumber ) ? 1 and Contacts::FistName = $t and
Contacts::LastName = $u and Contacts::ZipCode = $w;
Let( $theIDs = $theIDs & ¶ & Contacts::RecordID; “” );
Let(
[
$t = Contacts::FistName;
$u = Contacts::LastName;
$w = Contacts::ZipCode
];
“”
)
) ]
[ No dialog ]
Set Field [ Contacts::gStrainedList; $theIDs ]
Go to Related Record [ From table: “Contacts 2”; Using layout: “Contacts” (Contacts) ] [ Show only related records ]
Delete All Records [ No dialog ]
[/code]

–sd

So if you were looking at the first two examples, the fields you would check for are Products_dup::ID or Customers_dup::ID. If you find a value in those fields, then you know you have a duplicate record and you can just delete it.

If[ Not IsEmpty( duplicate_records::ID ) ]

What is duplicate_records? a table? relations can have a name?
The tables in your example are named products and Products_dup, so the notation duplicate_records::ID don´t make sense to me).

If you make the calculation unstored, then the search performance will be very poor in a medium-to-large table.

The method in this article will perform well on large tables, and although it might introduce some indexing overhead, the impact will be much less than a full duplication of all data.

x = field_1 & field_2 …field_n

Then sort all records based on the “checksum” field, so that identical records are next to each other.

Then simply compare the value of x of record 1 with the value of
x of record 2 - and if they are identical you can delete one of the two records.

If you have more than two duplicate records you simply run the script again - until the number of records stays constant.

As my Filemaker sadly does not allow me to export scripts I cannot show you the actual text of the script.

Now, in the find/delete dupes loop, just before deleting: add the key of the record being deleted to the new key field of all self related records, separated by a newline.

The end result will be that after all dupes are gone, all records formerly related to any now deleted record, will now be related to the ’surviving’ record effectively consolidating all related redords to the kept records…

Remember to add an auto-enter calc to the new key field which adds the real unique id, in case it is missing, so that the mechanism stays functional).

Hope this helps!