Comments on: Dealing with Damaged FileMaker Files

By: Geoff Coffey

Geoff Coffey — Wed, 17 Oct 2007 20:57:48 +0000

@bryan: The most likely explanation is that you do have some under-the-hood corruption that just has no visible impact. FileMaker 9 does something called a consistency check on the file when it is opened to look for corruption. It does this to help detect problems early before they’re likely to lead to major problems.

In your case, the consistency check is failing so 9 is complaining that the file is damaged, even though in 8, where no check is done, it runs “fine.”

In most cases, simply saving as a compacted version in 8 or 8.5, then re-opening in 9 will fix this problem. It sounds like that isn’t working in your case. You may need to try a more aggressive approach. Have you tried recovery (I know…bad word…try it anyway, at least to see if it gets the file open-able in 9). Also, sometimes it works to clone the file, then save the clone as a compacted copy, then re-import the data form the original. Do this all in 8 or 8.5, then try opening in 9.

Let us know what happens using the Contact Us button above.

By: Bryan

Bryan — Wed, 17 Oct 2007 20:29:07 +0000

Good dialogue here… in our case we did not see any errors or corruption until updating from 8.5 to 9.0. Does this imply an error that was lurking but not seen (likely from a previous Server crash), or does this imply some issue with FMP 9?

We haven’t had any luck with compacting or recovering… the database is functioning as best we can tell on 8.5.

Since this same behavior (only indicating corruption when opening in 9.0) on several files, makes me wonder.

By: Importing Records to a Backup - Brute Force at Six Fried Rice

Importing Records to a Backup - Brute Force at Six Fried Rice — Thu, 13 Sep 2007 19:19:52 +0000

[…] Dealing with Damaged FileMaker Files […]

By: mr_vodka

mr_vodka — Fri, 07 Sep 2007 19:56:23 +0000

Hey Geoff and Jesse,

Just as a FYI, Thatcher’s presentation is available for download on the DevCon update site now. The link is in the DevCon binder and is under “Conference Information” of the General Information tab just in case you werent aware.

By: Tom Gelin

Tom Gelin — Mon, 27 Aug 2007 20:53:39 +0000

I spent the better part of a week recovering a database with 2 million records. I didn’t compact or recover the original file. What I did was export 60,000 records at a time to an Xcel file. Then imported them into alphabetical FMP9 files. In the process I had numerous crashes but by importing the files alphabetically I only had to start over for the particular character. After I had all of the A-Z files I then imported them one at a time into a master file. Not a very pleasant experience, but it worked.

By: Beatrice Beaubien

Beatrice Beaubien — Mon, 27 Aug 2007 04:30:10 +0000

In his session, Jon Thatcher solicited our ideas about how the report after recovery could be improved. I’d take this offer seriously. For example, we could request a result that went into detail into what didn’t have to be changed. “This file did not appear to need recovery.” That would at least tell us nothing was changed. Or a verbose description as to what was actually changed. Or even a take on whether the data resulting was worth recovering, how many blobs were lost and how many were rescued, etc. The engineers get a fair amount of info back from the recovery process and (apparently) we can request that it be put into a report, like the Import or Conversion Report.

Your constructive ideas to the engineers can go into the Help -> Send us your feedback selection in FMPA9.

By: Geoff Coffey

Geoff Coffey — Thu, 23 Aug 2007 15:46:28 +0000

@winfried: First, thanks for the clarification. We never intended to say that the consistency check ensures a perfect file, and I have cleaned up some “casual” language that went too far. The key is that with Server 8, you could have damaged files that keep running for weeks. Now, in 9, you have a better chance of finding out early if the backup-time consistency check can catch it, so we recommend turning this on in most cases, at least for a once-nightly backup.

As for turning off indexing, I’m not aware of any case where removing an index causes a dependent calculation field to become unstored. If that can happen, it would certainly cause problems that are hard to track down later. But FileMaker is happy to store a calculation based on an unindexed field. As far as I understand it, this is only a concern if you make another calc field unstored. Are you sure removing an index can lead to this situation? What are the circumstances?

Again, thanks for the feedback and good info.

By: Winfried Huslik

Winfried Huslik — Thu, 23 Aug 2007 12:32:14 +0000

Find problems: damaged index

The problem with turning indexing off and on may be:

a dependent calculation becomes unstored as well and further down the road, Value Lists, Relations, or other things stop working. Indexing of depending fields may never be turned on again. So be careful with that!

FileMaker Server 9 and FileMaker Pro 9 Consistency Check

What it does, and what it NOT does

It tests if the blocks are linked correctly to the previous and next one

It tests whether the internal table of contents of IDs point to the right blocks

It does NOT assure the file is 100 % OK! Claiming so is plain wrong.

It NEITHER does check the consistency of the data NOR the so-called file structure (what makes up Tables, Fields, Scripts, Layouts, etc.)

FileMaker Server 9 Backups

The new perform consistency check box in your backup schedule

It simply runs the same tests as described above.

It does NOT assure the file is 100 % OK! Claiming so is plain wrong.

It NEITHER does check the consistency of the data NOR the so-called file structure (what makes up Tables, Fields, Scripts, Layouts, etc.)

Admitted it’s better than nothing and sure catches some common problems, but you may NOT rely on these tests for 100 % corruption free files.

Save a Copy As -> Compacted

Does it really remove corruption? Not really, and at occasions it makes things even worse.

This may become clear if we know what it really does:

It reads the file in its logical block sequence. It tests how much of the block is filled and if possible fills it with part of the stuff from the next block. Then that information is written to the new file.

If per some damage the block length of the previous block was not set correctly, then either a gap with random data appears, or part of the old data is overwritten. This can cause severe problems and crashes.

If there is already some damage within the block, this may be copied without further ado.

Comment on comment 10. by Geoff Coffey, 2007-08-17

You all may not be aware that help is on its way: FMDiff tells you that a file is damaged if it is - and where.

Read How to Cope with File Corruption

By: Geoff Coffey

Geoff Coffey — Thu, 23 Aug 2007 12:30:04 +0000

@all: Winfried’s article is well worth the read (I’ve read it many times). Just in case it wasn’t obvious, here’s the direct link:

How to Cope with File Corruption

Thanks, Winfried. Excellent stuff. I haven’t had the chance to use FMDiff for this purpose, but I’m sure I will some day.

By: Winfried Huslik

Winfried Huslik — Thu, 23 Aug 2007 09:30:15 +0000

The main questions remains unanswered: Is this file damaged?

Or more important: Are you sure the file is NOT damaged?

See website for an answer. I’ll also comment on some details of this blog later.

Winfried