external USB drive corruption

[ducman]

I'm running 10.4.10 on a Thinkpad T60p. Used JaS 10.4.8 DVD to install, and things have generally worked very well through 10.4.8, 10.4.9, 10.4.10. Of course, I have to hard-code my display resolution and I get some screen tearing, but the laptop works well enough to be my main machine at work.

 

However, I'm having a serious problem with files getting corrupted on an external USB drive. I've searched the forums and haven't seen other references to this problem, so I wonder whether it's a system problem or a problem with the specific drive/enclosure I'm using.

 

The problem most commonly appears when I'm running Azureus to download some files. After some amount of time (seems like has to be >1 hour) of running fine, I get an error about ejecting a disk while in use and the machine unmounts and remounts the external drive on it's own. I can reproduce the error by doing a large finder copy to the external USB drive, but Azureus is a better test 'cause it will recheck the files to find out whether any got corrupted. _Generally_ I can quit Azureus, check the drive with Disk Utility and Disk Warrior and find no problems, although _sometimes_ Disk Utility will find a problem, and Disk Warrior is always able to rebuild the directory.

 

After several days of playing around with it, I'm now convinced that the disk remount thing is not causing the file corruption, but must be a result of it. I've confirmed that I can copy a file from the internal to the external drive and verify that the file is correct. Then, after some amount of time, I can check the file, again, and find that it no longer matches the original file on the internal drive. Once I've detected that the file is corrupt, un/remounting the drive or rebooting doesn't help, and Disk Utility and Disk Warrior find no problems with the disk.

 

So, somehow, I'm getting completely silent corruption of files on the external drive, even if I don't use the files! This means that my backups to the external drive are completely useless! Has anyone else experience this problem? Any suggestions on how to determine, for sure, whether the problem is with the system as opposed to the drive/USB enclosure being used? Any USB enclosures that are know to be better/worse with MacOS X?

 

I believe I've been having this problem since my initial install. I'm currently using this kernel:

 

Darwin Kernel Version 8.10.1: Wed May 23 16:33:00 PDT 2007; Sochi2014:VoteForUs/RELEASE_I386 i386 i386

 

The USB section of my system_profiler output looks like this:

root# system_profiler SPUSBDataType

USB:

 

USB High-Speed Bus:

 

Host Controller Location: Built In USB

Host Controller Driver: AppleUSBEHCI

PCI Device ID: 0x27cc

PCI Revision ID: 0x0002

PCI Vendor ID: 0x8086

Bus Number: 0xfd

 

USB Mass Storage Device:

 

Capacity: 111.79 GB

Removable Media: Yes

Detachable Drive: Yes

BSD Name: disk1

Version: b0.08

Bus Power (mA): 500

Speed: Up to 480 Mb/sec

Manufacturer: Myson Century, Inc.

OS9 Drivers: No

Product ID: 0x8818

Serial Number: 100

S.M.A.R.T. status: Not Supported

Vendor ID: 0x04cf

Volumes:

ext_boot:

Capacity: 11.75 GB

Available: 11.72 GB

Writable: Yes

File System: Journaled HFS+

BSD Name: disk1s2

Mount Point: /Volumes/ext_boot

ext_data:

Capacity: 99.59 GB

Available: 41.91 GB

Writable: Yes

File System: Journaled HFS+

BSD Name: disk1s3

Mount Point: /Volumes/ext_data

Volumes:

disk1s2:

Capacity: 11.75 GB

Available: 11.72 GB

Writable: Yes

File System: Journaled HFS+

disk1s3:

Capacity: 99.59 GB

Available: 41.91 GB

Writable: Yes

File System: Journaled HFS+

 

USB Bus:

 

Host Controller Location: Expansion Slot

Host Controller Driver: AppleUSBUHCI

PCI Device ID: 0x27c8

PCI Revision ID: 0x0002

PCI Vendor ID: 0x8086

Bus Number: 0x1d

 

USB Bus:

 

Host Controller Location: Expansion Slot

Host Controller Driver: AppleUSBUHCI

PCI Device ID: 0x27c9

PCI Revision ID: 0x0002

PCI Vendor ID: 0x8086

Bus Number: 0x3d

 

USB Bus:

 

Host Controller Location: Expansion Slot

Host Controller Driver: AppleUSBUHCI

PCI Device ID: 0x27ca

PCI Revision ID: 0x0002

PCI Vendor ID: 0x8086

Bus Number: 0x5d

 

USB Bus:

 

Host Controller Location: Expansion Slot

Host Controller Driver: AppleUSBUHCI

PCI Device ID: 0x27cb

PCI Revision ID: 0x0002

PCI Vendor ID: 0x8086

Bus Number: 0x7d

 

BCM2045B:

 

Version: 1.00

Bus Power (mA): 500

Speed: Up to 12 Mb/sec

Manufacturer: Broadcom Corp

Product ID: 0x2110

Vendor ID: 0x0a5c

[ducman]

So I finally got around to doing some real testing. I found that a Finder copy of a large file often, but not always, failed. I started doing some command-line tests to try to see if it was just a Finder issue. A single copy of a 2.6GB test file always finished, but an md5sum check of the copied file NEVER matched the original! Three copies running at once would almost always cause the disconnect issue, and none of the three files ever matched the original.

 

By now I'm thinking either the drive or the USB enclosure must be complete junk. Then, for some reason, I tried reformatting the drive as "MS-DOS Filesystem" in Disk Utility. This time a single copy of the 2.6GB file worked, and the md5sum matched the original! Three copies running at once finished fine, too, and all three copies matched the original! So now I'm even more confused than ever. I assume an "MS-DOS Filesystem" is FAT32, and I thought FAT32 didn't support files bigger than 2GB.

 

Knowing that FAT32 has some significant limitations, even if it did handle my >2GB test file, I then tried reformatting the disk as UFS. Again, single and multiple copies were successfull, and the md5sum results checked out.

 

So, for now, I'm leaving the external drive in UFS format.

 

I can't imagine, though, why UFS and FAT would work and HFS+ wouldn't. It's definitely something about the USB connection, because I've never had any problems with the boot drive (SATA) or a second drive in the expansion bay (SATA), both of which are HFS+Journaling. But how on earth could a different filesystem result in what appears to be hardware-related problems?

[Randy Oo]

That's pretty weird, alright!

 

I'd go back to HFS+ and try again, just to confirm that it was the filesystem change, rather than formatting, that fixed things.

[ducman]

So I finally solved the USB corruption problem.

 

More intesive testing with different filesystems showed that the corruption did happen with every available filesystem. My initial results that seemed to favor FAT or UFS were just a fluke.

 

One thing I did learn is that HFS+Journaled is dramatically faster than FAT or UFS, at least on an external USB drive.

 

What I finally did is add a powered USB hub.

 

Apparently my lenovo ThinkPad T60p can't provide enough power, even when using the cable provided with the drive to draw power from two USB ports. I actually tried the external enclosure with two different drive mechanisms. One drive was labeled as requiring 1.0A, the other labeled as 0.75A, but both had the corruption problems. Connected to the powered hub though, either drive was fine in extensive testing of all available filesystems.