Re: [announcement] ntfs-3g: open source read-write driver

On Wed, 2006-07-26 at 17:58 +0300, Yura Pakhuchiy wrote:
> Hi Anton,
> 
> On Sun, 23 Jul 2006 at 22:49 +0100, Anton Altaparmakov wrote:
> > On Sun, 23 Jul 2006, Yura Pakhuchiy wrote:
> > > On Sun, 23 Jul 2006 at 08:33 +0300, Yura Pakhuchiy wrote:
> > > > On Sat, 15 Jul 2006 at 20:46 +0100, Anton Altaparmakov wrote:
> > > > > > > > Problem:    "No such file or directory" message when accessing 
> > > > > > > > some files:
> > > > > > > > Answer:     This is a known problem in libntfs inode attribute 
> > > > > > > > enumeration 
> > > > > > > >             when the same type of attributes are distributed 
> > > > > > > > across
> > > > > > > >             multiply mft records. Occures only very rarely.
> > > > > > > > Workaround: If the file is a hard link (usually yes) then first 
> > > > > > > > delete
> > > > > > > >             some other files hard linking to the same file then 
> > > > > > > > you'll
> > > > > > > >             be able to delete the problematic file as well.
> > > > > > > > Status:     High priority work.
> > > > > > > 
> > > > > > > Interesting.  This is the first time I hear of this problem.  Why 
> > > > > > > did you 
> > > > > > > not report it?  Do you have an example where this happens?
> > > > > > 
> > > > > > I identified this problem before yesterday. Sometimes I can 
> > > > > > reproduce it by
> > > > > > 
> > > > > >     cp -a /usr ntfs_volume
> > > > > >     rm -rf ntfs_volume/usr
> > > > > > 
> > > > > > Most ntfs volumes have also such cases usually, so if you rm -rf / 
> > > > > > then you 
> > > > > > can find them easily. 
> > > > > 
> > > > > Weird.  I wonder if it is something libntfs specific and not in the 
> > > > > kernel as I have never seen this problem...
> > > > 
> > > > This is kernel driver problem too. It seems our knowledge about
> > > > attribute list a bit incorrect. ntfs_external_attr_find written in mind
> > > > with that attributes in the MFT record follow in same order as they
> > > > follow in attribute list. Metadata (link to which I send in previous
> > > > email) demonstrates vice-versa. FILE_NAME attributes follow in order 2,
> > > > 3 (instance numbers) in the attribute list, but 3, 2 in the MFT record.
> > 
> > You are correct that instance numbers are in sequence in attribute list 
> > attribute and are not in order in the mft record.  This is because the 
> > sequence numbers are used for collation in the attribute list attribute 
> > and they are not used for collation in the mft record (instead the 
> > attribute value is used which in case of the file name attribute means 
> > that the sequence number is irrelevant and the name is relevant instead).
> > 
> > What I don't understand is how that is causing file not found problems...
> > 
> > /me reads through ntfs_external_attr_find().
> > 
> > /me sees stupendously stupid bug and bashes head against wall repeatedly.
> > 
> > /me fixes bug and commits it to ntfsprogs.
> > 
> > I cannot believe this has not been found earlier!  This bug has been there 
> > since I originally wrote ntfs_external_attr_find() which was quite a few 
> > years ago now!  Amazing!
> 
> This bug fix introduced new bug to ntfs_external_attr_find. :-(
> 
> Metadata that illustrates problem:
> http://data.linux-ntfs.org/attr_lookup_bug2.img.bz2
> 
> Mount it with ntfsmount and try to rename directory "SFU" to another
> name. ntfs_external_attr_find will complain about damaged inode
> (Probably you know, but for any case, you can receive debug output from
> ntfsmount using "-o debug"). Reverting your last patch makes it happy.
> Please, fix!

Argh.  You do evil things!  (-;  It now works again but please note that
I believe what you do still is have a bug and will randomly cause
corruption of the attribute list attribute because the attribute list
entries will be sorted incorrectly.  Basically if SFU was a file rather
than a directory and you instead of renaming it added hard links to it
you would find that at least some of the hard links are incorrectly
ordered in the attribute list attribute.  (It would be a complete per
chance event as to which will go where.)  This is arguably a bug in
ntfs_external_attr_find() or it could be argued to be a bug of
ntfs_attrlist_entry_add().

Also I think you have a potential bug in that you add the attribute to
the mft record before you have added it to the attribute list attribute
and then you call ntfs_attr_lookup() on what effectively is a corrupt
inode.  The only reason this works is a complete pure chance/luck and
stronger sanity checking would make this code break...

Best regards,

        Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://www.linux-ntfs.org/ & http://www-stu.christs.cam.ac.uk/~aia21/


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV