Re: [announcement] ntfs-3g: open source read-write driver

Hi Szaka,

On Fri, 14 Jul 2006, Szakacsits Szabolcs wrote:
> As part of the Linux-NTFS project, I'm happy to announce my contribution
> to ntfsmount and libntfs which resulted ntfs-3g, a read-write ntfs driver,
> capable for unlimited file creation and deletion.

Wow, well done!  Although "unlimited" is a bit of an exagerration as you 
yourself admit below.

I had a quick look at the code and it chickens out of a lot of corner 
cases which are quite hard...

Also I may be wrong as I did not spend enough time on it but if you run 
out of space on the partition in the middle of a complex add or delete 
operation you appear to leave a corrupt directory.  In general the error 
handling appears to leave things corrupt and does not attempt to roll back 
/ correct things.  Is this a fair description?

And of course you use recursion which you cannot use in kernel context so 
the code could never be moved to a kernel driver (even ignoring the fact 
that it does not do locking or endianness handling).

> The driver was successfully tested very exhaustively for a longer period of
> time by many ways and methods, creating and destroying millions of files and
> directories on newly created images, and on over 40 real, very diverse NTFS
> images collected over the last four years.
> 
> The README file is copied below which includes more general, performance and
> quality information with answers and future plans.
> 
> Please make a backup of your NTFS volume by ntfsclone and try to reproducible
> break ntfs-3g, if you can. If you are able to do so then please let us know,
> preferable providing your NTFS metadata image, if it couln't be reproduced by
> other ways.

I admit I haven't tried but from reading the coded just reaching ENOSPC on 
the partition and then continuing to operate may cause corruption.

> The driver can be downloaded from
>       
>       http://mlf.linux.rulez.org/mlf/ezaz/ntfs-3g-20070714-BETA.tgz
> 
> Please note that I'm leaving for an over 5,000 km long Far North and Arctic
> Ocean advanture on Sunday morning (European time) and won't be able to read
> and answer emails for almost a month (no, I'm not escaping, I plan to come
> back alive ;). Originally I planned to fix all the minor issues listed below
> but no more time left for me and I think it's better to release now, in case 
> I would get lost somewhere on the road ;) Well, actually I hope that most
> issues will be resolved when I'm back!

Wow!  Enjoy it!  That sounds like an amazing experience!  I hope you bring 
back some nice photos for us all to wonder at!  (-;

> Version  1.03    ------Sequential Create------ --------Random Create--------
>                  -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
>            files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
> reiserfs     16k 21459  99 +++++ +++ 17856  96 20172  98 +++++ +++ 16414  96
> jfs          16k  7015  13 +++++ +++  5868  10  3068  14 +++++ +++  1075   3
> ntfs-3g      16k  3021  99 14291  99  5226  99  3548  99 16149  99  5223  99
> xfs          16k  2401  17 +++++ +++  2095  15  2301  20 +++++ +++   347   2
> ext3         16k  1862  96 +++++ +++ +++++ +++  1914  96 +++++ +++  9695  98
> minix        16k  1450  97 +++++ +++ 18148  94  1694  97 +++++ +++  4847  98
> fat32        16k   366  97 +++++ +++  1809  97   428  97 +++++ +++  1361  97
> paragon ntfs 16k    58  98  1259  99   245  99    55  99 +++++ +++   832  99
> captive ntfs    Always crashes early on file creations and loses files.
>               This was also confirmed and fix denied by its developer.
> 
> We estimate the ntfs-3g driver performance could be increased by several 
> times.

Wow, nice looking numbers.  I haven't done anything like that for my code.  
I wonder how it would compare given it is in the kernel and IMHO a lot 
better optimized than your version...  E.g. I use binary search where you 
use sequential and I never have to repeat a search twice during a single 
operation as that is all cached the first time although I admit that the 
cache overhead may introduce a delay in itself so the benefit in my code 
may not be as much as one would expect initially...

> KNOWN ISSUES, TROUBLESHOOTING
> =============================
> 
> Problem:    "No such file or directory" message when accessing some files:
> Answer:     This is a known problem in libntfs inode attribute enumeration 
>             when the same type of attributes are distributed across
>             multiply mft records. Occures only very rarely.
> Workaround: If the file is a hard link (usually yes) then first delete
>             some other files hard linking to the same file then you'll
>             be able to delete the problematic file as well.
> Status:     High priority work.

Interesting.  This is the first time I hear of this problem.  Why did you 
not report it?  Do you have an example where this happens?

> Problem:    Why doesn't the driver work on 64-bit and bigendian systems?
> Answer:     We have no resource for that. Neither hardware, nor workforce.
> Status:     Low priority.

You do not need hardware.  If the little endian version works, then you 
can just run sparse over appropriately annotated code and it will tell you 
if you have screwed up somewhere.  Workforce: fair enough but in this 
state it cannot be added to ntfsprogs because it would break ntfsprogs on 
all but i386/x86_64 which is not acceptable.  It is a shame.  I hope you 
or someone fixes this one day...  ntfsprogs are in use on big endian 
architectures by people today and I am not willing to see that all 
broken...

Best regards,

        Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642