[announcement] ntfs-3g: open source read-write driver

Hello,

As part of the Linux-NTFS project, I'm happy to announce my contribution
to ntfsmount and libntfs which resulted ntfs-3g, a read-write ntfs driver,
capable for unlimited file creation and deletion.

The driver was successfully tested very exhaustively for a longer period of
time by many ways and methods, creating and destroying millions of files and
directories on newly created images, and on over 40 real, very diverse NTFS
images collected over the last four years.

The README file is copied below which includes more general, performance and
quality information with answers and future plans.

Please make a backup of your NTFS volume by ntfsclone and try to reproducible
break ntfs-3g, if you can. If you are able to do so then please let us know,
preferable providing your NTFS metadata image, if it couln't be reproduced by
other ways.

The driver can be downloaded from
        
        http://mlf.linux.rulez.org/mlf/ezaz/ntfs-3g-20070714-BETA.tgz

Please note that I'm leaving for an over 5,000 km long Far North and Arctic
Ocean advanture on Sunday morning (European time) and won't be able to read
and answer emails for almost a month (no, I'm not escaping, I plan to come
back alive ;). Originally I planned to fix all the minor issues listed below
but no more time left for me and I think it's better to release now, in case 
I would get lost somewhere on the road ;) Well, actually I hope that most
issues will be resolved when I'm back!

Happy testing, problem reporting and coding,

        Szaka


INTRODUCTION
============

The ntfs-3g driver is an open source, GPL licensed, third generation Linux 
NTFS driver for 32-bit, little-endian architectures which was implemented 
by the Linux-NTFS project. It provides full read-write access to NTFS, 
excluding access to encrypted files, writing compressed files, changing 
file ownership, access right.

Technically it's based on and a major improvement to the third generation 
Linux NTFS driver, ntfsmount. The improvements includes functionality, 
quality and performance enhancements. 

The driver currently is in BETA status: before release of this software we 
haven't experienced any driver crashes or data loss during our heavy quality 
testing, however we are aware of some minor issues which will be resolved
in the near future. We listed all of them in a below section. Please report 
any new problem to linux-ntfs-dev@xxxxxxxxxxxxxxxxxxxxx if it's not listed 
yet in the latest release of this software. 


QUICK INSTALLATION
==================
        
Make sure you have the basic Linux development tools and FUSE 
(http://fuse.sourceforge.net) is correctly installed on your computer 
then type:  

        ./configure
        make
        make install      # or 'sudo make install' if you aren't root.


USAGE
=====

If there was no error during installation then you can mount NTFS volumes 
for read-write access (unmount them if they were already mounted by 
another NTFS driver.)

        ntfs-3g /dev/hda1 /mnt/windows

If you want to give full access for all users, ignore permission related 
driver messages, and make national characters visible then use (drop or 
replace the below hu_HU.utf8 with the appropriate setting, and also the 
device and mount points, if needed).

        ntfs-3g /dev/hda1 /mnt/windows -o silent,umask=0,locale=hu_HU.utf8  

Please see the ntfs-3g manual page for more options and examples.

You can also make NTFS to be mounted during boot by ensuring that the FUSE 
kernel module is automatically loaded and by putting the below line into 
/etc/fstab 

        /dev/hda1 /mnt/windows ntfs-3g silent,umask=0,locale=hu_HU.utf8 0 0


PERFORMANCE
===========

Our highest priority is reliable functionality, performance comes only 
next. However acceptable performance is part of the functionality so we 
constantly keep an eye on it and consider any major performance issue to 
be a real functionality problem as well.

Below are the averaged file creation, deletion and access performance 
numbers got by running 'bonnie++ -s0' using several filesystems (the 
benchmark used 16,000 files per directory in each sessions):

Version  1.03    ------Sequential Create------ --------Random Create--------
                 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
           files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
reiserfs     16k 21459  99 +++++ +++ 17856  96 20172  98 +++++ +++ 16414  96
jfs          16k  7015  13 +++++ +++  5868  10  3068  14 +++++ +++  1075   3
ntfs-3g      16k  3021  99 14291  99  5226  99  3548  99 16149  99  5223  99
xfs          16k  2401  17 +++++ +++  2095  15  2301  20 +++++ +++   347   2
ext3         16k  1862  96 +++++ +++ +++++ +++  1914  96 +++++ +++  9695  98
minix        16k  1450  97 +++++ +++ 18148  94  1694  97 +++++ +++  4847  98
fat32        16k   366  97 +++++ +++  1809  97   428  97 +++++ +++  1361  97
paragon ntfs 16k    58  98  1259  99   245  99    55  99 +++++ +++   832  99
captive ntfs    Always crashes early on file creations and loses files.
                This was also confirmed and fix denied by its developer.

We estimate the ntfs-3g driver performance could be increased by several 
times.


KNOWN ISSUES, TROUBLESHOOTING
=============================

Problem:    Error message when mounting:
            fusermount: failed to open /dev/fuse: No such file or directory
Answer:     The FUSE kernel module isn't loaded. 
Workaround: Run 'modprobe fuse' as root.
Status:     High priority work.


Problem:    "Directory not empty" message when deleting "empty" directories: 
Answer:     If locale isn't set correctly then glibc can't convert
            some filenames, so they aren't visible and the directory 
            indeed isn't empty. Occures very rarely.
Workaround: Set your locale properly according to the used characters
Status:     High priority work.


Problem:    "No such file or directory" message when accessing some files:
Answer:     This is a known problem in libntfs inode attribute enumeration 
            when the same type of attributes are distributed across
            multiply mft records. Occures only very rarely.
Workaround: If the file is a hard link (usually yes) then first delete
            some other files hard linking to the same file then you'll
            be able to delete the problematic file as well.
Status:     High priority work.


Problem:    "Operation not supported" message when creating a file:
Answer:     This is a controlled refusal of file creation which can happen 
            when there are already about 100,000 files on the NTFS and the
            MFT needs to be extended in a way which isn't implemented yet.
Workaround: Delete files then you can create new ones.
Status:     High priority work.


Problem:    "Operation not supported" message when deleting a file:
Answer:     The support for certain very-very rare directory update 
            operations weren't fully tested yet, so they are not 
            included in the release. This is so rare that you probably
            will never see it ;-)
Workaround: Delete or create other files in the same directory until you
            can remove the file you originally wanted.
Status:     High priority work.


Problem:    Only Windows Administrator can access the files created by
            the driver.
Answer:     File permission and ownership handling isn't implemented yet.
Workaround: As Windows Administrator, set access permissions for other 
            users.
Status:     Solution exists but it needs to be revised and tested.


Problem:    Windows chkdsk complains about free space being marked in use.
Answer:     chkdsk sometimes optimizes the NTFS layout and later finds
            its own bugs (e.g. when index root attributes are moved from 
            extent mft records to the base one). This is not a problem 
            with ntfs-3g.
Workaround: No need, it's a chkdsk bug and it fixes itself.
Status:     Low priority work: we will optimize our algorithms too.


Problem:    Windows chkdsk reports the below messages (N is a number).
            Cleaning up N unused index entries from index $SII of file 0x9.
            Cleaning up N unused index entries from index $SDH of file 0x9.
            Cleaning up N unused security descriptors.
Answer:     These messages are part of an optimization process which is
            completely independent of ntfs-3g. Nothing to worry about them.
Workaround: No need.
Status:     Nothing to do.


Problem:    Windows chkdsk may report minor inconsistency.
Answer:     The allocation size of sparse files isn't set always correctly.
Workaround: No need.
Status:     Normal priority work.


Problem:    Why ntfs-3g CPU usage is always 100%?
Answer:     You have some application running which inefficiently "polls"
            for file modification events.
Workaround: Identify the software and try to upgrade it.
Status:     Normal priority work: we also intend to improve significantly 
            our CPU usage for such cases.

Problem:    Why doesn't the driver work on 64-bit and bigendian systems?
Answer:     We have no resource for that. Neither hardware, nor workforce.
Status:     Low priority.




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642