Subject: Re: UTF-8 sequences in filenames - msg#00014

Previous Message by Date:

Re: UTF-8 sequences in filenames

On 5 Mar 2003, Dmitry G. Mastrukov wrote: > Hi! > > I'd like to raise problem we (in Russian community) encountered with > filenames. > The problem exists in non-unicode locales (ru_RU.KOI8-R, ru_RU.CP1251) > with mamy GNOME apps (gedit, glade, nautilus at least). Files with > national filenames are viwed through file selector (G_BROKEN_FILENAMES > is set) and can be opened. But newly created files looks on filesystem > (in mc for example) as UTF-8 sequences. > We have some preliminary patches to gnome-vfs but we have no confidence > in that gnome-vfs is only source of problem. It should be good to find > out thougths of gnome-vfs developers about that. > We see two possibilities: > 1. Gnome-vfs is broken and need patching. Seems it is not using > g_filename_from_utf8/uri() functions from glib at all. Then where is > control point for possible conversion? In other words which public > functions work with "locale-dependant" filenames and which work with UTF > definitely? Brief look at gedit sources shows > gnome_vfs_get_local_path_from_uri is used to get filename to feed to > fopen(). So it should produces locale filename on output. Also may be > do_action() functions in method/file-method.c and some other methods > should use g_filename_fron_utf8/uri(). This is not right. gnome-vfs uris cannot be defined in a specific encoding, just like unix pathnames cannot be. You must be able to use gnome-vfs to access a file that has an invalid utf8 name for instance (maybe someone created it with G_BROKEN_FILENAMES in another locale), otherwise you can't rename it or delete that file. Similarily uri:s may reference remote systems that don't use utf8-encoded filenames, nor your local locale encoding. Basically, all URIs in gnome-vfs are in an filesystem-dependent unknown encoding (maybe even no/several encodings, such as on a system using G_BROKEN_FILENAMES with users in different locales), and it has to be that way for gnome-vfs to be able to access all files. However, the goal of Gnome filename handling is that all files created are in either UTF8 (default) or at the users choice (G_BROKEN_FILENAMES is set) in the current locale. In general the filename encoding issue this is a hard problem, and a full real solution will not be availible for a long time, until the whole world has switched over to a common encoding. Many sources of filenames just don't have a corresponding filename encoding specified, so until everyone use the same we have to guess. Take an ftp site for instance. How are you supposed to know the encoding it uses for filenames? > 2. Gnome-vfs is OK but developer do not use it power properly. Then it > should exist some document with description of right method of dealing > with filenames. That document should be widely known to developers. Yes. This is true, in fact I've fixed some nautilus bugs relating to this recently. For Gtk+ apps, when creating a file you're supposed to use g_filename_from_utf8() to get the actual file name to use from what the user entered (which is always utf8). -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Alexander Larsson Red Hat, Inc alexl@xxxxxxxxxx alla@xxxxxxxxxxxxxx He's a notorious vegetarian stage actor trapped in a world he never made. She's a strong-willed red-headed mermaid with an incredible destiny. They fight crime!

Next Message by Date:

Re: gnome-vfs-sftp 0.1.2

On 21 Feb 2003, Bradford Hovinen wrote: > Folks, > > I just put up version 0.1.2 of the sftp method, as earlier promised. > It's at > http://www.math.uwaterloo.ca/~bghovinen/gnome-vfs-sftp-0.1.2.tar.gz > > This version fixes several bugs, including the 'Cannot open "/"' bug and > some difficulties with renaming files. It adds some rudimentary error > checking (based on reading the stderr data from ssh), so failed logins > and whatnot will get reported as such. I also incorporated the > suggestions of Alex Larsson and others. > > Please try it out if you are interested, and let me know of any > problems. Thanks. I got a deadlock. I think this fixes it: --- sftp-method.c.deadlock 2003-03-05 12:28:35.000000000 +0100 +++ sftp-method.c 2003-03-05 12:30:22.000000000 +0100 @@ -1653,6 +1653,7 @@ if (status != SSH2_FX_OK) { buffer_free (&msg); g_free (write_req); + sftp_connection_unlock (handle->connection); return sftp_status_to_vfs_result (status); } @@ -1752,8 +1753,10 @@ if (options & GNOME_VFS_FILE_INFO_FOLLOW_LINKS) { res = get_real_path (conn, path, &real_path); - if (res != GNOME_VFS_OK) + if (res != GNOME_VFS_OK) { + sftp_connection_unlock (conn); return res; + } } else real_path = path; The error handling when you've note got the key for the server in ssh-agent needs to be better too. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Alexander Larsson Red Hat, Inc alexl@xxxxxxxxxx alla@xxxxxxxxxxxxxx He's an ungodly hunchbacked ex-con who must take medication to keep him sane. She's a ditzy wisecracking wrestler who believes she is the reincarnation of an ancient Egyptian queen. They fight crime!

Previous Message by Thread:

Re: UTF-8 sequences in filenames

В Срд, 05.03.2003, в 11:41, Alexander Larsson написал: > On 5 Mar 2003, Dmitry G. Mastrukov wrote: > > > Hi! > > > > I'd like to raise problem we (in Russian community) encountered with > > filenames. > > The problem exists in non-unicode locales (ru_RU.KOI8-R, ru_RU.CP1251) > > with mamy GNOME apps (gedit, glade, nautilus at least). Files with > > national filenames are viwed through file selector (G_BROKEN_FILENAMES > > is set) and can be opened. But newly created files looks on filesystem > > (in mc for example) as UTF-8 sequences. > > We have some preliminary patches to gnome-vfs but we have no confidence > > in that gnome-vfs is only source of problem. It should be good to find > > out thougths of gnome-vfs developers about that. > > We see two possibilities: > > 1. Gnome-vfs is broken and need patching. Seems it is not using > > g_filename_from_utf8/uri() functions from glib at all. Then where is > > control point for possible conversion? In other words which public > > functions work with "locale-dependant" filenames and which work with UTF > > definitely? Brief look at gedit sources shows > > gnome_vfs_get_local_path_from_uri is used to get filename to feed to > > fopen(). So it should produces locale filename on output. Also may be > > do_action() functions in method/file-method.c and some other methods > > should use g_filename_fron_utf8/uri(). > > This is not right. gnome-vfs uris cannot be defined in a specific > encoding, just like unix pathnames cannot be. You must be able to use > gnome-vfs to access a file that has an invalid utf8 name for instance > (maybe someone created it with G_BROKEN_FILENAMES in another locale), > otherwise you can't rename it or delete that file. Similarily uri:s may > reference remote systems that don't use utf8-encoded filenames, nor your > local locale encoding. > > Basically, all URIs in gnome-vfs are in an filesystem-dependent unknown > encoding (maybe even no/several encodings, such as on a system using > G_BROKEN_FILENAMES with users in different locales), and it has to be that > way for gnome-vfs to be able to access all files. However, the goal of > Gnome filename handling is that all files created are in either UTF8 > (default) or at the users choice (G_BROKEN_FILENAMES is set) in the > current locale. > > In general the filename encoding issue this is a hard problem, and a full > real solution will not be availible for a long time, until the whole world > has switched over to a common encoding. Many sources of filenames just > don't have a corresponding filename encoding specified, so until everyone > use the same we have to guess. Take an ftp site for instance. How are you > supposed to know the encoding it uses for filenames? > Thanks for explanation. My last tests show that properly written apps work fine. So seems there exist bugs in gedit and glade but not in gnome-vfs. Now i know it's problem of particular apps. Regards, Dmitry

Next Message by Thread:

Re: UTF-8 sequences in filenames

В Срд, 05.03.2003, в 10:29, Dmitry G. Mastrukov написал: > Hi! > > I'd like to raise problem we (in Russian community) encountered with > filenames. > The problem exists in non-unicode locales (ru_RU.KOI8-R, ru_RU.CP1251) > with mamy GNOME apps (gedit, glade, nautilus at least). Files with > national filenames are viwed through file selector (G_BROKEN_FILENAMES > is set) and can be opened. But newly created files looks on filesystem > (in mc for example) as UTF-8 sequences. > We have some preliminary patches to gnome-vfs but we have no confidence > in that gnome-vfs is only source of problem. It should be good to find > out thougths of gnome-vfs developers about that. > We see two possibilities: > 1. Gnome-vfs is broken and need patching. Seems it is not using > g_filename_from_utf8/uri() functions from glib at all. Then where is > control point for possible conversion? In other words which public > functions work with "locale-dependant" filenames and which work with UTF > definitely? Brief look at gedit sources shows > gnome_vfs_get_local_path_from_uri is used to get filename to feed to > fopen(). So it should produces locale filename on output. Also may be > do_action() functions in method/file-method.c and some other methods > should use g_filename_fron_utf8/uri(). More deep tests show that Nautius, grecord, gThumb with G_BROKEN_FILENAMES=1 work fine, recodin to locale occurs. Seems gnome-vfs is mostly OK, but gedit and glade put UTF8 sequences in filenames in any case. Regards, Dmitry