Subject: Re[2]: UTF-8 Encoding problem - msg#00072

List: bug-tracking.scmbug.user

Mail Archive Navigation:
by Date: Prev Next Date Index by Thread: Prev Next Thread Index

Guten Tag Kristis Makris,
am Samstag, 16. Dezember 2006 um 00:45 schrieben Sie:

> If the locale is somehow figured out through environment variables,
> this should still be a problem without Apache (but with Subversion)
> because Subversion clears the environment completely when running hooks.

I don't really understand that point. My codepage was CP850, a Windows
one, and the output of Subversion on the console was properly encoded
by Subversion. This was a problem for SCMBug because it has read that
CP850-encoded data and Perl ist used to handle UTF-8. It's no problem
with Javiers setting because Subversions output is already in UTF-8
because of his local settings and Perl cand handle it the right way, I
think. This couldn't be possible if Subversion would clear the
environment, wouldn't it? What's the benefit of doing so from the
Subversion part?

> And possibly other SCM systems we support in the future may do the same.

> open ( SVNLOOK_INFO, "env LANG=es_ES.UTF-8 LANGUAGE=es_ES.UTF-8
> svnlook info " . $svn_tools_argument . "\
> $svn_txn $svn_repository |" );

> Do we need to set both LANG and LANGUAGE ?

It could be enough to set LC_ALL to the appropriate language, I think.

>> IT WORKS!!!!!!!!

> Smashing.

Yor're the man! ;-) The problem sounds like my one with PuTTY and
character conversion, always searched at the wrong place.

> I don't know how this now affects Windows 'chcp'. We might still need a
> "different" way of doing this under Windows; the way Thorsten described.
> Ahhhhw... Windows...

First you need to know if the real problem is the wrong environment
provided to Subversion by Apache, isn't it? There's a difference in
the output data Subversion provides between Javiers scenario and mine,
which would make me think that a conversion from the read input by
SCMBug depending on the local codepage settings is always necessary.

Would a be a way to clarify this while giving another codepage to
Subversion in Javiers open-line? Something like ISO-8859-1, in which
his characters would mean problems to SCMBug, too, without any
conversion to the internally used UTF-8.

Mit freundlichen Grüßen,

Thorsten Schöning

--
Thorsten Schöning
AM-SoFT IT-Systeme - Hameln | Potsdam | Leipzig

eMail Thorsten.Schoening-K5ztjtdJPw2ELgA04lAiVw@xxxxxxxxxxxxxxxx

Telefon Potsdam...0331-743881-0
Telefon Mobil.....0178-8 9468-04

Thread at a glance:

Previous Message by Date:

Re: UTF-8 Encoding problem

Javier Sotomayor!!! On Fri, 2006-12-15 at 21:47 +0100, Javier Lafuente wrote: > So we only need to know where the locale is changed and why... > I think that Glue is used by Subversion, and subversion is used by > apache2 (in my system I use https to access my reps) so the problem > can be in apache2 using the incorrect locale (???) I remember I read > something about that in the past... You were using Apache ? It seems that Apache is using a different locale yes. If the locale is somehow figured out through environment variables, this should still be a problem without Apache (but with Subversion) because Subversion clears the environment completely when running hooks. And possibly other SCM systems we support in the future may do the same. > open ( SVNLOOK_INFO, "env LANG=es_ES.UTF-8 LANGUAGE=es_ES.UTF-8 > svnlook info " . $svn_tools_argument . "\ > $svn_txn $svn_repository |" ); Do we need to set both LANG and LANGUAGE ? > (only added the env command to the original line) > > and... > > IT WORKS!!!!!!!! Smashing. > It has been hard to hunt but we got it! YOU got it! Thanks SO MUCH. > Now we "only" nedd to know how to make this more generic... > Perhaps can you put a command in yout scripts to get the locale from > some of the config files and set the new environment at the begining > of the script?? Right. > I leave that to you... I don't know very much about what I'm > talking :-) I don't know how this now affects Windows 'chcp'. We might still need a "different" way of doing this under Windows; the way Thorsten described. Ahhhhw... Windows...

Next Message by Date:

(no subject)

Hi all, I just finished installing scmbug 0-18-3 on 2 machines : 1) CVS server 2) bugzilla (2.22.1)/mysql server On the bugzilla machine I got the following message when commiting on the CVS machine : "The changeset comment could not be added. This must be due to a bug in the bug-tracker backend" How could I test from where the problem is coming from ? My integration daemon is running on the bugzilla machine, and my daemon.conf contains : bugtracker => { type => 'Bugzilla', version => '2.22.1', database_location => 'localhost', database_port => '3306', database_vendor => 'mysql', database_name => 'bugs', database_username => 'bugs', database_password => 'toto', installed_locally => 1, installation_directory => '/var/www/html/bugzilla', bug_url_prefix => 'http://bugzilla/show_bug.cgi?id=' }, Thanks in advance

Previous Message by Thread:

Next Message by Thread:

Re[2]: UTF-8 Encoding problem

Guten Tag Javier Lafuente, am Dienstag, 12. Dezember 2006 um 19:03 schrieben Sie: > But my problem is that I work in UTF-8... What have I need to add? > Something like? $request->>{ original_log_message } = $original_log_message; $request->>{ original_log_message } = decode("UTF-8", $request->{ original_log_message }); > does this make sense? I would have tried the same but the point is that Perl already handles all intern Strings as UTF-8 encoded calues. The problem was that everything which was read bei SCMBug from the console was internally not handled as a proper Perl-String, this issue is adressed by the decode-line which decodes "anything" under specification into a proper Perl string. > Am I decoding from UTF-8 to UTF-8??? Thats what I would think, the encoding of the resulting Perl string and whatever SCMBug used before should be the same. The only difference is that with the decode-line you cane be sure that Perl handles the data internally as a proper Perl string and not just as a bunch of characters. This is important for any kind of conversion lateron. Kristis link should explain the difference, I have forgotten to attach that one, too. :-( There's one more thing to consider: Maybe Bugzilla handles UTF-8 encoding before writing it to the database in another way than RT. In my oppinion Bugzilla is directly sending whatever comes from within Perl, because it's assumed as UTF-8 encoded, to the database. That was the problem with SCMBug, because it sends the formerly CP850-encoded String from my console directly to AppendComment from Bugzilla which either sends it directly to the database. RTs behaviour may be completely different. Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning AM-SoFT IT-Systeme - Hameln | Potsdam | Leipzig eMail Thorsten.Schoening-K5ztjtdJPw2ELgA04lAiVw@xxxxxxxxxxxxxxxx Telefon Potsdam...0331-743881-0 Telefon Mobil.....0178-8 9468-04