[formulator-dev] using other encodings

[Joachim Schmitz]

Martijn Faassen wrote:
> Joachim Schmitz wrote:
> [ snip ]
> 
>>So why not use utf-8 as the hardcoded charset ? This does not work, then 
>>already the xml-parser breaks, with invalid token error, when entering a 
>>char like "ä".
>  
> I'm not entirely sure why this is happening in Formulator already.
> 
This does not show up in Formulator cause the xml-encoding is set to
<?xml version="1.0" encoding="iso-8859-1"?> in FormToXML.py
If you change this to "utf-8" and enter a "ä" in a title or description.
Then the parser fails, cause the 'ä' is entered in a textaerea on a 
page, where the encoding is set to "iso-8859-1" and the parser assumes 
it is "utf-8". So the right thing to (tm) is to set the encoding to 
currently used encoding in Zope. Is there a way to find this out ?

This will not help in Silva though, cause there all forms are defined in 
xml-textfiles, where the encoding set to "utf-8", since the editor I use 
outputs iso-8859-1, entering a "ä" there without changing the encoding 
to the one, which your editor produces.

By the way I found a very helpfull discussion of the topic here:

http://www.vandervossen.net/2003/07/unicode_in_python

> The best strategy to go forward I suspect is to store unicode strings
> in Formulator natively for the field properties where it makes sense.
> If Formulator outputs unicode though for instance for 'title' or
> 'description' current code would be broken though (just like my
> sample page template is). 
I made a testform and entered some umlauts in title, these display 
correctly and don't break anything. Even the modification in the xml 
source works, cause 
browser-encoding=xml-encoding=textaera-input-encoding=iso-8859-1

It should also work if you have other encodings as long as the above 
condition is met.

> There's also a Formulator upgrade issue. I am not
> quite clear yet on how to proceed there..
> 
I am not sure if there is an issue see above.
> I'm considering moving towards a Formulator 2.0 anyway at some point,
> which means we might accept at least the upgrade issue and a bit of
> code breakage. There's also a usability issue though; if you use *any* latin-1
> encoding in your page template use Formulator you'd get unicode errors if I 
> made title and description store their data as unicode...
> 
> Suggestions?
> 
> Regards,
> 
> Martijn


-- 
Mit freundlichen Grüßen                                Joachim Schmitz
......................................................................
AixtraWare eK ..Joachim Schmitz ..www.aixtraware.de ..t: +49-2464-8851
Hüsgenstr. 33a .....d-52457 Aldenhoven .............f: +49-2464-905163