Wrong Umlaut Encoding in Applications/ClassificationApp.ClassifiedTopic

The encoding of the category names in Applications/ClassificationApp.ClassifiedTopic seems to be handled incorrectly.

My Umlauts look like this: "Abbr�che".

Is there anything I can do about that, or is it a bug? As far as I have seen, 8-bit-chars in other places of my wiki are looking fine. There are also no issues with umlauts in category names in other places than inside that form.

-- BirgitNietsch - 04 Jun 2015

What is the encoding for your site set to?

Foswiki 1.1.x See LocalSite.cfg {Site}{CharSet}
Foswiki 1.2.0 beta see LocalSite.cfg {Store}{Encoding}

Some of the extensions might expect a utf-8 core. Foswiki 1.2 is making very major changes in this area for greatly improved utf-8 / unicode support.

-- GeorgeClark - 04 Jun 2015

$Foswiki::cfg{Site}{CharSet} = 'iso-8859-15';

I expect I would need to convert all of my Topics if I changed that into utf-8?

-- BirgitNietsch - 05 Jun 2015

yes. It's a major change to move to utf-8, but will probably be strongly recommended prior to migration to Foswiki 1.2.0. We just made the conversion last night on Foswiki.org. I'm not really all that familiar with ClassificationPlugin and we don't have it installed on Foswiki.org.

I asked MichaelDaum about this issue, his comment on IRC:

MichaelDaum: can't repro the bug works fine here with an utf8 site charset

As this seems to be related to Forms, it could very well be CGI related. Foswiki uses CGI:: methods to create the HTML for forms, and we've been discovering that there are major issues related to I18N character sets and form data, and Foswiki internals. This was a can of worms that pushed us into a major internal change for Beta 2. See some of the discussion in Tasks.Item13378. The bad news is that there is no quick fix here I don't think. But Foswiki 1.2.0 will be a huge improvement in overall character set handling.

-- GeorgeClark - 05 Jun 2015

I see. For the conversion to utf-8: CharsetConverterContrib, I presume?

Is it recommend to do the conversion on Foswiki 1.1.9., or would it be safer to live with this single ugly form field and wait for 1.2.0 to become the next stable version?

This (rather cosmetical?) bug is the only encoding problem I have come across up to now.

-- BirgitNietsch - 05 Jun 2015

If you are having character set issues, it's recommended to convert to utf-8. You can see a bit more information on the process we just went through for Foswiki.org in Utf8MigrationConsiderations. The recent changes we made to CharsetConverterContrib have not yet been released, so best to hold off until all that we learned during the f.o migration makes it into CharsetConverter.

Note we just ran into a corruption issue on 1.1.9 with utf8 and wysiwyg, so that is still being investigated.

-- GeorgeClark - 05 Jun 2015