Item13388: The %QUERY{...}% regex searches - fails on utf8 texts.
Priority: Normal
Current State: Closed
Released In: 2.0.0
Target Release: major
Applies To: Engine
Component:
Branches: master
The %QUERY% fails when for any
QuerySearch where the search is utf8 and
- the query contains a rexex with character class, such
:upper:
or the simple \w
- contains the ignore case modifier, e.g.
(?i}utf8text
- or contains the
lc
uc
for utf8
How to reproduce
Download the
attachment. It is an raw topic file and contains a demo.
Tested on the trunk (commit 9fde37a000da7d8e1e3dfe4ab4cbe5871d09fec2).
Also, other errors
Because don't want spam the Tasks web with many utf8 related errors, the demo-topic contains other errors too, such
- QUERY type
json
fails for utf8,
- any CALC command from the spreadheet plugin manipulating utf8-texts fails
- search/regex fails
- EDITTABLE plugin causes some strange errors and so on...
--
JozefMojzis - 30 Apr 2015
I updated the topic to be (
i hope ) self-contained. (my test web has many testing topics and many testing variables). The previous version
- used some %VARIABLES% what i has defined in the WebPreferences - now changed
- and in the QUERY tried to query another topic's preferences value - now corrected to query itself.
So,
GeorgeClark please download the topic and test it, not it should show: 15 errors. For ensure, i will attach the pdf version of the print... The page is intended to test clean
distro
testing (e.g. patternskin, etc..) so the Javascript could fail in other environments (i'm not an Javascript developer). It is tested with Mac's Safari and with Firefox/Firebug - and with fresh github clone. (Would be nice if someone could give some advices, how to make the JS part more universal.)
--
JozefMojzis - 04 May 2015
Ad comment from IRC
We'll probably document your Item13388 as a known issue.
Agree. IMHO we need release 1.2 :). Foswiki is now usable for editing UTF8 texts. Need document the issues:
- any %QUERY% or %SEARCH% will fail
- for
regex
searches containing character classes such \w
or [[:upper:]]
and such e.g. the case insensitive operator too (?i)
- for
lc
and uc
case conversions
- the string functions of the SpreadSheetPlugin doesn't works with utf8
- the %EDITABLE% macro somewhat broke the utf8 topic text... (only some characters). Maybe this issue got fixed with the Item13391 - not tested yet.
--
JozefMojzis - 04 May 2015
The example given is working as expected, on unicode core. Closing.
--
CrawfordCurrie - 19 May 2015