Item13931: ENCODING quotes sometimes fails with links

VarENCODE using type "quotes" sometimes fails when the encoded text includes an html link with double quotes. This was more of an issue with older version of WYSIWYG editor which created html links with double quotes. However, this is a case that still may be worth looking at.

Test: Define a variable that includes an html link and then apply VarENCODE to it:

• Local TESTTEXT = Here's some text with some "quotes" but also a troublesome link that will break VarENCODE (and some of this text will be lost).

Here%27s%20some%20text%20with%20some%20%22quotes%22%20but%20also%20a%20troublesome%20%3ca%20href%3d%22http://translate.foswiki.org/Sandbox/WebHome

-- LynnwoodBrown - 22 Jan 2016

I don't see how this can ever work. The issue is that with "inside-out left-right" expansion, TESTTEXT is fully expanded first, At that point, the parser tries to figure out the arguments to ENCODE, but the quotes completely break the parser. It misses the type="quotes" and defaults to URL encoding.

If this is intended to work, the parser probably has to be redesigned to fully parse the operands at each level. and then do the inside expansion. And that would break building macros with concatenation.

Here is the debug trace of the macro parser:

QUEUE:''
      '%'
      'ENCODE{"'
      '%'
      'TESTTEXT'
      '%'
      '" type="quote"}'
      '%'
      '
'
PROCESSING  
QUEUE:'%'
      'ENCODE{"'
      '%'
      'TESTTEXT'
      '%'
      '" type="quote"}'
      '%'
      '
'
PROCESSING % 
CONSIDER 
QUEUE:'ENCODE{"'
      '%'
      'TESTTEXT'
      '%'
      '" type="quote"}'
      '%'
      '
'
PROCESSING ENCODE{" 
QUEUE:'%'
      'TESTTEXT'
      '%'
      '" type="quote"}'
      '%'
      '
'
PROCESSING % 
CONSIDER %ENCODE{"
QUEUE:'TESTTEXT'
      '%'
      '" type="quote"}'
      '%'
      '
'
PROCESSING TESTTEXT 
QUEUE:'%'
      '" type="quote"}'
      '%'
      '
'
PROCESSING % 
CONSIDER %TESTTEXT
POP TESTTEXT
EXPANDED TESTTEXT -> Here's some text with some "quotes" but also a troublesome <a href="%SCRIPTURL{view}%/Sandbox/WebHome" title="Link to sandbox web">link</a> that will break 
QUEUE:'Here's some text with some "quotes" but also a troublesome <a href="'
      '%'
      'SCRIPTURL{view}'
      '%'
      '/Sandbox/WebHome" title="Link to sandbox web">link</a> that will break '
PROCESSING Here's some text with some "quotes" but also a troublesome <a href=" 
QUEUE:'%'
      'SCRIPTURL{view}'
      '%'
      '/Sandbox/WebHome" title="Link to sandbox web">link</a> that will break '
PROCESSING % 
CONSIDER Here's some text with some "quotes" but also a troublesome <a href="
QUEUE:'SCRIPTURL{view}'
      '%'
      '/Sandbox/WebHome" title="Link to sandbox web">link</a> that will break '
PROCESSING SCRIPTURL{view} 
QUEUE:'%'
      '/Sandbox/WebHome" title="Link to sandbox web">link</a> that will break '
PROCESSING % 
CONSIDER %SCRIPTURL{view}
POP SCRIPTURL
EXPANDED SCRIPTURL -> http://foswiki.mysite.com
QUEUE:'/Sandbox/WebHome" title="Link to sandbox web">link</a> that will break '
PROCESSING /Sandbox/WebHome" title="Link to sandbox web">link</a> that will break  
QUEUE:'" type="quote"}'
      '%'
      '
'
PROCESSING " type="quote"} 
QUEUE:'%'
      '
'
PROCESSING % 
CONSIDER %ENCODE{"Here's some text with some "quotes" but also a troublesome <a href="http://foswiki.mysite.com/Sandbox/WebHome" title="Link to sandbox web">link</a> that will break " type="quote"}
POP ENCODE
EXPANDED ENCODE -> Here%27s%20some%20text%20with%20some%20%22quotes%22%20but%20also%20a%20troublesome%20%3ca%20href%3d%22http://foswiki.mysite.com/Sandbox/WebHome
QUEUE:'
'
PROCESSING 
 
QUEUE:'Here'
      '%'
      '27s'
      '%'
      '20some'
      '%'
      '20text'
      '%'
      '20with'
      '%'
      '20some'
      '%'
      '20'
      '%'
      '22quotes'
      '%'
      '22'
      '%'
      '20but'
      '%'
      '20also'
      '%'
      '20a'
      '%'
      '20troublesome'
      '%'
      '20'
      '%'
      '3ca'
      '%'
      '20href'
      '%'
      '3d'
      '%'
      '22http://foswiki.mysite.com/Sandbox/WebHome
'
...

-- GeorgeClark - 08 Feb 2016

I wonder if a solution could be some sort of a "expand" option: %ENCODE{"$percentTESTTEXT$percent" type="quotes" expand="yes"}%

• expand option would decodeFormatTokens() and then expandCommonVariables()
• quote option would then encode the quotes in the expanded string

Anyway, this would need a feature request.

-- GeorgeClark - 08 Feb 2016

Commenting is disabled while not logged in