Page-Cache doesn't store INCLUDE macros

I believe I've discovered a shortcoming, if not a bug in the Page-Cache feature (under the "Tuning" configuration tab) of Foswiki. It appears the when recalculating the contents of a <dirtyarea> tag added to a page via an %INCLUDE%, Foswiki doesn't restore any macro values that might have been supplied to the included fragment at the time of the %INCLUDE%. Here's an example:

In !TopicA, place the following code:

Demonstration of a problem with page caching, !%INCLUDE%, !%SEARCH% and <dirty area>.

   * Set MYPREFIX=Topic

This page is built using !%INCLUDE% to include the results of a !%SEARCH% on the name of topics.  The desired search text is passed into the included section as a macro value given in the !%INCLUDE%.  The dynamic content of the include is enclosed in a <dirtyarea> tag so that it will be rebuilt each time the page is retrieved rather than cached.  When page caching is turned on, the macro value is not available at the time the dirty area is evaluated and the search fails.

Here are the search results generated via the included section:
---
%INCLUDE{"TopicB" section="SearchSection" PREFIX="%MYPREFIX%"}%
---

Here's the same search performed inline:

---
<dirtyarea>
%SEARCH{"^%MYPREFIX%.*$" scope="topic" type="regex" nonoise="on" format="$web.$topic$percntBR$percnt"}%
</dirtyarea>
---

In !TopicB place this code:

This page accompanies TopicA to demonstrate a problem with page caching, !%INCLUDE%, !%SEARCH% and <dirty area>.

---
%INCLUDE{"%TOPIC%" section="SearchSection" PREFIX="Topic"}%
---

<verbatim>
%STARTSECTION{SearchSection}%
<dirtyarea>

Value of !%PREFIX% Macro = %PREFIX{default="(No value is set)"}%

%SEARCH{"^%PREFIX%.*$" scope="topic" type="regex" nonoise="on" format="$web.$topic$percntBR$percnt"}%
</dirtyarea>
%ENDSECTION{SearchSection}%

&lt;/verbatim&gt;

As described in the example code, when the page-cache preference is turned on, the value of the %PREFIX% macro in the included section is not set, and the search fails.

Am I misunderstanding how caching should work, or is this a problem with the caching module?

-- PaulMerchantJr, 4 Nov 2015

Short-term fix: replace INCLUDE with DBCALL

Here the findings.

INCLUDE and DBCALL process named parameters (such as PREFIX in your example) differently with regards to the order when dirty areas are processed.

INCLUDE:

1. take out dirty areas
2. add named params to preference stack
3. expand common tags (including named parameters added in step 2)
4. put back dirty areas

(sidenote: step 3 does not use Foswiki::expandMacros but implements its own way of expanding macros in Foswiki::Macros::INCLUDE thus duplicating most of the logic in Foswiki::expandMacros. ... but thats a different issue of redundant code)

DBCALL:

1. insert all named parameters ... and only these
2. take out dirty areas
3. call Foswiki::Func::expandCommonVariables ... which boils down to Foswiki::expandMacros
4. put back dirty areas

As you can see steps 1 and 2 are different in both macros. Therefore in step 4 INCLUDE puts back the unexpanded named parameters (PREFIX) whereas DBCALL has expanded it to its value before already.

Further note that in any case included sections that contain dirty areas will then be evaluated in the context of the including topic, not the included topic. That's because the overall page and all of its macros are expanded completely on the initial cache hit, leaving behind the unprocessed dirty areas. On the second cache hit there is no trace where these dirty areas came from; they are processed as is: no context switch is performed as it is the case when computing an INCLUDE/DBCALL.

For instance, the value of %TOPIC% in a dirty area is different:

• without page caching it is the name of the included topic,
• with page caching it is the name of the base topic.

Basically, I'd consider the behaviour of DBCALL to be "more correct" than that of INCLUDE with respect to the order when named parameters are inserted into the included text fragment. I'd consider it a bug in INCLUDE. The fix is to first expand all named parameters and then take out dirty areas.

-- MichaelDaum - 05 Nov 2015

Commenting is disabled while not logged in