Item8239: Intermittent issues under FastCGI

Under Fast CGI, if I have two or more graphs in the same topic, only the 1st one will get rendered and that one will be used for every other instance.

-- AndrewJones - 13 Aug 2009

Could you run a test with two dot graphs, with the debug option enabled, and attach the debug log to this topic, or on pastebin? I expect it's a variable reuse issue, but not sure where the issue is.

-- GeorgeClark - 14 Aug 2009

Thanks for the quick response. I have attached the output from the debug log. This is from the released version, not including the changes you made earlier.

Hope it helps.

-- AndrewJones - 14 Aug 2009

The log looks normal - could you check a couple of things - it shows that 2 attachments previously existed and are cached - DirectedGraphPlugin_1.png and DirectedGraphPlugin_2.png. (The oldhash= newhash= message indicates if the dot input has changed and needs to be regenerated). If you access those files directly are they identical or are they correct. Also check the raw html to see if the two <img tags include the _1 and _2 images, or is one of the tags wrong? Finally if things look right, try clearing the browser cache.

Another thing you can do to get more information is to remove the attachments, or the hash files in the workarea. This will cause the plugin to regenerate the files, and with debug enabled, it will retain all of the intermediate files, so you can look at the input to and output from the GraphViz program. With those files, you can see exactly what the plugin extracted and processed for each image. That will also get a bit more debug information. As it stands now, because the hash codes match, the plugin never reruns GraphViz. So far from the log it appears to be working correctly.

-- GeorgeClark - 15 Aug 2009

After accessing the two files directly I can see that they are identical, so it looks like the problem is when the graphs are being created.

Checking the raw HTML shows that the two image tags are pointing a the right attachments.

I removed the hash files from the workarea and re-run the plugin. Where are all the intermediate files? They don't seem to be in the workarea or the pub directory.

I also attached the debug to this topic (directed-graph-debug2.txt).

Thanks.

-- AndrewJones - 17 Aug 2009

The working files should be left behind in the tmp directory. The plugin picks up the tmp setting from the Foswiki config. If it is not found there, it will try to use the system tmp path.

    if ( defined $Foswiki::cfg{TempfileDir} ) {
        $tempdir = $Foswiki::cfg{TempfileDir};
    }
    else {
        $tempdir = File::Spec->tmpdir();

I'm not really sure what is going on. I may need to try to set up FastCGI on one of my test servers.

-- GeorgeClark - 18 Aug 2009

I've installed fastcgi on a test server and installed a simple foswiki 1.0.6 install, the fastcgi engine, and DirectedGraphPlugin. I'm pretty sure that fastcgi is working - Not sure how to tell, other than seeing fastcgi initialize in the logs, and I used the ApacheConfigGenerator to build the configuration for Fastcgi.

I'm not seeing your symptoms. The System/DirectedGraphPlugin topic is correctly built, and all examples generated correctly. I then cut/pasted your examples from the error logs, and that worked as well.

My default fastcgi config for the virtual host is: FastCgiServer /var/www/foswiki106/bin/foswiki.fcgi -processes 3

Any other ideas what might be causing this? Are you running on apache on Linux? I installed www-apache/mod_fastcgi-2.4.6-r1 (Gentoo Linux)

-- GeorgeClark - 18 Aug 2009

After a lot of head scratching I think I have found the problem.

If I have the following two graphs in a topic, everything works fine under both under Fast CGI and plain CGI:

<dot>
digraph G {
node [fontname=Helvetica,shape=box,style=filled];
"ATest Framework" [fillcolor=yellow];
"TestSuite" [fillcolor="#d0d0ff"];
"Results" [fillcolor="#ffd0d0"];
"Global Context" [fillcolor="#d0ffd0"];
}
</dot>

---

<dot>
digraph G {
ranksep = 0.2;
node [fontname=Helvetica,shape=box];
compound=true
subgraph cluster0 {
node [fontname=Helvetica,fillcolor=white,color=black,style=filled];
n1 [label="gcc_ver\lmsvc_ver\lpvlib_home\lpvexamples_lib_dir\lmaxcore_home\lmaxview_home\limages_dir\l"];
#gcc_ver -> pvlib_home;
label = "Global Context\n(*.global_context and --set-global-attr)";
fillcolor = "#d0ffd0";
style=filled;
fontname=Helvetica;
}
{rank=same;
TestSuite;
test_list;
n1;
}
{rank=same;
atest;
atest_drivers;
}
{node [];
TestSuite [label="TestSuite\n(*.atest)",fillcolor="#d0d0ff",style=filled];
atest [style=filled, fillcolor=yellow, label="atest.py"];
atest_drivers [style=filled, fillcolor=yellow, label="Test Drivers\n(atest/z_\*.py)"];
atest_summary [style=filled, fillcolor=yellow, label="atest_summary.py"];
results [label="Results\n(*.atest_result)",fillcolor="#ffd0d0",style=filled];
log [label="Log Files\n(*.log)"];
tmp [style=dashed,label="Temporary Files\n(automatically deleted)"];
test_list [style=dashed,label="TestList"];
failed_tests [label="List of failed tests\n(or interesting tests)"];
}

TestSuite -> atest;
n1 -> atest [ltail=cluster0];

atest_drivers -> atest;
atest -> results;
atest -> log;
atest ->  tmp;
results -> atest_summary;
atest_summary -> failed_tests;
failed_tests -> test_list [style=dashed];
test_list -> atest [style=dashed];
}
</dot>

However, the following does not work properly under Fast CGI or Plain CGI, and the two graphs will be generated as the same:

<dot>
digraph G2 {
node [fontname=Helvetica,shape=box,style=filled];
"ATest Framework" [fillcolor=yellow];
"TestSuite" [fillcolor="#d0d0ff"];
"Results" [fillcolor="#ffd0d0"];
"Global Context" [fillcolor="#d0ffd0"];
}
</dot>

---

<dot>
digraph G3 {
ranksep = 0.2;
node [fontname=Helvetica,shape=box];
compound=true
subgraph cluster0 {
node [fontname=Helvetica,fillcolor=white,color=black,style=filled];
n1 [label="gcc_ver\lmsvc_ver\lpvlib_home\lpvexamples_lib_dir\lmaxcore_home\lmaxview_home\limages_dir\l"];
#gcc_ver -> pvlib_home;
label = "Global Context\n(*.global_context and --set-global-attr)";
fillcolor = "#d0ffd0";
style=filled;
fontname=Helvetica;
}
{rank=same;
TestSuite;
test_list;
n1;
}
{rank=same;
atest;
atest_drivers;
}
{node [];
TestSuite [label="TestSuite\n(*.atest)",fillcolor="#d0d0ff",style=filled];
atest [style=filled, fillcolor=yellow, label="atest.py"];
atest_drivers [style=filled, fillcolor=yellow, label="Test Drivers\n(atest/z_\*.py)"];
atest_summary [style=filled, fillcolor=yellow, label="atest_summary.py"];
results [label="Results\n(*.atest_result)",fillcolor="#ffd0d0",style=filled];
log [label="Log Files\n(*.log)"];
tmp [style=dashed,label="Temporary Files\n(automatically deleted)"];
test_list [style=dashed,label="TestList"];
failed_tests [label="List of failed tests\n(or interesting tests)"];
}

TestSuite -> atest;
n1 -> atest [ltail=cluster0];

atest_drivers -> atest;
atest -> results;
atest -> log;
atest ->  tmp;
results -> atest_summary;
atest_summary -> failed_tests;
failed_tests -> test_list [style=dashed];
test_list -> atest [style=dashed];
}
</dot>

The only difference between the two is the digraph G line at the top. In the first example, both graphs use digraph G and it works fine. In the second example, the first graph uses digraph G2 and the second uses digraph G3.

Do you get the same?

Other things that might help: I was previously using version 17659 (12 Dec 2008) on TWiki, and this works fine. I have not changed my GraphViz version or any of the other back end tools. I am using Red Hat Enterprise 4.

As I said, the same bug appears under plain CGI as well, so I changed the summary of this bug report.

Sorry, I should have been more thorough at the start, but I just assumed that it was a Fast CGI problem.

Thanks.

-- AndrewJones - 20 Aug 2009

Andrew, At this point I don't have a clue. I've pasted in your G2 / G3 failing example into a new topic, and it works just fine on my system under FastCGI. I've tried all four examples in a single topic, modifying some of the colors to force updates. Still everything is clean. I'm not sure what to try next.

If the content of the two files is the same, then all I can think of is maybe there is a files issue or some issue passing the files to the Graphviz dot command. If you run with debug enabled and then examine the tmp directory, you should see the input and output files generated from each execution of Graphviz.

• Are the two input files identical?
• Are there any errors in the temporary files

I'm running Graphviz 2.20.3 with the plugin Version 1.5 as downloaded from Foswiki.org using the bin/configure installer.

-- GeorgeClark - 21 Aug 2009

George, do you use $attachPath? IIRC I got the problem that Andrew describes too but it went away after I enabled $attachPath.

-- AndreasVoegele - 25 Aug 2009

Andreas, I did not set the attachPath in my FastCGI test. I installed a new Foswiki 1.0.6, and didn't touch anything in the DGP config except to edit and save the configuration once so that the switchboard entry would get created. It's a completely default install.

-- GeorgeClark - 26 Aug 2009

Sorry I did not update this. I set attachPath and it works fine, so must be a problem with the Foswiki attach API.

The problem was temperamental, I couldn't reproduce it reliably. It happened with both FastCGI and plain CGI, happened with any topic that had two or more graphs.

When I find time I will see if I can provide a reliable test case, or possibly a unit test.

-- AndrewJones - 29 Sep 2009

---

I've not observed anything like this in my testing. I did make a couple of changes to hopefully improve initialization in a persistent perl environment like mod_perl and FCGI. I'm close to a new release. When I upload the new release I'll close this, If you can recreate the issue on the new code please reopen it.

Note that the only time I've seen the "wrong graph" displayed is when the order of multiple graphs change on the page. Filenames get reused, but the browser cache might not load the modified files.

-- GeorgeClark - 01 Oct 2009

I've set this to closed. Please reopen if you are still running into this issue with the new version.

---

Turns out the issue was down to compiled regex's, similar to the EditTablePlugin's problems.

-- AndrewJones - 06 Dec 2010

Commenting is disabled while not logged in