Item14105: Could not create statistics for all webs using cron

Metadata

• Foswiki: 2.1.2
• Perl: 5.14.2
• Apache: 2.2.22
• RCS: 5.8.1

Problem

I want to update the statistics on a regular basis using cron. I executed the following call for testing on the command line:

user@host:/srv/foswiki/bin$ sudo su www-data -c './statistics -autocreate 1 -subwebs 1 -user (UserName|WikiGuest|AdminUser)'

with and without -user.

• I get the following error message for WikiGuest|AdminUser:

  [Mon Jul  4 13:38:56 2016] statistics: Use of uninitialized value in string eq at /srv/foswiki/lib/Foswiki/User
  s/TopicUserMapping.pm line 574, <DATA> line 558.
  Foswiki: Create Usage Statistics
  !Do not interrupt this script!
  (Please wait until page download has finished)
  * Statistics for Jul 2016
  * Reporting on Web1 web
    - view: 1266, save: 1, upload: 0
    - top view: 370 Topic1
    - top contributor:   1 Main.UserName2
  !  - ERROR: no permission to CHANGE statistics topic in Web1
  * Reporting on Web1/SubWeb web
    - view: 0, save: 0, upload: 0
  !  - ERROR: no permission to CHANGE statistics topic in Web1/SubWeb
  [...]
  End creating usage statistics

• Using an user from the admin group results in the following:

  Foswiki: Create Usage Statistics
  !Do not interrupt this script!
  (Please wait until page download has finished)
  * Statistics for Jul 2016
  * Reporting on WebA web
    - view: 618, save: 0, upload: 0
    - top view:  20 WebHome
    - Topic WebStatistics updated
  * Reporting on WebB web
    - view: 22, save: 0, upload: 0
    - top view:   9 WebHome
    - Topic WebStatistics updated
  * Reporting on WebC web
    - view: 908, save: 0, upload: 0
    - top view: 179 TopicC
  [Mon Jul  4 13:41:46 2016] statistics: Automatic recovery: breaking lock for /srv/foswiki/data/WebC/WebStatistics.txt at /srv/foswiki/lib/Foswiki/Store/Rcs/RcsWrapHandler.pm line 526.
  RCS: rcs  -l %FILENAME|F% failed:  at /srv/foswiki/lib/Foswiki/Store/Rcs/RcsWrapHandler.pm line 539.
  Foswiki detected an internal error - please check your Foswiki logs and webserver logs for more information.
  
  RCS: rcs  -l %FILENAME|F% failed: 

  The according error.log entry is:

  | 2016-07-04T13:41:46+00:00 warning | RCS: rcs  -l %FILENAME&vbar;F% failed:  at /srv/foswiki/lib/Foswiki/Store/Rcs/RcsWrapHandler.pm line 539.

  When using rlog to look at the history of the file, I get the following error:

  rlog: ../data/Documentation/WebStatistics.txt,v:4768: missing author
  rlog aborted

  The according line in the file is the following:

   4766 next    1.382;
   4767 1.382
   4768 date    70.01.01.00.00.00;      author ;        state Exp;
   4769 branches;
   4770 next    1.381;
   4771 1.381

  It seems that the date and author values are somehow broken. Change the values of the broken history file to something in between the other revisions, e.g.

   5958 1.383
   5959 date    2007.01.29.21.03.42;    author TWikiGuest;      state Exp;
   5960 branches;
   5961 next    1.382;
   5962 
   5963 1.382
   5964 date    2007.01.29.16.00.00;    author TWikiGuest;      state Exp;
   5965 branches;
   5966 next    1.381;
   5967 
   5968 1.381
   5969 date    2007.01.29.15.52.18;    author TWikiGuest;      state Exp;
   5970 branches;
   5971 next    1.380;

  Now, I can build statistics for all webs using my user account. When using something useful like WikiGuest, I get again the

  ERROR: no permission to CHANGE statistics topic in

  messages.

What parameters should I use to get this running without using my user account as -user parameter?

Comments

-- Main.Ehj52N - 18 Aug 2016 - 09:10

I now get the following error

ERROR: no permission to CHANGE statistics topic in

for my user, too, when executing the following command line using cron

((echo "\nStart: $(date)\n\n" && cd /srv/foswiki/bin && ./statistics -u MyUser -subwebs 1)  >> /tmp/foswiki-statistics.log)

. The output in the log:

Start: Thu Aug 18 06:42:01 UTC 2016


Foswiki: Create Usage Statistics
!Do not interrupt this script!
(Please wait until page download has finished)
* Statistics for Aug 2016
* Reporting on ExampleWeb web
  - view: 978, save: 1, upload: 0
  - top view:  78 WebHome
  - top contributor:   1 Main.MyUser
!  - ERROR: no permission to CHANGE statistics topic in ExampleWeb
End creating usage statistics

I can create the statistics manually for all webs when using the web interface from /System/SiteToolStatistics. Hence, I am confused because my user seems to have the right permissions from web but not from command line.

-- Main.Ehj52N

Where did you find the example of using a string like (UserName|WikiGuest|AdminUser) for the -user argument. You should pick one. Typically Statistics should be run as the admin user, which ensures that they have full access to the WebStatistics topics. Allowing guests or other users to update statistics is rather unusual. I'm pretty sure the default user for cli scripts is admin, so you should not need the -user parameter.

As far as the corrupted history, I'm guessing that the very unusual (UserName|WikiGuest|AdminUser) user string somehow broke the topic history. Probably the easiest thing to do is to just remove the RCS file for your WebStatistics topic. We recommend doing that occasionally anyway. RCS processing with 1000's of revisions is extremely slow.

-- GeorgeClark - 18 Aug 2016

I also believe the problem is with your command syntax. To run the statistics script from cli, I'd use simply:

sudo -u www-data ./statistics -autocreate 1 -subwebs 1

And for the cron job syntax, assuming you create the cron job for www-data user, would be (as per System.SiteToolStatistics#Configuring_for_automatic_operation):

0 0 * * * (cd /path/to/bin; ./statistics -subwebs 1 >/dev/null 2>&1)

One problem that may have triggered your errors above is that recent versions of Apache don't actually allow you to su to www-data user, however you can run a command as that user using sudo -u www-data command...

-- LynnwoodBrown - 18 Aug 2016

• Regarding the -u|user parameter: I was playing around with the command and when being used, the script worked but as I stated before, it broke.
• Regarding the corrupted history: that issue seems to be fixed, because the statistics can be generated from the WebUI.
• @ LynnwoodBrown Your first command resulted in the error message I posted in my first comment: !  - ERROR: no permission to CHANGE statistics topic in ExampleWeb
  The permission for the WebStatistics topic are always set to default.
• What additional information do you need or may help here?

-- Ehj52N - 19 Aug 2016

I am afraid that I really don't have any ideas where to go from here. The Command scripts should always run as the admin user by default, which will always have authorization to read / save all topics. If something is not working in that area and we have been unable to recreate it, we'll have to work on some debugging print statements in the perl code to try to figure out why your shell scripts are not getting admin authority.

-- GeorgeClark - 25 Sep 2016

Closing due to inactivity.

-- MichaelDaum - 12 Oct 2020

Commenting is disabled while not logged in