[dokuwiki] Re: http://wiki.splitbrain.org/wiki:compare
- From: Harry Fuecks <hfuecks@xxxxxxxxx>
- To: dokuwiki@xxxxxxxxxxxxx
- Date: Thu, 6 Oct 2005 10:04:59 +0200
On 10/5/05, Joe Lapp <joe.lapp@xxxxxxxxx> wrote:
> >From: Harry Fuecks <hfuecks@xxxxxxxxx>
> > Hitting a memory limit on displaying that page: Fatal error: Allowed
> > memory size of 8388608 bytes exhausted
>
> We really ought to make the parser use a nested output buffer -- most likely
> culprit, anyway. String cat'ing creates enormous amounts of string objects
> and new memory allocations. Excessive memory consumption is the #1 cause of
> poor server performance (peak capacity), so this is a good clue for us.
>
> ~joe
In this case the error message was coming from the handler - that
probably means the number of instructions the parser had produced
(which get stored in an array in the handler) had gobbled up the
memory. With the wiki:compare page, that makes some sense, as there's
alot of formatting plus it's fairly long.
This is probably a scenario that's worth preventing / controlling (at
least with some configuration option). To me think it would make sense
to force users to break up pages into multiple pages, if they start
getting big - the edit form returns an error to the user if it's too
big and the page isn't saved.
So how to check for "too big". A simple solution might be a limit on
the size of the raw text of the page - unfortunately that's not a very
accurate measure. The wiki:compare page is only 42k - would expect it
to be possible for a larger page (in terms of raw text) to consume
less memory in parsing if it has less wiki markup.
A more accurate approach would be to check memory usage a number of
times during parsing. PHP provides the memory_get_usage() function but
it's not always available - PHP installation specific (see notes on
workarounds - for Windows in particular, the most effective involve
calling an external program - not pretty / fast). It could be used to
compare current memory use with either PHP's configured limit or some
Dokuwiki specified value, during parsing - if the limit is reached,
parsing halts and the user is forced to break up the page.
An alternative approach might be to use a file instead of array in the
handler to write instructions, keeping only a small part of the data
in memory (fseek() etc). This will be tough to implement but doable
but may have a significant performance impact at "render" time,
especially when you have a slow disk. A tricky part will be how to
identify a single parser instruction in the file, given the file might
also contain wiki markup. The benefit, at least in theory, is this
effectively removes any memory limits - something equivalent to a swap
file.
--
DokuWiki mailing list - more info at
http://wiki.splitbrain.org/wiki:mailinglist
Other related posts:
