[dokuwiki] Re: about the Dokuwiki Lexer/Parser/Handler improvements....

> > Specifically I'm referring to inc/parser/lexer.php,
> > inc/parser/handler.php, inc/parser/renderer.php I have some ideas on
> > potential improvements
> 
> Hi Mike,
> Can you tell us more about your ideas ?
> 
> Regards
> cyrille.

I am interested in achieving 2 goals in the short term:
* improve the the lexing/parsing performance 
* prototype a parser/lexer based on javascript

Towards those goals, here is what I'm considering:
* take the code from Doku_LexerParallelRegex, Doku_Lexer->addPattern* 
functions, and all the Doku_Parser related classes 
   and move them into a Doku_Generate_Tokenizer class that would contain the 
tokenizer logic (doesn't need invokation on each request)
* modify Doku_Lexer to accept the data structure generated from 
Doku_Generate_Tokenizer as input

I am thinking that a common data interchange format could be used to store this 
lexer data. JSON seems like a good choice because it's built into PHP and can 
serialize/deserialize those ds pretty fast. JSON is also nice in that it would 
allow me to import the same lexing data into javascript very easily, and then 
all that would be required is to port the remainder of the lexer include the 
stack and the traversal. But with the static parts of the lexer/parser moved 
into Doku_Generate_Tokenizer, it's only a few hundred lines instead of a few 
thousand.

again, these are just ideas but I'd love to get feedback from you folks on if 
you see this being feasible, and offer the possibility of contributing this 
back to the Dokuwiki community if it's desired. Didn't want to start coding it 
and then find out that a super awesome refactorign of the lexer/parser/renderer 
were underway. :)

Thoughts? comments/suggestions/feedback/concerns highly appreciated!

-Mike

Other related posts: