Re: Data-dependent slowdown in loop involving io.lines()
- From: Tudor Bosman <tudorb@xxxxxxxxx>
- To: luajit@xxxxxxxxxxxxx
- Date: Fri, 07 Nov 2014 16:09:31 +0000
On Fri Nov 07 2014 at 4:20:02 AM Mike Pall <mike-1411@xxxxxxxxxx> wrote:
>
> A sparse hash inevitably shows worst case behavior like this. It's
> just a matter of trying hard enough to find such an example (with
> either Lua or LuaJIT).
>
The point is that we didn't try hard enough; the example I gave is
contrived, but that's only because I didn't want to show our full pathnames
in the example (also, because I didn't want to attach the original 98MB
file).
Lua doesn't make a distinction between symbols and strings, which
> makes it rather difficult to fully cater for the typical needs of
> both types (which are quite different).
>
Couldn't LuaJIT, for example, do full (dense) hashing for strings of length
of less than 100 or so bytes, and assume that all longer strings are data
and therefore likely to be unique (and intern them without any hashing /
comparison check)? Is there an assumption that two strings are equal if
*and only if* their interned ids are the same?
-Tudor.
Other related posts:
