Re: Low-overhead profiling for LuaJIT 2.1

  • From: <steve@xxxxxxxxxxxxxxx>
  • To: <luajit@xxxxxxxxxxxxx>
  • Date: Mon, 24 Jun 2013 20:43:20 +0100

So xperf profile counters works by walking the call-stack from (AFAIK) kernel of each thread which is executed every interval. The xperf system is extensible, so it would be possible to create a provider which gave a snapshot of the lua state's for a process at a similar interval, and with similar performance.


There is some documents for the kernel side of windows development (e.g. ddk) that explain how to generate these logs.

http://wychi.wordpress.com/2009/10/21/create-your-own-etw-provider-and-log-it-by-xperf/

http://msdn.microsoft.com/en-us/library/windows/desktop/dd996930(v=vs.85).aspx



The difficulties will be as follows.
a) joining the windows stack and the lua stack. Probably the most informative way of finding out what is going on is to see from both modes.
b) Having some way of marking the jit code as meaningful
c) Ensuring that the etl logs include the loaded vm "symbolic information" so the lua stacks can be decoded. d) Producing a viewer for the results - I suspect the xperfview views are not extensible (given they don't support wpp logging data) so although you get a performent data collector, you would still need to decode the logs and render the analysis extra.

I would expect the following data points to log.
a) function address pointers to identify which lua functions (bc) are loaded where.
b) jit => function mappings
c) frame id (e.g. 1,2,3,4,5,6,...) + function id (e.g from a, b ).

This is way outside my spare time limits to implement.

Hope this helps ....

Steve



-----Original Message----- From: agentzh
Sent: Monday, June 24, 2013 8:05 PM
To: luajit@xxxxxxxxxxxxx
Subject: Re: RFC: Low-overhead profiling for LuaJIT 2.1

Hello!

On Fri, Jun 21, 2013 at 7:37 AM, Dan Tull wrote:
I'd really like to do something along these lines:
  http://dtrace.org/blogs/brendan/2011/12/16/flame-graphs/

https://github.com/agentzh/nginx-systemtap-toolkit/blob/master/ngx-sample-lua-bt

However, I don't know of a way to do it across the platforms I need to support (Mac and Windows and iOS). The dtrace techniques used for the above involves ustack helpers
which I gather are broken on OS X. I haven't even dug into Windows yet.


It seems that the Windows folks are already doing (C-level) flame
graphs with Xperf:

http://randomascii.wordpress.com/2013/03/26/summarizing-xperf-cpu-usage-with-flame-graphs/

Not sure if it's possible or easy enough to add support for Lua-land
backtraces though :) I haven't touched Windows for anything serious
for years already.

Best regards,
-agentzh


Other related posts:

  • » Re: Low-overhead profiling for LuaJIT 2.1 - steve