[codeface] Re: AW: Re: AW: Re: AW: Re: AW: AW: Re: AW: Re: AW: Re: Unit tests - Exception in mailing list analysis executing batch.R
- From: Mitchell Joblin <joblin.m@xxxxxxxxx>
- To: codeface@xxxxxxxxxxxxx
- Date: Thu, 12 Feb 2015 20:32:05 +0100
On Thu, Feb 12, 2015 at 8:03 PM, Matthias Gemmer
<matthias.gemmer@xxxxxxxxxxxxxxxxxxxx> wrote:
>
>>________________________________________
>>Von: codeface-bounce@xxxxxxxxxxxxx <codeface-bounce@xxxxxxxxxxxxx> im Auftrag
>>von Matthias Dittrich <matthi.d@xxxxxxxxxxxxxx>
>>Gesendet: Donnerstag, 12. Februar 2015 19:06
>>An: codeface@xxxxxxxxxxxxx
>>Betreff: [codeface] Re: AW: Re: AW: Re: AW: AW: Re: AW: Re: AW: Re: Unit
>>tests - Exception in mailing list analysis executing batch.R
>
>>Hi,
>
>>On 12.02.2015 18:39, Matthias Gemmer wrote:
>>>> ________________________________________
>>>> Von: codeface-bounce@xxxxxxxxxxxxx <codeface-bounce@xxxxxxxxxxxxx> im
>>>> Auftrag von Mitchell Joblin <joblin.m@xxxxxxxxx>
>>>> Gesendet: Freitag, 30. Januar 2015 23:28
>>>> An: codeface@xxxxxxxxxxxxx
>>>> Betreff: [codeface] Re: AW: Re: AW: AW: Re: AW: Re: AW: Re: Unit tests -
>>>> Exception in mailing list analysis executing batch.R
>>>> On Fri, Jan 30, 2015 at 11:03 PM, Matthias Gemmer
>>>> <matthias.gemmer@xxxxxxxxxxxxxxxxxxxx> wrote:
>>>>> ________________________________
>>>>>> [codeface] Re: AW: AW: Re: AW: Re: AW: Re: Unit tests - Exception in
>>>>>> mailing list analysis executing batch.R
>>>>>> codeface-bounce@xxxxxxxxxxxxx
>>>>>> im Auftrag von
>>>>>> Mitchell Joblin <joblin.m@xxxxxxxxx>
>>>>>> Fr 30.01.2015 22:00
>>>>>> An:
>>>>>> codeface@xxxxxxxxxxxxx;
>>>>>> On Jan 30, 2015 9:26 PM, "Matthias Gemmer"
>>>>>> <matthias.gemmer@xxxxxxxxxxxxxxxxxxxx> wrote:
>>>>>>>
>>>>>>> ________________________________
>>>>>>> Von: codeface-bounce@xxxxxxxxxxxxx <codeface-bounce@xxxxxxxxxxxxx> im
>>>>>>> Auftrag von Matthias Gemmer <matthias.gemmer@xxxxxxxxxxxxxxxxxxxx>
>>>>>>> Gesendet: Freitag, 30. Januar 2015 20:35
>>>>>>> An: codeface@xxxxxxxxxxxxx
>>>>>>> Betreff: [codeface] AW: Re: AW: Re: AW: Re: Unit tests - Exception in
>>>>>>> mailing list analysis executing batch.R
>>>>>>>
>>>>>>>
>>>>>>> There is one error "left":
>>>>>>>
>>>>>>>
>>>>>>> ======================================================================
>>>>>>> ERROR: integration.test_exampleprojects
>>>>>>> (unittest.loader.ModuleImportFailure)
>>>>>>> ----------------------------------------------------------------------
>>>>>>> ImportError: Failed to import test module:
>>>>>>> integration.test_exampleprojects
>>>>>>> Traceback (most recent call last):
>>>>>>> File "/usr/lib/python2.7/unittest/loader.py", line 252, in _find_tests
>>>>>>> module = self._get_module_from_name(name)
>>>>>>> File "/usr/lib/python2.7/unittest/loader.py", line 230, in
>>>>>>> _get_module_from_name
>>>>>>> __import__(name)
>>>>>>> File
>>>>>>> "/home/matthias/codeface/codeface/test/integration/test_exampleprojects.py",
>>>>>>> line 25, in <module>
>>>>>>> from .example_projects import example_project_func
>>>>>>> File
>>>>>>> "/home/matthias/codeface/codeface/test/integration/example_projects.py",
>>>>>>> line 17, in <module>
>>>>>>> from .gitproject import GitProject
>>>>>>> File
>>>>>>> "/home/matthias/codeface/codeface/test/integration/gitproject.py",
>>>>>>> line 29
>>>>>>> def __str__(self):
>>>>>>> ^
>>>>>>> IndentationError: expected an indented block
>>>>>>>
>>>>>>> ----------------------------------------------------------------------
>>>>>>> Ran 3 tests in 0.086s
>>>>>>>
>>>>>>> FAILED (errors=1)
>>>>>>>
>>>>>>>
>>>>>>> Sorry, the syntax error above was made on my side.
>>>>>>>
>>>>>>> Unit tests raise:
>>>>>>>
>>>>>>>
>>>>>>> a) File "/home/matthias/codeface/codeface/cluster/cluster.py", line
>>>>>>> 1388,
>>>>>>> in performAnalysis
>>>>>>>
>>>>>>> emitStatisticalData(cmtlist, id_mgr, logical_depends, outdir,
>>>>>>> releaseRangeID,\
>>>>>>> UnboundLocalError: local variable 'logical_depends' referenced before
>>>>>>> assignment
>>>>>>>
>>>>>>> b) Exception: Command
>>>>>>> '/home/matthias/codeface/codeface/R/cluster/persons.r --loglevel devinfo
>>>>>>> --logfile /tmp/codeface_test_projectd3XgLf/.git/log.R.r0 -c
>>>>>>> /home/matthias/codeface/codeface_testing.conf -p
>>>>>>> /tmp/codeface_test_projectd3XgLfbYdrIG
>>>>>>> /tmp/codeface_test_projectd3XgLf/.git/results/codeface_test_projectd3XgLf/proximity/v0_release-v1_release
>>>>>>> 189' failed with exit code 1.
>>>>>> I can't be certain but to me it looks like you are not running the latest
>>>>>> code. Can you check that you fetched the latest and merged it to your
>>>>>> local
>>>>>> branch. Can you tell me the has of the latest commit you see on your
>>>>>> branch?
>>>>> The hash is 88db2d5.
>>>> That's good, you are on the latest. Please edit the project
>>>> configuration file (in the codeface/conf directory) by changing the
>>>> "tagging" line to "proximity" or "feature" instead of "tag". Please
>>>> let me know if those work. For example, the Linux conf file looks
>>>> like:
>>>> ==== Begin config file
>>>> project: linux
>>>> repo: linux # Relative to git-dir as specified on the command line
>>>> description: Linux kernel
>>>> mailinglists:
>>>> - {name: gmane.linux.kernel, type: dev, source: gmane}
>>>> revisions: ["v2.6.30", "v2.6.31", "v2.6.32", "v2.6.33", "v2.6.34",
>>>>> "v2.6.35", "v2.6.36", "v2.6.37", "v2.6.38", "v2.6.39",
>>>>> "v3.0", "v3.1", "v3.2", "v3.3", "v3.4", "v3.5", "v3.6",
>>>>> "v3.7", "v3.8", "v3.9"]
>>>> rcs: ["v2.6.30-rc1", "v2.6.31-rc1", "v2.6.32-rc1", "v2.6.33-rc1",
>>>> "v2.6.34-rc1", "v2.6.35-rc1", "v2.6.36-rc1", "v2.6.37-rc1",
>>>> "v2.6.38-rc1", "v2.6.39-rc1", "v3.0-rc1", "v3.1-rc1", "v3.2-rc1",
>>>> "v3.3-rc1", "v3.4-rc1", "v3.5-rc1", "v3.6-rc1", "v3.7-rc1",
>>>> "v3.8-rc1",
>>>> "v3.9-rc1"]
>>>> tagging: tag <=============================== Change here
>>>> ======End config file
>>>> Thanks,
>>>> Mitchell
>>> Sorry for the late reply.
>>> Obviously I was not on the latest version. Now I surely have the one with
>>> the hash 88db2d5.
>>>
>>> With "feature" and "tag" in the tagging-line of the config-file I get:
>>You need at least 0144984d08d73ac14d41577f88fdfab33ecb2665 if you want
>>to use the "tag" based analysis.
>>>
>>> $codeface run -c codeface.conf -p conf/linux-kernel.conf ../analysis/
>>> ../projects/
>>> ...
>>> Pass 1/2: 100% |#################################################| Time:
>>> 0:06:58
>>> Traceback (most recent call last):
>>> File "/home/matthias/.local/bin/codeface", line 9, in <module>
>>> load_entry_point('codeface==0.2.0', 'console_scripts', 'codeface')()
>>> File "/home/matthias/codeface/codeface/cli.py", line 198, in main
>>> return run(sys.argv)
>>> File "/home/matthias/codeface/codeface/cli.py", line 194, in run
>>> return args.func(args)
>>> File "/home/matthias/codeface/codeface/cli.py", line 112, in cmd_run
>>> args.profile_r, args.jobs, args.tagging)
>>> File "/home/matthias/codeface/codeface/project.py", line 110, in
>>> project_analyse
>>> endmsg=prefix + "Commit analysis done."
>>> File "/home/matthias/codeface/codeface/util.py", line 109, in add
>>> func(*args, **kwargs)
>>> File "/home/matthias/codeface/codeface/cluster/cluster.py", line 1900, in
>>> doProjectAnalysis
>>> rc_range)
>>> File "/home/matthias/codeface/codeface/cluster/cluster.py", line 1806, in
>>> performAnalysis
>>> populatePersonDB(cmtdict.values(), id_mgr, link_type)
>>> File "/home/matthias/codeface/codeface/cluster/cluster.py", line 1357, in
>>> populatePersonDB
>>> ID = id_mgr.getPersonID(cmt.get
>>> AuthorName())
>>> File "/home/matthias/codeface/codeface/cluster/idManager.py", line 141,
>>> in getPersonID
>>> self._cache[(name, email)] = self._query_user_id(name, email)
>>> File "/home/matthias/codeface/codeface/cluster/idManager.py", line 123,
>>> in _query_user_id
>>> jsond = json.loads(result)
>>> File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
>>> return _default_decoder.decode(s)
>>> File "/usr/lib/python2.7/json/decoder.py", line 365, in decode
>>> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>>> File "/usr/lib/python2.7/json/decoder.py", line 383, in raw_decode
>>> raise ValueError("No JSON object could be decoded")
>>> ValueError: No JSON object could be decoded
>>> ----------------------------
>>To me that looks like the id_service is not running, something else is
>>running on its port (8080) or something is blocking the connection
>>(firewall?).
>>>
>>> When I set "proximity" as tagging, there is repeatedly the warning:
>>> ignoring unsupported tag ...
>>> Finally a memory error appears.
>>I thing I had similar problems before and just used 4gb a swap file (8gb
>>total). No more problems since then. How much memory is available on
>>your system?
>
> Thanks for your fast answer, Mitchell.
>
> $ free -m
> total used free shared buffers cached
> Mem: 3965 1853 2111 0 117 950
> -/+ buffers/cache: 786 3179
> Swap: 0 0 0
>
> The size of a page is 4MB.
> What can/shall I do?
I have not experienced this issue before, sorry. Can you please post
the full error message or at least the call stack.
Thanks,
Mitchell
>
>
>>Hope this helps,
>> Matthias
>>>
>>> Best regards,
>>> Matthias
>>>>>
>>>>>> Thanks,
>>>>>> Mitchell
>>>>>>> -- Matthias
>>>>>>>
>>>>>>>
>>>>>>> ________________________________
>>>>>>> Von: codeface-bounce@xxxxxxxxxxxxx <codeface-bounce@xxxxxxxxxxxxx> im
>>>>>>> Auftrag von Mitchell Joblin <joblin.m@xxxxxxxxx>
>>>>>>> Gesendet: Freitag, 30. Januar 2015 20:33
>>>>>>> An: codeface@xxxxxxxxxxxxx
>>>>>>> Betreff: [codeface] Re: AW: Re: AW: Re: Unit tests - Exception in
>>>>>>> mailing
>>>>>>> list analysis executing batch.R
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Jan 30, 2015 8:22 PM, "Matthias Gemmer"
>>>>>>> <matthias.gemmer@xxxxxxxxxxxxxxxxxxxx> wrote:
>>>>>>>> I ran the analysis (with the updated code) for jquery and qemu. In both
>>>>>>>> cases I got:
>>>>>>> Can you run the tests without error now?
>>>>>>>
>>>>>>> --Mitchell
>>>>>>>
>>>>>>>> Traceback (most recent call last):
>>>>>>>> File "/home/matthias/.local/bin/codeface", line 9, in <module>
>>>>>>>> load_entry_point('codeface==0.2.0', 'console_scripts',
>>>>>>>> 'codeface')()
>>>>>>>> File "/home/matthias/codeface/codeface/cli.py", line 191, in main
>>>>>>>> return run(sys.argv)
>>>>>>>> File "/home/matthias/codeface/codeface/cli.py", line 187, in run
>>>>>>>> return args.func(args)
>>>>>>>> File "/home/matthias/codeface/codeface/cli.py", line 106, in cmd_run
>>>>>>>> args.profile_r, args.jobs)
>>>>>>>> File "/home/matthias/codeface/codeface/project.py", line 93, in
>>>>>>>> project_analyse
>>>>>>>> endmsg=prefix + "Commit analysis done."
>>>>>>>> File "/home/matthias/codeface/codeface/util.py", line 104, in add
>>>>>>>> func(*args, **kwargs)
>>>>>>>> File "/home/matthias/codeface/codeface/cluster/cluster.py", line
>>>>>>>> 1418, in doProjectAnalysis
>>>>>>>> rc_range)
>>>>>>>> File "/home/matthias/codeface/codeface/cluster/cluster.py", line
>>>>>>>> 1388, in performAnalysis
>>>>>>>> emitStatisticalData(cmtlist, id_mgr, logical_depends, outdir,
>>>>>>>> releaseRangeID,\
>>>>>>>> UnboundLocalError: local variable 'logical_depends' referenced before
>>>>>>>> assignment
>>>>>>>> ________________________________________
>>>>>>>> Von: codeface-bounce@xxxxxxxxxxxxx <codeface-bounce@xxxxxxxxxxxxx> im
>>>>>>>> Auftrag von Mitchell Joblin <joblin.m@xxxxxxxxx>
>>>>>>>> Gesendet: Freitag, 30. Januar 2015 16:34
>>>>>>>> An: codeface@xxxxxxxxxxxxx
>>>>>>>> Betreff: [codeface] Re: AW: Re: Unit tests - Exception in mailing list
>>>>>>>> analysis executing batch.R
>>>>>>>>
>>>>>>>> Hi Matthias,
>>>>>>>>
>>>>>>>> On Fri, Jan 30, 2015 at 3:08 PM, Matthias Gemmer
>>>>>>>> <matthias.gemmer@xxxxxxxxxxxxxxxxxxxx> wrote:
>>>>>>>>> I cloned the latest commit of the branch "mitchell-updates" and
>>>>>>>>> installed codeface.
>>>>>>>>> There are the following failures running unit tests:
>>>>>>>> The test suite is broken in the version you pulled, I just merged some
>>>>>>>> new code that disables the broken tests. Some tests broke because of
>>>>>>>> changes to R dependencies. Despite the failing tests, the analysis
>>>>>>>> will perform correctly. The tests need to be updated to account for
>>>>>>>> the interface changes and this will happen soon. Please give the
>>>>>>>> latest code a try and let me know if you experience any other issues.
>>>>>>>>
>>>>>>>> If you are going to run code on the mitchell-updates branch then you
>>>>>>>> also need to install doxygen. Sorry that wasn't mentioned in the
>>>>>>>> installation guide, once it's merged to master I will change that.
>>>>>>>>
>>>>>>>> Kind regards,
>>>>>>>>
>>>>>>>> Mitchell
>>>>>>>>
>>>>>>>>> 1) IOError: [Errno 2] No such file or directory:
>>>>>>>>> '/tmp/does_not_exist_42_42'
>>>>>>>>> 2015-01-30 15:57:19 [codeface.configuration] MainProcess ERROR:
>>>>>>>>> Could not parse configuration file '/tmp/tmp4cq_OI'
>>>>>>>>>
>>>>>>>>> 2) [codeface.util] MainProcess ERROR: Command 'Rscript -e
>>>>>>>>> library(testthat); if (test_dir(".")$n > 0) stop("Some tests
>>>>>>>>> failed.")'
>>>>>>>>> failed with exit code 1.
>>>>>>>>>
>>>>>>>>> 3) [codeface.configuration] MainProcess CRITICAL: Malformed
>>>>>>>>> configuration: revision and rcs list lengths differ! Found 5
>>>>>>>>> revisions and 4
>>>>>>>>> release candidates.
>>>>>>>>>
>>>>>>>>> 4) [codeface.VCS] MainProcess CRITICAL: doxygen analysis error[Errno
>>>>>>>>> 2] No such file or directory - defaulting to Ctags
>>>>>>>>>
>>>>>>>>> 5) [codeface.util] MainProcess ERROR: Command
>>>>>>>>> '/home/matthias/codeface/codeface/R/cluster/persons.r --loglevel
>>>>>>>>> devinfo
>>>>>>>>> --logfile /tmp/codeface_test_project9FdvGm/.git/log.R.r0 -c
>>>>>>>>> /home/matthias/codeface/codeface_testing.conf -p
>>>>>>>>> /tmp/codeface_test_project9FdvGm3NuXGh
>>>>>>>>> /tmp/codeface_test_project9FdvGm/.git/results/codeface_test_project9FdvGm/proximity/v0_release-v1_release
>>>>>>>>> 153' failed with exit code 1.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Matthias Gemmer
>>>>>>>>>
>>>>>>>>> ________________________________________
>>>>>>>>> Von: codeface-bounce@xxxxxxxxxxxxx <codeface-bounce@xxxxxxxxxxxxx> im
>>>>>>>>> Auftrag von Wolfgang Mauerer <wm@xxxxxxxxxxxxxxxx>
>>>>>>>>> Gesendet: Dienstag, 27. Januar 2015 13:16
>>>>>>>>> An: codeface@xxxxxxxxxxxxx
>>>>>>>>> Betreff: [codeface] Re: Unit tests - Exception in mailing list
>>>>>>>>> analysis executing batch.R
>>>>>>>>>
>>>>>>>>> Am 27/01/2015 12:55, schrieb Mitchell Joblin:
>>>>>>>>>> On Tue, Jan 27, 2015 at 9:03 AM, Wolfgang Mauerer
>>>>>>>>>> <wm@xxxxxxxxxxxxxxxx> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Am 26/01/2015 17:33, schrieb Matthias Gemmer:
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> I am using R version 3.1.2. All required packages should be
>>>>>>>>>>>> installed
>>>>>>>>>>>> correctly.
>>>>>>>>>>>>
>>>>>>>>>>>> Running the unit tests still raises an exception, something goes
>>>>>>>>>>>> wrong
>>>>>>>>>>>> in the mailinglist analysis:
>>>>>>>>>>>>
>>>>>>>>>>>> Exception: Command '/home/matthias/codeface/codeface/R/ml/batch.r
>>>>>>>>>>>> --logfile /tmp/codeface_test_projectH8ZL0F/.git/log.R.ml.0
>>>>>>>>>>>> --loglevel
>>>>>>>>>>>> devinfo -c /home/matthias/codeface/codeface_testing.conf -p
>>>>>>>>>>>> /tmp/codeface_test_projectH8ZL0F/.git/testproject.conf -j 1
>>>>>>>>>>>>
>>>>>>>>>>>> /tmp/codeface_test_projectH8ZL0F/.git/results/codeface_test_projectH8ZL0F/ml
>>>>>>>>>>>> /tmp/codeface_test_projectH8ZL0F/.git
>>>>>>>>>>>> codeface_test_projectH8ZL0F.dev1'
>>>>>>>>>>>> failed with exit code 1.
>>>>>>>>>>>>
>>>>>>>>>>>> The log file says:
>>>>>>>>>>>> $cat /tmp/codeface_test_projectH8ZL0F/.git/log.R.ml.0
>>>>>>>>>>>> ...
>>>>>>>>>>>> 2015-01-26 14:49:40 [codeface.R.ml.analysis] INFO: Starting
>>>>>>>>>>>> mailinglist
>>>>>>>>>>>> analysis
>>>>>>>>>>>> 2015-01-26 14:49:40 [codeface.R] INFO: Traceback:
>>>>>>>>>>>> 0:
>>>>>>>>>>>> 1: config.script.run({
>>>>>>>>>>>> positional.args <- list("resdir", "mldir", "listname
>>>>>>>>>>>> 2: withCallingHandlers(expr, error = function(e) {
>>>>>>>>>>>> if (!interactive()) {
>>>>>>>>>>>>
>>>>>>>>>>>> 3: dispatch.all(conf, conf$mldir, conf$resdir)
>>>>>>>>>>>> 4: gen.forest(conf, repo.path, resdir)
>>>>>>>>>>>> 5: gen.corpus(conf$listname, repo.path, suffix = ".mbox", marks
>>>>>>>>>>>> =
>>>>>>>>>>>> c("^_{10,}",
>>>>>>>>>>>> 6: preprocess(corp)
>>>>>>>>>>>> 7: sapply(corp, Author)
>>>>>>>>>>>> 8: match.fun(FUN)
>>>>>>>>>>>> 2015-01-26 14:49:40 [codeface.R] CRITICAL: object 'Author' not
>>>>>>>>>>>> found
>>>>>>>>>>> package tm changed the interface in the latest release (0.6) and
>>>>>>>>>>> does
>>>>>>>>>>> not pro function Author() any more. The replacement call is meta(),
>>>>>>>>>>> see
>>>>>>>>>>> http://cran.r-project.org/web/packages/tm/tm.pdf.
>>>>>>>>>>> The call to Author() needs to be changed (or a replacement provided
>>>>>>>>>>> via meta()); can you please file an appropriate bug? Or better
>>>>>>>>>>> still, submit a patch :)
>>>>>>>>>> No need to send a patch for this. I fixed this a while back in
>>>>>>>>>> eef75f98 on the mitchell-updates branch, it just hasn't been merged
>>>>>>>>>> to
>>>>>>>>>> master yet. Now that I can commit to master I will merge that very
>>>>>>>>>> soon.
>>>>>>>>> great, I did not realise that. I agree that it's high time we merge
>>>>>>>>> your updates branch. I'm fine with the branch.
>>>>>>>>>
>>>>>>>>> Cheers, Wolfgang
>>>>>>>>>> --Mitchell
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Thanks & best regards, Wolfgang Mauerer
>>>>>>>>>>>
>>>>>>>>>>>> 2015-01-26 14:49:40 [codeface.R] INFO: Error dump was written to
>>>>>>>>>>>> 'error.dump.rda'.
>>>>>>>>>>>> 2015-01-26 14:49:40 [codeface.R] INFO: To debug, launch R and run
>>>>>>>>>>>> 'load("error.dump.rda"); debugger(error.dump)'
>>>>>>>>>>>>
>>>>>>>>>>>> Attached you can find the error dump.
>>>>>>>>>>>>
>>>>>>>>>>>> Kind regards,
>>>>>>>>>>>> Matthias
>
Other related posts:
