[edm-discuss] Re: edm-discuss Digest V7 #6

  • From: Qing_sheng Zhang <qingshengzhng@xxxxxxxxx>
  • To: edm-discuss@xxxxxxxxxxxxx
  • Date: Fri, 8 Mar 2013 15:18:51 +0800

Dear all,
            Some clues are very interesting. I don't have any related
experience (available toolkit)in this field.
            However, I alway wonder how to measure difficulty of webpages
in web-based online course for students. Can we measure text complexity of
individual webpage in web-based online course, further measure domain
knowledge strength (level) of individual webpage.
            If someones are interested in this idea, can we setup a
discussion thread in EDM?

Dr. Qingsheng Zhang
lecturer
Xi'an University of Posts and Telecommunications, Shannxi, China


On Fri, Mar 8, 2013 at 2:09 PM, FreeLists Mailing List Manager <
ecartis@xxxxxxxxxxxxx> wrote:

> edm-discuss Digest      Thu, 07 Mar 2013        Volume: 07  Issue: 006
>
> In This Issue:
>                 [edm-discuss] Anyone work on web mining and feature
> generati
>                 [edm-discuss] Re: Anyone work on web mining and feature
> gene
>                 [edm-discuss] Re: Anyone work on web mining and feature
> gene
>                 [edm-discuss] data mining
>                 [edm-discuss] Re: Anyone work on web mining and feature
> gene
>                 [edm-discuss] Re: Anyone work on web mining and feature
>
> ----------------------------------------------------------------------
>
> From: "Joseph E. Beck" <josephbeck@xxxxxxx>
> Date: Wed, 6 Mar 2013 18:24:55 -0500
> Subject: [edm-discuss] Anyone work on web mining and feature generation?
>
> Hello, we're working on a project determining the educational efficacy of
> webpages.  I am wondering if anyone knows of a resource for computing
> properties of the webpage itself.  Even relatively simple-sounding
> concepts, such as whether there is a movie, can be difficult to compute.
>  So we'd prefer to leverage off of someone else's work :-)   Has anyone
> come across such tools in their work?
> Thanks.
>
> joe
>
> --
> Joseph E. Beck
> Assistant Professor
> Computer Science Department, Fuller Labs 138
> Worcester Polytechnic Institute
>
>
>
> ------------------------------
>
> From: Nidhi Chopra <nidhi.chopra@xxxxxxxxx>
> Date: Thu, 7 Mar 2013 18:15:29 +0530
> Subject: [edm-discuss] Re: Anyone work on web mining and feature
> generation?
>
> In TTS (text to speech) mp3 files are opened in Visual C++ to view
> contents, after changing extension name of the file. Then code can be
> written in C/C++ to read the files & perform other operations. This is the
> summary of my 6 months project in I did in my Masters.
> Thinking on these line, you have to open the saved page in notpad/txt and
> read contents, look for keywords (TAGS in HTML language) that specify type
> of file. Then write code to do what you are doing manually using ctrl
> function. Or have you tried this already?
>
> Nidhi Chopra
> Delhi, India
>
> On Thu, Mar 7, 2013 at 4:54 AM, Joseph E. Beck <josephbeck@xxxxxxx> wrote:
>
> > Hello, we're working on a project determining the educational efficacy of
> > webpages.  I am wondering if anyone knows of a resource for computing
> > properties of the webpage itself.  Even relatively simple-sounding
> > concepts, such as whether there is a movie, can be difficult to compute.
> >  So we'd prefer to leverage off of someone else's work :-)   Has anyone
> > come across such tools in their work?
> >
> > Thanks.
> >
> > joe
> >
> > --
> > Joseph E. Beck
> > Assistant Professor
> > Computer Science Department, Fuller Labs 138
> > Worcester Polytechnic Institute
> >
>
>
>
> ------------------------------
>
> Date: Thu, 07 Mar 2013 08:05:57 -0500
> From: Vincent Aleven <aleven@xxxxxxxxxx>
> Subject: [edm-discuss] Re: Anyone work on web mining and feature
> generation?
>
> Just heard a talk by Ranjitha Kumar from Stanford who (together with
> Scott Klemmer and others) is doing interesting work on analyzing web
> pages. Not necessarily for educational purposes but nonetheless it might
> be relevant.
>
> Vincent
>
>
> On 3/6/13 6:24 PM, Joseph E. Beck wrote:
> > Hello, we're working on a project determining the educational efficacy
> > of webpages.  I am wondering if anyone knows of a resource for
> > computing properties of the webpage itself.  Even relatively
> > simple-sounding concepts, such as whether there is a movie, can be
> > difficult to compute.  So we'd prefer to leverage off of someone
> > else's work :-)   Has anyone come across such tools in their work?
> >
> > Thanks.
> >
> > joe
> >
> > --
> > Joseph E. Beck
> > Assistant Professor
> > Computer Science Department, Fuller Labs 138
> > Worcester Polytechnic Institute
>
>
> ------------------------------
>
> From: Nidhi Chopra <nidhi.chopra@xxxxxxxxx>
> Date: Thu, 7 Mar 2013 19:00:24 +0530
> Subject: [edm-discuss] data mining
>
> i m analyzing enrollment data. need some algorithms to apply. some idea.
> n. chopra
> delhi, india
>
>
>
> ------------------------------
>
> From: Peter Swire <swirepe@xxxxxxxxx>
> Date: Thu, 7 Mar 2013 13:17:13 -0500
> Subject: [edm-discuss] Re: Anyone work on web mining and feature
> generation?
>
> One tool I've used in the past is scrapy. It's a little more friendly than
> mechanize and its kin, if you know what you are looking for on a page:
> http://doc.scrapy.org/en/latest/intro/overview.html
> I haven't personally used Pattern, but I've heard good things:
> https://github.com/clips/pattern
>
> From there, you have text mining packages like the nltk for python, or tm
> and tau for R.  Good luck!
>
> Best,
> Peter
>
> On Thu, Mar 7, 2013 at 8:05 AM, Vincent Aleven <aleven@xxxxxxxxxx> wrote:
>
> > Just heard a talk by Ranjitha Kumar from Stanford who (together with
> Scott
> > Klemmer and others) is doing interesting work on analyzing web pages. Not
> > necessarily for educational purposes but nonetheless it might be
> relevant.
> >
> > Vincent
> >
> >
> >
> > On 3/6/13 6:24 PM, Joseph E. Beck wrote:
> >
> >> Hello, we're working on a project determining the educational efficacy
> of
> >> webpages.  I am wondering if anyone knows of a resource for computing
> >> properties of the webpage itself.  Even relatively simple-sounding
> >> concepts, such as whether there is a movie, can be difficult to compute.
> >>  So we'd prefer to leverage off of someone else's work :-)   Has anyone
> >> come across such tools in their work?
> >>
> >> Thanks.
> >>
> >> joe
> >>
> >> --
> >> Joseph E. Beck
> >> Assistant Professor
> >> Computer Science Department, Fuller Labs 138
> >> Worcester Polytechnic Institute
> >>
> >
> >
> >
>
>
>
> ------------------------------
>
> Date: Fri, 08 Mar 2013 10:46:25 +1100
> From: "Karen Blackmore" <Karen.Blackmore@xxxxxxxxxxxxxxxx>
> Subject: [edm-discuss] Re: Anyone work on web mining and feature
>
> Not sure whether I'm understanding what you are after correctly, but
> Outwit Hub has proved very useful for me in the past to scrape websites.
> Can be automated using macros and there is a very easy GUI to build your
> own scrapers. It is quite useful for exploring domain trees. It can pull
> content quickly and export to a .csv - you might be able to get the info on
> website properties you are interested in?
>
> I free trial version is available.
>
> Hope this helps.
>
> Cheers,
> Karen
>
>
>
> ----------------------------------------------
> Dr Karen Blackmore
> Lecturer
> School of Design, Communication and Information Technology
> Room MCG18, McMullin Building
> The University of Newcastle
> University Drive
> Callaghan NSW 2308
>
> Karen.Blackmore@xxxxxxxxxxxxxxxx
> Ph: +61 2 492 15206
> Fax: +61 2 492 15896
>
> Researcher Profile:
> http://www.newcastle.edu.au//staff/research-profile/Karen_Blackmore/
>
> Member of the Applied Informatics Research (AIR) Group:
> http://silverbullet.newcastle.edu.au/air/
> ----------------------------------------------
>
>
>
>
>
>
>
>
>
>
> >>> "Joseph E. Beck" <josephbeck@xxxxxxx> 03/07/13 10:24 AM >>>
> Hello, we're working on a project determining the educational efficacy of
> webpages.  I am wondering if anyone knows of a resource for computing
> properties of the webpage itself.  Even relatively simple-sounding
> concepts, such as whether there is a movie, can be difficult to compute.
>  So we'd prefer to leverage off of someone else's work :-)   Has anyone
> come across such tools in their work?
>
> Thanks.
>
> joe
>
> --
> Joseph E. Beck
> Assistant Professor
> Computer Science Department, Fuller Labs 138
> Worcester Polytechnic Institute
>
>
>
> ------------------------------
>
> End of edm-discuss Digest V7 #6
> *******************************
>
>

Other related posts:

  • » [edm-discuss] Re: edm-discuss Digest V7 #6 - Qing_sheng Zhang