[Ilugc] Webpage layout analyzer
- From: steve@xxxxxxxxxxxx (steve)
- Date: Fri Mar 19 14:11:49 2010
Hi,
On 03/19/2010 09:59 AM, JAGANADH G wrote:
On Fri, Mar 19, 2010 at 9:53 AM, Shrinivasan T<tshrinivasan@xxxxxxxxx>wrote:
Hi,
>
> Is there any webpage layout detection tool available in FOSS !!
What do you mean by this?
A web page will be having left pane, right pane etc.. . As like a wiki pedia
article page. I have to extract the article only . Not the content from left
pane etc. I downloaded the Tamil wikipedia html dump and some blog pages. I
have to extract content from this .
Firstly, what you are speaking about (extracting only some elements from a web
page) is commonly referred to as web scraping. There are various libraries and
tools available to web page scraping. Depending on how you wish to do it (ie:
single pages, multiple pages, choice of programming language ...etc). Do a
google and if you need more help in narrowing down the choices ask again with
more specifics of what type of tools you would prefer.
Secondly, if you intend to get large amount of content from wikipedia, it is
recommended that you /do not/ use an automated tool:
http://en.wikipedia.org/wiki/Wikipedia_database#Please_do_not_use_a_web_crawler
Instead use one of the alternate methods mentioned in the page above.
Thirdly, if you just want to remove unnecessary elements from a page and save
only the content, while browsing, I would suggest using one of these Firefox
tools:
Aardvark:
http://karmatics.com/aardvark/
Readability:
http://lab.arc90.com/experiments/readability/
hth,
regards,
- steve
--
random new spiel:
http://lonetwin.net/
random old spiel:
http://lonetwin.blogspot.com/
what i'm stumbling into:
http://lonetwin.stumbleupon.com/
Other related posts:
