Brent Sordyl's blog

Goose – article extractor

May 16, 2011 by Brent Sordyl 0 Comments

Project Goose is an article extractor written in Java using Maven for the dependencies. It’s an open source project born from Gravity Labs http://gravity.com, Its goal is to take a webpage, perform calculations and extract the main text of the article as well as make recommendations on what image might be the most relevant image on the page. Goose aims to create an easy to use, scalable extractor that can plug into any application that needs to extract structure from unstructured web pages.

(Full Story: Goose – article extractor)

github, java, scraping, summary

Overview: Extracting article text from HTML documents

March 28, 2011 by Brent Sordyl 0 Comments

In the world of web scraping, text mining and article reading utilities (readability bookmarklet) there is an ever growing demand for utilities that are capable of distinguishing parts of a HTML document which represent an article apart from other common website building blocks like menus, headers, footers, ads etc.

(Full Story: Overview: Extracting article text from HTML documents)

datamining, scraping

Maltego – data forensics application

March 28, 2011 by Brent Sordyl 0 Comments

Maltego is an open source intelligence and forensics application. It will offer you timous mining and gathering of information as well as the representation of this information in a easy to understand format.

(Full Story: Maltego – data forensics application)

data, opensource, scraping

Needlebase – where data comes together

March 28, 2011 by Brent Sordyl 0 Comments

Needle platform for acquiring, integrating, cleansing, analyzing and publishing data on the web. Using Needle through a web browser, without programmers or DBA

(Full Story: Needlebase – where data comes together)

data, scraping, tools