Opensource, Search, Clustering

OSU Open Source Lab

http://osuosl.org/

Search Tools with Open Source Code

http://www.searchtools.com/tools/tools-opensource.html

Hadoop

http://lucene.apache.org/hadoop/

info: Hadoop is a software platform that lets one easily write and run applications that process vast amounts of data.

Here's what makes Hadoop especially useful:

* Scalable: Hadoop can reliably store and process petabytes.

* Economical: It distributes the data and processing across clusters of commonly available computers. These clusters can number into the thousands of nodes.

* Efficient: By distributing the data, Hadoop can process it in parallel on the nodes where the data is located. This makes it extremely rapid.

* Reliable: Hadoop automatically maintains multiple copies of data and automatically redeploys computing tasks based on failure

Nutch

http://lucene.apache.org/nutch/

Effort to implement a prototype of an open source web-search engine.

Creator Wandex the very first search engine :): Matthew Gray

http://gray.org/blog/

info: bio: Matthew Gray is the author of the World Wide Web Wanderer, the first autonomous agent on the web, used to track web growth since 1993. He is the worldwide authority on historical growth of the web. He set up one of the first 100 web servers in the world, www.mit.edu, in June 1993. In 1994, he founded a company, net.Genesis, to develop tools for web developers. With net.Genesis, he co-authored a book "Build A Web Site", published by Prima Publishing. Additionally, he was a member of the Apache Group, a non-profit group of volunteers who wrote Apache, the worlds most popular web server. In January, 1996 he left net.Genesis, which still continues to operate, to return to MIT to complete his degree in physics. He received is undergraduate degree in physics in 1997 and his masters degree from the MIT Media Lab in 1999. From 2001 to 2006, he was CTO of Newbury Networks, Inc., a provider of wireless location technology. He is currently a software engineer at Google.