I am looking for a way to query multiple search forms and aggregate results. The forms are NOT from common websearch engines (google and such) but from several editor database.
Most forms have to be submitted in POST method.
Apple's Sherlock used to let users develop their own search plugins that could query multiple sites with GET or POST, but 1. Sherlock is no longer supported by Apple 2. The only app that is supposed to run these plugins on Windows is an old french app called Glooton (www.glooton.com) which anyway doesn't work (I tried it with the supplied plugins, but I can't get any results). 3. We would prefer to have a web application (php or perl)
Do anyone has an idea?
We are a public library and would eventually like to offer our readers with a way to search the multiple ebook and reference databases we have. Developping a metasearch for these databases will be complicated, since the editors are unwilling to let other engines search their contents. Therefore, we would first like to find a quick solution and then wait to see how the market for these ebooks evolves.
> I am looking for a way to query multiple search forms and aggregate > results. The forms are NOT from common websearch engines (google and > such) but from several editor database.
> Most forms have to be submitted in POST method.
If the "searches" are keyword searches then a good web programmer could put this together quickly. What you are talking about is what database developers often refer to as a "database mediator:"
A high level query is sent to the mediator. The mediator re-sends the query to N-different databases and then collects N-different results in a a single, concatenated answer, which is returned to the original posting form.
When the query is a complex statement, that has to be translated into N-different dialects of SQL, and then sent off to a N-different relational databases, in that case building the mediator is a major undertaking.
But if the query is just a list of comma-delimited keywords, then there is no query translation step to go through. A journeyman web programmer should be able to put a form system like that together in an hour or two.
> I am looking for a way to query multiple search forms and aggregate > results. The forms are NOT from common websearch engines (google and > such) but from several editor database.
> Most forms have to be submitted in POST method.
If the "searches" are keyword searches then web programmer could put this together quickly. What you are talking about is what database developers often refer to as a "database mediator:"
A high level query is sent to the mediator. The mediator sends that query to N-different databases and then collects N-different results in a a single, concatenated answer, which is returned to the original posting form.
When the query is a complex statement, that has to be translated into N-different dialects of SQL, and the sent to a N-different relational databases, in that case building the mediator is an undertaking.
But if the query is just a list of comma-delimited keywords, then there is no query translation step to go through. A journeyman web programmer should be able to put a form system like that together in an hour or two.
> I am looking for a way to query multiple search forms and aggregate > results. The forms are NOT from common websearch engines (google and > such) but from several editor database.
<?php
class keyword_query_mediator { var $buffer; var $keywords; var $servers = ( "http://abc.org"=>"/queryform.php", "http:/xyz.net"=>"/cgi-bin/db.cgi", "http://www.ugabuga.com"=>"/db/db.php" );
function keyword_query_mediator($keywords) { $this->keywords = $keywords; $this->buffer=''; $this->makeQueries(); }
function makeQueries() { while(list($host,$path) = each($this->servers)) { $this->makeQuery($host,$path); } }
function makeQuery($host,$path) { $this->buffer .= $this->sendToHost($host,'post',$path,$this->keywords,0); }
// from http://www.faqts.com/knowledge_base/view.phtml/aid/12039/fid/51 function sendToHost($host,$method,$path,$data,$useragent=0) { // Supply a default method of GET if the one passed was empty if (empty($method)) { $method = 'GET'; } $method = strtoupper($method); $fp = fsockopen($host, 80); if ($method == 'GET') { $path .= '?' . $data; } fputs($fp, "$method $path HTTP/1.1\r\n"); fputs($fp, "Host: $host\r\n"); fputs($fp,"Content-type: application/x-www-form- urlencoded\r\n"); fputs($fp, "Content-length: " . strlen($data) . "\r\n"); if ($useragent) { fputs($fp, "User-Agent: MSIE\r\n"); } fputs($fp, "Connection: close\r\n\r\n"); if ($method == 'POST') { fputs($fp, $data); }