This forum is in READ-ONLY mode.
You can look around, but if you want to ask a new question, please use the new forum.
Home » plugins » Search plugins » Lucene Order By Score
Lucene Order By Score [message #97519] Sat, 17 April 2010 02:08 Go to next message
FeelLikeANut  is currently offline FeelLikeANut
Messages: 12
Registered: March 2010
Junior Member
Hi. I was wondering if anyone knew a good trick for ordering doctrine results by order of a Lucene score.

Right now I have something like...

    $pks = array();
    foreach ($this->getLuceneIndex()->find($query) as $hit)
    {
      $pks[] = $hit->pk;
    }

    return $this->createQuery('a')
           ->whereIn('a.id', $pks);


How can I get the results in the same order as the IDs in the $pks array?
Re: Lucene Order By Score [message #98997 is a reply to message #97519 ] Thu, 13 May 2010 12:29 Go to previous messageGo to next message
erisds  is currently offline erisds
Messages: 7
Registered: April 2010
Junior Member
I am also looking for a way to do this.

The Symfony Jobeet tutorial provides code like what you have shown above, where you first get the Lucene hits, translate them to Symfony Primary Keys and use a WHERE IN sql query to get the results.

The problem, as you have noted is that WHERE IN does not honour the order of the PKs in the IN clause, and such the relevance information from Lucene is totally lost.

As I understand it, this is not how Lucene is intended to be used. This very old article is very similar to the Jobeet tutorial, but instead uses Lucene's stored values to generate the results list in the correct order.

So, instead of fetching the objects based on the PKs using a WHERE IN clause, you change your updateLuceneIndex() function to store the values you want to return in your search results like this:

$doc->addField(Zend_Search_Lucene_Field::Text('title', $this->getTitle(), 'utf-8'));
$doc->addField(Zend_Search_Lucene_Field::Text('excerpt', $this->getExcerpt(), 'utf-8'));
$doc->addField(Zend_Search_Lucene_Field::Text('date', $this->getCreatedAt('l, jS F, Y'), 'utf-8'));
$doc->addField(Zend_Search_Lucene_Field::UnStored('body', $this->getBody(), 'utf-8'));


Here I am storing the title, excerpt and date of a news/blog item in the lucene index, but using UnStored for the body as I do not need this.

Then your getForLuceneQuery function becomes the following:

static public function getForLuceneQuery($query)
{
  $hits = self::getLuceneIndex()->find($query);
 
  return $hits;
}


And in your template you can echo out the values direct from the Lucene hit object:

<?php foreach($hits as $hit): ?>
<h2><?php echo $hit->title; ?></h2>
<p class="date><?php echo $hit->date; ?></p>
<p><?php echo $hit->excerpt; ?></p>
<?php endforeach; ?>


This works in the case I am using it at the moment, because I have a small data set and don't want to put too much info into the search result listing.

However I have another case with a much larger dataset and where I also want to display lots of information, including thumbnails in the search results list and so this is a less appropriate solution.

So if anyone knows of a clever way of fetching the Symfony objects in the correct order, I'd super appreciate some help!
Re: Lucene Order By Score [message #102828 is a reply to message #98997 ] Tue, 20 July 2010 22:16 Go to previous messageGo to next message
conrad  is currently offline conrad
Messages: 9
Registered: July 2009
Location: Düsseldorf, Germany
Junior Member
I'm having the same trouble with lucene and would like to use the doctrine collection for the output.. this is a little approach of an idea I had - definately not perfect and quite dowdy.. but it works, at least with an array holding doctrine objects

    public function getForLuceneQuery($query)
    {
      // find hits
      $hits = self::getLuceneIndex()->find($query);
      // fetch primary keys of result for db query
      $pks    = array();
      $score  = array();
      foreach ($hits as  $i => $hit) {
        // pks for doctrine query
        $pks[]    = $hit->pk;
        // save the score temporarily
        // COMMENT: another Datatype than String?
        $score[]  = (String)$hit->score;
      }

      if (empty($pks)) {
        return array();
      }

      // get result
      $q = $this->createQuery('Seminare s')
        ->whereIn('s.id', $pks)
        ->leftJoin('s.Rubriken r');

      $resTmp = $q->execute();

      // run through hits again to add it to the final result
      $res = array();
      foreach($hits as $i => $hit) {
        // fetch doctrine object from collection
        $object = $resTmp->get($i);
        // store the lucene reference
        // (added it to the class, not neccessary)
        $object->lucene = $hit;
        // probably a lucene score exists twice or more often, so
        // add a new number to the end to avoid losing one entity
        // COMMENT: would decrementing be better?
        $resKey = $score[$i];
        $k = 0;
        do {
          $resKey .= $k;
          $k++;
        }while(array_key_exists($resKey, $res));
        // finally add the object to the array with the new key
        $res[$resKey] = $object;
      }
      // sort by keys ASC to have the appropriate order again
      krsort($res, SORT_NUMERIC);

      return $res;
    }


comments?
Re: Lucene Order By Score [message #103121 is a reply to message #102828 ] Tue, 27 July 2010 11:13 Go to previous messageGo to next message
trontank  is currently offline trontank
Messages: 179
Registered: July 2008
Location: Germany
Senior Member
Maybe this helps: http://brentertainment.com/2010/07/20/short-order-result-set s-by-an-array/
Re: Lucene Order By Score [message #103123 is a reply to message #103121 ] Tue, 27 July 2010 11:49 Go to previous message
conrad  is currently offline conrad
Messages: 9
Registered: July 2009
Location: Düsseldorf, Germany
Junior Member
definitely a more sophisticated solution. thank you.
Previous Topic:I need a plugin to search
Next Topic:Culture problems when using Lucene
Goto Forum:
  

powered by FUDforum - copyright ©2001-2004 FUD Forum Bulletin Board Software