Rails, Thinking_sphinx, Delta Index

Question

Hi,

I'm using thinking_sphinx and am delta indexing a model.

The delta index works but there is small bug. When I create a new product it is index. However, when I update that product it is not getting index right away. I have to update or create a new product before that old updated product is indexed.

Not quite sure where to start.

Answer 1

My recommendation would be to use delayed_delta indexing instead of straight delta indexing (which can be slow and if you have a few updates in a few seconds, can cause you all kinds of problems).

It takes two steps:

1. Change your define_index block to have a set_property :delta => :delayed
2. Create a short script to make sure the delayed indexing jobs get run. Here's the one I use:

#!/usr/bin/env ruby
## this script is for making sure and delayed_jobs get run
##   it is used by thinking sphinx
require File.dirname(__FILE__) + '/../config/environment'

# you can also put the definition of this in config/environments/*.rb so it's different for test, production and development
JobRunnerPidFile = "#{RAILS_ROOT}/tmp/pids/job_runner.pid" 

if File.exists?(JobRunnerPidFile)
  old_pid = File.read(JobRunnerPidFile).to_i
  begin
    if Process.getpgid(old_pid) > 0
      # still running, let's exit silently...
      exit(0)
    end
  rescue
    # looks like nothing is running, so let's carry on
  end
end

File.open(JobRunnerPidFile, "w") {|f| f.write "#{$$}\n" }

Delayed::Worker.new.start

You can run that script from cron every 5 minutes (it'll only run one instance) or if you have a monitoring service (e.g., monit) you can have it make sure it's running.

Make sure to restart that script when ever you deploy a new version of your code.