New search hotness!
For years we've gotten by with a fairly primitive approach to search. There was a single full-text indexed field in the database called "vitals" with every possible word related to a product or part crammed into it. Search keywords would be matched against it, and you got what you got. It actually worked very well most of the time, but every now and then it would surprise you with an oddity that went against expectation (eg. a search for "browning" would turn up toys with "brown" in the name ahead of "browning"). There were also troubling technical limitations, namely that the only storage engine with full text search capability was one that did not support transactions (which means that if something were to go wrong in a chain of database queries involving that table, it was impossible to roll back the changes).
To make a long story short, I've installed a dedicated search product called "Sphinx". Sphinx has a number of qualities that make it a better choice for shmax.com:
- Separation of search index and database.This will allow me to migrate key tables over to a transaction-protected storage engine.
- Manual weighting We now have the ability to assign weight modifiers to specific searchable fields. Product names, for instance, now far outweigh part colors, thus solving the "browning" issue mentioned earlier.
- Automatic stemming. Stemming refers to the practice of matching the stem of a word when searching for a variant (eg. "autobots" will match "autobot"). We had stemming support before, but it was done as a pre-processing step on the search keywords--using a fairly cheesy algorithm--that also left us subject to frustrating relevance conflicts
- Improved performance Sphinx searches are around 20 times faster than what we had before. To be fair, search time was never much of a bottleneck on shmax.com, but now we can move ahead with projects that do depend on super speedy search (such as an auto-complete feature for the search box at the top of the page).
- Character mappings Shmax.com has always supported unicode and unicode searches, but search was very strict about it, and unless you knew how to type special characters there were some things that were difficult to search for. With sphinx, I'm able to map certain unicode characters to regular ascii characters, such that a search for "aeria" now matches "Aéria".
And a bunch of other stuff, but those are the highlights. While we evaluate the new system I've left the old search functionality enabled, as well. To revert back to the old search for the duration of your session, add "SPHINX=0" to the url (eg. "
www.shmax.com?SPHINX=0"). Use "SPHINX=1" to switch it back on, again.
Let me know what you think!
