Saturday, February 25, 2012

Flavor of the month- Neo4j and Heroku, part 2

Continuing from the previous post, here are the features we used for the first version of Flavorwocky:

Set up:
When the application starts up, we check if the database is empty. To do this, we perform a traversal from the reference node looking for any categories connected to it. If there are none, then we go ahead and create the set of categories by first creating their nodes, and then a relationship to the reference node. We also create the index "ingredients", which is used to index ingredient names. Source code: https://github.com/luanne/flavorwocky/blob/master/grails-app/conf/BootStrap.groovy

Add pairing:
Adding a pairing involves
  • Checking whether either node already exists to avoid recreating it
  • Creating both nodes and linking them to their categories in a single transaction
  • Creating a relationship between them in the same transaction above
To accomplish this, we used the Batch operation (note, this is still experimental).
Source: https://github.com/luanne/flavorwocky/blob/master/grails-app/controllers/com/herokuapp/flavorwocky/FlavorwockyController.groovy (fetchOrCreateNodes())

Auto complete: This was just an index lookup matching the partially entered ingredient by name. Source: https://github.com/luanne/flavorwocky/blob/master/grails-app/controllers/com/herokuapp/flavorwocky/FlavorwockyController.groovy (autosearch())

Visualization:
We used d3.js to provide two visualizations for the search results. The "Explore" visualization is based on the Node-Link tree; we used a Cypher query to find all ingredients that pair with the searched ingredient up to 3 levels deep, transform it into the appropriate data structure, and render it as JSON. Note, although the visualization interactive, the fetching of data is not. The entire set of data for 3 levels is grabbed at once- a future enhancement would be fetching children only when you expand a node.
Source: https://github.com/luanne/flavorwocky/blob/master/grails-app/controllers/com/herokuapp/flavorwocky/FlavorwockyController.groovy (getSearchVisualizationAsTreeJson())

Although the tree is pretty slick, one shortcoming is that it is a tree- so if children are linked to each other, then you see multiple instances of that ingredient in the tree. Hence we tried another visualization to capture the interconnections between ingredients and also surface interesting facts such as flavor trios- there's a pretty high chance that if you see a triangle in the network visualization, those three ingredients can be combined together well.
We used the Force directed graph for this. Again, a Cypher query came to the rescue (this time 5 levels deep to produce a richer model).
Source: https://github.com/luanne/flavorwocky/blob/master/grails-app/controllers/com/herokuapp/flavorwocky/FlavorwockyController.groovy (getSearchVisualizationAsNetworkJson())

In both examples, the affinity of the pairing is used to calculate the length of the connector between ingredients, indicating shorter connections have ingredients that pair much better than those with longer connections.

That's about it! There's so much more than can be done with this application, but it's going to have to wait for a bit.
Again, please about Flavorwocky if you like it or just want me to win. Voting also helps!

-Luanne

No comments: