« Roolastic - Part One | Main | Roolastic - Part... »
Mittwoch Mrz 24, 2010

Roolastic - Part Two

Part One

Part Three

Part Four

Now that we have the basics running let's have a look at some interesting bits of code (have a look at the roo.log to see how I build what is there so far using roo). Now the only domain object we have so far is Image. This is the object that holds the data we want to persist for each uploaded image and it's simple enough to understand. The two interesting properties of this class' source are that there are no bean style getters and other boilerplate code, as Roo has generated that and 'outsourced' it into AspectJ .aj files (like this). That is one of Roos main design decisions. The other notable thing here is the @JsonIgnoreProperties annotation - but we get to that later
CRUD operation for this object are handled by the ImageController which I created by first running 

controller scaffold --class ~.web.ImageController --entity ~.model.Image
in the Roo Console and then copying the methods that Roo generated in the .aj file directly into my Java class (which required a bit of search and replace to get it working). Here is my first gripe with Roo: If you want to override a generated controller method, Roo gets rid of the whole .aj file, which is inconvenient - I might have overlooked something here but that's what it looked like to me.
Now my controller still basically does what the autogenerated controller did, it just adds a bit of code for the file upload handling (for which to work I obviously had to change the form definition in the form view and add the 'enctype="multipart/form-data"'. That code also extracts the image metadata using the metadata extractor library.
OK, so where does elasticsearch come in now? The answer is: asynchronously, but almost in realtime ;-)
In a recent project I had some problems (which I still don't fully understand) with Compass (which is another project by the creator of elasticsearch), whereby the in-transaction indexing of domain objects was causing issues which in turn caused my database transactions to roll back - pants. Anyway hence my ventures into asychronicity
So how do we do this here? First we register a listener with hibernate (our JPA provider); to do this we add the following to our persistence.xml:
<property name="hibernate.ejb.interceptor" value="de.woerd.blogs.roolastic.persistence.HibernateInterceptor"/>
Then in that listener we override the postFlush method, which is the point in time when in the persistence lifecycle we already have a primary key set for our model object (which we want to know when we add it to the elastic search index). We then take our model objects/entities and pass them on to IndexService (to the static method 'index', as I can't think of a good way on how to have an IndexService bean instance injected into the HibernateInterceptor).
Now another technology comes into play. To really be asynchronous we use JMS to decouple the indexing from the persisting. so IndexService just stuffs the content to be indexed into a JMS queue which we setup by simply running 
jms setup --provider ACTIVEMQ_IN_MEMORY
in the Roo shell. We then create a listener that listens on that queue by doing this (Roo shell again)
jms listener class --class ~.search.IndexerSlave
That class IndexerSlave then uses ElasticSearchTemplate to stuff the content into the index. ElasticSearchTemplate is supposed to be the equivalent to Springs Jdbctemplate, JmsTempate etc. It should provide a simple API to interact with ElasticSearch. It currently uses ElasticSearch' TransportClient but Shay Banon advises on rather using the embedded server client (which I'll try soon). OK that's indexing done. In the next part we look into running searches against the index.

Posted at 05:31PM Mrz 24, 2010 by joerg in Allgemein  |  Kommentare[5]

Kommentare:

Hi,

The integration between Roo generated applications and ElasticSearch is a rather nice idea. Are you considering to create a Roo addon out of this work? I think there are endless opportunities to make this a truly great addon. I would be happy to assist you in this venture if you are interested.

One quick comment on your use of HibernateInterceptor as discussed above. This ties your solution into Hibernate only whereas Roo uses JPA (with your choice of Hibernate, EclipseLink or openJPA implementations). Maybe you want to consider using JPA lifecycle callback methods which would need to be annotated with @PostPersist, @PostUpdate, etc. This would then work with any of the JPA implementations I mentioned.

Cheers,

Stefan Schmidt
Software Engineer Spring Roo
SpringSource, a division of VMware

Gesendet von Stefan Schmidt am März 24, 2010 at 10:13 PM CET #

[Trackback] This post was mentioned on Twitter by joeslow: I started a little blog series on #spring #roo and #elasticsearch http://bit.ly/9sE51G and http://bit.ly/b1DLzK

Gesendet von uberVU - social comments am März 24, 2010 at 11:08 PM CET #

Stefan,
I would love to think about and discuss a Roo elasticsearch plugin - I've been meaning to contribute to one or the other OSS project for a long time and this combination would be close to my heart ;-)
I'm in the middle of a potential career change and it might turn out to be quite challenging - so I don't know right now how much time I'll have in the coming months.
I was aware of the JPA Lifecycle callbacks but went for interceptor route because it felt more general (all entities would then automatically be persisted) - but I guess it actually gives you some nice configurability using the annotations per entity - at the cost of having to remember adding the callback to each entity you want to have indexed. (It would solve the problem of not being able to have the indexer injected into the interceptor, because you can have the indexer injected into the entity, can't you?)

Gesendet von Jörg Erdmenger am März 25, 2010 at 09:13 AM CET #

Ah, just read up on JPA a bit more, no need for callbacks in the entity - just link to entityListener with @EntityListeners - that's even simpler - but the indexer reference issue again :-(

Gesendet von Jörg Erdmenger am März 25, 2010 at 09:22 AM CET #

Hi,

Thanks for considering to generalize your work into a Roo addon (even if it can't happen immediately).

I have started some work on a similar Roo addon a few weeks back (Solr to be specific), but have not found the time to finish it yet. Maybe I'll get to it soon so you can take a look at my approach.

About your question, you can easily add callback methods for each entity through an ITD (the developer would most likely not even see them). As you noted, this approach does offer great flexibility in terms of configurability. Further, if you configure your indexer as a Spring bean you can inject it easily into any entity.

Another comment on your note above (If you want to override a generated controller method, Roo gets rid of the whole .aj file). This should not happen. If you 'push in' an individual controller method Roo should still maintain the ITD for the other methods. See the docs for details. If this is not the case, we would need to see some details. Maybe the Roo forum or Jira would be a better place for that though.

Cheers,

Stefan Schmidt
Software Engineer Spring Roo
SpringSource, a division of VMware

Gesendet von Stefan Schmidt am März 25, 2010 at 09:27 AM CET #

Senden Sie einen Kommentar:
Kommentare sind ausgeschaltet.