Sweepstakes with JBoss Infinispan, ModeShape, and others.

Tuesday, June 8th, 2010

Don’t wait for Ed McMahon to come knocking on your door for this Sweepstakes.  Instead, make sure you grab the CITYTECH mobile app “Summit World 2010″ which will be released to the Apple AppStore and Google MarketPlace sometime before JBoss World on June 22, 2010 (keeping fingers crossed on AppStore acceptance).     Once you have the mobile app and the conference starts,  you can start answering trivia questions served up by our “Trivia Server” running in the Amazon Cloud using JBoss ModeShape and Infinispan.    Questions are mostly technical in nature and each correct answer is worth specific points.  The more points you earn, the better your chances of winning great prizes in the sweepstakes drawing at the end of the conference.    I know the marketing folks are planning to top the Kindle prizes from last year, so make sure to stop by the CITYTECH booth to see what they have in mind.

Our goal for JBoss World 2010 was to  build a distributed and highly-scalable system with JBoss Community projects to show it could be done without the need for expensive commercial software.  Truth be told, we also wanted to get the chance to really put some of these newer technologies to the test and make sure they delivered on their promise.   We figured this Trivia Sweepstakes would be a good way to exercise the system because it gives everyone the chance to show off their technical expertise in answering the questions AND to win some cool prizes.

Technical Overview

This was a team project at CITYTECH with several people specializing in the different technologies.  As such, I don’t plan to cover each piece in depth, but I will discuss at a high level what technologies were used, where, and why.   Some of the other team members will be adding their own specific blogs in the coming weeks leading up to the conference so keep an eye on those.   I would also invite anyone at the conference to stop by the booth and grab someone from the development team during the conference to talk details.

The diagram below outlines a handful of the technologies used and what function they served:

Let me discuss these each briefly and how they played a part in the overall system.  A shopping list of the major technologies used in the system include:

JBossAS 6

Used as the container for the RESTEasy web app which includes ModeShape and Infinispan as well in this case.  JBossAS6 is included as part of CirraAS and can be monitored OOTB by RHQ, so was pretty much an automatic choice to use.  Note that we also use Jetty for quick integration testing via Maven, so pretty much any servlet container will do the job.

JBoss ModeShape

ModeShape provides a JCR API to our data and let’s us choose one or more back-end datasources.  In our case, we only had a single datasource (Infinispan) in mind so we did not need the federation capabilities (yet), but wanted to have it to front an Infinispan Cache.    We chose to add this layer for several reasons:  One, it provides a data access layer that is standardized (JCR).  Two, our team has plenty of JCR experience from working with CRX and Jackrabbit.  Finally, we wanted to take advantage of the JCR search capabilities across our data.  ModeShape has several connectors included so wiring it to Infinispan is included; however, we did encounter a few hurdles on this task that I promise to elaborate on in an upcoming blog.

JBoss Infinispan

Infinispan provides us with an in-memory cache to store the content used in the system (users, questions, answers, etc).  We are running Infinispan in the distributed mode to allow for the greatest scalability of the system without impacting performance.  We have several Infinispan instances running in EC2 and via the Infinispan configuration we indicate how many copies of the content we want spread across the cluster.  We always want at least 2, to ensure a single node failure does not result in the loss of data.  Infinispan itself determines which nodes get which copies and handles forwarding requests to the proper nodes when the node in question does not have a copy of that content.   This approach allows us to scale the system by adding Infinispan nodes as needed without degrading performance the way a replication approach would do.

A quick note on Infinispan in EC2.  UDP communications are not allowed in EC2, so we have Infinispan using TCP and an Amazon S3 bucket that is accessible by all nodes as a workaround.   The JGroups included with Infinispan already has an S3Ping option and Infinispan comes with a sample configuration that is ready to go.  Check out the Infinispan blog at http://infinispan.blogspot.com/2010/05/infinispan-ec2-demo.html for details.

JBoss RESTEasy

Though ModeShape has a RESTful API, those are more data level services of a true RESTful nature.  That was not exactly what we needed for this system.  Instead, we needed to expose coarse-grained business services over HTTP and that is where the RESTEasy framework came into play for us.  The services exposed by our RESTEasy web app allow cross-platform consumption.  For example, we have native iPhone clients, native Android clients, JavaScript clients, and Flex clients all using these same services.    The services in turn handle the data access via ModeShape’s JCR API.

JBoss HornetQ

The system uses HornetQ for a high performance messaging system to capture all “system events” to allow for later consumption.

RHQ

RHQ is the management console with associated agents installed on all nodes.  RHQ is setup automatically if you use CirrAS AMI’s in Amazon as part of the Mgmt AMI instance.  RHQ is required if you use CirrAS and allows for the management and monitoring of the other JBoss instances.

JBoss CirrAS

CirrAS is a sub-project of the StormGrind project.  It provides 3 AMI types using the Fedora OS that allow you to have a typical JBoss cluster running in EC2.  The 3 AMIs include: a “front-end” AMI that has Apache HTTPD with Mod_Cluster and JGroups configured for EC2, a “back-end” AMI that autostarts JBoss6 with a clustered profile, and a “mgmt” AMI that runs the RHQ server already mentioned.  We decided to use CirrAS because it provided almost everything we needed and saved us some time setting things up.  The trade-off to all this automation is that you still have to dig in and figure out exactly what the CirrAS scripts are doing.  As CirrAS is still a beta version, this can be a challenge as the documentation is still pretty light.

Amazon Cloud

The products available in AWS allow the system to provide the needed scalability.  EC2 is used to power up JBoss instances within minutes.  We have also researched using the Elastic Load Balancer to front the Apache CirrAS AMI and the Amazon RDS to serve as the write-behind CacheLoader for our Infinispan cache.

PhoneGap

The PhoneGap framework was used to generate a cross-platform mobile application for both the iPhone and Android platforms.  Note that a web-based client is also available for conference attendees without iPhone or Android devices.

Adobe Flex

A Flex application is used in the system to provide an RIA for the CITYTECH booth at the conference.  The Flex app uses the RESTEasy services hosted in EC2 to access the data in ModeShape as well as BlazeDS to pull messages from the HornetQ server in EC2.

As I mentioned before, keep an eye on the CITYTECH blogs as other team members will most likely be discussing specific areas of the system in greater detail in the coming weeks.  Also please stop by our booth to say hello if you are at the conference later this month.

Finally, I want to thank all the  community folks who were very active in their support through the forums.  As mentioned before, several of these technologies are relatively new, so having an active and responsive community to provide support is essential.

Good luck with the Trivia Sweepstakes!

JBoss World – The First full day

Thursday, September 3rd, 2009

Just wrapped up the first full day of JBoss World here in Chicago and wanted to share some of my thoughts.  To start, this is my first time attending the event so I did not really know what to expect.  Attendance seems to be very good and probably more than I had expected given the economy.  However, it is hard to judge who is there on the RHEL track versus JBoss given it is a shared event.  In fact, that seems to be the leading question in small talk at lunch and in the pavilion!  “So…., you here for RHEL or JBoss?”

Picking my highlights of the day:

• Attended a Seam session co-hosted by Dan Allen, author of Seam in Action and now part of the JBoss team.  Gathered some good tips/tricks on Seam but also glad to meet Dan since I really appreciate his book.  In it, he has no qualms about pointing out both the strong and weak points in the technology.

• My colleague, Matt Van Bergen, dragged me over to the eXo booth for something “I had to see”.  He was right.  They are doing some great work with both the portal and wcm products I saw and sounds like they are ready to release their first beta product after partnering with JBoss to embed the JBoss portal container inside their product.  Besides their polished and advanced UI, eXo also adds a clusterable JCR implementation to the mix, something that you did not get (at least out of the box) with the Jackrabbit implementation in the current JBoss Portal.  I am looking forward to release of this project on JBoss Community in the next day or so.  More on the eXo / JBoss partnership can be found here.

There was plenty more during the day, including lunch with two attendees who had come in from Australia the night before.  It was their first time to Chicago and they were awed by the “IMAX-sized landscape of the downtown buildings”.  The guy next to me was a New Yorker and just chuckeled.

Part 2 – Alfresco Integration with JBoss Portal

Tuesday, April 14th, 2009

This blog is an addendum to last week’s blog found here.  I had not planned to do a follow-up or second part to that blog, but after working to display the CMIS objects in a tree structure, I wanted to share what I had learned.

Let’s start with the result first and work backwards.  I have enhanced the Browser tab to display the CMIS objects from Alfresco in a hierarchical tree structure.  See here:

To do this, I used the RichFace recursiveTreeNodesAdaptor component to build a dynamic tree based on the CMIS calls to Alfresco.  As it’s name implies, you feed it CMIS objects in a recursive fashion to display as a tree.  In my case, I then cached the objects in Seam’s Conversational state to make subsequent retrievals faster.    The component also allows different UI looks, which I based on the CMIS object’s baseType property.

Couple “gotchas” I hit on the way.

• I was originally using the JBoss Portlet Bridge v1.0.0.B6.  However I could not get the tree to expand beyond the root children.  After some time wasted thinking I was implementing the tree incorrectly, a quick search found this was a reported bug and fixed in v1.0.0.RC1.  So… I upgraded to that version.  That “almost” fixed the issue.  The tree started to work, but would not expand beyond 2 nodes deep when in the switchType=”ajax” mode.  This was the mode/behavior I wanted, but as a workaround for now, I changed the switchType for the tree to “server”.    This bridge is still a tech preview, so I guess some bugs are to be expected.

• I also upgraded from Seam 2.1.0 to 2.1.1.GA as that is the version tested with the Bridge version I was using.  This was a simple replacement of the Seam-related JAR files with the newer versions.

• Upgraded RichFaces from 3.3.0.GA to 3.3.1.B2 as again, that was what was listed as tested for the new Bridge version.  This was also a simple replacement of the three associated JAR files with the latest versions.

• Upgraded JBoss Portal 2.7.0 to 2.7.1 for the same reasons as above.

Sorry to say I am not sure if the Seam, RichFaces, and Portal updgrade were necessary as I just went with the “big bang” approach based on what was tested for the Bridge version I needed.

FYI – I discovered the tree issue was caused by the Bridge because I could run the Seam application outside the portal as a stand-alone webapp and things worked fine.  Once I determined that, I was able to focus on the Bridge as the cause, and not a problem I was causing implementing the tree or CMIS.

Hope this follow-up helps if you are planning a similar approach.

Alfresco Integration with JBoss Portal

Wednesday, April 8th, 2009

A common use case we see is clients wishing to integrate Alfresco ECM with or within their existing portal products.  In the past, the most common way to do this was to place the entire Alfresco WAR within a portal container and deploy the product as a portlet.  Although this approach works, it is not useful in many cases as you are limited in the customization of the look and feel with the rest of the portal.  Plus, the repository itself is now tied directly to the portal container in terms of scalability.

With the release of Alfresco v3.x a few months ago, the options for integrating with a portal or other custom web application have greatly increased.  I will discuss a few of the approaches that are now available as well as my thoughts on the benefits and drawbacks of each.   To be sure we are on the same page, I am using Alfresco 3.0.1 Enterprise Edition and JBoss Enterprise Portal Platform 4.3.

Approaches

1. Alfresco Presentation Web Script and JSR-168 Portlet

With this approach, you call an Alfresco “web script” from within a portlet.  The web script is hosted in Alfresco’s repository, which can be local / remote from portal container.  Before I go any further, a quick summary on web scripts:

A web script is Alfresco’s terminology for what is basically a RESTful request to the repository over HTTP.  A web script can return XML, HTML, JSON, etc.  Alfresco comes with some web scripts out of the box and as of v3.0, that number has greatly increased.  Alfresco refers to a web script that returns HTML as a”presentation web script” and if it returns raw data in XML or some other format, it is a “Data Web Script”.  More on web scripts.  You can also write your own web scripts as needed.

In my example, I called a provided web script that returned HTML and displayed the HTML in the portlet.  Here is the gist of the code needed:

And here is a screen shot of the resulting portlet running:

Thoughts:

• Easy with quickly had something to show.  I had this running with something to show in less than 20 minutes.
• Portal developer has almost no control over the look-feel.  Of course the web script can be customized, but that work must be done in the Alfresco environment, not the portal.  This can be a pro or a con depending upon your work environment and preference for portlet development tooling.  Do you want to use JSP or some of the web frameworks that are standard across portals or do you want to use Alfresco’s approach and create presentation web scripts with FreeMarker?
• The paging and links do not stay within the portlet.  This is because the HTML is generated outside of the portlet rendering life-cycle.  I am not sure if this could be routed through some sort of portlet-bridge to make it part of the life-cycle.  I did not spend much time investigating beyond the JSR-168 portlet proxy class Alfresco provides (see mention of this in my Summary below).

2. Alfresco Data Web Script and JSR-168 Portlet

Like the approach above, this option also calls a web script from a portlet.  The web script can be hosted on a local or remote Alfresco instance.  In this case, the web script returns some format such as XML or RSS.  The portlet developer is then responsible for formatting the data as desired within the portlet itself.

Thoughts:

• Data can be formatted as desired and even placed in some sort of web framework used to produce the other portlets – IE: Grails, Seam, Struts, plain JSP, etc.
• Loose coupling between portal and Alfresco.  No Alfresco code has to be deployed to the portal, which I see as a good thing.
• Need to code custom parsing of web script responses in portlet.  Because there is no standardization in web scripts, the author can form the XML in any manner.  That means, the portlet developer may have to write custom parsing logic for each web script they call.  This is probably controllable in most environments, so not a huge deal and something to be aware of.
• I like this approach better than #1 for sure, but things can get better.

3. CMIS & Seam Portlet

The direct web script access as described in options #1 or #2 both also have a potential drawback in that they are both proprietary to Alfresco. If you ever switch or add an additional content management system to the mix, you would be starting from scratch.

However, the very good news is that Alfresco v3.0 introduced one of the first CMIS implementations and I see it is now available as a technology preview in Alfresco v3.1.  Content Management Interoperability Services (CMIS) is a specification developed by EMC, IBM, Microsoft, and a few other vendors including Alfresco.  The purpose of CMIS is to allow developers to use a common API to access any content management system that offers as CMIS implementation.  This is a bit like using SQL to access any relational database – really a no-brainer I think.

Alfresco implements their CMIS API on top of both web scripts, which we discussed above, and SOAP web services.  So, you have a choice to use CMIS via RESTful or SOAP web services.  Per the specification, the RESTful binding uses the ATOM Publishing Protocol.  For my purposes, I chose to use the REST/ATOM binding.

I mentioned that Alfresco based their CMIS REST binding on top of their web scripts platform.  However, unless you look under the covers, you really don’t know or care that this is the case.  As a client, you deal only with the CMIS API.  This means your client portlet code is not tied to Alfresco.  It will work with any system that provides a CMIS implementation.    Further, using CMIS means your XML parsing can be standardized and re-usable since it must follow the ATOM protocol.  You are no longer tied to parsing whatever XML the web script author decided to deliver.

In my example, I decided to use a Seam application to act as the Alfresco client using the CMIS REST/ATOM binding.  I then exposed the Seam application as a portlet within the JBoss Portal.  I chose Seam because I knew I was going to need the paging, caching, and session management it provides and I did not want to write or worry about all that – I had my hands full coming up to speed first on the CMIS specification and then on the ATOM/APP protocol.  I used JBoss Enterprise Portal Platform 4.3 which contains a tech preview the required JSR-301 portlet bridge to expose the Seam application as a portlet.  This should also work in the latest JBoss Community portal or as a standalone web application running in JBoss, Tomcat, or any of the other major app vendors.

For the Seam portlet application, I used Java and Apache Abdera to assist with the binding/parsing of the ATOM XML.  If I had to do it again, I would maybe look to using Groovy inside the Seam application instead of Java.  For the simple reason that I was mainly dealing with XML parsing, which I think Groovy excels at.

Here are a couple screen shots showing the portlet.  I basically took the same approach (read: blatantly copied) that my co-worker Shane Johnson took with his CMIS Explorer project.  To date, I have only implemented the getRepository(), getTypes(), and the 1st level on the getChildren() CMIS operations.  However, that was enough for a POC and also proved to me that this was the approach to take.

getRepository()

getTypes()

getChildren()

Note in the screen shots I have paging and sorting.  These were some of the things I wanted to get from using Seam instead of writing myself.  I also cached the repo and type data in Seam’s Session context since that is very static data.  Right now I have the actual browser tab data in Seam’s conversational session, but I may take that out and not cache the children info at all so it remains 100% real-time.

Thoughts:

• I like this approach because it can work against any content management system that has a CMIS implementation, not just Alfresco.  This is a big deal, because many of our clients have a mix of CMS systems in their enterprise.  With this approach, the same portlet can access them all with the same code base.
• You will need to become familar with CMIS, ATOM/APP, and then Abdera if you choose to use it.
• This is new technology so that introduces some risk.  CMIS is only at 0.5 version and will probably change a bit.  It is also not supported by Alfresco as part of the enterprise edition and only provided as a preview.   Note that the Seam approach I took is supported by RedHat as part of their Tech Preview, but does not have an SLA attached.
• CMIS is only intended to cover the common CMS use cases.  You may need to do things that are not covered or not yet covered by CMIS.  In that case, I would use direct data web script access and write them to return the ATOM protocol.  This way your approach remains as close as possible to the CMIS-covered use cases.
• In my opinion, the benefits of having a solution that can cross CMS vendors outweighs the risks of adopting a new technology.

4. Other Options

Even though I would recommend #3 above, there are other options for portal integration that I will mention:

• With Alfresco 3.x, they WebScript framework was split from the UI code.  So, it is possible to host just the webscripts code inside the portal and access web scripts using Alfresco’s JSR-168 proxy portlet as described here.  I am not crazy about this approach because you still end up putting quite a bit of Alfresco code into your portlet and what you need exactly is not documented from what I could find.  I started down this path and added the Webscript framework.jar and then started adding missing Spring JARs, etc on a trial and error basis, until I finally gave up.  Alfresco makes a point that they split this out to allow WebScript Runtimes, yet they do not document what is required for actual client deployment?  Maybe I am just missing something obvious so if anyone knows where this is documented, let me know.
• You could stick with the “old” approach of deploying the entire Alfresco WAR as a portlet.  This may make sense if you truly need to reproduce the Alfresco Explorer web app inside your portal.

Summary

The CMIS approach outlined in #3 is by far my favorite.  If you or the organization is not comfortable using that new technology, then #2 will certainly work.  For #2, you can access most of the data web scripts Alfresco provides OOB with the ATOM format, so you can still standardize your client code as long as you can control that format is also used for any custom web scripts you may need.

I hope to get a chance to complete a few of the other CMIS operations for the Seam portlet.  If I do, I will post an update here.

Thanks to my colleagues Shane Johnson and Sten Anderson for putting up with my barrage of CMIS-related questions.  I would recommend you take a look at some of the leading work they have done with CMIS.

Existing Seam application converted to a JBoss Portlet

Wednesday, February 25th, 2009

Recently, I had the need to port a standalone web application that was written in Seam to be a portlet within JBoss Portal.   Overall, I was surprised at how easy this process was, but also wanted to write about a “gotcha” to maybe save someone else some time.

Some background:  I have a standalone web application written in Seam and the client wanted to add some basic content management around this web site.   One option was to integrate a JCR-compliant repository directly into the application in the same manner as my colleague did for another client and discussed here.  However, I chose to go with JBoss Portal because I was wanting more out-of-the-box functionality related to CMS.  IE: JBoss Portal is already wired to Apache Jackrabbit and has the CMS-related portlets already created that I would need.  This would also allow the client to eventually add in some additional applications in the future with a single sign-on for the users.

Porting the application from standalone to a portlet is misleading because what you are really doing is just adding the ability for the application to run as a portlet.  It can still run as a standalone web application as well.  Take note of that even if you don’t ever plan to do it in production.  The face that a developer can test and troubleshoot the application as a standalone application without introducing the additional portal complexities is a very useful practice.

How does this all work?

The JBoss Portal Bridge did the heavy lifting for me.  The bridge is an implementation of JSR-301 and goes beyond vanilla JSF integration to include Seam and Richfaces JSF applications as well.   I simply followed the directions included with the bridge download for integrating the Bridge, which is just 2 JAR files in my application’s WEB-INF/lib directory.  If you are not familiar with JBoss Portal, realize that a portlet is deployed as a WAR within the JBoss /deploy directory.   So, I had to add these 2 portletbridge JAR files to the existing Seam project.  I also had to add a few XML files required by any portal.  These are spelled out in detail with examples in the short documentation included with the bridge download so I won’t repeat them here.

So, I followed the directions in the documentation to update my existing WAR with 2 new JAR files and some additional XML and deployed to a JBoss Portal 2.7 instance instead of the plain JBoss App Server it had been running on.  I then brought the site up directly as a standalone site still in order to verify I had not broken anything by adding these new JARS and XML files.  All worked great.  Then, I switched over to the /portal context and as an admin I created a new portlet instance based on my new portlet and added it to a page.  However, when I accessed that new portlet within the portal, I received a not-so-pretty exception stack.

The “Gotcha” – or why I should read all the instructions first!

The stack trace was caused by a “method does not exist” error in a Richfaces class called by one of the bridge classes.  Looking into this, I realized the JBoss Portal Bridge was expecting and actually came bundled with a newer version of the RichFaces libraries than I had been using in the existing Seam application (Seam 2.1.0_SP1).   This is all spelled out in the wiki site for the Bridge – which I did not bother to read until I saw the problem!    Once I updated my Richfaces libraries to the versions provided with the Bridge things worked fine.  Note – if I had used the proper version of Seam tested with the Bridge version to begin with I would have been fine.

Final

Based on what Red Hat presented at the JBoss Virtual Event on February 11th, my understanding is that the next version of JBoss Enterprise Portal Platform will include support for the JBoss Portlet Bridge.  Plain JSF based portlets would have full support and Seam-based portlets will be supported as a Technology Preview, meaning they will not have an SLA attached to that support.

JBoss Tools 3.0.0 RC2 released today

Thursday, January 29th, 2009

I have been using RC1 and this next release seems to be mostly fixes.  However, it does have some new functionality as well.  The drag-n-drop to deploy is especially interesting.

Here is the link to the interesting stuff: http://docs.jboss.org/tools/whatsnew/

JBoss Seam support in latest IntelliJ release

Monday, November 10th, 2008

IntelliJ IDEA v8 now has Seam support out of the box. For a list of capabilities included see:

http://www.jetbrains.com/idea/features/newfeatures.html

They seem to cover everything you would expect in an IDE in terms of Seam-specific functionality.  I was happy/suprised to see the visual jPDL pageflow support.  First thing I want to do is to see if this also supports the jPDL process definitions as well.  I suspect it does, since the only difference is some schema naming, but will report back once I verify this.

From my perspective, the inclusion of tooling into the IntelliJ product is a key indicator of the traction the Seam framework has generated in the industry.

To recap, developers now have 3 choices for Seam support in an IDE:

Alfresco Share in the Enterprise

Friday, October 31st, 2008

Alfresco 3.0 Enterprise was released this week.  If you have been holding off previewing the new functionality while still in the Labs edition, you are in for a treat.   The thing that will catch your eye immediately is the new Share application that is included in Alfresco.

Nothing against the “old” Alfresco web client, but it is really geared towards administrators and document management back-office work.  With the addition of Share, Alfresco delivers a “Web 2.0″ application leveraging Flash and AJAX with a polished interface any business person will enjoy.

Share is heavily focused on Collaboration tasks and includes integration with popular Blogging, Wiki, and Forum/Discussion products out of the box.  And, of course, Share provides great interface into your more traditional document management library (think folders) as well.

But what really caught me eye was Share’s ability for user site creation.  Any business user, given the proper authorities, can quickly set up a “Site” in Share, invite users to the site, and assign permissions to users within that Site.   We have gotten requests for this in past from clients and always had to build a custom-type solution to accommodate this need.  Now, Share provides this out of the box.   A few places where I see this fitting in rather nicely are:

• Universities – Instructors can create a site for specific classes and students could then create sub-sites for team projects going on in the class.
• Laboratories – A site can be created for any project going on that requires collaboration.  Site members could include internal as well as external members.
• Manufacturers – Product development would benefit by having a site for each major product under design.  Engineers, vendors, and customers could use the site to collaborate on the specifications, testing, and design.

Keep in mind, all Site content and documents are stored in the Alfresco repository still.  So they are secured, versioned, searchable, and auditable as well.

Check out the screen shots here as they quickly give you a taste of the improvements Alfresco has made.

I plan to post on the other features of 3.0 in the future, so stay tuned.  Also, Kudos to the Alfresco team on this major release!

A Treat for Halloween – Performance Improvements in Alfresco Task Management

Friday, October 31st, 2008

I wanted to share an Alfresco customization that greatly increased the usability and performance when dealing with the workflow tasks.  First some background and then I will get into the solution.   On a recent project, we had the task of designing workflows to manage the client’s contract approval process.  We implemented these with several custom Advanced Workflows within Alfresco using the bundled JBoss jBPM tool.   It was the nature of the business that there were several parallel paths resulting in “lots” of active tasks at a given time.

We quickly found that by the time the number of tasks reached a couple thousand, Alfresco began having serious performance issues in querying and returning the tasks for management.   For instance, if someone wanted to page through all the “active” tasks, this took minutes and at some point completely failed with a Hibernate exception.

Paging through a large set of data is a common use case; however, Alfresco’s WorkflowService interface has no mechanism to handle this.  Instead it relies on returning all results to the client and it is up to the client to do the paging logic if needed.   In all fairness, this is probably because a majority of task management solutions are not dealing with thousands of tasks concurrently.   If however, your application does deal with many tasks or you just prefer a more efficient way to handle this, Alfresco can be extended without too much work.

The Treat
One of the simplest ways to visually page through data efficiently is to let the database do the heavy lifting for you and limit the number of rows returned.  Because both Alfresco and jBPM use Hibernate, the easiest solution I saw was to use Hibernate’s setMaxResults() and setFirstResult() functions.    These functions allow Hibernate to leverage the underlying database to return a large result set in chunks that work nicely within a UI page.

Step 1
Created a custom class called CustomWorkflowTaskQuery in order to pass the values for both MaxResults and FirstResult properties.  This class extends the org.alfresco.service.cmr.workflow.WorkflowTaskQuery .

public class CustomWorkflowTaskQuery extends WorkflowTaskQuery {

	Integer maxResults = null;
	Integer firstResult = null;

	public Integer getMaxResults() {
		return maxResults;
	}

	public void setMaxResults(Integer maxResults) {
		this.maxResults = maxResults;
	}

	public Integer getFirstResult() {
		return firstResult;
	}

	public void setFirstResult(Integer firstResult) {
		this.firstResult = firstResult;
	}
}

Step 2
Note that Alfresco class org.alfrecso.repo.workflow.jbpm.JBPMEngine is a plug-in implementation of the generic BPMEngine.  This is where the code dealing directly with the JBoss jBPM API resides.   This is the class we need to extend with our own customized version.   I created the CustomJBPMEngine class that extends Alfresco’s JBPMEngine class.  I really just need to “append” a few lines of code to the existing method JBMEngine.createTaskQueryCriteria().  If Alfresco had made this method protected, as one would maybe expect out of an extendable product, I could have overridden it, called super(), added my few lines and been handing out candy to the kids already.  Sadly they used private instead.   That means my subclass has no visibility to it and I am forced to copy the entire method and add my logic to the end.

For the same reason, I am also forced to copy the JBPMEngine.queryTasks() as well since it calls createTaskQueryCriteria() and would automatically use the original private method if run from within the parent JBPMEngine class.  See below:

public class CustomJBPMEngine extends JBPMEngine {

 /**
* exact copy from JBPMEngine - had to copy since the other methods were made private.
 */
public List queryTasks(final WorkflowTaskQuery query) {
	try {
		return (List) jbpmTemplate.execute(new JbpmCallback() {
		 public List doInJbpm(JbpmContext context) {
		  Session session = context.getSession();
		  Criteria criteria = createTaskQueryCriteria(session, query);
		  List tasks = criteria.list();

		 // convert tasks to appropriate service response format
		 List workflowTasks = new ArrayList(tasks.size());
		 for (TaskInstance task : tasks) {
		  WorkflowTask workflowTask = createWorkflowTask(task);
		  workflowTasks.add(workflowTask);
		 }
		 return workflowTasks;
	        }
		});
	}
	catch (JbpmException e) {
	  throw new WorkflowException("Failed to query tasks", e);
	}
}

/**
 * Had to override the Alfresco version since it does not allow for Hibernate offsets that we need for
* performance.
* Copied from JBPMEngine except for the very bottom of the method.
 */
private Criteria createTaskQueryCriteria(Session session, WorkflowTaskQuery query) {
  Criteria process = null;
  Criteria task = session.createCriteria(TaskInstance.class);

  // task id
  if (query.getTaskId() != null) {
 	task.add(Restrictions.eq("id", getJbpmId(query.getTaskId())));
  }

// task state
 if (query.getTaskState() != null) {
	WorkflowTaskState state = query.getTaskState();
	if (state == WorkflowTaskState.IN_PROGRESS) {
		task.add(Restrictions.eq("isOpen", true));
		task.add(Restrictions.isNull("end"));
	}
	else if (state == WorkflowTaskState.COMPLETED) {
		task.add(Restrictions.eq("isOpen", false));
		task.add(Restrictions.isNotNull("end"));
	}
}

// task name
if (query.getTaskName() != null) {
	task.add(Restrictions.eq("name", query.getTaskName().toPrefixString(namespaceService)));
}
// task actor
if (query.getActorId() != null) {
	task.add(Restrictions.eq("actorId", query.getActorId()));
}

// task custom properties
 if (query.getTaskCustomProps() != null) {
	Map props = query.getTaskCustomProps();
	if (props.size() > 0) {
	   Criteria variables = task.createCriteria("variableInstances");
          Disjunction values = Restrictions.disjunction();
		for (Map.Entry prop : props.entrySet()) {
			Conjunction value = Restrictions.conjunction();
			value.add(Restrictions.eq("name", mapQNameToName(prop.getKey())));
			value.add(Restrictions.eq("value", prop.getValue().toString()));
			values.add(value);
		}
		variables.add(values);
	}
}

// process active?
if (query.isActive() != null) {
	process = (process == null) ? task.createCriteria("processInstance") : process;
	if (query.isActive()) {
 	 process.add(Restrictions.isNull("end"));
	}
	else {
 	 process.add(Restrictions.isNotNull("end"));
	}
}

// process id
if (query.getProcessId() != null) {
	process = (process == null) ? task.createCriteria("processInstance") : process;
	process.add(Restrictions.eq("id", getJbpmId(query.getProcessId())));
}

// process name
if (query.getProcessName() != null) {
	process = (process == null) ? task.createCriteria("processInstance") : process;
	Criteria processDef = process.createCriteria("processDefinition");
	processDef.add(Restrictions.eq("name",    query.getProcessName().toPrefixString(namespaceService)));
}

// process custom properties
if (query.getProcessCustomProps() != null) {
	// TODO: Due to Hibernate bug http://opensource.atlassian.com/projects/hibernate/browse/HHH-957
	// it's not possible to perform a sub-select with the criteria api. For now issue a
	// secondary query and create an IN clause.
	Map props = query.getProcessCustomProps();
		if (props.size() > 0) {
		// create criteria for process variables
			Criteria variables = session.createCriteria(VariableInstance.class);
			variables.setProjection(Property.forName("processInstance"));
			Disjunction values = Restrictions.disjunction();
			for (Map.Entry prop : props.entrySet()) {
 			  Conjunction value = Restrictions.conjunction();
				value.add(Restrictions.eq("name", mapQNameToName(prop.getKey())));
				value.add(Restrictions.eq("value", prop.getValue().toString()));
				values.add(value);
			}
			variables.add(values);

			// retrieve list of processes matching specified variables
		List processList = variables.list();
		Object[] processIds = null;
		if (processList.size() == 0) {
			processIds = new Object[] { new Long(-1) };
		}
		else {
			processIds = new Object[processList.size()];
			for (int i = 0; i < processList.size(); i++) {
				processIds[i] = processList.get(i).getId();
			}
		}

		// constrain tasks by process list
		process = (process == null) ? task.createCriteria("processInstance") : process;
		process.add(Restrictions.in("id", processIds));
       }
}

// order by
if (query.getOrderBy() != null) {
	WorkflowTaskQuery.OrderBy[] orderBy = query.getOrderBy();
	for (WorkflowTaskQuery.OrderBy orderByPart : orderBy) {
		if (orderByPart == WorkflowTaskQuery.OrderBy.TaskActor_Asc) {
			task.addOrder(Order.asc("actorId"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskActor_Desc) {
			task.addOrder(Order.desc("actorId"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskCreated_Asc) {
			task.addOrder(Order.asc("create"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskCreated_Desc) {
			task.addOrder(Order.desc("create"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskDue_Asc) {
			task.addOrder(Order.asc("dueDate"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskDue_Desc) {
			task.addOrder(Order.desc("dueDate"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskId_Asc) {
			task.addOrder(Order.asc("id"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskId_Desc) {
			task.addOrder(Order.desc("id"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskName_Asc) {
			task.addOrder(Order.asc("name"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskName_Desc) {
			task.addOrder(Order.desc("name"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskState_Asc) {
			task.addOrder(Order.asc("end"));
		}
		else if (orderByPart == WorkflowTaskQuery.OrderBy.TaskState_Desc) {
			task.addOrder(Order.desc("end"));
		}
	}
}

// custom code starts here.
if (query instanceof CustomWorkflowTaskQuery){
	CustomWorkflowTaskQuery tempQuery = (CustomWorkflowTaskQuery) query;
	if (tempQuery.getMaxResults() != null) {
		task.setMaxResults(tempQuery.getMaxResults().intValue());
	}
	if (tempQuery.getFirstResult() != null) {
		task.setFirstResult(tempQuery.getFirstResult().intValue());
	}
}  // end of custom code.

return task;
}

/**
 * Straight copy from JBPMEngine since we have no visibility to it.
 */
private String mapQNameToName(QName name) {
	String nameStr = name.toPrefixString(this.namespaceService);
	return nameStr.replace(':', '_');
}
}

Step 3
The final step is to handle the Spring bean configuration files to ensure CustomJBPMEngine is used instead of JBPMEngine.   To do this, I created a custom-workflow-context.xml file under the Alfresco extension directory.  I copied the jbpm_engine bean configuration from Alfresco’s config/workflow-context.xml file and changed it to use my custom class instead.  This way, the custom class is registered with the generic BPM

 <bean id="jbpm_engine" class="org.cityofchicago.dhs.repo.workflow.jbpm.CustomJBPMEngine" parent="bpm_engine">
      	<property name="engineId" value="jbpm"/>
      	<property name="JBPMTemplate" ref="jbpm_template"/>
      	<property name="dictionaryService" ref="DictionaryService"/>
      	<property name="namespaceService" ref="namespaceService"/>
     	<property name="nodeService" ref="nodeService"/>
      	<property name="personService" ref="personService"/>
      	<property name="authorityDAO" ref="authorityDAO"/>
      	<property name="serviceRegistry" ref="ServiceRegistry"/>
      	<property name="companyHomeStore"><value>${spaces.store}</value></property>
      	<property name="companyHomePath"><value>/${spaces.company_home.childname}</value></property>
	<property name="unprotectedSearchService" ref="searchService"/>
   </bean>

Step 4
Now, when you want to use this, just call Alfresco’s workflowService.queryTasks() as you normally would, but pass in the new CustomWorkflowTaskQuery created in Step 1 above.  Remember to set the maxResults to the number you want to show per page and then set firstResult to the page# your UI wants to display.

Happy Halloween

BTW – As this seems to be a common use case, I opened an enhancement request with Alfresco along these lines.  If you agree, please cast your vote there.  https://issues.alfresco.com/jira/browse/ETHREEOH-460

Presenting at Alfresco Chicago MeetUp on Feb. 12th

Friday, February 8th, 2008

The Alfresco Community Meetup is a chance to learn about Alfresco and talk to your peers about how they are using Alfresco in their business space. CityTech will be teaming with Matria Healthcare to present a use case of how Alfresco is being used to solve their business challenges.

There will also be several informal (IE: laptop on pub table) demonstrations during the happy hour portion of the MeetUp as well. If you are interested in the Alfresco’s Advanced Workflow functionality or in how Alfresco can be integrated into your operation’s current application monitoring system, please stop by one the tables during a break and take a look.

The event is at a great Chicago location – Columbia Yacht Club, so we hope to see you there. You can register for the event at:

http://webcms.meetup.com/43/calendar/6927116/ 

Welcome to the blog of:

Jeff Brown