Day Communique (CQ), Amazon S3, & Mule – Together, at last.
Thursday, October 30th, 2008
This week I integrated CQ with Amazon S3 via the JCR observation API and Mule.
First, the business need. The client was making extensive use of the CQ media library, and that is just fine. As media library assets are requested, they are cached on the web server(s). The issue at hand is file size and bandwidth. The client wanted to push the larger files to S3 to save on bandwidth, they also wanted to continue using the CQ authoring interface to manage media assets. Absolutely nothing wrong with that.
Second, the architecture. The idea is that when an author creates, updates, or deletes a media library asset we will send a request to a service. That service will fetch the asset and upload it to S3. Then it will return the new S3 URL to said asset in the response. Finally, upon receiving the response we will extract the S3 URL and add a new property to the media library asset with the value of said S3 URL. Now, whever we generate a media library asset URL we first check to see if this property is present. If it is, we return its value. The S3 URL. If it is not, we generate our standard CQ media library asset URL.
Now for the fun part. The technical implementation.
Step 1: Send a request whenever an creates, updates, or deletes a media library asset.
How are we going to do this? The JCR observation API. Whenever an author performs any of these actions, the repository will automatically fire events. The two of interest to me are ‘property added’ and ‘property updated’. In my case, we are focusing on specific assets. Downloads. As in not images or media. All I have to do is look for ‘property added’ or ‘property updated’ events (on a particular path in my case) where the property name is ‘File’. Whenever an author creates, updates, or deletes a media library ‘Download’ asset the ‘File’ property is created, updated, or deleted.
As I mentioned in a previous post, I have created a custom JCR application that exposes CQ content via RESTful services. It is Spring based and thus takes advantage of the Spring Modules JCR module. So, you can register JCR event listeners within the Spring configuration. My event listener listens for the particular events I just mentioned and sends a request to a service using HttpClient 4 when they are fired. I make a POST and I add two parameters. One for the CQ media library asset URL. One for the action (create/update/delete).
Spring Config
<bean id="myListener" class="com.company.MyEventListener">
<property name="jcrTemplate" ref="jcrTemplate"/>
</bean>
<bean id="jcrSessionFactory"
class="org.springmodules.jcr.JcrSessionFactory">
...
<property name="eventListeners">
<list>
<bean class="org.springmodules.jcr.EventListenerDefinition">
<property name="listener" ref="myListener"/>
<property name="absPath" value="/etc/medialib/somepath"/>
</bean>
</list>
</property>
</bean>
Event Listener
public void onEvent(EventIterator eventIterator) {
try {
while (eventIterator.hasNext()) {
final Event event = eventIterator.nextEvent();
if (event.getType() == Event.PROPERTY_ADDED ||
event.getType() == Event.PROPERTY_CHANGED) {
final String action;
if (event.getType() == Event.PROPERTY_ADDED)
action = "create";
else
action = "update";
final Property prop =
(Property) session.getItem(event.getPath());
final Node node = prop.getParent();
if (prop.getName().equals("File"))
postEvent(node, action, getMediaLibURL(prop));
}
}
}
catch (Exception ex) {
LOGGER.error(ex);
}
}
Step 2: Upload the asset to Amazon S3.
I don’t have any details on this particular step. My colleague implemented the service in Mule for me. I just send him the URL to the media library asset and the action (create/update/delete), and he takes care of the S3 business. For create/update actions, he uses the CQ URL to fetch the asset and upload it to S3. For a delete, he’ll use the URL (as a filename) to remove the asset from S3. Finally, he sends me a response with the new S3 URL in the case of a create/update action or just a 200 in the case of a delete.
Step 3: Update the media library asset with the new S3 URL.
The event listener sends the request using HttpClient 4. It then extracts the S3 URL from the response. Finally, it creates a new property on the media library asset node and sets the value to said S3 URL. Typically, with CQ, we have to update the CSD to add a new attribute. However, that isn’t really necessary. We usually update the CSD, because we want to update the dialog as well. In this case, this is not a user managed property. It is managed by the system. Also, one of the great things about the JCR API is that we can dynamically add properties to nodes. In my case I did not update the media library asset CSD. I just add the property programatticaly.
HttpClient (postEvent from above code)
private void postEvent(final Node node, final String action,
final String path) {
final HttpClient client = new DefaultHttpClient();
final HttpPost post =
new HttpPost("http://www.company.com/s3service");
final NameValuePair actionParam =
new BasicNameValuePair("action", action);
final NameValuePair pathParam =
new BasicNameValuePair("path", url);
final List<NameValuePair> params =
new ArrayList<NameValuePair>();
params.add(actionParam);
params.add(pathParam);
try {
final UrlEncodedFormEntity entity =
new UrlEncodedFormEntity(params);
post.setEntity(entity);
final HttpResponse response = client.execute(post);
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode == HttpStatus.SC_OK) {
final ResponseHandler<String> handler =
new BasicResponseHandler();
final String s3URL = handler.handleResponse(response);
node.setProperty("s3URL", s3URL);
session.save();
}
else
LOGGER.debug("It failed: (" + statusCode + ").");
}
catch (Exception ex) {
LOGGER.error(ex);
}
}
Step 3: Take advantage of S3.
When I work with CQ applications, I usually write a utility class for generate CQ URLs to media library assets. I then use that utility throughout the application. All did in this case was update that utility to first check if that asset has an S3 property. If it does, I use its value for the URL. If not, I continue building a CQ URL.
So until next time, good fight, good night.
3 Responses to “Day Communique (CQ), Amazon S3, & Mule – Together, at last.”
Leave a Reply
Pages
Calendar
| M | T | W | T | F | S | S |
|---|---|---|---|---|---|---|
| « Oct | ||||||
| 1 | 2 | 3 | 4 | 5 | ||
| 6 | 7 | 8 | 9 | 10 | 11 | 12 |
| 13 | 14 | 15 | 16 | 17 | 18 | 19 |
| 20 | 21 | 22 | 23 | 24 | 25 | 26 |
| 27 | 28 | 29 | 30 | |||
Very nice work by you and your colleague
–Whaley
how can i register EventListner with CRX 1.4.2
We just implements EventListner ……….
–>register with ObserverManager —> now we wont this should run in CRX as service ? how it possibal