I also met up with a number of old friends, met some people with whom I’d only had email or FriendFeed conversations and made new friends. We had a Blue Obelisk dinner, with Christoph awarding a Blue Obelisk to Nina Jeliazkova. This time round, we got a whole restaraunt to ourselves, thanks to Christoph, so conversations was much easier! Also the financial contribution towards the dinner from Bob Belford and Harry Pence was very much appreciated.

At this meeting I finally got round to making use of Twitter – it turns out it was quite useful for keeping running notes during a talk, as well as keeping track of other parallel sessions. Thanks to Egon for those extra tweets (though maybe Egon and I were being a bit obssesive!?). A quick hack I put together just before the meeting allowed Tweeters to visualize the Twitter stream emanating from the ACS meeting as a word cloud. Obviously, it works better with more people tweeting, but cute nonetheless.

As always, CINF hosts some great receptions and this meetings’ ones were no exception. Though the weather didn’t cooperate, the convention center was pretty nice in providing free wireless. This came in especially useful as we had a speaker with three talks in our program but was unable to make it to the meeting. With the wireless available, we successfully connected with him over Skype, and with me switching slides, were able to have him present (audio and video) his work. Definitely not a trend we want to encourage, but for emergencies, “Skype talks” are great!

At this meeting I also organized an experimental symposium consisting of lightning talks – 8 minutes talks on arbitrary (but hopefully interesting) topics in chemical information and cheminformatics. While we only had 5 speakers, we had a great set of talks – I’m still amazed at how Richard West got through 24 slides in 7 minutes so smoothly! While it could have been publicized better, we got a lot of good feedback and will be running a revamped version in Denver, next fall.

Overall a pretty good meeting, and my last meeting as CINF Program Chair. I had a great time in this role, and with the help of a very capable Program Committee, I think we were able to successfully develop interesting multidisciplinary programs over the last four meetings. As I step down, Rachelle Bienstock from the NIEHS will take over as Program Chair, and I wish her all the best. However, I’m not done with CINF just yet I’ve been elected as Chair-Elect of CINF for 2011 so will be switching roles (though I certainly hope to continue contributing to the CINF program in the future).

I’ll end with three suggestions for the ACS: 1. Seriously consider letting divisions to drop Thursdays 2. Reduce registration fees & do a better job on hotel rates 3. Fix the meetings to one or two places (preferably San Francisco).

Update: I had misstated Anthony Nicholl points from his presentation. The post is updated to the correct that.

]]> http://blog.rguha.net/?feed=rss2&p=743 0 http://blog.rguha.net/?p=738 http://blog.rguha.net/?p=738#comments Wed, 11 Aug 2010 04:31:12 +0000 Rajarshi Guha http://blog.rguha.net/?p=738 Sometime back John Van Drie and I had developed the Structure Activity Landscape Index (SALI), which is a way to quantify activity cliffs – pairs of compounds which are structurally very similar but have significantly different activities. In preparation for a talk on SALI at the Boston ACS, I was looking for SAR datasets that contained cliffs. It turns out that ChEMBL is a a great resource for SAR data. And with the EBI providing database dumps it’s very easy to query across the entire collection to find datasets of interest.

For the purposes of this talk, I wanted to see what the datasets looked like in terms of the presence (or absence of cliffs). Given that the idea of an activity cliff is only sensible for ligand receptor type interactions, I only considered compound sets associated with binding assays. Furthermore, I only considered those assays which involved human targets, had a confidence score greater than 8 and contained between 75 and 500 molecules. (If you have an Oracle installation of ChEMBL then this SQL snippet will get you the list of assays satisfying these constraints).

This gives us 31 assays, which we can now analyze. For the purposes of this note, I evaluated the CDK hashed fingerprints and used the standardized activities to generate the pairwise SALI values for each of the datasets (performing the appropriate log transformation  of the activities when required). The matrices that represent the pairwise SALI values are plotted in the heatmap montage below (the ChEMBL assay ID is noted in each image) where black represents the minimum SALI value and white represents the maximum SALI value for that dataset. (See the original paper for more details on this representation.) Clearly, the “roughness” of the activity landscape differs from dataset to dataset.

At this point I haven’t looked in depth into each dataset to characterize the landscapes in more detail, but this is a quick summary of multiple datasets. (Though a few datasets contain cliffs which are derived from stereoiomers and hence may not actually be real cliffs – since their activity difference may be small, but will look structurally identical to the fingerprint).

An alternative and useful representation is to convert the SALI values for a dataset into an empirical cumulative distribution function to provide a more quantitative view of how cliffs are distributed within a landscape. I’ll leave those details for the talk.

Sometime back, my colleague Trung posted an ad for a software engineer position, primarily working on our chemogenomics data application. Now, we’re also looking for a research informatics scientist. See the detailed ad for more information. For both positions, see the ads themselves for contact details. If you’d like to chat face to face I’ll be at the ACS in Boston this month, so drop me a line and we can chat in Boston.

Overall, the hybridization only fingerprint performs comparably to the extended fingerprint and better than the standard one. But at a small percentage of the database screened, it appears that this fingerprint outperforms both. Of course, this is only one dataset, and more MUV datasets should be analyzed to get a more comprehensive view.

Message queue systems

A number of such MQ systems are available such as ActiveMQ, RabbitMQ and so on. See here for a comparison of different MQ systems. Given that we already use Oracle for our backend databases, we use Oracle Advanced Queue (AQ). One advantage of this is that we can store the messages in the database, allowing us to keep a history of a screen as well as use SQL queries to retrieve messages if desired. Such storage can obviously slows things down, but our message throughput is low enough that it doesn’t matter for us.

In this post I’ll briefly describe how I set up a queue on the database side and show the code for a Java application to send a message to the queue and retrieve a message from the queue. The example will actually use the JMS API, which Oracle AQ implements. As a result, the code can trivially swap out AQ for any other JMS implementation.

Creating queues & tables

The first step is to create a queue table and some queues in the database. The PL/SQL to generate these is

So we’ve created a queue table called test_qt which will hold a queue called input_q. The plan is that we’ll have a process listening on this queue and processing each message as it comes and another process that will send a specified number of messages to the queue. The queue_payload_type argument to the create call, indicates that we can store any of the standard JMS message types (though we’ll be focusing on the text message type). We’ve also specified that for the input_q queue, messages will be retained in the database indefinitely. This is useful for debugging and auditing purposes.

Message producers & consumers

OK, with the queues set up, we can now write some Java code to send messages and receive them. In this example, the receiving code will actually run continuously, blocking until messages are received.

This example extends TimerTask. The strategy is that when the listener receives a message, it will create a new instance of this task and schedule it immediately on a new thread. As a result the message processing logic is contained within the run method. At this stage, we only consider messages that are of type TextMessage. If that’s the case we simply extract the payload of the message and print it to STDOUT.

You’ll note that we also create a unique listener ID and include that in the output. This is handy when we run multiple listeners and want to check that messages are being received by all of them.

Before looking at sending and receiving messages we need to initialize the connection to the message queue

The next step is to listen for messages and dispatch them for processing. The method below initializes the queue if it isn’t already initialized. After creating a consumer object, we simply wait for messages to come in. The receive method is blocking, so the program will wait for the next message. Once a message is received it creates an instance of this class and schedules it – when the thread starts, the run method will execute to process the message.

The final component is to send messages. For this simple example, it’s primarily boiler plate code. In this case, we specify how many messages to send. The DeliveryMode.PERSISTENT indicates that the messages will be stored (in this case in the DB) until a consumer has received it. Note that after receipt by a consumer the message may or may not be stored in the database. See here for more details.

In the code below, we can set a variety of properties on the message. For example, we’ve set an “application id” (the JMSXAppID property) and a correlation id. Right now, we ignore this, but it can be used to link messages or even link a message to an external resource (though that could also be done via the payload itself). Another useful property that could be set is the message type via setJMSType. Using this one can assign a MIME type to a message allowing the message processing code to conditionally handle the message based on the type. For more details on the various properties that can be set see Message documentation.

Running

The complete source code can be found here. To compile it you’ll need an OJDBC jar file as well as the following jar files (that come with the Oracle installation)

• $ORACLE_HOME/rdbms/jlib/aqapi.jar
• $ORACLE_HOME/rdbms/jlib/jmscommon.jar
• $ORACLE_HOME/jlib/jndi.jar
• $ORACLE_HOME/jlib/jta.jar
• $ORACLE_HOME/rdbms/jlib/xdb.jar
• $ORACLE_HOME/lib/xmlparserv2.jar

Once the code has been compiled to a jar file, we first start the listener:

In another terminal we send some messages

Switching to the previous terminal we should see something like

The fun starts when we instantiate multiple listeners (possible on different machines). It’s simple enough to execute the first invocation above multiple times and watch the output as we send more messages. If you send 10 messages, you should see that some are handled by one listener and the remainder by another one and so on. if the actual message processing is compute intensive, this allows you to easily distribute such loads easily.

Next steps

The code discussed here is a minimalistic example of sending and receiving messages from a queue. In the next post, I’ll discuss how we can represent messages in the database using a custom message type (defined in terms of an Oracle ADT) and send and receive such messages using Java. Such custom message types allow the Java code to remain object oriented, with the AQ libraries handling serialization and deserialization of the messages between our code and the queue.

One of the downsides that I see with Oracle AQ is that the only clients supported are PL/SQL, C and Java. While AQ implements the JMS API, it employs its own wire protocol. The lack of support for  AMQP means that a lot of client libraries in other languages cannot be used to send or retrieve messages from AQ. If anybody knows of Python packages that work with Oracle AQ I’d love to hear about them. (Looks like stomppy might support AQ?)

In a number of situations, this approach is unavoidable as the data processing stage requires human intervention (plate corrections, switching controls, etc.). But in some situations, we can automate the whole process – primary screen, automated analysis & compound selection and secondary screen. Given that most screens at NCGC are dose response screens, we can refine an automated pipeline by processing individual plate series (i.e. a collection of plates representing a titration series) rather than waiting for all the plates to be completed.  Another important point to note is that the different steps being considered here take different times. Thus screening a plate series might take 15 minutes, processing the resultant data and making selections would take 3 minutes and performing the secondary screen might take 10 minutes. Clearly the three steps have to proceed in the given order – but we don’t necessarily want to wait for each preceding step to be complete. In other words, we need the steps to proceed asynchronously, yet maintain temporal ordering.

One approach to automating such a process is the use of a message queue (MQ). The fundamental idea behind a MQ is that one creates a queue on some machine and then starts one or more processes (likely on some other machines) to send messages to the queue. These messages can then be retrieved by one or more listener processes. MQ systems provide a number of useful features beyond the core functionality of storing and distributing messages – these include message persistence, security policy, routing, batching and so on.

In our case, when a plate series is screened, the robot sends a message to the queue. Some process will be listening to the queue and when it sees a message, pulls it of the queue and processes the data from the screen for that plate series. Once processing is complete, the process sends another message to the queue (or another queue) from which yet another process (this one running on another robot) can pull it off and start the secondary screen on the selected compounds. Thus, as soon as a plate series is finished in the primary screen, we can start the processing and follow up, while the next plate series gets started. A message queue approach is also useful since messages can remain on the queue until the appropriate listener pulls them of for processing. A good queue system will ensure that such messages are delivered reliably and don’t get lost.

The diagram below highlights this approach. The solid lines represent the traditional workflow. Given that we’d manually process the screening data, we’d wait till all plate series are run. The dashed lines represent a message based workflow, in which we can process each plate series independently.

In the next few posts I’ll describe such a message queue based workflow that I’ve been working on these past few days. Currently it’s specific to a screen that we’re going to be running. The infrastructure is written in Java and makes use of Oracle Advanced Queue (AQ) to provide message queues and the facilities for receiving and sending message. I’ll describe a minimal implementation that makes use of Java Messaging Services (JMS) and the standard JMS message types and then follow on with an example using a custom message type that maps to a Oracle user defined type, allowing for more “object oriented” messages.

While the ACS meetings usually have some pretty interesting symposia and talks, it’s not always the best venue for getting breaking news. So to address this, CINF will be holding a short session consisting of lightning talks – 6 minute talks, strictly timed (think of loud bells and maybe even a fog horn) on any topic related to cheminformatics and chemical information.

Lightning talks are certainly fun (see Ignite for example). But it wouldn’t be fun if we had to hear 6 minute synopses of old work. So, for this session, we’re not going through PACS. In fact we’re going to accept submissions from July 17 to Aug 7. The expectation being that speakers are going to talk about recent developments and not rehash old work.

This is an experimental symposium, so we’ll likely have just 8 to 10 speakers. But I’m excited as this a brand new format and should be a lively session. Hopefully, the cheminformatics crowd can put on a good show! (Or as the Ignite motto states: ‘Enlighten us, but make it quick‘).

I’ve included the full announcement below

For the 2010 Fall meeting in Boston, CINF will be running an experimental session of lightning talks – short, strictly timed talks. The session does not have a specific topic, however, all talks should be related to cheminformatics and chemical information. One of the key features of this session is that we will not be using the traditional ACS abstract submission system, since that system precludes the inclusion of recent work in the program.

So, since we will be accepting abstracts directly, the expectation is that they be about recent work and developments, rather than rehashes of year-old work. In addition, talks should not be verbal versions of posters submitted for this meeting. Given the short time limits we don’t expect great detail – but we are expecting compact and informative presentations.

That’s the challenge.

What

• Talks should be no longer than 6 minutes in length. At 6 minutes, you will be asked to stop.
• Use as many slides as you want, as long as you can finish in 6 minutes
• Talks should not be rehashes of poster presentations
• Talks will run back to back, and questions & discussion will be held of off until the end

If you haven’t participated in these types of talks before here are some suggestions:

• No more than three slides for a 5 minute talk (but if you can pull of 20 slides in 6 minutes, more power to you)
• Avoid slides with too much text (and don’t paste PDF’s of papers!)
• A single chart per slide and make sure labels are readable at a distance

When

Aug 23, 2:45 PM

Submissions run from July 17 to Aug 7

Where

Boston Convention & Exhibition Center, Room 155. You can get a map of the concourse here

How

• Send in an abstract of about 100 – 120 words to cinf.flash@gmail.com
• We will let you know if you will be speaking by Aug 15 and we will need slide decks by Aug 20
• You must be registered for the meeting
• Note that the usual publication/copyright rules apply
• We will encourage live blogging and tweets (if we have net access)