This interesting read addresses the problem of discovering conversations within dialog blocks. Parallel conversational threads occur in many scenarios, such as within an online forum, in an instant message session, and in emails. Being able to tease these threads apart from one another is an important first step, before we try to understand the conversations within each thread.

There is, however, a dearth of suitable corpora that can be used to train systems to perform this task (which the paper terms DOC, or discovering ongoing conversations). The authors propose that we overcome this with the use of theatrical plays. It may be counterintuitive, but it turns out that theatrical plays may approximate conversation logs better than, say, extracts from a forum. The paper dissects each source into a set of summary statistics, and uses them to show why this is the case.

The authors also share a possible solution to this DOC task. They make use of a linear online learning algorithm (passive-aggressive online learning) over a combination of stylistic, syntactic, and semantic features. Particularly interesting is the use of distributed trees to capture syntactic features. These are actually low-dimension vectors that model sub-trees alongside a set of weights. This builds on the concept of word embeddings; instead of having vectors encode words, distributed trees encode sub-trees within each vector.

DOC is a difficult but worthwhile problem to tackle. The use of a theatrical plays dataset will definitely help kick-start more work in this area. What do you think of the use of plays as a proxy dataset? Are you curious about how word embeddings can be generalized into distributed trees? Then this paper would be worth your time.