The million-fold increase in computing power over the last few decades has made possible types of quantitative analysis that were previously available only to rulers with access to large amounts of menial but highly precise labor. Because the humanities generally tend to trail the sciences and the social sciences in adopting such new-fangled techniques. Instead, academics have preferred to write about such new technologies using existing frameworks: see the now-forgotten hypertext boomlet of the early-90s led by Stuart Moulthrop.

Here, I want to contrast two scholars who have seriously used quantitative analysis in literary criticism. My negative example is Franco Moretti; my positive example is Brian Vickers.

This piece is something of an appendix to my n+1 piece “The Stupidity of Computers,” because I found that the conclusions I presented there held just as true for literary analysis as they did for data mining, search engines, and online dating. Computers are still dumb, so we must be sure to be extra-smart in using them.

Moretti’s Materialism

Franco Moretti caused something of a splash in the last fifteen years by advocating a quantitative model for tracking literature’s paths. He seeks, in his notorious phrase, a “materialist sociology of literary form.” I’m not fond of the phrase, but it gives an idea of the overlapping circles that are at work here: quantitative, materialist (in the Marxist and positivist sense), sociological, taxonomic.

I do not endorse a never-the-twain-shall-meet split between natural sciences and human sciences, the Naturwissenschaften and the Geisteswissenschaften. Many literature departments and continental philosophers cling to the separation as though their jobs depended on it–which indeed they might. But the great inadequacy of quantitative methods applied to artistic and social forms (e.g., economics) still serves as a reminder that the human sciences are vastly more imprecise and variable than the most successful of the natural sciences.

I believe that the reasons for this, as I discussed at some length in my recent n+1 piece, “The Stupidity of Computers,” primarily have to do with language and the endlessly fine-grained distinctions and variances it forces upon us. The practical result is that almost any quantitative approach falls down, and I think Moretti’s is a particularly good example of such a failure.

Moretti’s work has foundered on two main points:

1. Presuppositions of taxonomies, ontologies, and evidentiary relevance that are then set up to be “confirmed” by the statistical model at work.
2. Sheer lack of evidence, caused by the vast scope on which Moretti is working.

I have written on the first issue of presuppsitions with regard to political issues: attempts to discern how “liberal” and “conservative” blogs link to one another rely on purely human classification of blogs into the two-category taxonomy selected by the researchers themselves, rendering the results highly dubious.

In the political sphere, complicated charts analyzing networks of links from one political blog to another show clusters of linkages tightly within sets of “conservative” and “liberal” blogs. Another chart from a separate analysis shows clusters using a different taxonomy: progressive, independent, and conservative. Who decided on these categories? Humans. And who assigned individual blogs to each category? Again humans. So the humans decided on the categories and assigned the data to the individual categories—then told the computers to confirm their judgments. Naturally the computers obliged.

“The Stupidity of Computers“, n+1 13

The same principle is at work in classifying books and sentences, as Moretti does. This sort of ontological question begging is so ubiquitous it should be the first question anyone asks on seeing a pretty chart or graph purporting to represent some aspect of human society.

The second issue is more particular to Moretti, which is that even with unprecedented processing power at his command, I gather Moretti still doesn’t have the resources to do the requisite analysis at the level which would be required.

Cosma Shalizi, expert statistician, has written extensively on these failings with regard to Moretti’s two main books, and his articles elaborate the problems sufficiently that I won’t go into much detail here. He criticized the statistical methods of Atlas of the European Novel, but his review of Graphs, Maps, Trees, which attempted to make a statistical case that new genres of novel tended to occur in bunche, cuts closer to the theoretical problems at hand:

[Moretti should] give actual causal accounts of how macroscopic patterns emerge from the interaction of many material bodies (notably, people and books), of the sort we know to exist, endowed with the kinds of abilities we know them to have.

This commitment may sound harmless, because contentless, but it does actually have implications.  It means that you have to do a lot of work to justify functionalist explanations (though it’s not impossible). It should make you very dubious about ideal types.  It should make you more interested in exploring variation, and not dismissing it.  It should make you very dubious about “practices” and other shared mental objects, at least as ordinarily conceived.  And it suggests a lot of productive directions, investigating communication, cognition, and the collective patterns they produce.

Graphs, Trees, Materialism, Fishing

If anything, Shalizi understates the difficulty. There needs to be a huge amount of work, and it must be done with partial, often inaccurate data from biased observers. The hermeneutic circle holds sway here. It’s not impossible. But the data analysis is vastly easier than the data collection, which computers still can’t do.

Vickers’ Wissenschaften

I have a positive example, however. Studies of authorship attribution are nothing new. They have frequently employed quantitative methods, albeit frequently in a haphazard and unsystematic way. In English, at least, the writer who has been subject to more of them than any other is William Shakespeare.

There has not been a good track record. Two attributions of new (and rather poor) works to Shakespeare in the last couple decades, “Shall I die?” and “A Funeral Elegy,” seem almost certainly incorrect. In the second case, the original claimant, Don Foster, has retracted the claim. In the first, it appears that no one except the original claimant supports the attribution anymore. Unfortunately, the original claimant, Gary Taylor, is the editor of the Oxford Shakespeare, so you can find “Shall I die?” there. What are you going to do?

Against that, though, there is Brian Vickers and particularly his recent work Shakespeare, A Lover’s Complaint, and John Davies of Hereford. (A short overview is available behind the TLS paywall.) This is a distinctive case because it’s trying to pull authorship of A Lover’s Complaint away from Shakespeare. The long (329 lines) poem was published in the same book as the sonnets in 1609, attributed to Shakespeare, but it has never garnered too much attention. There have been some doubts but, over the last 50 years, general endorsement of Shakespeare’s authorship.

Work had already been done to identify Shakespeare’s co-authors on Titus Andronicus, Pericles, Timon of Athens, and others, but this is the first instance I know of pulling a poem away from Shakespeare.

Vickers is upfront in his biases: he thinks the poem is lousy, too lousy to be Shakespeare’s, uninventive and moralistic. He attributes it instead to a mediocre but prolific poet named John Davies, whom I’d never heard of. But his method is far more solid than most. He recognizes the need for a holistic/hermeneutic understanding of the period and its literature, and so Vickers’ book takes both a top-down and a bottom-up approach, deriving general regulative and evaluative guidelines from a comprehensive knowledge of the period, then using atomic, discrete metrics to attempt to make meaningful distinctions. I believe this bi-directional method to be the only one that has a chance of success. (Moretti, in contrast, is entirely top-down, while much philological work is bottom-up.)

Here is a brief sample, taken from the TLS overview:

Davies did so on over 160 occasions. The author of the Complaint used “th’-” elisions frequently, and twice clumsily, not on words beginning with a vowel -as in Shakespeare’s “Th’expense of spirit in a waste of shame” -but before a consonant: “th’wel doing”, “th’smallest teene”. Davies regularly contracted the definite article without any concern for euphony, with such equally clumsy elisions as “th’reversions”, “th’reprobates” or “th’Gospel”. Critics have judged the contractions in “A Lover’s Complaint” to be a sign of Shakespeare’s later style, but here is a different explanation. The poet of the “Complaint” ended two lines with the formulaic phrase “forme receive”, once an infinitive (using the pleonastic do): “which did no forme receive”, and once a present tense: “all straing formes receives”, both times rhyming on “leave”. In John Davies’s poetry there are twenty-one instances of the rhyme word “deceive”; ten of them need a pleonastic do, twice rhyming with “leave”. The author of the “Complaint” took liberties with the English language to obtain a rhyme, coining the nonce-words “sawne” and “loverd”. In one poem Davies rhymed “wander” and “gander”, adding a marginal note justifying “The word . . . Gander for the Rime’s necessity”.

Brian Vickers, “A rum ‘do'”

So while the first section of the book, “Background,” is somewhat traditional literary criticism and close reading, the second section, “Foreground,” is tabulation across many metrics. Vickers downplays one of the most common (and easiest) forms of analysis, that of uncommon word occurrence, owing to the ambiguity of word placement and meaning. Vickers does use this analysis, but not to the exclusions of others, as some of his predecessors had in making the case for Shakespeare’s authorship.

Regardless, A Lover’s Complaint is only 329 lines long, so the sample size for any feature is quite small. Because of this, doubt should accompany any supposed distinction established along any one axis. Aware of this, Vickers is definitely trying for a preponderance of evidence, attacking the poem from different levels and angles. Those in favor of the attribution to Shakespeare (John Kerrigan and Katherine Duncan-Jones are the two most vocal, I believe) have not done a comparable study from the other side, which is what would be necessary to counter Vickers’ claims.

Similarly, statistical analysis is somewhat down-played, because the margins of error are so great. There are lists and collections, but they are presented in the sense of outlying features that distinguish the poem from other work. They are also presented in the sense of non-outlying features that are actually typical of work in the period against common knowledge. The presence of both types of claims is reassuring.

And it underscores something to bear in mind when reading the work of Moretti and others: beware statistical analysis on incomplete evidence. The challenge is in deciding on the metrics and applying them in the most exacting fashion, something that remains in the human sphere, not the computer. What computers have enabled is the ability to run some of the metrics on a large corpus rather quickly: a significant evidentiary boon, but not a paradigm shift. The most satisfactory uses are considerably less sexy than Moretti’s graphs: for example, searching for particular words and variations of a word across all (or most) Elizabethan literature.

(Again, I refer back to my n+1 article, where computers proved themselves most useful in dumb tasks performed in great quantity: lexical analysis rather than semantic analysis.)

For any analysis, there is also the ever-present problem of negative evidence. The significance of what words aren’t co-occurring is considerably harder to analyze, and so is frequently ignored. The case study Vickers gives is that of neologistic word formation with the prefix un-. Shakespeare indeed did form an awful lot of new words by attaching un- to existing words, a feature of A Lover’s Complaint. But Vickers cites Juergen Schaefer’s work in showing that the OED overrepresented Shakespeare (for assorted reasons) and that other writers had coined similar un- neologisms at a similar rate. Two of them were Thomas Nashe and…John Davies (133).

And in fact, when it comes to neologisms, A Lover’s Complaint introduces 11 Latin neologisms in 329 lines, including wackiness like annexions, fluxive, and Cautills (151). This is hardly sufficient evidence but it is the sort of heuristic metric that gives reason for doubt. Moreover, Vickers uses it to bash those who have claimed that the supposed lexical inventiveness of A Lover’s Complaint was unique to Shakespeare, when in fact it seems to have been a common feature among many writers of the period. Context is everything.

Above lexical and grammatical analysis, Vickers performs more complex rhetorical analysis. One striking example is Davies’ almost compulsive use of asyndeton (omission of conjunctions), helping to produce what’s been called the “cramped, gritty, discontinuous quality” of A Lover’s Complaint. Vickers has some fun citing instances of Davies piling on up to ten verbs in a row in his other work.

A high level of elision of other parts of speech and peculiar inversions of word order is also present. Here the qualitative element kicks in, as distinguishing “awkward” and “unnatural” inversions from “good” ones is simply not something that is uncontestable. Here Vickers returns to close reading and attempts to see what purpose inversions serve in the function and rhythm of the poem. This is not a positivistic method, but I’m with him in thinking it is more convincing than attempting to taxonomize inversions on a purely grammatical level as “unnatural” or not.

This alternation between the quantitative and the qualitative continues throughout the rest of Vickers’ book, and because Vickers is forceful in his aesthetic judgments, they do not always stand by themselves, but in tandem with the statistical evidence, they do gain a certain amount of force. Because I am not an expert in rhetoric and not an expert on the literature of the period, I cannot say how contestable some of the judgments are. But the work is there to be evaluated. Vickers also shows how metrics have been abused in the past to make other wrongful attributions, and so points out the pitfalls.

For rhetorical devices in particular, it helps a great deal that Vickers is immersed in the rhetorical taxonomy that Shakespeare and Davies themselves would have learned, and so applies metrics that they themselves would have used in constructing their works. This is where historical knowledge is crucial; the ready-made ontological categories that happen to be popular at a given time (structuralist tropes, for example) are less likely to line up as well as taxonomical devices that authors would have knowingly applied to their work at the time of creation.

Let’s look at some of the general metrics Vickers applies to A Lover’s Complaint, in order to see how that middle-ground is navigated:

• A set of six rare words occurring in Davies work and in the Complaint
• 21 common phrases like “high and low,” “wake and sleep,” and “gainst plus admirable moral principle” occurring disproportionately in Davies work and present in the Complaint
• Six instances of poetic diction unique to A Lover’s Complaint and Davies other work
• A reflexive fondness for a a handful of rhetorical figures, particularly the overuse of the word “all” (often twice in a line), occurring in the complaint and in Davies’ other work (and nowhere else save sometimes John Donne)
• A certain overlap in metaphorical vocabulary, such as the “congestion” and “compounding” of individual appetites into desire
• Of rhyme-word pairs in the Complaint not used by Shakespeare, 25% of those pairings do occur in Davies’ work

The last one gives an example of how inexhaustible the work is, since the analysis is only performed on rhyme-word pairs not occurring in Shakespeare and is not compared to other authors. Similar objections can be made to other metrics, but the thing to remember is that Vickers has still set the bar vastly higher than usual, and since such work is a progressive process, the details are at least there to be rebutted by someone who wants to perform some of the many remaining analyses. To use an apt legal analogy, such matters are always decided by the standard of a preponderance of evidence, not a K.O., and right now it appears that Vickers has presented the preponderance.

Harold Love, in the TLS, in fact criticized Vickers for not using more sophisticated statistical and computational tools, and disputed the certainty of the Davies’ attribution. Vickers’ response displays, in my opinion, the right attitude to take toward such tools and analyses:

Your readers are told that Vickers uses “an old-fashioned kind of testing”, and “lacks real understanding” of modern computational stylistics, such as “John Burrows’s Delta algorithm, based on complex statistical probing of lengthy frequency lists”.

To reply in brief: I am perfectly familiar with modern stylometry, but the Delta algorithm uses small vocabulary samples, such as the fifty most frequently occurring words, which are then treated as individual counters, deprived of semantic identity and grammatico-syntactical relationships with other words. It has achieved some success, but it is open to suspicion that two or more writers might favour the same fifty words. I am currently working with a team of medical statisticians who are applying to computational stylistics a technique developed for measuring irregular heart beats, which is able to use all the words in a text. It lists shared words in descending frequency and then uses a phylogenetic algorithm to create a tree, grouping similar texts on nearby branches and dissimilar texts on distant branches. (Intertextual distances are measured, applying a weighting function which is the sum of Shannon’s entropy–details on request.)

Brian Vickers, 13 July 2007

Love definitely erred in endorsing the Delta algorithm, which is indeed dangerously arbitrary in its metrics. Endorsing any one metric, in fact, is liable to be dangerously arbitrary because of evidentiary limitations. Fancy statistics run the danger of obscuring the anecdotal component of the work.

Vickers impresses, ultimately, because he bears three main points in mind: all such analyses are to be treated as incomplete, no one metric should ever be seen as definitive, and conclusions must be based on a pluralistic methodology utilizing both quantitative and qualitative metrics.

In conclusion: the human sciences are inexact and dismal, much like humans.

_________________

Afterword: Duncan-Jones and Kerrigan’s rebuttals to Vickers’ de-attribution were not sufficiently substantive to need addressing here. Kerrigan’s failure should not obscure my esteem of his panoramic study Revenge Tragedy: Aeschylus to Armageddon.)

In addition, I should say that Vickers’ critical sympathies are not necessarily my own. While I’ve learned a great deal from his work on the history of rhetoric, someone who can confidently announce of the sonnets, “These are, to state the obvious again, not homosexual poems”–and then cite Sonnet 20 as evidence–either sees friendship as a wildly fertile ground for manic jealousy, or else is using such a narrow definition of homosexuality as to make the observation tautological. I prefer Samuel Butler’s witty (though coy) formulation:

Fresh from the study of the other great work in which the love that passeth the love of women is portrayed as nowhere else save in the Sonnets, I cannot but be struck with the fact that it is in the two greatest of all poets that we find this subject treated with the greatest intensity of feeling. The marvel, however, is this, that whereas the love of Achilles for Patroclus depicted by the Greek poet is purely English,absolutely without taint or alloy of any kind, the love of the English poet for Mr W. H. was, though only for a short time, more Greek than English. I cannot explain this.

Samuel Butler