Be more precise about uncertainty

Note: The following is the latest in the current blog series about analysis and CI. It is excerpted from my new book, Competitive Intelligence: Improving Law Firm Strategy and Decision Making. You can learn more about the book and purchase it here.

The CI discipline has spawned some pithy maxims, one of which says intelligence does not predict the future. Another one tells us that intelligence need not survey every tree in the competitive forest to understand how the forces within that forest may affect our ability to perform there.

Heeding the spirit of these two maxims, intelligence analysts develop high tolerance for ambiguity and uncertainty. Consequently, intelligence reports tend to reflect these ambiguous environments by the inclusion of qualifier terms like certainly, almost certainly, highly likely, good chance, probable, likely, improbable, unlikely, highly unlikely, may, maybe, most, few, some, we doubt, we are confident, we believe, etc.

Although these qualifiers reflect analysts' inability to make crystal ball predictions, unnecessary confusion develops when analysts and decision makers fail to agree -- sometimes without realizing they disagree -- over the meaning of terms like "probable." One often-cited study about the interpretation of intelligence revealed that 23 decision makers interpreted the likelihood of a "probable" event's occurrence ranging from 25% to 91% (by Sherman Kent, "Words of Estimated Probability," in Donald P. Steury, ed., Sherman Kent and the Board of National Estimates: Collected Essays, published by the CIA, Center for the Study of Intelligence, 1994).

Other common qualifiers found in intelligence reports are phrases having to do with time, e.g., soon, often, usually, seldom, infrequent, at this time. An analyst who knows what she or he means by these phrases should replace them with more specific judgments, e.g., during 2011, before the economy turns around, by 2Q 2012. More specific opinions like these improve the intelligence and its usefulness.

Analysts must offer conclusions that are as precise as but no more precise than their analyses will permit. Therefore, a good way to further reduce uncertainty is to highlight all the qualifiers in a nearly final intelligence report. Then ask your colleagues with fresh eyes (those who did not work on the report) to tell you what these terms mean to them in more specific terms. If you learn from this exercise that the intelligence they received was not the intelligence you thought you had provided, edit your report accordingly.

Another opportunity to reduce uncertainty in intelligence reports appears as you prepare your findings to decision makers. Anticipate their most likely questions, but do not wait for the questions to be asked. Include the questions and your best answers in the report itself. By such simple means, intelligence reports can become more meaningful and useful to law firm CI clients.

Say what?

I'm blogging again, thanks to Russell Lawson who emailed me and said, "Just do it." The following is the first in a series of posts on the topic of analysis -- that much too mysterious process that converts information into intelligence.

Last week, as often happens, I was on an airplane. I was seated next to a nice man, an intelligent man.

Early in the flight during light conversation, he proffered the news that young people are spending too much time looking at computer screens instead of books and that their brains are being rewired in new ways. Studies were proving this to be true, he said.

"Oh," I said, "What do the studies say is happening to their brains?"

"I don't know," he responded, "but it can't be good."

Several hours later, he offered me more information. "I read an article recently that said 80% of Americans did not read a single book last year."

"Wow," I said. "How many Americans used to read one or more books a year?"

He frowned and said, "I don't know."

The above anecdote suggests an autumn meditation for law firm CI practitioners and their clients: What would improve your own firm's decision making -- more information or more analysis?

Prudence and analysis

Lately I’ve been thinking a lot about analysis, both in terms of competitive intelligence and in terms that affect me directly—like investment decisions, career management, whom to vote for, how my industry will be affected by the events on Wall Street and in Washington—things like that.

Anyone who has a passing acquaintance with competitive intelligence knows that analysis lies at its heart. Analysis is everything CI professionals do with information they gather to identify patterns and produce intelligence their decision-making clients use to improve their decisions.

With these issues on my mind, a paragraph in a recent New York Times column by David Brooks, "Why Experience Matters," caught my eye:

“What is prudence? It is the ability to grasp the unique pattern of a specific situation. It is the ability to absorb the vast flow of information and still discern the essential current of events — the things that go together and the things that will never go together. It is the ability to engage in complex deliberations and feel which arguments have the most weight.”

Hmmm, I thought, this sounds like analysis. In fact, I like very much Brooks’s definition of prudence as a synonym for analysis. His prudence is no Nervous Nellie of a virtue, but sagacity, shrewdness, intellectual discipline, and good judgment rolled up into one helluva risk-minimizing Athena-like attribute.

Brooks goes on to posit that prudence is acquired through experience. He says one gains experience through personal involvement and the study of history. He says prudent leaders have the ability to build and test models “… and apply those [models] to current circumstances to evaluate what’s important and what’s not, who can be persuaded and who can’t, what has worked and what hasn’t.”

I don’t recall seeing anywhere any better description of analysis and decision-making. I’m only surprised it came in the form of a political column. (Memo to self: Keep inspecting those assumptions.)

Finally, these connections among prudence, experience, and analysis also inform another ongoing argument I have with myself: When law firms build and staff their CI units, should they (1) hire CI professionals from other industries who have strong analytical skills, but no legal industry experience or (2) reassign people within the law firm to this unit who have strong industry knowledge and experience, demonstrated analytical instincts, but no CI analytical training or experience? Although this is a topic for another column, I already think the best answer will not be a simple “either-or” one.