Chapter Seven: A Framework for Evaluating

The main aim of education is practical and reflective judgment, a mind trained to be critical everywhere in the use of evidence.

—Brand Blanshard, Four Reasonable Men

We now arrive at the portion of the book that is most important for good reasoning, the portion that Parts One and Two have been pointing toward: the evaluation of arguments.

In Part One we saw that good reasoning is ultimately a matter of cultivating the intellectual virtues, including the virtues of critical reflection, empirical inquiry, and intellectual honesty. This requires close attention to arguments, since cultivating each of these virtues is greatly enhanced by skill in clarifying and evaluating arguments. And close attention to arguments is shorthand, really, for close attention to whether arguments have the four merits of clarity, true premises, good logic, and conversational relevance.

In Part Two we saw that clarity is the starting point. This starting point is not only a matter of asking whether an argument is clear, but is also a matter of enhancing the argument’s clarity through the clarifying process. This process includes two general procedures: outlining the argument in standard clarifying format, and, at the same time, paraphrasing the argument for greater clarity. Paraphrasing should accomplish three things: streamlining, specifying, and structuring. And it should be governed by two general principles. The principle of loyalty tells you to imagine that the arguer is looking over your shoulder, checking to be sure that the paraphrased argument reflects the arguer’s intentions. The principle of charity applies if the context does not indicate the arguer’s intentions; it tells you to paraphrase in a way that makes the arguer as reasonable as possible—to paraphrase according to what you probably would have meant had you expressed the same words under similar circumstances.

The whole point of clarifying is to make it simpler to determine whether arguments have the other three merits. The remainder of the book has to do with asking these three questions of clarified arguments: Are the premises true? Is the logic good? And, to a lesser extent, Is the argument conversationally relevant?

7.1 Standard Evaluating Format

Just as there is a standard clarifying format, so is there a standard evaluating format. It systematically links your evaluation to the clarified argument and provides a framework for considering the questions about truth, logic, and conversational relevance. Let’s start with the clarified argument from Scientific American about the air sacs that are spread throughout the bodies of most birds:

Standard evaluating format provides a simple system for discussing the truth of each premise, the logic of the argument, and (where appropriate) the conversational relevance of the argument. Begin with the main heading EVALUATION, and under it provide at least three subheadings: TRUTH, LOGIC, and SOUNDNESS.  (In some cases you will need a fourth subheading, CONVERSATIONAL RELEVANCE.)

Under TRUTH, provide an entry for each premise, and for each premise do two things: state whether you judge the premise to be true, and provide your defense of that evaluation. For the air sac argument, this part of the evaluation would look something like this:

EVALUATION

TRUTH

Premise 1. This premise is probably true, assuming large enough quantities of carbon monoxide are involved, since carbon monoxide is known to be poisonous to any animal when breathed in sufficient quantities.

Premise 2. This premise is probably true, since this is reported in Scientific American, known to be a highly reliable publication on topics of this sort, and there is no reason to doubt this particular report.

Under the next subheading, LOGIC, do the same things: state whether you think the logic of the simple argument is good, and provide a defense of that evaluation. The evaluation would continue roughly as follows:

LOGIC

The argument is valid, since it has the form denying the consequent.

It isn’t important at this point in the text that you understand the exact technical meaning of expressions like valid or denying the consequent. For now you only need to know what is intuitively obvious—that expressions like valid, very strong, and fairly strong are ways of saying that the logic is good, while expressions like invalid, fairly weak, and very weak are ways of saying that it is bad.

After this, under the heading SOUNDNESS, provide your summary judgment—sound or unsound—based on the two preceding sections of the evaluation. If you judge that the argument is not sound, state whether this is owing to a problem either with a premise or with the argument’s logic. But if you judge that it is sound, there is no need for further explanation since saying it is sound is the same as saying the premises are true and the logic is good. In our sample air sac case, it would look like this:

SOUNDNESS

The argument is probably sound.

Notice the argument is judged probably sound. Your judgment of the argument’s soundness cannot be any better than the poorest thing said under TRUTH and LOGIC. While under the heading of LOGIC the logic of the air sac argument is judged to be good, under the heading of TRUTH each of the premises is judged merely probably true. Thus, the argument cannot be evaluated as any better than probably sound.

We have not yet provided a place in the format for conversational relevance. This can be an extremely important question, but it will turn out that the majority of the arguments you evaluate will not appear to be defective in this way. My suggestion—which we will follow in this text—is that a fourth subheading, CONVERSATIONAL RELEVANCE, be optional. Include it when an argument is conversationally flawed, and under the heading explain how the argument is thus flawed. But otherwise omit it, simply for the practical reason that it will save you extra writing.

The air sac argument is, so far as I can tell, conversationally relevant. Without any context, there is no good reason to think that it begs the question or misses the point. We could imagine, however, contexts in which it would be conversationally flawed. Suppose, for example, that the same argument had been put forward by a laboratory assistant who was asked by the laboratory director to look into whether air sacs in birds played any role in their breeding. We would in that case add a fourth subheading, as follows:

CONVERSATIONAL RELEVANCE

Even though it is sound, the argument commits the fallacy of missing the point, since the point is to show whether the air sacs play any role in breeding, but the argument only addresses whether they play a role in breathing.

By following this format, you can develop the habit of systematically asking all the right evaluative questions of an argument, and you will always have a straightforward way of presenting your judgments.

7.1.1 The Conclusion

You may have noticed there is no place in this format for evaluating whether the main conclusion of any argument is itself true. This may initially strike you as a serious oversight. But it is not. What we are evaluating here is not the truth of the conclusion, but the quality of the reasoning for the conclusion. Suppose you decided that an argument was utterly unsound, yet at the same time suspected that the conclusion was true. That would be no problem. Recall that for any true statement, it is possible to offer a bad argument in the attempt to support the statement. (If it then turned out that you were especially interested in such a conclusion, it would be up to you to see if you could come up with a better argument for it.) On the other hand, suppose you had a strong hunch that a conclusion was false, even though the argument itself appeared to be sound. This would give you good reason to check more carefully for a flaw in the argument, one that may have initially escaped your notice. Or you could end up changing your mind and accepting the initially implausible conclusion.

7.2 Complex Arguments

There is no important difference between evaluating a simple argument and a complex one. If the argument is complex—that is, if it is a series of linked simple arguments—then, after the clarification of the complex argument, evaluate separately each simple argument that makes up the complex argument. Instead of the heading EVALUATION, use the heading EVALUATION OF ARGUMENT TO N, where n identifies the relevant subconclusion or conclusion. To illustrate, suppose other air sac experiments by Professor Soum, reported in the same Scientific American story, had independently narrowed down the role of air sacs in birds to either flight-enhancement or breathing-enhancement. Suppose further that the air sac argument had been the first part of a larger argument that was designed to settle this issue, concluding thus:

Therefore, since air sacs in birds are known to play a role in either flight or breathing, we can conclude that they play a role in flight.

The complex argument would be clarified thus (adding premise 4 and a new conclusion):

The evaluation, framed in standard evaluation format, would then look like this:

EVALUATION OF ARGUMENT TO 3

TRUTH

Premise 1. This premise is probably true, assuming large enough quantities of carbon monoxide are involved, since carbon monoxide is known to be poisonous to any animal when breathed in sufficient quantities.

Premise 2. This premise is probably true, since this is reported in Scientific American, known to be a highly reliable publication on topics of this sort, and there is no special reason to doubt this particular report.

LOGIC

The argument is valid, since it has the form of denying the consequent.

SOUNDNESS

The argument is probably sound.

EVALUATION OF ARGUMENT TO C

TRUTH

Premise 3 is probably true, since it is supported by an argument that we have seen is probably sound (see evaluation of argument to 3).

Premise 4 is probably true, since (according to my hypothetical addition to the actual story, for the sake of this illustration) the experiments are reported in Scientific American, known to be a highly reliable publication on topics of this sort, and there is no reason to doubt this particular report.

LOGIC

The argument is valid, since it has the form of the process of elimination.

SOUNDNESS

The argument is probably sound.

The evaluation of the first simple argument remains exactly the same, except for expanding the heading to say EVALUATION OF THE ARGUMENT TO 3. And we add to it the evaluation of the second simple argument—the evaluation of the argument to C. Premise 3 is the subconclusion of the complex argument—so it is both the conclusion of the argument to 3 and a premise in the argument to C. When evaluating its truth (under the heading EVALUATION OF THE ARGUMENT TO C) it is good to point out that the premise is supported by an argument that you have just evaluated as probably sound.

7.2.1 When a Simple Argument within the Complex Argument Is Unsound

In a complex argument, when one simple argument is sound it has an important effect on your entire evaluation. When you evaluate the subconclusion as a premise in the next simple argument, the soundness of the preceding simple argument serves as a good defense for judging its subconclusion to be true.

But this ripple effect does not naturally occur if the simple argument is unsound. Obviously, its unsoundness would not be something to appeal to in defense of the truth of the subconclusion. But—note this carefully—neither would it be something to appeal to in defense of the falsity of the subconclusion. Any statement, whether true or false, can have an unsound argument offered for it.

This presents an interesting problem: in a complex argument, you can evaluate a simple argument as unsound without its affecting your evaluation of the next simple argument. Thus, in a complex argument, you may evaluate as perfectly sound the argument to the main conclusion, even though the previous simple arguments have been unsound. This is as it should be. But at the same time, since the arguer has presented the complex argument as a whole, there should be some way of indicating earlier problems when you evaluate later simple arguments.

The solution is this: in a complex argument, when one simple argument is unsound and the next one is sound, qualify your evaluation of it as sound but not shown. In this way, you indicate that even though the simple argument is, in your judgment, sound, the arguer has failed to carry out the job of showing it to be sound by the previous simple arguments.

Here is an easy-to-understand example:

You have to be extremely good-looking to get hired as a lifeguard. Not many people are that good-looking, so it’s very tough to land such a job. For that reason, even though it would be great to work on the beach, most people should probably try to find some other sort of summer job.

This argument can be clarified as follows:

The subconclusion—Not many people qualify for a job as a lifeguard—seems clearly to be true. And even though the simple argument offered in its support is a bad one (premise 1, despite evidence you might gather from Baywatch reruns, is surely false), the simple argument from premises 3 and 4 to the main conclusion is a pretty good one. A very brief evaluation might take this rough form:

EVALUATION OF ARGUMENT TO 3

TRUTH

Premise 1. This premise is certainly false; it isn’t looks, but experience and ability, which qualify you for a job as a lifeguard.

Premise 2. This premise is very probably true. My observations are that most people are average-looking (it may even be that average-looking just means the way most people look).

LOGIC

The argument is valid, since it has the form of singular denying the consequent.

SOUNDNESS

The argument is unsound, due to the falsity of premise 1.

EVALUATION OF ARGUMENT TO C

TRUTH

Premise 3 is certainly true. Most people need a lifeguard just because they aren’t qualified to be one. Qualifying to be a lifeguard requires that you be in excellent physical shape, that you be able to swim well, and that you have extensive training. (Before completing the evaluation, note that even though the argument to 3 has just been evaluated as unsound, I have nevertheless defended here the truth of 3—but for entirely different reasons than those offered in the argument to 3.)

Premise 4 is probably true. Under most circumstances, it doesn’t make good practical sense for people to apply for a job if their chances of getting it are extremely low.

LOGIC

The argument is valid, since it has the form of singular affirming the antecedent.

SOUNDNESS

The argument is probably sound, but is not shown to be so by the rest of the argument.

Note that the argument to C is judged as probably sound (since the poorest thing said about it under TRUTH and LOGIC is that its premises are probably true). But, to reflect the unsoundness of the simple argument used to lead into it, it is noted that it was not shown to be sound by the preceding simple argument.

7.3 A Reasonable Objector Over Your Shoulder

Whenever you write anything, it is crucially important that you know who your audience is. You may be writing for introductory students, your professor, your parents, a customer, a friend, your professional colleagues, or the general public. Different writing is designed for different audiences. And this applies to argument evaluations. Often they are directed at the arguer, whom you may hope to prove wrong. When doing the exercises in this text, you will be aiming them to your professor, who will grade your paper. When you do them on your job, you may be aiming them to a potential customer, whom you may hope to convince of the flaws in your competitor’s product.

But in the background, your primary audience should always be you. You should be aiming to arrive at the best evaluation you can for your own sake—the evaluation that is most likely to result in your arriving at knowledge and the one most likely to cultivate the habits that would continue to be conducive to your arriving at knowledge. In short, always evaluate arguments with a view to being the most honest, critically reflective, and inquisitive thinker you can be.

It may not always be easy to think in this way when evaluating an argument. It can be much easier to think in terms of an opponent who must be won over. And this can be turned to your advantage. Recall that an important guideline for clarifying is to imagine the arguer looking over your shoulder, checking your paraphrase for loyalty to the arguer’s intentions. I now recommend that you be similarly accompanied while evaluating the argument. In evaluating, though, imagine that looking over your shoulder is a reasonable person who disagrees with your evaluation. This reasonable objector has roughly the same evidence that you have and possesses the intellectual virtues of honesty, critical reflection, and inquiry. What reasons are most likely to persuade this person to accept your evaluation? What objection is this person most likely to raise? Be sure to express your defense in a way that defeats—or ultimately agrees with—the objections of this hypothetical adversary. In this way, you are more likely to exemplify the intellectual virtues yourself.

7.4 Summary of Chapter Seven

Frame your evaluation of every argument in the standard evaluation format, thereby ensuring that you appropriately present and defend your evaluation of the truth of every premise, the success of the argument’s logic, and, when necessary, the conversational relevance of the argument. The key judgment in every case is whether the argument is sound—that is, whether it is successful with respect to both truth and logic. Failure in either respect makes the argument unsound; and the poorest judgment in either respect should be reflected in your evaluation of the argument’s soundness. (Thus, for example, an argument that is logically successful and with a premise you have judged to be probably true can, at best, be probably sound.)

When the argument is complex, separately evaluate each component simple argument. If one of the simple arguments other than the argument to the main conclusion is unsound, and if a later simple argument is sound, be sure the earlier failure is reflected by noting that even the sound argument has not been shown to be sound in the preceding portion of the complex argument.

While thinking about and writing your evaluation, imagine that a reasonable objector—an intellectually virtuous person who has roughly the same evidence you have but disagrees with you—is watching over your shoulder and must be persuaded by your evaluation.

7.5 Guidelines for Chapter Seven

• Evaluate the clarified argument in standard evaluating format.

• Your evaluation of the argument’s soundness should be no better than the poorest evaluation you have provided of its logic and of the truth of its premises.
• If you have judged an argument to be sound, but you find that you still have doubts about the truth of the conclusion, carefully examine the argument again. You may initially have overlooked a flaw.
• Evaluate separately each simple argument that serves as a component of a complex argument.
• In a complex argument, if one simple argument is unsound and a later one is sound, qualify your evaluation of the sound one by saying that it is sound but not shown. This applies only to complex arguments.
• While writing your evaluation, imagine there is a reasonable objector looking over your shoulder, one whom you must persuade.

7.6 Glossary for Chapter Seven

Reasonable objector—someone who has approximately the same information you have, who exhibits the virtues of honesty, critical reflection, and inquiry, yet who disagrees with your evaluation. Imagine that this is your audience for every evaluation you write.

Sound but not shown—evaluation to use under the SOUNDNESS subheading in a complex argument when one simple argument is sound but a preceding simple argument, on which it depends, is unsound. Using this terminology reflects the fact that even though this simple argument happens to be sound, the arguer has failed to show it to be so, by virtue of having supported it with an unsound argument.