I had (...) decided it might be worth quantifying the rulings that PolitiFact gives. It was a way (in my view) to take them one step further: the best way to review a person’s rulings is individually because it’s important to see all the nuances, but calculating averages can give you a better view of that person among peers, and from one group to another (Democrats versus Republicans). So that is another part of my blog, called Politi-Score.Projects like Politi-Score are useful for partisan game-playing and little else. Selection bias renders the calculated averages worthless for every scientific inquiry other than measuring PolitiFact's selection bias (more on that here).
More from Street:
I (...) realized that there were several patterns emerging. One of those is the “six guidelines” or reasons which are referred to in the “Grading PolitiFact” condensed “Critique the Critique” matrix. These are common, general reasons for issues I have noticed seem to manifest with each critique.Apparently this means that Street thought of ways to systematically excuse PolitiFact. Let's see how they stack up:
Reason (1) “That’s not what we SAID we were looking for." What is PolitiFact’s (PF) method and goal in determining the truth of the fact? Do they state it in the article? If the person says we are not checking the underlying argument, just the statement, then Bryan’s claims are moot as to *ignoring the underlying argument.* PolitiFact may have implicitly stated it was not checking that in the article. And vice versa.1) When the PolitiFact author says only the literal truth of the statement will be checked it does not render my criticism moot. Rather, it provides compelling evidence that I am correct that the underlying argument was ignored. When PolitiFact examines the underlying argument in some cases but not in others this tends to translate to unequal application of standards and results in selection bias. A fact check should always employ the same standards regardless. That is the surest way to help ensure objectivity. Layered instances of selection bias increase the likelihood that political bias will taint the results.
Reason (2) “We can’t go there.” Would checking *caveats* or *giving a more charitable interpretation* force PF to move into ideological territory, which again, is precisely what it doesn’t want to do?2) Employing the same standards every time (see #1), including by always checking caveats and always employing the principle of charitable interpretation represent the best way of keeping PolitiFact from moving into ideological territory. When PolitiFact repeatedly fails to apply equal standards it brings into question whether entering ideological territory "is precisely what it doesn’t want to do."
Reason (3) “What is the focus of‘charitable’?” What is the agenda of Bryan’s frequent use of words charitable and uncharitable? Amusingly and ironically, according to the hesaurus, one of the words that can be used to replace *charitable* is *liberal.* Charitable also means “open handed” “sympathetic” and/or *helpful.* An antonym is “unforgiving.” Charitable can also be construed as, FAVORABLE. Bryan wants a more FAVORABLE interpretation….so, to what ends? Again, implying a more favorable interpretation should have been employed….favorable to who or what? Because making a more “charitable” interpretation might shift us into reason (2)—We can’t go there because it’s too ideological.3) These "reasons" trend toward inquisition, don't they? The goal of charitable interpretation is obtaining the best understanding of an attempt to communicate. Where I advocate charitable interpretation for all (which I do), the agenda is the best understanding of whoever happens to be writing or speaking.
Reason (4) “Too much information.” Does PolitiFact limit the length (number of words) of its article/ evaluations? Because providing all the context and detail (“to support the underlying argument” as Bryan might say) Bryan demands might not be possible. This is the reason I include a word count for the article on the condensed critique table.4) Obviously space is a consideration for journalists even on the Internet (and that goes triple or more for print). On the other hand, I do not ask for exhaustive ("all the context and detail") presentation of context. I simply point out such things as places where additional context significantly changes the picture. And there are ways of communicating an accurate picture, or at least a more accurate picture, without offending restrictions on space. If Karen ever asserts that my demands aren't possible it is reasonable to expect a demonstration. I doubt that a suitable case will ever occur.
Reason (5) “This is AS IS--NO returns.” If a pundit or politician makes a statement and then corrects it, should PolitiFact stand with its rating of the original statement even when the pundit or politician makes the correction? For example, when Rudolph Guiliani made the gaffe of saying that there were no attacks on America under G.W. Bush, Bryan wrote in his synopsis “…Sharockman's failure to note Giuliani's full explanation is nearly as perplexing as Giuliani's failure to make explicit note of the 9-11 attacks during his GMA comments. And contrary to Sharockman's opinion, Giuliani's explanation is quite defensible.” So, would going with Bryan’s opinion of Guiliani’s explanation make it any less biased?5) Another question! The answer is "That depends." In a case like Giuliani's, charitable interpretation should have made it clear both that he know of and viewed as an exception to his remarks the 9-11 attacks. Aside from that, charitable interpretation obligates us to accept any reasonable interpretation, including the reasonable one offered by Giuliani. Sharockman's dismissal of Giuliani's explanation was not reasonable, and was just as amenable to evaluation as Giuliani's statements. As a result, the answer to Street's final question is yes, because my supposed "opinion" of the reasonableness of Giuliani's explanation was based on sound reason while Sharockman's was not. Street's question contained a false premise. Naughty, naughty. If it's all opinion then there's no such thing as "PolitiFact." Let them rename it "PolitOpinion."
Reason (6) “What the hell do you expect, anyway?” Too much “DIPing”—Demanding Impossible Perfection….What would a reasonable expectation be of conclusions reached by an organization such as PolitiFact when evaluating statements by political figures? Do claims of possible ideological bias or journalistic errors or omissions as noted by one who IS biased invalidate the stated goal of their work? We read the complaints coming from both sides of the aisle, that PF is too left wing or too right wing. This should in and of itself prove they are not biased, or at least *trying* not to be. Even Bryan admits their bias is unintentional (and his own, intentional); so what does he expect them to do? There’s always going to be a certain small amount of subjectivity present. Bryan is also critical of their journalistic technique, but that should be a separate issue. If journalistic technique was a complaint that is problematic for a website, then the majority of right wing and left-wing websites have serious issues as well. So we can’t trust him as the judge of this.6) Please excuse Street's overlap with her reason #4. Everyone is biased, so the claim that anyone is unbiased is the most suspicious claim. And anyone of any level of bias can attempt to provide unbiased reporting. PolitiFact flouts the standards of objective reporting regularly (adverbs, snark); it is through the behavior that one judges the bias in reporting, not simply by noting the bias of the authors at the outset. "We read the complaints coming from both sides of the aisle, that PF is too left wing or too right wing. This should in and of itself prove they are not biased, or at least *trying* not to be." I encounter this fallacy with astonishing regularity. Indeed, it is exceptionally common coming from journalists. Most comments come from the extremes on any position (those nearer the center of the bell curve tend to care insufficiently to express themselves). Criticism from both sides does not indicate a lack of bias. It simply indicates that people may be found both left and right of the view expressed. I have criticism from the left and from the right. That does not remove my bias. It doesn't even reasonably suggest that I'm trying to be unbiased. It's long past time to put this mistake to rest forevermore.
Worth repeating:
If journalistic technique was a complaint that is problematic for a website, then the majority of right wing and left-wing websites have serious issues as well. So we can’t trust him as the judge of this.If what a majority of right wing and left wing Web sites do invalidates my judgment then it also invalidates Street's judgment ... so we can't trust Street to judge that I can't be trusted as the judge. Street has achieved self-stultification with admirable aplomb.
Summary:
In sum we have six obstacles placed on the road to truth courtesy of Karen Street. In each case, she would discourage the application of the best standards of judgment by providing excuses for the uneven application of standards.
What good is a blog that fundamentally opposes its own supposed purpose? Certainly a blog like that is what Bad Blogs' Blood was created to recognize.
Afters:
I offer thanks to Street for her charitable evaluation of my humble talents (referring to portions of the "about" page I did not elect to quote). If Street is not capable of doing a considerably better job with her blog than I am seeing thus far, then it counts against my judgment of her capabilities. I look forward to better work in the near future, hopefully including an extensive revision of the "about" page at Politi-Psychotics.
No comments:
Post a Comment