Rational Discussion of Controversial Topics
Economics Education Ethics Foreign Policy Government History Politics Religion Science

ClearerThinking’s Fact-Checking 2.0

Stefan Schubert          22 October 2015 01:25 PM

Cross-posted from Huffington Post (slightly amended). If anyone is interested in contributing to this project, e.g. by annotating debates in your own country, please get in touch.

Debating season is in full swing, and as per usual the presidential candidates are playing fast and loose with the truth. Fact-checking sites such as PolitiFact and have had plenty of easy targets in the debates so far. No doubt we are going to see more of that in the rest of the presidential campaign.

Research shows that fact-checking actually does make a difference. Incredible as it may seem, the candidates would probably have been even more careless with the truth if it weren't for the fact-checkers. To some extent, fact-checkers are a deterrent to politicians inclined to stretch the truth.

At the same time, the fact that falsehoods and misrepresentations of the truth are still so common shows that this deterrence effect is not particularly strong. This raises the question how we can make it stronger. Is there a way to improve on PolitiFact's and's model - Fact-Checking 2.0, if you will?

Spencer Greenberg of ClearerThinking and I have developed a tool which we hope could play that role. Greenberg has created an application to embed videos of recorded debates and then add subtitles to them. In these subtitles, I point out falsehoods and misrepresentations of the truth at the moment when the candidates make them.

We think that reading that a candidate's statement is false just as it is made could have quite a striking effect. It could trigger more visceral feelings among the viewers than standard fact-checking, which is published in separate articles. To over and over again read in the subtitles that what you're being told simply isn't true should outrage anyone who finds truth-telling an important quality.

Another salient feature of our subtitles is that we go beyond standard fact-checking. There are many other ways of misleading the audience besides playing fast and loose with the truth, such as evasions, ad hominem-attacks and other logical fallacies. Many of these are hard to spot for the viewers. We must therefore go beyond fact-checking and also do argument-checking, as we call it. If fact-checking grew more effective, and misrepresenting the truth less viable a strategy, politicians presumably would more frequently resort to Plan B: evading questions where they don't want the readers to know the truth. To stop that, we need careful argument-checking in addition to fact-checking.

So far, I've annotated the entire CNN Republican Debate, a 12 minute video from the CNN Democratic Debate (more annotations of this debate will come) and nine short clips (1-3 minutes) from the Fox News Republican Debate (August 6). My aim is to be as complete as possible, and I think that I've captured an overwhelming majority of the factual errors, evasions, and fallacies in the clips. The videos can be found on ClearerThinking.

What is perhaps most striking is the sheer number of falsehoods, evasions and fallacies the candidates make. The 2hr 55 min long CNN Republican debate contains 273 fact-checking and argument-checking comments (many of which refer to fact-checking sites). In total, 27 % of the video is subtitled. Similar numbers hold for the other videos.

Conventional wisdom has it that politicians lie and deceive on a massive scale. My analyses prove conventional wisdom right. The candidates use all sorts of trickery to put themselves in a better light and smear their opponents.

All of this trickery is severely problematic from several perspectives. Firstly, it is likely to undermine the voters' confidence in the political system. This is especially true for voters on the losing side. Why be loyal to a government which has gained power by misleading the electorate? No doubt many voters do think in those terms, more or less explicitly.

It is also likely to damage the image of democracy. The American presidential election is followed all over the world by millions if not billions of people. Many of them live in countries where democracy activists are struggling to amass support against authoritarian regimes. It hardly helps them that the election debates in the U.S. and other democratic countries look like this.

All of these deceptive arguments and claims also make it harder for voters to make informed decisions. Televised debates are supposed to help voters to get a better view of the candidates' policies and track-records, but how could they, if they can't trust what is being said? This is perhaps the most serious consequence of poor debates, since it is likely to lead to poorer decisions on the part of the voters, which in turn will lead to poorer political leadership and poorer policies.

Besides functioning as a more effective lie deterrent to the candidates, improved fact-checking could also nudge the networks to adjust the set-up of the debates. The way the networks lead the debates today hardly encourages serious and rational argumentation. To the contrary, they often positively goad the candidates against each other. Improved fact-checking could make it more salient to the viewers how poor the debates are, and induce them to demand a better debate set-up. The networks need to come up with a format which incentivizes the candidates to argue fairly and truthfully, and which makes it clear who has not. For instance, they could broadcast the debate again the next day, with fact-checking and argument-checking subtitles.

Another means to improve the debates is further technological innovation. For example, there should be a video annotation equivalent to, the web application which allows you to annotate text on any webpage in a convenient way. That would be very useful for fact-checking and argument-checking purposes.

Fact-checking could even become automatic, as Google CEO Eric Schmidt predicted it would be within five years in 2006.. Though Schmidt was over-optimistic, Google algorithms are able to fact-check websites with a high degree of accuracy today, whilst Washington Post already has built a rudimentary automatic fact-checker..

But besides new software applications and better debating formats, we also need something else, namely a raised awareness among the public what a great problem politicians' careless attitude to the truth is. They should ask themselves: are people inclined to mislead the voters really suited to shape the future of the world?

Politicians are normally held to high moral standards. Voters tend to take very strict views on other forms of dishonest behavior, such as cheating and tax evasion. Why, then, is it that they don't take a stricter view on intellectual dishonesty? Besides being morally objectionable, intellectual dishonesty is likely to lead to poor decisions. Voters would therefore be wise to let intellectual honesty be an important criterion when they cast their vote. If they started doing that on a grand scale, that would do more to improve the level of political debate than anything else I can think of.

Thanks to Aislinn Pluta, Doug Moore, Janko Prester, Philip Thonemann, Stella Vallgårda and Staffan Holmberg for their contributions to the annotations.

Would you like to read similar articles in the future?
-2 -1 0 +1 +2

                  Post Comment                   

Recommended for You Optimates Populares Centrists

Show comments            Sort by        

melian 22 October 2015 04:53 PM

How can you make sure that commenters go equally hard on all speakers and their personal political sympathies do not affect their comments?


Stefan Schubert 23 October 2015 03:42 AM

Yes, that can happen, though I think that the problem is made somewhat smaller by pointing out all fallacies, falsehoods, etc. That way, you can't cherry-pick fallacies and falsehoods that you want to focus on.

Another method would of course be to have several independent checkers, though that would be time-expensive. I had some people commenting on the comments draft, but the final decision was my own.


melian 23 October 2015 05:01 AM

I think several independent checkers might be a bit more reliable provided they do not share the same political beliefs. The problem with pointing out all falsehoods is that people are highly subjective in deciding what is false. For example, if a candidate said that there is not enough evidence for the global warming, a republican commenter might consider this a true statement, a democrat commenter might write that this is false and the independent might decide that this is a matter of opinion.


is4junk 23 October 2015 05:00 PM

Politicians are normally held to high moral standards.

Are you sure?

They are at 7% lower then car salesman!


is4junk 23 October 2015 05:40 PM

The politican's job is to lie, to re-frame, and use every underhanded rhetorical device they can get away with. I would suggest that ones who aren't good at this don't get the job. Can you imagine a politician who didn't see a major policy question as a one-sided debate? Can you name one that got elected?

[s] Now the job of fact-checkers is purity, light, and goodness for all. Am I right? [/s]


ChristianKl 24 October 2015 01:02 PM

Can you imagine a politician who didn't see a major policy question as a one-sided debate? Can you name one that got elected?
I don't think that true.
Most politicians see a lot of issues as not one-sided.

Often they don't argue that way in TV debates but that's more because it's not effective, than because it's reflective of the views of the politican.


tauta_krypta 31 October 2015 03:11 PM

It's worth noting that even immediately saying that facts are wrong might not prevent them from being persuasive:
Do We Believe Everything We're Told?


VoiceOfRa 23 October 2015 06:06 AM

You failed to address, or even acknowledge the question, of who fact-checks the fact-checkers. For example, you mention PolitiFact, it has a acquired a reputation for downplaying some politicians lies, and in some cases even outright classifying true statements as lies by others.

In general, this proposal is just silly. After all the media is supposed to fack-check politicians but it is rather notorious for its own biases and even occasional lies. Why would we expect self-proclaimed fact-checkers to be any better?


ChristianKl 22 October 2015 01:43 PM

How do you deal with the copyright issues of hosting those videos on ClearerThinking?


Stefan Schubert 22 October 2015 02:23 PM

ClearerThinking's laywer said it should be fine to embed videos in this way. We don't adjust the video in any way, unlike many other annotation applications.