NOTE! This section doesn’t apply to primary sources, only to secondary.
Vetting of authors of secondary sources is a bit more complicated. Secondary source authors are analyzing, interpreting, and deconstructing events to find patterns and then tell you their conclusions as if it were fact. Sometimes secondary sources just extrapolate their favorite theories and pet peeves and present them as divine canonical works from above. You must be on the lookout for a dusting of bullshit, partial bullshit, and full-on steaming bullshit.
In the case of books, there is always the bias to make the book sell-able. This is why treasure books are so untrustworthy as research sources.
For evaluation secondary source authors, I just look for specific things and mentally add and subtract bonuses to their scorecard in my head. However, I will try to quantify it for those of you who want to try to understand how I evaluate my secondary source authors. I’ve broken it down into a scorecard method.
+6 if author is known to publish in peer-reviewed scholarly journals
+5 if written with citations that go to primary sources or +2 for secondary citations
+3 article is hosted on a government or university’s website (.gov or .edu)
+2 article hosted by a reputable news media (ie. NY Times, Washington Post)
+2 if author publishes other articles in non-peer-reviewed journals
+2 if author publishes contact mailing address
+1 if author publishes contact email
+1 appears written for a scholarly audience
+2 originates from government report ( modern or vintage, doesn’t matter)
-1 refers to any “they” as an vague Illuminati
-2 if hosted on a ‘free’ web hosting or free blog like xxxxxx.blogspot.com
-4 for no author name given to an article (government report/police report- no penalty as they are sometimes author-less reports)
-4 Writing is poorly organized, there are spelling and grammatical errors, all suggestive of superficial thinking and research
I start all articles at a default ‘5’ and add the bonus or penalty from there.
How to interpret the total:
0-3 or below – the author is to be almost ignored, don’t let them walk your dog, let alone tell you history.
4-7 the author may have some credibility but be cautious
8 – 14 it is safe to put faith in the authors credibility
15 and up, believe them
Of course, you should check some of the cited sources. If they turn out bogus, meaningless, or inappropriate, that is a major penalty. If I were to put a number on it, I’d say -15. This is why I discredit ‘treasure sign’ books which cite other known-bad treasure sign books. Garbage in will only get you garbage out.
If you really, really get into source authority, you may want to see the five levels of authority that the Journal of the Association for History and Computing (JAHC) has developed. Just google ‘JAHC authority levels’ or see the link below
http://quod.lib.umich.edu/j/jahc/3310410.0007.204?rgn=main;view=fulltext
