The other day I asked the Internet for a ranked list of Bible verses. Not just the top 10 or top 100 or top 1000 verses (many sites offer those kinds of lists): I don't care what "top" means to some random website; I want to find out what verses are referred to the most (and least) in the great big modern textual medium of Internetwebbery.
Completely un-find-able.
No problem: I'll just run some kind of quick query. Iterate over every verse in every chapter in every book in the minimal Protestant canon (I'll get to the apocryphal and deuterocanonical books later) - and ask the Internet how many times people are talking about it. Make a list, two dimensions wide: Verse versus popularity.
Two problems immediately emerged: First, that asking the Internet to compare two search terms may be easy (Google Trends and, deeper, Google Correlate will do this) - but asking to count results many terms is hard. Simply invoking google.com/search?q=some+term will fail in most kinds of automated script: You must be an interactive human sitting at a real web browser for it to work (it's part of their terms of service). Other search engines have similar restrictions: Multiple on-demand comparison metrics is part of what makes an Internet search engine valuable and it is usually not allowed to be requested in bulk and on-demand for proprietary reasons if not for simply practical ones. (How difficult would it be to offer a free Internet search service to the entire world if everyone who felt like running a million-curl-requests-a-second script was permitted to?)
I should have anticipated this first obstacle. (It is such an obstacle - and drifts interestingly near willfully violating search engine terms of service that I will reserve its solution for follow-up discussion.)
The second problem is more surprising: That it is hard to find a simple, plaintext list of every Bible verse.
I thought surely someone had a simple, flat enumeration - the whole set - an entire collection of "Genesis 1:1" on through to "Revelation 22:21".
Nope.
So here you go:
7 under 8!
9 years ago