9
$\begingroup$

Search engines are increasingly being relied on as information gatekeepers, yet the criteria used by search engines to rank results is opaque to users. How can users be sure their results aren't biased or tampered with in some way to benefit some interest at the expense of search result quality?

Governments routinely demand that search providers remove or lower the ranking of websites deemed politically undesirable. Businesses may pay providers to boost certain results over others to increase their revenues. Firewalls may meddle with results before they're transmitted back to users.

Even seemingly innocuous changes to ranking algorithms that might not on the surface appear to be biased, could actually be deviously designed to harm websites that share some common attribute (unrelated to actual quality).

Is it possible to detect search engine bias, by say monitoring results over a period of time and evaluating whether some "hidden variable" (perhaps a political affiliation) is a driving factor in the change in website rankings?

A sneaky provider may gradually over time lower the ranking of targeted websites (and perhaps random websites as well to distract users). What are the limits on how much bias a provider can introduce without detection? Or is it possible to always conceal such interference by deviously selecting weighted ranking criteria that incidentally produce the intended result (by way of "data snooping").

Does any of this change if the ranking criteria is made public? Do we need to open-source the criteria search engines use?

This reminds me of the result that detecting whether or not a complex financial instrument such as a CDO has been tampered with by the seller is equivalent to solving the densest-subgraph problem:

http://www.cs.princeton.edu/~rongge/derivative.pdf

Thanks!

Improve this question
$\endgroup$
5
  • 4
    $\begingroup$ this is a cool question, but I would revise it by making sure to only ask one cstheory related question. The most obvious is to make this a reference request and ask "has anyone already looked at this?". If you are sure nobody has, then something like "how can this be modeled formally?" might be a good question. If you keep too many questions around, with some of them being potentially non-cstheory related, then it might be closed as "not a real question". $\endgroup$
    – Artem Kaznatcheev ♦
    Jan 31, 2012 at 1:25
  • $\begingroup$ Note that making a ranking scheme public opens it up to attack by spammers. An interesting variant would be: "is there a 'public-key' equivalent for rankings" $\endgroup$
    – Suresh Venkat
    Jan 31, 2012 at 16:45
  • $\begingroup$ @SureshVenkat "making a ranking scheme public opens it up to attack" sounds like you are suggesting <s>security</s> unbiasedness through obscurity ;). $\endgroup$
    – Artem Kaznatcheev ♦
    Jan 31, 2012 at 18:13
  • $\begingroup$ no, but that's why I asked about public-key versions of ranking schemes. $\endgroup$
    – Suresh Venkat
    Feb 1, 2012 at 3:04
  • $\begingroup$ Since non of the parties participating in the search process are supposed to be malicious users, a normal solution is to model the process as a game with selfish users. If modeled correctly, we can find out whether it is beneficial for the search engines to do such a thing or not. Then we can design a mechanism to prevent such a tampering. $\endgroup$
    – Helium
    Feb 2, 2012 at 7:49

1 Answer 1

Reset to default
3
$\begingroup$

This is obviously a very open ended question, but so as to stay on topic, here is one CS theory approach to the idea of "fairness" and how to enforce it.

"Fairness Through Awareness" Dwork, Hardt, Pitassi, Reingold, Zemel http://arxiv.org/abs/1104.3913

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.