Home > Trust across Disciplines > Reputation in trust - Regarding computer science

Trust across Disciplines

Reputation in trust - Regarding computer science

Reputation is one of the sources of trust: someone who has a good reputation is very likely trustworthy. As Kramer reported, research on trust development has shown that individuals' perceptions of others' trustworthiness and their willingness to engage in trusting behavior when interacting with them are largely history-dependent processes. Reputation therefore is the opinion which is publicly formed and held.

This topic has been studied in computer science. From this point of view, researchers developed computational systems that are able to manage reputational information among agents, in order to help them to make trust decision. Several approaches have been designed or even implemented but we can consider them in two main groups: one which require a central system that keeps traces for every information related to agents' past behaviours and third parties feedback, and one that has no need of a single database but chooses a decentralized design.

Another interesting direction for research derived from reputation is given by trust within social networks (Singh et al.). Other researchers pointed out some problems in automatic reputation management (Pavlov et al.). A completely different approach (Shrobe) considers reputation when applied to computers rather than agents, as strenght against hackers or viruses attacks.

Paul Resnick, Richard Zeckhouser, Eric J. Friedman, Ko Kuwabara

When you interact with someone over time, the history of past interactions informs you about the other party's abilities and disposition. [...] Moreover, the expectation of reciprocity or retaliation in future interactions creates an incentive for good behavior. [...] Between strangers, on the other hand, trust is much harder to build, and understandably so. Strangers do not have known past histories or the prospect of future interactions, and they are not subject to a network of informed individuals who will punish bad and reward good behavior toward any of them. [...] Reputation systems seek to restore the shadow of the future to each transaction by creating an expectation that other people will look back upon it. [...]

There remain significant challenges requiring further research and commercial development. Consider each of the phases of operation for such systems: eliciting, distributing, and aggregating feedback. Eliciting feedback encounters three related problems. The first is that people may not bother to provide feedback at all. [...] Second, it is especially difficult to elicit negative feedback, [...] only really bad performances are reported. [...] The third difficulty is assuring honest reports. One party could blackmail another - that is, threaten to post negative feedback unrelated to actual performance. At the other extreme, in order to accumulate positive feedback a group of people might collaborate and rate each other positively, artificially inflating their reputations. Distributing feedback, the second phase, also poses challenges. The first is name changes. At many sites, people choose a pseudonym when they register. If they register again, they can choose another pseudonym, effectively erasing prior feedback. A second difficulty in distributing feedback stems from lack of portability between systems. [...] Finally, there is also a potential difficulty in aggregating and displaying feedback so that it is truly useful in influencing future decisions about who to trust. [...]

Despite their theoretical and practical difficulties, it is reassuring that reputation systems appear to perform reasonably well. Systems that rely on the participation of large numbers of individuals accumulate trust simply by operating effectively over time.

Sandip Sen

There can be various combinations of prior and experiential knowledge that an agent can use to make interaction or partner selection decisions. It can also use reputation mechanisms to decide on who to interact with. Such reputation mechanisms assume the presence of other agent who can provide ratings for other agents that are reflective of the performance or behavior of the corresponding agents. An agent can use such social reputation mechanisms to select or probe possibly fruitful partnerships. Reputation based service and product selection has proved to be a great service for online users. Well-known sites like e-Bay and Amazon, for example, provide recommendations for items, ratings of sellers, etc. A host of reputation mechanism variants are used at various other Internet sites. Significant research efforts are under way, both in the academia and industrial research labs, that allow users to make informed decisions based on peer level recommendations. Most of this research develops and analyses collaborative filtering techniques. [...]

[Sen's] work is motivated by a complementary problem. [He] assume[s] that a user has identified one of several agents that can provide a service that it needs. The performance of the service providers, however, varies significantly, and the user is interested in selecting one of the service providers with high performance. As it lacks prior knowledge of the performances of the different providers, the user polls a group of other users who have knowledge about the performances of the service providers. The ratings provided by the other users constitute reputations for the service providers. An agent trusts, or selects to interact with, the service providers who have higher reputation. [...]

[Sen] develop[ed] a trust mechanism that selects the number of agents to query to ensure, with a given probabilistic guarantee, that it is selecting a high-performing service provider. The mechanism uses the knowledge of the percentage of agents in the population that is expected to be such deceitful agents.

Suzanne Barber & Joonoo Kim

Reputation is closely related to trust. [Barber and Kim's] definition of reputation is the amount of trust an information source has created for itself through interactions with other agents. If an information source consistently meets the expectations of other agents by delivering trustworthy information, it will increase its reputation. Likewise, not satisfying other agents' expectation due to either incompetence or maliciousness will decrease its reputation among agents. [...]

An agent can assume there must be the expected cost of recreating an equivalent reputation if it discards the current one. In this sense, reputation could also be viewed as an asset, something that provides a monetary flow to its owner. Cronk reports to "follow the example of animal behavior studies in seeing communication more as a means to manipulate others than as a means to inform them." In other words, most communication serves the purpose of social influence, defined as "change in one person's beliefs, attitudes, behavior, or emotions brought about by some other person or persons." If we accept this premise then the reputation of an information source not only serves as means of belief revision under uncertainty, but also serves as social law that mandates staying trustworthy to other agents. Though an agent can send unreliable information to other agents or even lie, the agent risks the reputation it has been building among agents. Agents with consistently low reputations will eventually be isolated from the agent society since other agents will rarely accept their justifications or arguments and interact with them. [...]

An agent learns reputations of other agents using (1) dissimilarity measures calculated from the belief revision processes (Direct Trust Revision) and/or (2) communicated trust information that contains reputations (Recommended Trust Revision). Agents utilize this model (1) to detect fraudulent information and (2) to identify potential deceptive agents as a form of Social Control, in which an individual member is responsible for security, not some global or special authority. [...]

In [Barber and Kim's] system, there exist two paths through which an agent can acquire the reputation of other information sources: 1. Indirect: [...] when there are conflicts among acquired knowledge, an agent initiates the reputation revision process. 2. Direct: when communication is available, an agent can ask the other agents about the reputations of third party information sources. [...]

Jordi Sabater & Carles Sierra

The use of previous direct interactions is probably the best way to calculate a reputation but, unfortunately this information is not always available. This is especially true in large multi-agent systems where interaction is scarce. [Moreover, an] agent could be a newcomer to a society [and the] society could be very large. Therefore, when the interactions with another agent are scarce it is not possible to assign it a reputation based just on direct experiences. It is in these situations when the social dimension of an agent may help by using information coming from other agents.

[Alongside to what is called "direct trust" or trust derived from direct experiences, an agent can use reputation in order to supply lack of information about a partner. Sabater and Sierra's] reputation model is divided into three specialized types of reputation depending on the information source that is used to calculate them. If the reputation is calculated from the information coming from witnesses we talk about the "witness reputation", if the reputation is calculated using the information extracted from the social relations between partners we are talking about the "neighbourhood reputation". Finally reputation based on roles and general properties is modelled by the "system reputation". [...]

All these model [direct trust and the three types of reputation] work together to offer a complete trust model based on direct knowledge and reputation. However the modular approach in the design of the system allows the agent to decide which parts it wants to use. [...]

Another advantage of this modular approach is the adaptability that the system has to different degrees of knowledge. [...] The system is operative even when the agent is a newcomer and it has important lack of information. As long as the agent increases its knowledge about the other members of the community and the social relations between them, the system starts using other modules to improve the accuracy of the trust and reputation values.

The last element [the authors consider] is the "ontological structure" [that] provides the necessary information to combine reputation and trust values linked to simple aspects in order to calculate values associated to more complex attributes.

Pinar Yolum & Munindar Singh

[Yolum and Singh] consider trust in open environments - large-scale, decentralized systems consisting of autonomous agents. The key problem is how an agent should trust another agent. [...]

Multiagent approaches take an empirical stance on trust, attempting to create trust based on evidence of some sort. The evidence could be local or social. Social trust means trusting an agent based on information from individual witnesses or from a reputation agency. The credential of the witnesses or reputation agencies are crucial for interpreting this second-hand information correctly. Hence, the trustwortiness of the information sources must be estabilished as well. Referrals are a powerful way to of ensuring that the sources themselves are trustworthy. Local trust means considering previous direct interations. There are valuable - since the trustor itself evaluates the interactions, the results are more reliable.

Previous approaches for trust emphasize either its local or its social aspects. By contrast, [Yolum and Singh's] approach takes a strong stance from both aspects. [...They] study peer-to-peer service networks consisting of autonomous agents who seek and provide information services [among their neighbours to fulfill their needs.] Here, the agents track each other's trustworthiness locally and can give and receive referrals. This enables [them] to address two properties of trust that are not adequately addressed by [other] current approaches. One, trust often builds up over interactions. That is, you might trust a stranger for a low-value transaction, but would only trust a known party for a high-value transaction. Two, trust often flows across service types. That is, you might assume that a party who is trustworthy in one kind of dealing will also be trustworthy in related kinds of dealings. [...]

Not all agents are service providers and hence all the agents depend on the few service providers present. This causes a potential bottleneck for three reasons. One, because of the incomplete connectivity of the agents in the network, many times not all consumers can reach the service providers. Two, if the providers are reachable by all, then the providers can become overloaded with queries. Three, if the agents do not learn the services offered and if their principal requests that service again, they will have to repeat the process of service location. These shortcomings motivate [agents] to find a way by which [they] can reuse some of the services already offered by some providers.

Information services can be easily cached by the agent, so that the providers need not generate them again. Caching aids the search for information since an agent that is looking for information can find it in some cache - its own cache or the cache of another agent. [...] A cache consists of a set of entries that contain the answer as well as some information about the quality or appropriateness of the given answer. A cache entry is hence a (query, answer, quality) triple. The quality of the cache entry is modeled based on how appropriate it is to the owner of the cache based on its interest. [...]

Even with small caches and fewer number of service providers, the agents can find answers successfully. If the agents are selective in the answers they cache or the answers they serve from their caches, then the chances of circulating poor answers are reduced. Further, caching results in the agents with similar interests to group together.

Elan Pavlov, Jeffrey S. Rosenschein, Zvi Topol

Previous studies have been suggestive of the fact that reputation ratings may be provided in a strategic manner for reasons of reciprocation and retaliation, and therefore may not properly reflect the trustworthiness of rated parties.

[...] When feedback providers' identities are publicly known, reputation ratings can be provided in a strategic manner for reasons of reciprocation and retaliation, not properly reflecting the trustworthiness of the rated parties. For example, a user may have an incentive to provide a high rating because he expects the user he rates to reciprocate, and provide a high rating for either the current interaction or possible future ones.

This type of strategic manipulation in the process of feedback provision is likely to occur also in decentralized reputation systems. [...]

It thus appears that supporting privacy of feedback providers could improve the quality of their ratings. [...] The logic of anonymous feedback to a reputation system is thus analogous to the logic of anonymous voting in a political system. It potentially encourages truthfulness by guaranteeing secrecy and freedom from explicit or implicit influence. Although this freedom might be exploited by dishonest feedback providers, who tend to report exaggerated feedbacks, it seems highly beneficial for honest ones, protecting the latter from being influenced by strategic manipulation issues as described above.

Howard Shrobe

The traditional approaches to building survivable systems assume a framework of absolute trust requiring a provably impenetrable and incorruptible Trusted Computing Base (TCB). Unfortunately, we don't have TCB's, and experience suggests that we never will. [Shrobe proposes that] we must instead concentrate on software systems that can provide useful services even when computational resource are compromised. Such a system will estimate the degree to which a computational resources may be trusted using models of possible compromises. [...] This, in turn, depends on the ability of the application, monitoring, and control systems to engage in rational decision making about what resources they should use in order to achieve the best ratio of expected benefit to risk.

References

Resnick, P., Zeckhauser, R., Friedman E. and Kuwabara, K. (2000). Reputation Systems: Facilitating Trust in Internet Interactions. Communications of the ACM, 43 (12). 45-48. Electronic version available.

To read another contribution of Resnick and friedman about pseudonyms and reputation related feedbacks, check the Security and Privacy in Trust page.

Biswas, A., Mundhe, M., Sen, S. & Debnath, S. (1999). A Bayesian Network Based Approach For Modeling Agent Relationships. Proceedings of the second workshop on Deception, Fraud and Trust in Agent Societies. Seattle, USA.

Sen, S. & Sajja, N. (2002). Robustness of reputation-based trust: Boolean Case. Proceedings of the first international joint conference on Autonomous agents and multiagent systems. Bologna, Italy.

Barber, S. & Kim, J. (2000). Belief Revision Process Based on Trust: Agents Evaluating Reputation of Information Sources. Proceedings of the third workshop on Deception, Fraud and Trust in Agent Societies. Barcelona, Spain. Electronic version available.

Barber, S. & Kim, J. (2001). Belief Revision Process Based on Trust: Simulation Experiments. Proceedings of the fourth workshop on Deception, Fraud and Trust in Agent Societies. Montreal, Canada. Electronic version available.

To see other works about belief revision and the role of sources of information, read our Sources role in Trust - Regarding Computer Science page, with a summary of the research at the LIPS Laboratory.

Sabater, J. & Sierra, C. (2002). Reputation and Social Network Analysis in MultiAgent Systems. In Proceedings of First International Joint Conference on Autonomous Agents and Multiagent Systems. 475-482.

Sabater, J. (2003). Trust and reputation for agent societies. Monografies de l'institut d'investigaciò en intelligencia artificial, 20.

Yolum, P. & Singh, M. (2003). Emergent Properties of Referral Systems. In Proceedings of The Second International Joint Conference On Autonomous Agents and Multiagent Systems, ACM Press.

Yolum, P. & Singh, M. (2003). Ladders of success: An Empirical Approach to Trust. In Proceedings of The Second International Joint Conference On Autonomous Agents and Multiagent Systems, ACM Press.

Udupi, Y.B., Yolum, P. & Singh M. (2003). Agent-Based Peer-to-Peer Service Networks: A Study of Effectiveness and Structure Evolution. In Proceedings of the Workshop on Agent Oriented Information Systems.

If you like to read more about referral systems, see a preliminary work by Singh and Yu in our Sources role in Trust - Regarding Computer Science page.

Pavlov, E., Rosenschein, J.S. & Topol, Z. (2004). Supporting Privacy in Decentralized Additive Reputation Systems. In Christian D. Jensen, Stefan Poslad, Theodosis Dimitrakos (Eds.): Trust Management, Second International Conference, iTrust 2004, Oxford, UK, March 29 - April 1, 2004, Proceedings. Lecture Notes in Computer Science 2995, Springer. Electronic version available. A more detailed summary of authors' proposal for privacy can be found in our Security and Privacy in Trust - Regarding Computer Science page.

Shrobe, H. & Doyle, J. (2000). Active Trust Management for Autonomous Adaptive Survivable Systems. In Self-Adaptive Software , P. Robertson, H. Shrobe, and R. Laddaga, editors, Berlin: Springer Verlag, 2001, pp. 40-49. Retrieved online.

For further references about Sen's publications, see his page at Tulsa University. It is also useful to visit the DAI Hards website, a research group led by Sen focused on MAS and machine learning.

If you like to read more about trust within social networks, you can download a paper by Bin Yu and Munindar Singh, Towards a Probabilistic Model of Distributed Reputation Management, from our AAMAS 2001 page.

For information about the Active Trust Management, visit the website at AI Lab, MIT.

If you want to know more about eBay reputation management mechanisms, visit the eBay feedback forum or read In Community We Trust: Online Security Communication at eBay by Josh Boyd, which develops a deep analysis of how users' reputation is managed through feedback ratings and other users' comments.

Other perspectives on reputation

This topic is studied also in economics and sociology. You may also want to read our contribution about reputation and its role in trust decisions.