Quantifying unique information

Publikationstyp
Journal Article
Date Issued
2014-04-15
Sprache
English
Author(s)
Bertschinger, Nils  
Rauh, Johannes  
Olbrich, Eckehard  
Jost, Jürgen  
Ay, Nihat 
TORE-URI
http://hdl.handle.net/11420/14055
Journal
Entropy  
Volume
16
Issue
4
Start Page
2161
End Page
2183
Citation
Entropy 16 (4): 2161-2183 (2014)
Publisher DOI
10.3390/e16042161
Scopus ID
2-s2.0-84899154558
ArXiv ID
1311.2852v2
Publisher
MDPI
We propose new measures of shared information, unique information and synergistic information that can be used to decompose the multi-information of a pair of random variables (Y,Z) with a third random variable X. Our measures are motivated by an operational idea of unique information which suggests that shared information and unique information should depend only on the pair marginal distributions of (X,Y) and (X,Z). Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures.
Subjects
Information decomposition
Mutual information
Shannon information
Shared information
Synergy
Computer Science - Information Theory
Computer Science - Information Theory
Mathematics - Information Theory
94A15, 94A17
DDC Class
004: Informatik
510: Mathematik