lojix/learning/src/main/com/thesett/aima/math/InformationTheory.java
public static double remainder(double[] pA, double[][] pGgivenA)
    {
        double result = 0.0d;

        // Loop over the probabilities for all the symbols of A calculating the contribution of each to
        // the expected value.
        for (int v = 0; v < pA.length; v++)
        {
            double phv = pA[v];

            // Calculate the expected information content of the distribution of the symbols of G given the symbol hv
            // and scale its contribution to the total by the probability of hv occuring.
            result += phv * expectedI(pGgivenA[v]);
        }

        // There is no need to convert the result from nats to bits and the expected information function is
        // already in bits.
        return result;
    }
public static double remainder(double[] pA, double[][] pGgivenA)
    {
        double result = 0.0d;

        // Loop over the probabilities for all the symbols of A calculating the contribution of each to
        // the expected value.
        for (int v = 0; v < pA.length; v++)
        {
            double phv = pA[v];

            // Calculate the expected information content of the distribution of the symbols of G given the symbol hv
            // and scale its contribution to the total by the probability of hv occuring.
            result += phv * expectedI(pGgivenA[v]);
        }

        // There is no need to convert the result from nats to bits and the expected information function is
        // already in bits.
        return result;
    }
public static double remainder ( double [ ] pA , double [ ] [ ] pGgivenA ) { double result = 0.0d ; // Loop over the probabilities for all the symbols of A calculating the contribution of each to // the expected value. for ( int v = 0 ; v < pA . length ; v ++ ) { double phv = pA [ v ] ; // Calculate the expected information content of the distribution of the symbols of G given the symbol hv // and scale its contribution to the total by the probability of hv occuring. result += phv * expectedI ( pGgivenA [ v ] ) ; } // There is no need to convert the result from nats to bits and the expected information function is // already in bits. return result ; }
Supposing a stream generates pairs of symbols. Let the first be A and the second be G. A ranges over a set of
symbols, {h1, ... , hv} and G ranges over a set of symbols, {g1, ..., gn}.

<p>G alone has a probability distribution which tells us what the expected information content of knowing what G
is, measured in bits.

<p>This remainder function estimates how much more information is needed, if we know A, in order to know G.

<p>If A and G are completely independent then this will be equal to the expected information content of G. In
other words A tells us nothing about G. If G is completely dependant on A then this will be equal to zero. In
other words as we know G already from knowing A, learning what G is will tell us nothing extra.

<p>Remainder(A|G) = Sum i from 1 to v {P(hi) * expectedInformation(P(g1|hi), ..., P(gn|hi)}.

<p>Where: p(g|h) = the probability of symbol g (of G) occuring given symbol h (of A).

@param  pA       The probability distribution of the symbols of A. This will be an array of size v.
@param  pGgivenA The probability distribution of the symbols of G given a particular symbol of A. This will be an
array of size n by v (v arrays of size n, array is indexed as [v][n]).

@return The expected remaining information needed to know G from knowing A.
Supposing a stream generates pairs of symbols . Let the first be A and the second be G . A ranges over a set of symbols { h1 ... hv } and G ranges over a set of symbols { g1 ... gn } .