Forecast ergodicity and instantaneous active information.

This dissertation introduces two new information theory tools for machine learning. Forecast ergodicity models prediction using algorithmic information theory, stating that future data can be forecasted if it has the same structure as past data. The novelty of this approach is that it is model-free, based only on data. The other tool is instantaneous active information. It measures the active information at each step of iterative searches that are assisted by oracles. The instantaneous active information characterizes the difficulty of each subsequent step of a search. Knowing the difficulty of each search step helps to make real-time decisions about conflicting goals. As a demonstration, this search analysis is applied to a phased array optimization problem where there exists a tradeoff between the time devoted to parameter optimization and signal transmission.