The past decade has seen an explosion of machine learning research and appli-
cations; especially, deep learning methods have enabled key advances in many
applicationdomains,suchas computervision,speechprocessing,andgameplaying.
However, the performance of many machine learning methods is very sensitive
to a plethora of design decisions, which constitutes a considerable barrier for
new users. This is particularly true in the booming field of deep learning, where
human engineers need to select the right neural architectures, training procedures,
regularization methods, and hyperparameters of all of these components in order to
make their networks do what they are supposed to do with sufficient performance.
This process has to be repeated for every application. Even experts are often left
with tedious episodes of trial and error until they identify a good set of choices for
a particular dataset.
Pattern recognition has its origins in engineering, whereas machine learning grew
out of computer science. However, these activities can be viewed as two facets of
the same field, and together they have undergone substantial development over the
past ten years. In particular, Bayesian methods have grown from a specialist niche to
become mainstream, while graphical models have emerged as a general framework
for describing and applying probabilistic models. Also, the practical applicability of
Bayesian methods has been greatly enhanced through the development of a range of
approximate inference algorithms such as variational Bayes and expectation propa-
gation. Similarly, new models based on kernels have had significant impact on both
algorithms and applications.
