HMM-basierte Online Handschrifterkennung : ein integrierter Ansatz zur Text- und Formelerkennung
This thesis deals with different aspects of automatic online handwriting recognition, comprising methods for the entire recognition process, such as pre-processing, handwriting normalization, feature extraction and Hidden Markov Model (HMM) based modeling techniques applied to text and formula recognition. The objectives of the developed pre-processing steps are basically the normalization of writer dependent writing characteristics (e.g. writing speed, character size and inclination). In order to achieve these objectives, a shape conserving re-sampling has been developed in combination with an entropy based slant and skew normalization. The normalizing scaling of the handwriting is based on an iterative region detection and a subsequent scaling to a standard character core height. The investigated feature extraction methods concern trajectory features as well as bitmap features. Several trajectory and bitmap based feature extraction methods have been developed and evaluated. Five trajectory and three bitmap features have finally been tested and presented in more detail and an optimal combination of the different feature types has been proposed. Considering the dynamic characteristics of online sampled handwriting, the HMM framework offers a couple of important advantages. Consequently, a further chapter is dedicated to the question of the optimal HMM paradigm for the modeling of handwriting. Another important aspect is the investigation of a context dependent impact on the handwritten characters. Significant character variations have been observed with varying adjacent characters. In order to cope with these inconsistencies, multiple models have been introduced for a single character, depending on it's context characters. A drawback of this approach is the sparseness of the available training data for most of the introduced contextual models, which requires an appropriate model tying. For the purpose of parameter tying, a selective approach, a data driven approach and a decision tree based approach have been proposed and compared. All considered aspects have been considered under the assumption that a large or very large vocabulary (up to 200000 words) has to be used for recognition. Although a very large vocabulary may yield high coverage rates, a closed vocabulary with a pre-defined set of words demands always a restriction in terms of system usability. In order to relax this restriction, a wide span statistical description of character strings has been proposed and tested as a substitution for a closed vocabulary. Beside a precise recognition and high vocabulary coverage, the flexible and efficient usage of a handwriting interface demands also the opportunity to enter not only characters, words and sentences, but also additional document elements such as figures, formulae and mathematical expressions. Consequently, an approach to the complex recognition and processing of mathematical formulae has been presented. This approach makes use of the automatic segmentation capabilities of the HMM framework. It has been presented of how the recognition results, combined with the segmentation information can be exploited for a subsequent parsing of the two dimensional structure of mathematical formulae. As a result, the system is capable of converting the handwritten input into a LaTeX document. Finally, the presented methods including text and formula recognition have been integrated into a realtime demonstration system.