This thesis deals with an integrated approach to the segmentation and classification of patterns in images and image sequences. The approach is based on so-called Hidden Markov Models (HMMs). The special recognition procedure for HMMs, based on Viterbi decoding, provides an automatic alignment of features and model states, which can be interpreted as a segmentation. Furthermore, the Viterbi-algorithm provides an estimate for the probability that a given pattern has been generated by the HMM. This probability can be used in order to classify the pattern. Although the integrated HMM-based segmentation and classification approach has been utilized in speech recognition since the early 80s, the application of this approach to image and image sequence recognition can be considered to be new. The application of HMMs to the task of two-dimensional pattern recognition is a challenging task, due to the fact that it becomes necessary to extend the ne-dimensional topology of the HMMs. Image sequences are even more demanding, due to their three-dimensional structure. The thesis presents a novel rotation-invariant modeling technique of pictograms and planar objects using conventional one-dimensional Hidden-Markov-Models. To verify the proposed approach, experiments with an image database system have been carried out, where users are enabled to provide simple sketches in order to retrieve images from the database. Thereafter, a statistical approach is presented, which allows the spotting and classification of two-dimensional patterns in complex scenes. Finally, novel pseudo three-dimensional HMMs are introduced, which allow the recognition of image sequences. Many experiments utilizing the novel modeling techniques are described in this theses. The achieved results demonstrate that the approaches can be used to solve problems in man-machine-communication as well as multimedia applications. Thus, the huge application potential of HMMs for image and image sequence recognition is shown.