The article discusses some of the problems concerning the interpretation of the concept of probability. It is emphasized that the mathematical theory of probability, being a part of mathematics, is not affected by such considerations, but that there are problems with subjective and frequentist interpretations when it comes to applications. In particular the frequentist requirement that the experimental conditions remain the same for all repetitions is in conflict with deterministic theories of nature. A deterministic nature does not per se imply the inapplicability of probability theory. It can be shown mathematically that complex sequences of numbers and certain deterministic chaotic systems behave like random systems and can be well described using probability theory. Statistics is one form of applied probability theory and is consequently more influenced by problems of interpretability than is probability theory itself. In its pure form Bayesian statistics is concerned with degrees of belief and how these should be altered in the light of experimental evidence. Frequentist statistics is more concerned with providing procedures which are optimal for repeated applications under similar circumstances. In practice both pure forms are tempered by practical necessities. The article describes two special areas of statistics, robust statistics and nonparametric regression. Robust statistics is concerned with the detection and accommodation of outlying or aberrant observations as they occur for example in interlaboratory tests and the analysis of variance. Non-parametric regression is concerned with estimating a functional relationship between two variables when this is contaminated by noise. The example discussed comes from the area of thin film physics where the intensity of diffracted x-rays depends on the angle of diffraction.