The teaching of probability, with special regard to the development of "stochastic thinking" requires a deeper consideration of pertinent applications. This is the only way to gain an appropriate understanding of the probability concept. Otherwise, however, the theoretical character of this concept is indispensable within the context of applications. Each measurement of probability presupposes this concept itself. The paper discusses these difficulties by analyzing the historical development of Bernoulli's theorem: The passage from a primarily combinatorical to a more "statistical" concept of probability requires to take into account this concept's theoretical character. The main intentions of this paper focus on an epistemological analysis of Bernoulli's theorem: this is followed by some more general didactical conclusions, which will require further investigation for concrete problems in curriculum construction.