Algorithmic systems may appear opaque to users. This can hinder users from making informed decisions about the use of such systems. To combat this, explanations intend to make them more transparent. However, explanations are typically informed by the system properties. We argue that they also need to consider the user understanding in order to be more comprehensible to users. To achieve these user-informed explanations, this qualitative work aims to (a) compare how experts and lay users understand algorithmic systems and (b) derive implications for creating user-informed explanations. We conducted an expert focus group (N = 3) and semistructured in-depth interviews with experts (N = 10) and lay users (N = 11), including a drawing task. Reflexive thematic analysis by the first author revealed group-specific and common themes: Experts understood algorithms as a decision-making process and were aware of the context dependency of algorithms. Lay users, in turn, understood algorithms as intelligence and as data structuring. They focused on the tangible and visible elements of algorithmic systems. Both groups also understood algorithms as a sequence of actions. The different understandings might be driven by group-specific experiences and purposes to use algorithmic systems. Based on our results, we argue that user-informed explanations could consider the context dependency of algorithmic systems and highlight their limitations.