Report · Page 50

50
Center for Research on Foundation Models (CRFM)
Metaphysics of understanding. Philosophy of language offers a number of alternatives for
what it is to understand natural language.25 Simplifying the landscape for the sake of brevity, the
following three broad classes of views all have connections with research lines in AI and NLP:26
• Internalism: Language understanding amounts to retrieval of the right internal representa-
tional structures in response to linguistic input. Thus, language understanding is not even a
possibility without a rich internal conceptual repertoire of the right kind.
• Referentialism: Roughly, an agent understands language when they are in a position to
know what it would take for different sentences in that language to be true (relative to a
context). That is, words have referents and (declarative) utterances are truth-evaluable, and
understanding involves a capacity to evaluate them relative to presentation of a situation or
scenario.
• Pragmatism: Understanding requires nothing in the way of internal representations or
computations, and truth and reference are not fundamental. Rather, what matters is that the
agent be disposed to use language in the right way. This might include dispositions toward
inference or reasoning patterns, appropriate conversational moves, and so on. Crucially, the
relevant verbal abilities constitute understanding.27
While this is a simplified picture of the space of possibilities, we already see how they relate in
quite different ways to the goals mentioned above. On the pragmatist view, for instance, achiev-
ing language understanding does not imply anything about our ability to trust or interpret the
system, insofar as it guarantees nothing about the agent’s internal structure or its relation to the
(non-linguistic) world. On the internalist view, by contrast, a fairly robust kind of internal/causal
interpretability is at least strongly suggested. The question of whether or not a foundation model
could understand language in principle takes on a very different character depending on which of
these metaphysical characterizations we adopt.
Internalism and referentialism can both be cast as defining a mapping problem: to associate a
linguistic sign with a “meaning” or a “semantic value”. For internalism this will be a representation or
concept, a program for computing a value, or some other type of internal object. For referentialism,
it might be a mapping from a word to an external referent, or a mapping from a situation to a truth
value (all relative to a context). Could self-supervision suffice for achieving the desired mapping
in a foundation model? Here, the nature of the training examples might be relevant. If the model
receives only linguistic inputs, then its capacity to learn this mapping might be fundamentally
limited in ways that prevent it from learning to refer in the relevant sense. (Indeed, Merrill et al.
[2021] identify some theoretical limits, albeit under very strong assumptions about what it means to
learn the meaning of a symbol.) However, if the input symbol streams include diverse digital traces
of things in the world – images, audio, sensors, etc. – then the co-occurrence patterns might contain
enough information for the model to induce high-fidelity proxies for the required mapping.28 For
25Relatedly, there is a sizable literature in philosophy of science focused on the concept of understanding, mainly as it
relates to scientific explanation. See Grimm [2021