LLMs, Google Translate, & Language1

LLMs, Google Translate, and their relation to language is a topic with many things to chew on and consider. I am particularly coming at this from a linguistic angle, specifically “Tarski’s Semantic Theory of Truth”2 and “Embodied Cognition.”

To abbreviate heavily but still illustrate why I value these concepts. The “Semantic Theory of Truth” means that the “meaning” of a statement in a “model language” is not found purely in its “syntactical form.” Tarski gives an example of truth in the English language: ‘snow is white’ is true if and only if snow is white and ‘Schnee ist weiß’ is true if and only if snow is white.

“Embodied Cognition” is what determines if snow is white is cognitively true. Conceptually, it means that the “semantics” of any “sentence” is grounded in sensorimotor experience. There’s a neat rabbit hole to go down here with discussing if languages are “objective” to the world.

Like, are “model-theoretic” truths fundamental to reality, or is it something we define to express some “truths”? Are models like formal logic, ZFC set theory, music theory, physics, economic theory, art formalism, data visualization, and for ai, probability starting from Kolmogorov “objective”? Is it both?

It’s not a new question either; see What the Tortoise said to Achilles. That isn’t the point of this post, though. The real aim for me is to explore why LLMs are incredibly similar to Google Translate.

Large Language Models

The whole discussion of Tarski is stating something I, if not everyone, who has used an LLM, has done before. Ask it to write an essay, code, or some other script.

The “groundbreaking” part of coding is that we can express the semantics of whatever thing in natural language, and by some wizardry, it is translated into the language we specified it to write in. Plus the “agential” features that people seem to have attributed to the “LLM” itself and not the GOFAI running along-side it.

My point is, besides the chatbot functionality of LLMs, they are functionally similar to Translation Engines in regards to how the two engines are trained to convert symbols. The “program,” in a sense, knows the correct interpretation of the symbols in order to output the syntactically equivalent statement in another language.

The only difference, and I concede that it is a big difference, is the auto-regressive capabilities of LLMs. But the point I want to highlight is that they both approximate syntactic truth rather than semantic truth, as a direct outcome of how these models were created3.

And here is when I abruptly realized/forgot my thoughts aren’t novel: The Chinese Room. And my thoughts are even less novel: P-Zombies. These concepts accurately illustrate why there is a lot of “mysticism” around AI.

Pragmatically, they are useful, but don’t treat their “understanding” as your own. Especially when you work in a domain that requires conceptual clarity, like operating systems, anything generally “low-level” and “safety critical.”


  1. Hear a British person talk about it. Basically, 1:1 in the concepts covered. Also, yet again, my thoughts aren’t novel or, as stated earlier in his video, “historically creative.” ↩

  2. https://en.wikipedia.org/wiki/Semantic_theory_of_truth ↩

  3. https://en.wikipedia.org/wiki/Universal_approximation_theorem ↩