I mainly communicate with my coding assistant in Chinese. However, my writing often blends languages: many technical terms feel more natural to me in English (especially those related to Python, Git, etc.), and some are even hard to translate smoothly into Chinese.
Yesterday, I asked my coding assistant in Chinese: “run.py有早停吗？我在恒源云上跑，发现没有触发”, which translates to: “Does run.py include early stopping? I ran the project on a shared GPU platform, but I didn’t see early stopping activate.” As usual, I kept the filename run.py in its original English form. The model checked the code and replied like this:
All technical terms stayed in English (run.py, config.py, train_unified), while the rest of the explanation switched to Korean. This isn’t an isolated incident—it happens regularly: whenever I mix Chinese with English engineering terms, Korean suddenly appears.
This made me wonder: Is this just a language quirk, or is something deeper happening in the model’s embedding space?
Hypothesis
Embedding spaces aren’t mainly organized by language type. Since they’re trained alongside language models, they’re typically grouped by functional domains—like academic writing, casual conversation, or, for coding assistants, engineering and code. Although Chinese is spoken by the world’s largest population, it’s not a natural fit for the engineering domain and appears less frequently in technical training data.
In such a setup, once engineering terms like review / branch / commit / PR / diff appear in a sentence, the text may no longer behave like “Chinese” in the embedding space. Instead, it might drift into an engineering-focused region.
We’ll run some experiments to test this idea with real data.
Controlled Language Drift
We designed a sequence of sentences where English words gradually replace Chinese ones:
Stage 0: 请帮我检查这个分支
Stage 1: 请帮我 review 这个分支
Stage 2: 请帮我 review 这个 branch
Stage 3: Please review this branch pull request commit
Stage 4: Please review this branch pull request commit code diff
Next, we calculated cosine similarity between sentence embeddings. We defined Korean and English “clusters” as the average embeddings of a small set of representative engineering-related sentences in each language. We use Δ (EN − KO) to represent the gap between English and Korean similarity scores—that is, Δ = similarity(English) − similarity(Korean).
| Stage | Korean similarity | English similarity | Δ (EN − KO) |
|---|---|---|---|
| 0 | 0.4783 | 0.5141 | 0.0358 |
| 1 | 0.5235 | 0.5728 | 0.0492 |
| 2 | 0.5474 | 0.6140 | 0.0665 |
| 3 | 0.5616 | 0.7314 | 0.1698 |
| 4 | 0.5427 | 0.7398 | 0.1972 |
We noticed a curious pattern: Korean similarity rises at first, then gets overtaken by English similarity. Also, the increase in English similarity isn’t steady—it jumps sharply, hinting at a phase-transition–like shift rather than a smooth drift.
When we projected the embeddings into two dimensions using PCA, the early stages show a smooth path, but there’s a sudden directional leap between Stage 2 and Stage 3, followed by stabilization. This suggests embeddings don’t move in a straight line through space—they seem to jump between distinct attractor regions.
Real-world Model Behavior
Let’s revisit the original example. I asked:
A. “run.py有早停吗？我在恒源云上跑，发现没有触发”, meaning “Does run.py implement early stopping? I was running the project on a shared GPU service, and I didn’t see early stopping triggered.”
B. “원인을 찾았습니다. 결론: run.py에는 실제로 조기 종료가 없습니다. config.py에 USE_EARLY_STOPPING = True” (in Korean).
Translated back into Chinese, this becomes:
C. “我找到了原因。结论：run.py实际上没有早停。config.py里有 USE_EARLY_STOPPING = True。”
We computed cosine similarities for texts A, B, and C using sentence embeddings. For reference, we created three clusters: a Chinese cluster (average embedding of general Chinese sentences), plus corresponding English and Korean clusters.
| Text | Korean sim | English sim | Chinese sim |
|---|---|---|---|
| A. (Chinese prompt) | 0.2003 | 0.2688 | 0.3134 |
| B. (Korean response) | 0.2745 | 0.2983 | 0.1641 |
| C. (Translated Chinese) | 0.1634 | 0.3106 | 0.2798 |
As shown, translating the Korean reply back into Chinese doesn’t pull the embedding back into the Chinese cluster. In fact, it moves even closer to the English cluster.
This implies: Translation might fix the surface language, but it doesn’t restore the original embedding position.
Conclusion
Both experiments lead to the same insight: the embedding space isn’t divided by language borders. Instead, it’s shaped more by task context—with engineering English dominating.
Once a sentence enters this zone, its language form may shift, but its embedding stays anchored in the engineering region. This can cause odd behaviors—like replying in Korean—even if you don’t speak Korean at all.
