Representational Isomorphism and Alignment of Multilingual Large Language Models
| Authors | |
|---|---|
| Publication date | 2024 |
| Host editors |
|
| Book title | The 4th Workshop on Multilingual Representation Learning : proceedings of the workshop |
| Book subtitle | MRL 2024 : November 16, 2024 |
| ISBN (electronic) |
|
| Event | 4th Workshop on Multilingual Representation Learning |
| Pages (from-to) | 293-297 |
| Number of pages | 5 |
| Publisher | Kerrville, TX: Association for Computational Linguistics |
| Organisations |
|
| Abstract | In this extended abstract, we investigate the capability of Large Language Models (LLMs) to represent texts in multilingual contexts. Our findings reveal that sentence representations derived from LLMs exhibit a high degree of isomorphism across languages. This existing isomorphism facilitates representational alignments in few-shot settings. Specifically, by applying a contrastive objective at the representation level with only a small number (e.g., 100) of translation pairs, we significantly improve models’ performance on Semantic Textual Similarity (STS) tasks across languages. |
| Document type | Conference contribution |
| Language | English |
| Related publication | Representational Isomorphism and Alignment of Multilingual Large Language Models |
| Published at | https://doi.org/10.18653/v1/2024.mrl-1.24 |
| Downloads |
2024.mrl-1.24
(Final published version)
|
| Permalink to this page | |