ELSA: Acoustic Event-Level Semantic Alignment for Fine-Grained Reference-Free Text-to-Audio Evaluation

Abstract

Text-to-audio (TTA) generation, synthesizing audio from natural language, has been widely studied for its ability to capture precise user intent. To effectively advance TTA models, it is essential to reliably evaluate generated audio without relying on costly human subjective ratings, motivating the development of automatic evaluation metrics that correlate well with human judgments. While recent CLAP-based metrics provide practical reference-free solutions, their coarse-grained text–audio similarity matching often correlates poorly with human ratings.

To address this, we propose ELSA, a reference-free evaluation metric for fine-grained text–audio alignment. ELSA decomposes generated audio guided by distinct acoustic events derived from the text query and assesses event-level alignment. Experiments across four TTA benchmarks show that ELSA reveals a higher correlation with human subjective ratings than prior metrics, highlighting its effectiveness for reliable TTA evaluation.

Method

ELSA hierarchically evaluates global text–audio matching and fine-grained acoustic-event alignment by combining shared text–audio embeddings with event-level representations extracted via a text parser and a language-queried audio source separation model.

Qualitative Results

Text Query

"Strong blizzard wind in the background with a bell ringing followed by animal footsteps."

Generated Audio

Acoustic
Event Parser

LASS

Event 01

"Animal footsteps"

Event 02

"A bell ringing"

Event 03

"Blizzard wind blowing"

ELSA Score

0.54

Quantitative Results

Table 1: Correlation with human ratings across AudioCaps, Clotho, MusicCaps, and RELATE. ELSA consistently achieves the highest rank-correlation, improving Kendall's tau on REL by +13.1, +4.8, +1.5, and +4.5 points over the strongest baseline.

Table 2: Compositional benchmark results on RELATE (IS/OS) and CompA (attribute/order). ELSA outperforms baselines on RELATE and wins 3 out of 4 CompA evaluation settings, showing stronger event-level compositional alignment.

Table 3: Component ablation on Clotho. Performance changes are relatively small across LASS variants, while changing the text-audio embedding space causes much larger drops, indicating ELSA is more sensitive to embedding quality than separator choice.

Figure 3: Sensitivity analysis of metric-REL correlation with respect to the number of acoustic events in the text query. ELSA maintains stronger correlation than CLAPScore variants across different event counts, indicating robust fine-grained alignment.

Figure 4: Relationship between REL and ELSA on AudioCaps. ELSA follows REL's rank tendency but with a lower score scale, reflecting stricter event-level evaluation than coarse global matching.

Citation

@inproceedings{suzuki2026elsa,
  title = {ELSA: Acoustic Event-Level Semantic Alignment for Fine-Grained Reference-Free Text-to-Audio Evaluation},
  author = {Shuntaro Suzuki and Kento Tokura and Daichi Yashima and Kanon Amemiya and Komei Sugiura and Shinnosuke Takamichi},
  year = {2026},
  booktitle = {The 27th Interspeech Conference (INTERSPEECH 2026)},
}