ELSA: Acoustic Event-Level Semantic Alignment

Abstract

Text-to-audio (TTA) generation, synthesizing audio from natural language, has been widely studied for its ability to capture precise user intent. To effectively advance TTA models, it is essential to reliably evaluate generated audio without relying on costly human subjective ratings, motivating the development of automatic evaluation metrics that correlate well with human judgments. While recent CLAP-based metrics provide practical reference-free solutions, their coarse-grained text–audio similarity matching often correlates poorly with human ratings.

To address this, we propose ELSA, a reference-free evaluation metric for fine-grained text–audio alignment. ELSA decomposes generated audio guided by distinct acoustic events derived from the text query and assesses event-level alignment. Experiments across four TTA benchmarks show that ELSA reveals a higher correlation with human subjective ratings than prior metrics, highlighting its effectiveness for reliable TTA evaluation.

Method

ELSA hierarchically evaluates global text–audio matching and fine-grained acoustic-event alignment by combining shared text–audio embeddings with event-level representations extracted via a text parser and a language-queried audio source separation model.

Qualitative Results

Input Text

"Strong blizzard wind in the background with a bell ringing followed by animal footsteps."

Input Audio

Text branch

GPT-5.2

Animal footsteps A bell ringing Blizzard wind blowing

Audio branch

SAMaudio

Separated Tracks

Event 01Animal footsteps

Event 02A bell ringing

Event 03Blizzard wind blowing

ELSA Predicted Score

0.54

Quantitative Results

Table 1: Correlation with human ratings across AudioCaps, Clotho, MusicCaps, and RELATE. ELSA consistently achieves the highest rank-correlation, improving Kendall's tau on REL by +13.1, +4.8, +1.5, and +4.5 points over the strongest baseline.

Table 2: Compositional benchmark results on RELATE (IS/OS) and CompA (attribute/order). ELSA outperforms baselines on RELATE and wins 3 out of 4 CompA evaluation settings, showing stronger event-level compositional alignment.

Table 3: Component ablation on Clotho. Performance changes are relatively small across LASS variants, while changing the text-audio embedding space causes much larger drops, indicating ELSA is more sensitive to embedding quality than separator choice.

Analysis

Figure 3: Sensitivity analysis of metric-REL correlation with respect to the number of acoustic events in the text query. ELSA maintains stronger correlation than CLAPScore variants across different event counts, indicating robust fine-grained alignment.

Figure 4: Relationship between REL and ELSA on AudioCaps. ELSA follows REL's rank tendency but with a lower score scale, reflecting stricter event-level evaluation than coarse global matching.

Citation


To be appeared.