NeFut Logo NeFut
中 Admin Login

[CS.AI] CIIR - Class-Indexed Interface Representation Learning

Published at: 2026-10-19 22:00 Last updated: 2026-10-20 00:49
#Machine Learning #LLM #Artificial Intelligence

Multimodal learning often assumes paired or synchronized observations, yet independently collected datasets rarely provide such correspondence. We introduce CIIR, a modular encoder‑decoder framework that links heterogeneous datasets via a class‑indexed address space while preserving each dataset's native input format and output module. A known local index selects address $k$, and an input‑dependent 128‑dimensional address representation $Z_k(x)$ is written to that address. To demonstrate CIIR, we learn this address space from handwritten digits, clothing images, spoken‑digit spectrograms, and hand‑sign images without synchronized or instance‑matched samples. Newly trained label‑free receivers recover the requested addresses with 94.7% macro‑average Address Realization Accuracy. We then attach the unseen SVHN dataset after freezing the base address space and train only lightweight adapters, achieving 90.2% MNIST→SVHN and 82.3% SVHN→MNIST end‑to‑end Address Transfer Accuracy without target‑side decoder labels. To show CIIR's generalizability, we reuse the ten addresses as collision‑free arithmetic codes and construct hierarchical address codes for all 100 CIFAR‑100 classes, evaluating direct arithmetic‑code realization through the existing receivers. These results establish CIIR as a compact interface for modular cross‑dataset communication and post‑hoc dataset attachment without exhaustive paired multimodal supervision.

Review: CIIR demonstrates a powerful way to connect disparate modalities through class‑indexed addresses, achieving high performance even without paired data and offering a scalable blueprint for future modular multimodal systems.

Original Source: https://arxiv.org/abs/2610.23151

[h] Back to Home