Existing visual‑preference evaluation relies on costly human annotations or preference‑reward models, while transaction prices in online marketplaces are readily available and reflect actual purchasing behavior, suggesting a low‑cost supervision signal. However, prices also embed non‑visual factors such as provenance, demand and competition, making overall accuracy insufficient to separate model errors, noisy price signals and protocol confounds. To tackle this identifiability issue, we introduce the MarketImage diagnostic framework, which aims to determine when transaction prices reflect visual preferences, when they fail, and whether multimodal language models can recover human‑supported visual‑preference signals. MarketImage consists of three components:
- MarketImageArt supplies 266 six‑image comparison sets with transaction prices and price‑blind human rankings;
- MarketImageLens uses a tie‑preserving protocol to partition 3,990 within‑set image pairs into price‑human agreement, conflict and price‑tie regions, while separately auditing presentation‑order effects;
- MarketValuator offers six elicitation methods covering direct comparison, independent scoring, budget constraints and multi‑agent aggregation to evaluate signal recovery by static visual scorers and three multimodal language models.
Experiments reveal that only 24.2% of image pairs exhibit a non‑tied price direction, of which 34.7% agree with the human majority. Within this region, HPSv2 achieves a recoverable accuracy of 66.4%, leading the pack; Qwen’s budgeted and aggregated variants fall behind baselines, and GPT’s interface gains remain statistically unresolved. Thus, transaction prices encode recoverable visual preferences only under limited conditions and should be treated as uncertain proxies rather than ground truth.
Review: Transaction prices hold promise as a visual‑preference cue but are heavily contaminated by non‑visual factors. Their reliability is confined to a small subset of cases, suggesting that future research should focus on noise reduction or complementary low‑cost signals to improve multimodal models’ ability to capture visual preferences.