Hide and Seek (HnS) is a concept‑level attribution method that requires neither model weights nor a labeled concept dataset. Given a target concept and a set of images, HnS first uses a vision‑language model to retrieve images aligned with the concept semantics, then localizes the concept region in each image, masks that region, and finally measures the change in the black‑box model’s output between the original and masked images to quantify the concept’s contribution. This direct measurement of model behavior yields local (image‑level) attributions.
We evaluated HnS on CelebA and COCO‑Stuff, where it achieved higher agreement with ground‑truth attributions than existing concept attribution methods. Additional demonstrations include image captioning and satellite image segmentation—tasks for which no prior concept attribution technique was applicable. The anonymous code is provided in the supplementary material.
Review