FindMyText is an open-source Python package designed to efficiently assess whether a given text appears, in part or in full, within a text corpus. The tool builds on prior techniques for document fingerprinting but extends them with a novel mechanism to explicitly capture sequences of matching fingerprints.
By identifying such chains, FindMyText can more reliably detect near-verbatim copies of a given text rather than mere textual similarities. This makes it particularly suited for verifying the presence of copyrighted material in a corpus.
Leveraging a distributed, disk-based indexing framework, the system scales to large web-crawled datasets. Using a new benchmark for evaluating text containment methods, we show that FindMyText outperforms alternative approaches across three datasets (ArXiv papers, Wikipedia, and generic web content).
Blogger's Review: The innovation of FindMyText lies in its extension of document fingerprinting techniques, enabling more precise identification of copyrighted materials. This method not only enhances detection accuracy but also ensures good scalability when handling large datasets, making it a valuable tool with broad application potential. Its open-source nature invites further experimentation and improvement by researchers.