Language agents are increasingly operating for users on the open web, and finance is the domain where trust is hardest to earn. Data must be scraped from the web, the strategy lives only in the user’s mind, and correctness depends on a strict time cutoff, so an agent can fail silently: it may follow a strategy the user never asked for, rely on unverified data, or use future yields in a backtest, yet produce a report that looks as convincing as a correct one. Existing benchmarks close the world by fixing the task, the data, or the timeline, thereby removing these failure modes instead of measuring them.\ FINCOSMOS introduces an open‑world evaluation paradigm. A simulated user holds a hidden strategy sampled from a compositional grammar and reveals it only when queried. The agent receives no initial data, must acquire information from the web, perform its own backtest under a data cutoff, and deliver verifiable results. Grading requires no reference answer: programs verify point‑in‑time correctness and recompute every reported metric, the delivered strategy is checked against the hidden specification, and a run is accepted only when the user is satisfied and none of fifteen veto checks fails.\ Across four coding agents and a tool‑less model in 150 sessions, no session was accepted: every session signed off by the simulated user failed a program check, and agents disclosed only 54%–64% of the details they were not told. Human experts attributed 88.33% of failures to the agents themselves. With FINCOSMOS feedback, an agent equipped with persistent memory learned to solve all ten tasks.\ Review: FINCOSMOS captures the most subtle risks of financial agents by requiring hidden strategies and real‑time data acquisition, offering a scalable benchmark for future safety assessments.