In causal inference, nuisance‑function estimators are often judged by prediction error, yet the link between this error and the performance of the resulting causal estimator can vary across evaluation metrics. We investigated this issue in a partially linear model using Monte Carlo simulations, comparing ordinary least squares (OLS), generalized additive models (GAM), XGBoost, and Double Machine Learning with XGBoost (DML‑XGBoost). The metrics considered were nuisance‑function prediction error, bias, root‑mean‑square error (RMSE), and 95 % confidence‑interval coverage. We also examined a simple joint‑error measure defined as the absolute cross‑product of the exposure‑nuisance error $\Delta_e$ and the outcome‑nuisance error $\Delta_y$: $$J = |\Delta_e \cdot \Delta_y|$$. Across simulated scenarios, XGBoost achieved the lowest RMSE among non‑oracle methods, while DML‑XGBoost generally yielded better interval coverage. Prediction error did not consistently track causal bias across methods and settings, and the method with the best point‑estimate performance did not necessarily provide the best confidence‑interval coverage. The joint‑error measure was only weakly associated with causal bias and did not serve as a useful standalone performance metric. These findings suggest that prediction error remains useful for assessing nuisance‑function estimation, but it should not be treated as a direct proxy for the quality of the causal estimator.
Review