Efficient management of spare parts inventory is crucial in the automotive aftermarket, where demand is highly intermittent, and uncertainty drives substantial cost and service risks. Forecasting is therefore central, but the quality of a forecasting model should be judged not by statistical accuracy (e.g., MAE, RMSE, IAE) but rather by its impact on key operational performance indicators (KPIs), such as total cost and service level. Yet most existing work evaluates models exclusively using accuracy metrics, and the relationship between these metrics and operational KPIs remains poorly understood. To address this gap, we propose a decision-centric simulation software framework that enables the systematic evaluation of forecasting models in realistic inventory management settings. The framework comprises: (i) a synthetic demand generator tailored to spare-parts demand characteristics, (ii) a flexible forecasting module that can host arbitrary predictive models, and (iii) an inventory control simulator that consumes the forecasts and computes, based on selected inventory control policy, operational KPIs. This closed-loop setup enables practitioners and researchers to evaluate models not only in terms of statistical error but also in terms of their downstream implications for inventory decisions. Using a wide range of simulation scenarios, we show that improvements in conventional accuracy metrics do not necessarily translate into better operational performance, and that models with similar statistical error profiles can induce markedly different cost–service trade-offs. We analyze these discrepancies to characterize how specific aspects of forecast performance affect inventory outcomes and to derive actionable guidance for model selection. Overall, the framework operationalizes the link between demand forecasting and inventory management, shifting evaluation from purely predictive accuracy towards operational relevance in the automotive aftermarket and related domains. An open-source implementation of the software, including all experimental results, is available at https://github.com/caisr-hh/TruckParts-Demand-Inventory-Simulator/releases/tag/IDA_2026. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.