The last decade has witnessed the proliferation of network function virtualization (NFV) in the telco industry, thanks to its unparalleled flexibility, scalability, and cost-effectiveness. However, as the NFV infrastructure is shared by virtual network functions (VNFs), sporadic resource contentions are inevitable. Such contention makes it extremely challenging to guarantee the performance of the provisioned network services, especially in high-speed regimes (e.g., Gigabit Ethernet). Existing solutions typically rely on direct traffic analysis (e.g., packet-or flow-level measurements) to detect performance degradation and identify bottlenecks, which is not always applicable due to significant integration overhead and systemlevel constraints. This paper complements existing solutions with a lightweight, non-intrusive framework for online performance inference that easily adapts to drift (i.e., a change over time of the actual state of our system). Instead of direct data-plane collection, we reuse hardware features in the underlying NFV infrastructure, introducing negligible interference in the data-plane. Our Drift-Resilient and Self-Tuning (DRST) framework can be integrated into existing NFV systems with minimal engineering effort and operates without the need for predefined traffic models or VNF-specific customization. DRST is deployed via a lightweight MLOps pipeline that automates the adaptation under runtime drift. We show how DRST can deliver accurate performance inference or diagnose run-time bottleneck diagnose, as demonstrated through comprehensive evaluation across diverse NFV scenarios.