The recent exponential growth in artificial intelligence (AI) deployment has outpaced regulatory and testing frameworks, presenting challenges in critical sectors like healthcare, manufacturing and business, where stakes are high. This white paper delves into the vital aspects of AI system performance and robustness, providing an overview of current practices in evaluating AI accuracy and ensuring reliability, as well as a discussion on the unresolved issues and emerging challenges.