On safety assessment of artificial intelligence

Abstract

Aim. In this paper we discuss how systems with Artificial Intelligence (AI) can undergo safety assessment. This is relevant, if AI is used in safety related applications. This holds also for railway systems, where AI is expected to take a role in railway automation. Methods. The focus of this paper is on safety assessment of AI rather than on AI itself. Taking a deeper look into AI models, we show that many models of artificial intelligence, in particular machine learning, are statistical models. Safety assessment would then have to concentrate on the model that is used in AI, besides the normal assessment procedure. Results. Part of the budget of dangerous random failures for the relevant safety integrity level needs to be used for the probabilistic faulty behavior of the AI system. We demonstrate our thoughts with a simple example and propose a research challenge that may be decisive for the use of AI in safety-related systems. Conclusion. The method of safety assessment of systems with AI is presented in this article.