On safety assessment of artificial intelligence
https://doi.org/10.21683/1729-2646-2020-20-4-25-34
Abstract
Aim. In this paper we discuss how systems with Artificial Intelligence (AI) can undergo safety assessment. This is relevant, if AI is used in safety related applications. This holds also for railway systems, where AI is expected to take a role in railway automation. Methods. The focus of this paper is on safety assessment of AI rather than on AI itself. Taking a deeper look into AI models, we show that many models of artificial intelligence, in particular machine learning, are statistical models. Safety assessment would then have to concentrate on the model that is used in AI, besides the normal assessment procedure. Results. Part of the budget of dangerous random failures for the relevant safety integrity level needs to be used for the probabilistic faulty behavior of the AI system. We demonstrate our thoughts with a simple example and propose a research challenge that may be decisive for the use of AI in safety-related systems. Conclusion. The method of safety assessment of systems with AI is presented in this article.
About the Authors
Jens Braband
Siemens Mobility GmbH
Germany
Germany
Jens Braband, Dr. rer. nat., Principal Key Expert for RAMSS at Siemens Mobility GmbH, and Honorary Professor, TU Braunschweig
Braunschweig
Hendrik Schäbe
TÜV Rheinland
Germany
Germany
Hendrik Schäbe, Dr. rer. nat. habil., Chief Expert on Reliability, Operational Availability, Maintainability and Safety
Cologne
References
1. Anscombe F.J. Graphs in Statistical Analysis // American Statistician. 1973. 27(1). P. 17–21.
2. Braband J., Gall H., Schäbe H. Proven in Use for Software: Assigning an SIL Based on Statistics / Mahboob Q., Zio E., editors. Handbook of RAMS in Railway systems – Theory and Practice. Boca Raton, Taylor and Francis. 2018.
3. Brunette E.S., Flemmer R.C., Flemmer C.L. A review of artificial intelligence // Proc. 4th International Conference on Autonomous Robots and Agents. Feb. Wellington. 2009. P. 385-392.
4. Chen S.H., Jakeman A.J., Norton J.P. Artificial Intelligence techniques: An introduction to their use for modelling environmental systems // Mathematics and Simulation. 2008. Vol. 78. P. 379-400.
5. Corni M. Is Artificial Intelligence Racist? (And Other Concerns). URL : https://towardsdatascience.com/is-artificial-intelligence-racist-and-other-concerns817fa60d75e9 [accessed October 25, 2018].
6. Cybenko G. Approximations by superpositions of sigmoidal functions // Mathematics of Control, Signals, and Systems. 1989. № 2(4). P. 303–314.
7. Hättasch N., Geisler N. The Deep Learning Hype: Presentation at 36C3. 2019. URL: https://www.youtube.com/watch?v=FomrN5XHQhY.
8. EN 50128. Railway applications – Communication, signalling and processing systems – Software for railway control and protection systems; 2011.
9. EN 50129. Railway applications – Communication, signalling and processing systems – Software for railway control and protection systems; 2018.
10. IEC 61508. Functional safety of electrical/electronic/ programmable electronic safety-related systems; 2010.
11. Ivanov A.I., Kuprianov E.N., Tureev S.V. Neural network integration of classical statistical tests for processing small samples of biometrics data // Dependability. 2019. № 19(2). P. 22-27.
12. Pearl J., Mackenzie D. The Book of Why. Penguin Science, 2018.
13. Putzer H. Ein strukturierter Ansatz für funktional sichere KI. Presentation at DKE Funktionale Sicherheit. Erfurt; 2019. (in Ger.)
14. Schäbe H. SIL Apportionment and SIL Allocation. / Mahboob Q., Zio E., editors. Handbook of RAMS in Railway systems – Theory and Practice. Boca Raton, Taylor and Francis. 2018. P. 69-78.
15. Underwriter Laboratories: Standard for Safety for the Evaluation of Autonomous Products. Draft UL 4600; 2019.
16. Wang J., Ma Y., Zhang L., et al. Deep learning for smart manufacturing: Methods and Applications // Journal of Manufacturing Systems. 2017. № 48. P. 144-156.
17. Wigger P. Independent Safety Assessment – Process and Methodology / Mahboob Q., Zio E., editors. Handbook of RAMS in Railway systems – Theory and Practice. Boca Raton, Taylor and Francis. 2018. P. 475-485.
Review
For citations:
Braband J., Schäbe H. On safety assessment of artificial intelligence. Dependability. 2020;20(4):25-34. https://doi.org/10.21683/1729-2646-2020-20-4-25-34
Views:
1362
ISSN 1729-2646 (Print)
ISSN 2500-3909 (Online)
ISSN 2500-3909 (Online)
Email this article 