sustainНадежностьDependability1729-26462500-3909RAMS Journal Limited liability company10.21683/1729-2646-2021-21-3-54-65sustain-433Research ArticleБЕЗОПАСНОСТЬ. УПРАВЛЕНИЕ РИСКАМИ. ТЕОРИЯ И ПРАКТИКАSAFETY. RISK MANAGEMENT. THEORY AND PRACTICEИнтеллектуальные методы повышения точности прогнозирования редких опасных событий на железнодорожном транспортеIntelligent methods for improving the accuracy of prediction of rare hazardous events in railway transportationПроневичО. Б.PronevichO. B.

Ольга Борисовна Проневич – начальник отдела

ул. Нижегородская, д. 27, стр. 1, Москва, 109029

тел +7 (495) 786-68-57 

Olga B. Pronevich, Head of Unit

27, bldg 1 Nizhegorodskaya St., Moscow, 109029

phone: +7 (495) 786 68 57

obpronevich@gmail.comЗайцевМ. В.ZaitsevM. V.

Михаил Вадимович Зайцев – ведущий специалист

ул. Нижегородская, д. 27, стр. 1, Москва, 109029

тел +7 (495) 786-68-57

Mikhail V. Zaytsev, Lead Specialist

27, bldg 1 Nizhegorodskaya St., Moscow, 109029

phone: +7 (495) 786 68 57 

m.v.zaicev@mail.ruАО «НИИАС»РоссияJSC NIIASRussian Federation2021210920212135464Copyright © Проневич О.Б., Зайцев М.В., 20212021Проневич О.Б., Зайцев М.В.Pronevich O.B., Zaitsev M.V.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.dependability.ru/jour/article/view/433

Цель статьи – рассмотреть подходы к методам повышения качества прогнозирования и классификации несбалансированных данных и выбрать методы, позволяющие повысить точность классификации редких событий. При прогнозировании появления редких событий методами машинного обучения ученые сталкиваются с проблемой несоответствия качества обученных моделей их реальной способности правильно спрогнозировать появление редкого события. Предмет исследования в статье – обучение моделей при исходных несбалансированных данных. Объект исследования – информация об инцидентах и опасных событиях на объектах железнодорожного электроснабжения. Проблема несбалансированных данных выражается заметной диспропорции между типами наблюдаемых событий – количествами представителей различных классов. Методы. При работе с несбалансированными данными, в зависимости от характера задачи, качества и объема исходных данных, применяют различные методы повышения качества моделей классификации и прогнозирования Data Science. Часть этих методов направлена на работу с признаками и параметрами моделей классификации. К ним относятся методы FAST, CFS, нечёткие классификаторы, GridSearchCV и другие. Другая группа методов ориентирована на формирование репрезентативных подмножеств из исходного массив данных – сэмплов. Методы сэмплинга данных позволяют исследовать влияние пропорции классов на качество машинного обучения. В частности, в рамках настоящей статьи подробно рассматривается метод NearMiss. Результаты. Проблема дисбаланса классов при анализе количества инцидентов на объектах железнодорожного транспорта существуют с 2015 года. Несмотря на снижение доли опасных событий на объектах железнодорожного электроснабжения в течении трех лет с 2018 года, не исключен рост количества таких событий. Статистика долей опасных событий на уровне месяца демонстрирует отсутствие тренда на снижение и наличие пиков. В таких условиях эффективным периодом наблюдений за количеством инцидентов и опасных событий является месяц. Визуализация соотношения классов показала отсутствие выраженной границы между представителями класса большинства (инцидентами) и класса меньшинства (опасные события). Исследовалось соотношение классов в двух и трех измерениях в натуральных величинах и с применением метода главных компонент. Такая «близость» классов является одной из причин ошибок прогноза. В рамках работы проведен анализ имеющегося исследовательского опыта повышения качества машинного обучения при работе с несбалансированными данными. Определены и уточнены используемые для описания степени дисбалансов классов термины. Изучены сильные и слабые стороны различных методов работы с такими данными, приведено описание сильных и слабых сторон 50 методов. Из методов работы с количеством представителей классов при решении задачи классификации (прогнозирования появления) редких опасных событий на железнодорожном транспорте выбран метод NearMiss. Указанный метод позволяет проводить эксперименты с пропорциями представителей классов и методами отбора представителей классов. По результатам серии экспериментов удалось добиться повышения точности классификации редких опасных событий от 0 до 70-90%.

The paper Aims to examine various approaches to the ways of improving the quality of predictions and classification of unbalanced data that allow improving the accuracy of rare event classification. When predicting the onset of rare events using machine learning techniques, researchers face the problem of inconsistency between the quality of trained models and their actual ability to correctly predict the occurrence of a rare event. The paper examines model training under unbalanced initial data. The subject of research is the information on incidents and hazardous events at railway power supply facilities. The problem of unbalanced data is expressed in the noticeable imbalance between the types of observed events, i.e., the numbers of instances. Methods. While handling unbalanced data, depending on the nature of the problem at hand, the quality and size of the initial data, various Data Science-based techniques of improving the quality of classification models and prediction are used. Some of those methods are focused on attributes and parameters of classification models. Those include FAST, CFS, fuzzy classifiers, GridSearchCV, etc. Another group of methods is oriented towards generating representative subsets out of initial datasets, i.e., samples. Data sampling techniques allow examining the effect of class proportions on the quality of machine learning. In particular, in this paper, the NearMiss method is considered in detail. Results. The problem of class imbalance in respect to the analysis of the number of incidents at railway facilities has existed since 2015. Despite the decreasing share of hazardous events at railway power supply facilities in the three years since 2018, an increase in the number of such events cannot be ruled out. Monthly statistics of hazardous event distribution exhibit no trend for declines and peaks. In this context, the optimal period of observation of the number of incidents and hazardous events is a month. A visualization of the class ratio has shown the absence of a clear boundary between the members of the majority class (incidents) and those of the minority class (hazardous events). The class ratio was studied in two and three dimensions, in actual values and using the method of main components. Such “proximity” of classes is one of the causes of wrong predictions. In this paper, the authors analysed past research of the ways of improving the quality of machine learning based on unbalanced data. The terms that describe the degree of class imbalances have been defined and clarified. The strengths and weaknesses of 50 various methods of handling such data were studied and set forth. Out of the set of methods of handling the numbers of class members as part of the classification (prediction of the occurrence) of rare hazardous events in railway transportation, the NearMiss method was chosen. It allows experimenting with the ratios and methods of selecting class members. As the results of a series of experiments, the accuracy of rare hazardous event classification was improved from 0 to 70-90%.

машинное обучениередкие событиядисбаланс классовповышение точности прогнозированиясэмплинг данныхбалансировка классовmachine learningrare eventsclass imbalancebetter accuracy of predictionsdata samplingclass balancingReferencesСевастьянов Л.А, Шетиние Е.Ю. О методах повышения точности многоклассовой классификации на несбалансированных данных // Информатика и ее применение. 2020. Том 14. № 1. С. 63-70.Sevastianov L.A, Shetinin E.Yu. On methods for improving the accuracy of multiclass classification on imbalanced data. Informatics and applications 2020;14(1):63-70. (in Russ.)Садов М.А. Исследование методов классификации текстов для несбалансированных данных // Полиматис. 2016. № 2. С. 28-41.Sadov M.A. Study of the methods of text classification for unbalanced data. Polymathis Scientific Journal 2016;2:28-41. (in Russ.)Маслихов С.Р., Мохов А.С., Толчеев В.Ю. Построение сбалансированных классов в задаче классификации запросов пользователей // Сборник трудов V Международной научно-практической конференции «Дистанционные образовательные технологии». 2020. С. 245-248.Maslikhov S.R., Mokhov A.S., Tolcheev V.Yu. [Building balanced classes in respect to user query classification]. [Proceedings of the 5th International Science and Practice Conference Remote Education Technologies]; 2020. P. 245-248. (in Russ.)Шипицын А.В., Журавлева Н.В. Оценка онлайнзаявок на ипотечный кредит с помощью алгоритмов Machine Learnng // Вестник Белгородского университета кооперации, экономики и права. 2016. № 4(60). С. 199-209.Shipitsyn A.V., Zhuravleva N.V. Evaluation of online mortgage applications with machine learning algorithms. Herald of the Belgorod University of Cooperation, Economics and Law 2016;4(60):199-209. (in Russ.)Chawla N.V., Bowyer W.B., Hall L.O. et al. SMOTE: Synthetic Minority Over-sampling Technique // Journal of Artificial Intelligence Research. 2002. № 16. P. 321-357.Chawla N.V., Bowyer W.B., Hall L.O. et al. Smart: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research 2002;16:321-357.Ali A., Shamsuddin S.M., Ralescu A. Classification with class imbalance problem: a review // International Journal of Advances in Soft Computing. 2013. № 7. P. 176-204.Ali A., Shamsuddin S.M., Ralescu A. Classification with class impact problem: a review. International Journal of Advances in Soft Computing 2013;7:176-204.Mladenic D., Grobelnik M. Feature selection for unbalanced class distribution and naive bayes // Proceedings of the Sixteenth International Conference on Machine Learning (ICML 1999), Bled, Slovenia, June 27–30, 1999. P. 258–267.Mladenic D., Grobelnik M. Feature selection for unbalanced class distribution and national scores. Proceedings of the Sixteenth International Conference on Machine Learning (ICML 1999). Bled (Slovenia); 1999. p. 258-267.Yang T.-N., Wang S.-D. Robust algorithms for principal component analysis // Pattern Recognition Letters. 1999. 20(9). P. 927-933.Yang T.-N., Wang S.-D. Robust algorithms for principal component analysis. Pattern Recognition Letters 1999;20(9):927-933.Yu H., Ni J., Zhao J. ACOSampling: An ant colony optimization-based undersampling method for classifying imbalanced DNA microarray data // Neurocomputing. 2013. 101(0). P. 309-318.Yu H., Ni J., Zhao J. ACOSampling: An ant colony optimization-based undersampling method for classifying imbalanced DNA microarray data. Neurocomputing 2013;101(0):309-318.Chawla N.V. SMOTE: synthetic minority oversampling technique // arXiv:1106.1813, 2002.Chawla N.V. SMOTE: synthetic minority oversampling technique. arXiv:1106.1813; 2002.García S., Herrera F. Evolutionary undersampling for classification with imbalanced datasets: Proposals and taxonomy // Evolutionary Computation. 2009. 17(3). P. 275-306.García S., Herrera F. Evolutionary undersampling for classification with imbalanced datasets: Proposals and taxonomy. Evolutionary Computation 2009;17(3):275-306.Yin L. Feature selection for high-dimensional imbalanced data // Neurocomputing. 2013. 105(0). P. 3-11.Yin L. Feature selection for high-dimensional imbalanced data. Neurocomputing 2013;105(0):3-11.Sun Y. Cost-sensitive boosting for classification of imbalanced data // Pattern recognition. 2007. 40(12). P. 3358-3378.Sun Y. Cost-sensitive boosting for classification of imbalanced data. Pattern Recognition 2007;40(12):3358-3378.Luukka P. Nonlinear fuzzy robust PCA algorithms and similarity classifier in bankruptcy analysis // Expert Systems with Applications. 2010. 37(12). P. 8296-8302.Luukka P. Nonlinear fuzzy robust PCA algorithms and similarity classifier in bankruptcy analysis. Expert Systems with Applications 2010;37(12):8296-8302.Zheng Z., Wu X., Srihari R. Feature selection for text categorization on imbalanced data // ACM SIGKDD Explorations Newsletter. 2004. 6(1). P. 80-89.Zheng Z., Wu X., Srihari R. Feature selection for text categorization on imbalanced data. ACM SIGKDD Explorations Newsletter 2004;6(1):80-89.Visa S., Ralescu A.L. Fuzzy Classifiers for Imbalanced Data Sets. University of Cincinnati, Computer Science Dept. Cincinnati, OH, United States, 2007. 157 p.Visa S., Ralescu A.L. Fuzzy Classifiers for Imbalanced Data Sets. University of Cincinnati, Computer Science Dept. Cincinnati (OH, United States); 2007.Imam T., Ting K., Kamruzzaman J. z-SVM: An SVM for Improved Classification of Imbalanced Data // AI 2006: Advances in Artificial Intelligence. 19th Australian Joint Conference on Artificial Intelligence, Hobart, Australia, December 4-8, 2006. Proceedings. P. 264-273.Imam T., Ting K., Kamruzzaman J. z-SVM: An SVM for Improved Classification of Imbalanced Data. AI 2006: Advances in Artificial Intelligence. 19th Australian Joint Conference on Artificial Intelligence. Hobart (Australia); 2006. p. 264-273.Fernández A., M.J. del Jesus, Herrera F. Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets // International Journal of Approximate Reasoning. 2009. 50(3). P.561-577.Fernández A., M.J. del Jesus, Herrera F. Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets. International Journal of Approximate Reasoning 2009;50(3):561-577.Kriminger E., Principe J.C., Lakshminarayan C. Nearest Neighbor Distributions for imbalanced classification // The 2012 international joint conference on neural networks (IJCNN), Brisbane, QLD, Australia, 10–15 June 2012. P. 1–5.Kriminger E., Principe J.C., Lakshminarayan C. Nearest Neighbor Distributions for imbalanced classification. The 2012 international joint conference on neural networks (IJCNN). Brisbane (QLD, Australia): 2012. p. 1–5.Li Y., Zhang X. Improving k nearest neighbor with exemplar generalization for imbalanced classification // Advances in knowledge discovery and data mining: 15th Pacific-Asia Conference, PAKDD 2011, Shenzhen, China, May 24-27, 2011, Proceedings, Part II. P. 321-332.Li Y., Zhang X. Improving k nearest neighbor with exemplar generalization for imbalanced classification. In: Proceedings of Advances in knowledge discovery and data mining: 15th Pacific-Asia Conference, Part II. Shenzhen (China); 2011. p. 321-332.Candès E.J. Robust principal component analysis // Journal of the ACM (JACM). 2011. 58(3). P. 11.Candès E.J. Robust principal component analysis. Journal of the ACM (JACM) 2011;58(3):11.Japkowicz N., Myers C., Gluck M. A novelty detection approach to classification // IJCAI. Vol. 1. 1995. P. 518–523.Japkowicz N., Myers C., Gluck M. A novelty detection approach to classification. IJCAI 1995;1:518–523.Jolliffe I. Principal component analysis // Encyclopedia of Statistics in Behavioral Science. John Wiley & Sons, Ltd, 2005.Jolliffe I. Principal component analysis. Encyclopedia of Statistics in Behavioral Science. John Wiley & Sons, Ltd; 2005.Cao P., Zhao D., Zaiane O. An Optimized CostSensitive SVM for Imbalanced Data Learning // Advances in Knowledge Discovery and Data Mining: 17th Pacific-Asia Conference, PAKDD 2013, Gold Coast, Australia, April 14-17, 2013, Proceedings, Part II. P. 280-292.Cao P., Zhao D., Zaiane O. An Optimized Cost-Sensitive SVM for Imbalanced Data Learning. In: Proceedings of Advances in Knowledge Discovery and Data Mining: 17th Pacific-Asia Conference. Part II. Gold Coast (Australia); 2013. p. 280-292.Cao P., Zhao D., Zaiane O. A PSO-based CostSensitive Neural Network for Imbalanced Data Classification // Trends and Applications in Knowledge Discovery and Data Mining: PAKDD 2013 International Workshops: DMApps, DANTH, QIMIE, BDM, CDA, CloudSD, Gold Coast, QLD, Australia, April 14-17, 2013, Revised Selected Papers. P 452–463.Cao P., Zhao D., Zaiane O. A PSO-based Cost-Sensitive Neural Network for Imbalanced Data Classification. In: Revised Selected Papers of Trends and Applications in Knowledge Discovery and Data Mining. International Workshops: DMApps, DANTH, QIMIE, BDM, CDA, CloudSD. Gold Coast (Australia); 2013. p. 452-463.Wang X., Shao H, Japkowicz N et al. Using SVM with Adaptively Asymmetric Misclassification Costs for Mine-Like Objects Detection // 2012 11th International Conference on Machine Learning and Applications, Boca Raton, FL, USA, 12-15 Dec. 2012. P. 78–82.Wang X., Shao H, Japkowicz N, et al. Using SVM with Adaptive Asymmetric Miseducation Costs for Minelike objects Detection. In: Proceedings of the 11th International Conference on Machine Learning and Applications. Boca Raton (Florida, USA); 2012. p. 78-82.Yang P., Liu W, Zhou B.B. et al. Ensemble-based wrapper methods for feature selection and class imbalance learning // Advances in Knowledge Discovery and Data Mining: 17th Pacific-Asia Conference, PAKDD 2013, Gold Coast, Australia, April 14-17, 2013, Proceedings, Part I. P. 544-555.Yang P., Liu W, Zhou B.B. et al. Ensemble-based wrapper methods for feature selection and class imbalance learning. In: Proceedings of Advances in Knowledge Discovery and Data Mining: 17th Pacific-Asia Conference, Part I. Gold Coast (Australia); 2013. p. 544-555.Yu E., Cho S. Ensemble based on GA wrapper feature selection // Computers & Industrial Engineering. 2006. 51(1). P.111-116.Yu E., Cho S. Ensemble based on GA wrapper feature selection. Computers & Industrial Engineering 2006;51(1):111-116.Liao J.-J. An ensemble-based model for two-class imbalanced financial problem // Economic Modelling. 2014. 37(0). P.175-183.Liao J.-J. An ensemble-based model for two-class imbalanced financial problem. Economic Modelling 2014;37(0):175-183.Liu Y., AnA., Huang X. Boosting prediction accuracy on imbalanced datasets with SVM ensembles // Advances in Knowledge Discovery and Data Mining, 10th Pacific-Asia Conference, PAKDD 2006, Singapore, April 9-12, 2006, Proceedings. P. 107-118.Liu Y., AnA., Huang X. Boosting prediction accuracy on imbalanced datasets with SVM ensembles. In: Proceedings of Advances in Knowledge Discovery and Data Mining, 10th Pacific-Asia Conference. Singapore; 2006. p. 107-118.Seiffert C. RUSBoost: A hybrid approach to alleviating class imbalance // Systems, Man and Cybernetics. Part A: Systems and Humans. IEEE Transactions. 2010. 40(1). P.185-197.Seiffert C. RUSBoost: A hybrid approach to alleviating class imbalance. Systems, Man and Cybernetics. Part A: Systems and Humans. IEEE Transactions 2010;40(1):185-197.Chawla N.V. SMOTEBoost: Improving prediction of the minority class in boosting, // Proceedings of Conference: Knowledge Discovery in Databases: PKDD 2003, 7th European Conference on Principles and Practice of Knowledge Discovery in Databases, Cavtat-Dubrovnik, Croatia, September 22-26, 2003. P. 107-119.Chawla N.V. SMOTEBoost: Improving prediction of the minority class in boosting. In: Proceedings of Knowledge Discovery in Databases: PKDD 2003, 7th European Conference on Principles and Practice of Knowledge Discovery in Databases. Cavtat-Dubrovnik (Croatia); 2003. p. 107-119.Wasikowski M., Chen X.-W. Combating the small sample class imbalance problem using feature selection // Knowledge and Data Engineering. IEEE Transactions. 2010. 22(10). P.1388-1400.Wasikowski M., Chen X.-W. Integrating the small sample class impact problem using feature selection. Knowledge and Data Engineering. IEEE transactions 2010;22(10): P.1388-1400.Martino M.D. Novel Classifier Scheme for Unbalance Problems // Pattern Recognition Letters. 2013. Vol. 34. Issue 10. P. 1146–1151.Martino M.D. Novel Classifier Scheme for Unbalance Problems. Pattern Recognition Letters 2013;34(10):1146–1151.Fernández A. A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets // Fuzzy Sets and Systems. 2008. 159(18). P. 2378-2398.Fernández A. A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets. Fuzzy Sets and Systems 2008;159(18):2378-2398.Le X., Mo-Yuen C., Taylor L.S. Power Distribution Fault Cause Identification With Imbalanced Data Using the Data Mining-Based Fuzzy Classification E-Algorithm // Power Systems. IEEE Transactions. 2007. 22(1). P. 164-171.Le X., Mo-Yuen C., Taylor L.S. Power Distribution Fault Cause Identification With Imbalanced Data Using the Data Mining-Based Fuzzy Classification E-Algorithm. Power Systems. IEEE Transactions 2007;22(1):164-171.Soler V. Imbalanced Datasets Classification by Fuzzy Rule Extraction and Genetic Algorithms // Data Mining Workshops 2006. ICDM Workshops 2006. Sixth IEEE International Conference 2006.Soler V. Imbalanced Datasets Classification by Fuzzy Rule Extraction and Genetic Algorithms. Data Mining Workshops. ICDM Workshops. Sixth IEEE International Conference; 2006.Hung C.-M. Huang Y.-M. Conflict-sensitivity contexture learning algorithm for mining interesting patterns using neuro-fuzzy network with decision rules // Expert Systems with Applications. 2008. 34(1). P. 159-172.Hung C.-M., Huang Y.-M. Conflict-sensitivity contexture learning algorithm for mining interesting patterns using neuro-fuzzy network with decision rules. Expert Systems with Applications 2008;34(1);159-172.Jeatrakul P., Wong K.W., Fung C.C. Classification of imbalanced data by combining the complementary neural network and SMOTE algorithm // Proceedings of the 17th international conference on Neural information processing: models and applications. Volume Part II. 2010. SpringerVerlag: Sydney, Australia. P. 152-159.Jeatrakul P., Wong K.W., Fung C.C. Classification of imbalanced data by combining the complementary neural network and SMOTE algorithm. Proceedings of the 17th International Conference on Neural Information Processing: Models and Applications. Part II. Sydney (Australia); Springer-Verlag. p. 152-159.Malof J.M., Mazurowski M.A., Tourassi G.D. The effect of class imbalance on case selection for case-based classifiers: An empirical study in the context of medical decision support // Neural Networks. 2012. 25(0). P. 141-145.Malof J.M., Mazurowski M.A., Tourassi G.D. The effect of class imbalance on case selection for case-based classifiers: An empirical study in the context of medical decision support. Neural Networks 2012;25(0):141-145.Mazurowski M.A. Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance // Neural networks. 2008. 21(2-3). P. 427-436.Mazurowski M.A. Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural networks 2008;21(2-3):427-436.Culotta A. Sorensen J. Dependency tree kernels for relation extraction // Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 2004.Culotta A., Sorensen J. Dependency tree kernels for relation extraction. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics; 2004.Drummond C., Holte R.C. Exploiting the cost (in) sensitivity of decision tree splitting criteria // ICML. 2000.Drummond C., Holte R.C. Exploiting the cost (in) sensitivity of decision tree splitting criteria. ICML; 2000.Al-Shahib A., Breitling R., Gilbert D. Feature selection and the class imbalance problem in predicting protein function from sequence // Applied Bioinformatics. 2005. 4(3). P. 195-203.Al-Shahib A., Breitling R., Gilbert D. Feature selection and the class imbalance problem in predicting protein function from sequence. Applied Bioinformatics 2005;4(3):195-203.Koziarski M. Radial-Based Undersampling for imbalanced data classification // Pattern Recognition. 2020. Vol. 102.Koziarski M. Radial-Based Undersampling for imbalanced data classification. Pattern Recognition 2020;102.He H., Bai Y., Garcia E.A. et al. ADASYN: adaptive synthetic sampling approach for imbalanced learning // IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). 2008. P. 1322-1328.He H., Bai Y., Garcia E.A. et al. ADASYN: adaptive synthetic sampling approach for imbalanced learning. In: Proceedings of IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence); 2008. p. 1322-1328.Mani I., Zhang I. kNN approach to unbalanced data distributions: a case study involving information extraction // Proceedings of Workshop on Learning from Imbalanced Datasets. 2003. Vol. 126.Mani I., Zhang I. kNN approach to unbalanced data distributions: a case study involving information extraction. In: Proceedings of Workshop on Learning from Imbalanced Datasets. Vol. 126; 2003.Wilson D.L. Asymptotic properties of nearest neighbor rules using edited data // IEEE Trans. Syst. Man Cybern. 1972. 2 (3). P. 408-421.Wilson D.L. Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans. Syst. Man Cybern. 1972;2(3): 408-421.Tomek I. Two modifications of CNN // IEEE Trans. Syst. Man Cybern. 1976. 6. P. 769-772.Tomek I. Two modifications of CNN. IEEE Trans. Syst. Man Cybern. 1976;6:769-772.Hart P. The condensed nearest neighbor rule // IEEE Trans. Inf. Theory. 1968. 14(3). P. 515-516.Hart P. The condensed nearest neighbor rule. IEEE Trans. Inf. Theory 1968;14(3):515-516.Махсотова Ц.В. Исследование методов классификации при несбалансированности классов // Научный журнал. 2017. №5 (18). URL: https://cyberleninka.ru/article/n/issledovanie-metodov-klassifikatsii-prinesbalansirovannosti-klassov (дата обращения: 5 июля 2020 г.).Makhsotova Ts.V. [Study of classification methods in the case of class imbalance]. Science Magazine 2017;5(18). (accessed July 5, 2020). Available at: https://cyberleninka.ru/article/n/issledovanie-metodov-klassifikatsii-pri-nesbalansirovannosti-klassov. (in Russ.)Каврин Д.А., Субботин С.А. Методы количественного решения проблемы несбалансированности классов // Радіоелектроніка, інформатика, управління. 2018. №1 (44). URL: https://cyberleninka.ru/article/n/metody-kolichestvennogo-resheniya-problemynesbalansirovannosti-klassov (дата обращения: 6 июля 2020 г.).Kavrin D.A., Subbotin S.A. The methods for quantitative solving the class imbalance problem. Radio Electronics, Computer Science, Control 2018;1. (accessed July 6, 2020). Available at: https://cyberleninka.ru/article/n/metody-kolichestvennogo-resheniya-problemy-nesbalansirovannostiklassov. (in Russ.)Yi Lu, Hong Guo, Feldkamp L. Robust neural learning from unbalanced data samples // 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227), Anchorage, AK. 1998. Vol. 3. P. 1816-1821.Yi L., Hong G., Feldkamp L. Robust neural learning from unbalanced data samples. In: Proceedings of IEEE International Joint Conference on Neural Networks. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227). Vol. 3. Anchorage (USA); 1998. p. 1816- 1821.Al-Stouhi S., Reddy C.K. Transfer learning for class imbalance problems with inadequate data // Knowledge and Information Systems. 2016. 48. P. 201–228.Al-Stouhi S., Reddy C.K. Transfer learning for class imbalance problems with inadequate data. Knowledge and Information Systems 2016;48:201-228.Near-Miss – version 0.9.0.dev0. API reference. URL: https://imbalanced-learn.org/dev/references/generated/imblearn.under_sampling.NearMiss.html (дата обращения: 10 июля 2021 г.).Near-Miss – version 0.9.0.dev0. API reference. (accessed July 10, 2021). Available at: https://imbalanced-learn.org/dev/references/generated/imblearn.under_sampling.NearMiss.html.Sun Y. Cost-Sensitive Boosting for Classification of Imbalanced Data // Pattern Recognition. 2007. Vol. 40. Issue 12. P. 3358–3378.Sun Y. Cost-Sensitive Boosting for Classification of Imbalanced Data.Pattern Recognition2007;40(12):3358-3378.

The authors declare that there are no conflicts of interest present.