The Experts below are selected from a list of 84 Experts worldwide ranked by ideXlab platform

Giuliano Casale - One of the best experts on this subject based on the ideXlab platform.

  • Artificial neural networks based techniques for anomaly detection in Apache Spark
    Cluster Computing, 2019
    Co-Authors: Ahmad Alnafessah, Giuliano Casale
    Abstract:

    Late detection and manual resolutions of performance anomalies in Cloud Computing and Big Data systems may lead to performance violations and financial penalties. Motivated by this issue, we propose an artificial neural network based methodology for anomaly detection tailored to the Apache Spark in-memory processing platform. Apache Spark is widely adopted by industry because of its speed and generality, however there is still a shortage of comprehensive performance anomaly detection methods applicable to this platform. We propose an artificial neural networks driven methodology to quickly sift through Spark logs data and operating system monitoring metrics to accurately detect and classify anomalous behaviors based on the Spark resilient distributed dataset characteristics. The proposed method is evaluated against three popular machine learning algorithms, decision trees, nearest neighbor, and support vector machine, as well as against four variants that consider different monitoring datasets. The results prove that our proposed method outperforms other methods, typically achieving 98–99% F-scores, and offering much greater accuracy than alternative techniques to detect both the period in which anomalies occurred and their type.

Ahmad Alnafessah - One of the best experts on this subject based on the ideXlab platform.

  • Artificial neural networks based techniques for anomaly detection in Apache Spark
    Cluster Computing, 2019
    Co-Authors: Ahmad Alnafessah, Giuliano Casale
    Abstract:

    Late detection and manual resolutions of performance anomalies in Cloud Computing and Big Data systems may lead to performance violations and financial penalties. Motivated by this issue, we propose an artificial neural network based methodology for anomaly detection tailored to the Apache Spark in-memory processing platform. Apache Spark is widely adopted by industry because of its speed and generality, however there is still a shortage of comprehensive performance anomaly detection methods applicable to this platform. We propose an artificial neural networks driven methodology to quickly sift through Spark logs data and operating system monitoring metrics to accurately detect and classify anomalous behaviors based on the Spark resilient distributed dataset characteristics. The proposed method is evaluated against three popular machine learning algorithms, decision trees, nearest neighbor, and support vector machine, as well as against four variants that consider different monitoring datasets. The results prove that our proposed method outperforms other methods, typically achieving 98–99% F-scores, and offering much greater accuracy than alternative techniques to detect both the period in which anomalies occurred and their type.

Casale G - One of the best experts on this subject based on the ideXlab platform.

  • Artificial neural networks based techniques for anomaly detection in Apache Spark
    'Springer Science and Business Media LLC', 2019
    Co-Authors: Alnafessah A, Casale G
    Abstract:

    Late detection and manual resolutions of performance anomalies in Cloud Computing and Big Data systems may lead to performance violations and financial penalties. Motivated by this issue, we propose an artificial neural network based methodology for anomaly detection especially for the Apache Spark in-memory processing platforms. Apache Spark has become widely adopted by industry because of its speed and generality, however there is still a shortage of comprehensive performance anomaly detection methods applicable to this platform. We propose artificial neural networks driven methodology to quickly sift through Spark logs data and operating system monitoring metrics to accurately detect and classify anomalous behaviors based on the Spark resilient distributed dataset (RDD) characteristics. The proposed method is evaluated against three popular machine learning algorithms, decision trees, nearest neighbor, and support vector machine (SVM), as well as against four variants that consider different monitoring datasets. The results prove that our proposed method outperforms other methods, typically achieving 98%-99% F-scores, and offering much greater accuracy than alternative techniques to detect both the period in which anomalies occurred and their type

Santanu Kumar Rath - One of the best experts on this subject based on the ideXlab platform.

  • fast computing of microarray data using resilient distributed dataset of apache spark
    2016
    Co-Authors: Ransingh Biswajit Ray, Mukesh Kumar, Santanu Kumar Rath
    Abstract:

    Microarray technology is one of the emerging technologies in the field of genetic research that many biologists use to monitor expression levels of genes in a given organism. Microarray experiments are used to investigate genome-wide expression changes in health care aspects. Analysis of large Microarray datasets has become a challenging task, as the number of genes available in commercial probe sets and the number of test samples for an experimental set increases. The colossal amount of raw gene expression data often leads to computational and analytical challenges including feature selection and classification of the dataset into correct group or class. In this paper, statistical method (test), i.e., ANOVA based on Spark framework is proposed to select the pertinent features. After feature selection, various classifiers i.e., Naive Bayes (sf-NB) and Logistic Regression (sf-LoR) based on Spark framework are applied to classify the Microarray dataset. A detail comparative analysis in terms of execution time and accuracy is done on these feature selection and classifier methodologies that are based on Spark framework and conventional system respectively.

Raju Kumar Mishra - One of the best experts on this subject based on the ideXlab platform.