A Literature Review

The following section is a summary of paper Using Machine Learning to Generate Test Oracles: A Systematic Literature Review. This paper summarizes the studies in this field, which published on Aug 24, 2021.

Background and Statistics

Problems concerned

1. Types of oracles generated
2. Researchers goal using ML
3. Which specific ML technique
4. How such skill is trained and validated
5. How the success of the generation process is accessed
6. Limitations

Basic Statistics

1. Goal
   • Test verdict: 18%
   • Metamorphic relation: 27%
   • Expected output: 55%
2. Types of ML
   • Supervised learning: 96%
3. Input
   • Labeled system execution logs
   • Source code metadata
4. Method used
   • NN (59%): Backpropagation NN, Multipalyer Perceptrons, RBF NN, probabilistic NN, DNN
   • SVM (23%)
   • DT (5%)
   • Adaptive boosting (5%)
5. Evaluation
   • Mutation Score (55%)
   • Accuracy (18%)
   • Number of correct classifications (18%)
   • ROC (5%)
6. Limitations
   • Quantity of training data
   • Hard to cope with multiple output functions - DL and ensemble should be explored
   • Lack of common benchmarks.