Package org.moeaframework

Class Analyzer

java.lang.Object
org.moeaframework.Analyzer
All Implemented Interfaces:
Displayable
public class Analyzer extends Object implements Displayable
Performs basic end-of-run analysis. This includes evaluating the selected performance indicators, summarizing the indicator values with descriptive statistics (min, median, max, inter-quartile range, etc.), and determining if the results of each algorithm are statistically similar.

For example, the following demonstrates its typical use. First construct and configure the analyzer: 

   Analyzer analyzer = new Analyzer()
       .withProblem("DTLZ2_2")
       .includeGenerationalDistance()
       .includeInvertedGenerationalDistance()
       .includeAdditiveEpsilonIndicator()
       .includeContribution()
       .showAggregate()
       .showStatisticalSignificance();
The problem must always be specified. Next, add the data to be analyzed: 
   Executor executor = new Executor().withProblem("DTLZ2_2");
   analyzer.add("NSGAII", executor.withAlgorithm("NSGAII").run());
   analyzer.add("eMOEA", executor.withAlgorithm("eMOEA").run());
Lastly, print the results of the analysis: 
   analyzer.display();

  • Constructor Details

    • Analyzer

      public Analyzer()
      Constructs a new analyzer initialized with default settings.

  • Method Details

    • withSameProblemAs

      public Analyzer withSameProblemAs(org.moeaframework.ProblemBuilder builder)

    • usingProblemFactory

      public Analyzer usingProblemFactory(ProblemFactory problemFactory)

    • withProblem

      public Analyzer withProblem(String problemName)

    • withProblem

      public Analyzer withProblem(Problem problemInstance)

    • withProblemClass

      public Analyzer withProblemClass(Class<?> problemClass, Object... problemArguments)

    • withProblemClass

      public Analyzer withProblemClass(String problemClassName, Object... problemArguments) throws ClassNotFoundException
      Throws:
      ClassNotFoundException

    • withEpsilon

      public Analyzer withEpsilon(double... epsilon)

    • withEpsilons

      public Analyzer withEpsilons(Epsilons epsilons)

    • withReferenceSet

      public Analyzer withReferenceSet(File referenceSetFile)

    • withReferenceSet

      public Analyzer withReferenceSet(NondominatedPopulation referenceSet)

    • includeHypervolume

      public Analyzer includeHypervolume()
      Enables the evaluation of the hypervolume metric.
      Returns:
      a reference to this analyzer

    • includeGenerationalDistance

      public Analyzer includeGenerationalDistance()
      Enables the evaluation of the generational distance metric.
      Returns:
      a reference to this analyzer

    • includeGenerationalDistancePlus

      public Analyzer includeGenerationalDistancePlus()
      Enables the evaluation of the generational distance plus metric.
      Returns:
      a reference to this analyzer

    • includeInvertedGenerationalDistance

      public Analyzer includeInvertedGenerationalDistance()
      Enables the evaluation of the inverted generational distance metric.
      Returns:
      a reference to this analyzer

    • includeInvertedGenerationalDistancePlus

      public Analyzer includeInvertedGenerationalDistancePlus()
      Enables the evaluation of the inverted generational distance plus metric.
      Returns:
      a reference to this analyzer

    • includeAdditiveEpsilonIndicator

      public Analyzer includeAdditiveEpsilonIndicator()
      Enables the evaluation of the additive ε-indicator metric.
      Returns:
      a reference to this analyzer

    • includeMaximumParetoFrontError

      public Analyzer includeMaximumParetoFrontError()
      Enables the evaluation of the maximum Pareto front error metric.
      Returns:
      a reference to this analyzer

    • includeSpacing

      public Analyzer includeSpacing()
      Enables the evaluation of the spacing metric.
      Returns:
      a reference to this analyzer

    • includeContribution

      public Analyzer includeContribution()
      Enables the evaluation of the contribution metric.
      Returns:
      a reference to this analyzer

    • includeR1

      public Analyzer includeR1()
      Enables the evaluation of the R1 indicator.
      Returns:
      a reference to this analyzer

    • includeR2

      public Analyzer includeR2()
      Enables the evaluation of the R2 indicator.
      Returns:
      a reference to this analyzer

    • includeR3

      public Analyzer includeR3()
      Enables the evaluation of the R3 indicator.
      Returns:
      a reference to this analyzer

    • includeAllMetrics

      public Analyzer includeAllMetrics()
      Enables the evaluation of all metrics.
      Returns:
      a reference to this analyzer

    • showAll

      public Analyzer showAll()
      Enables the output of all analysis results.
      Returns:
      a reference to this analyzer

    • showIndividualValues

      public Analyzer showIndividualValues()
      Enables the output of individual metric values for each seed.
      Returns:
      a reference to this analyzer

    • showAggregate

      public Analyzer showAggregate()
      Enables the output of the metric value of the aggregate approximation set, produced by merging all individual seeds.
      Returns:
      a reference to this analyzer

    • showStatisticalSignificance

      public Analyzer showStatisticalSignificance()
      Enables the output of statistical significance tests. If enabled, it is necessary to record multiple seeds for each entry.
      Returns:
      a reference to this analyzer

    • showStatistic

      public Analyzer showStatistic(org.apache.commons.math3.stat.descriptive.UnivariateStatistic statistic)
      Specifies the UnivariateStatistics calculated during the analysis. If none are specified by the user, then the default statistics are displayed.
      Parameters:
      statistic - the statistic to calculate
      Returns:
      a reference to this analyzer

    • withSignifianceLevel

      public Analyzer withSignifianceLevel(double significanceLevel)
      Sets the level of significance used when testing the statistical significance of observed differences in the medians. Commonly used levels of significance are 0.05 and 0.01.
      Parameters:
      significanceLevel - the level of significance
      Returns:
      a reference to this analyzer

    • withIdealPoint

      public Analyzer withIdealPoint(double... idealPoint)
      Sets the ideal point used for computing the hypervolume metric.
      Parameters:
      idealPoint - the ideal point
      Returns:
      a reference to this analyzer

    • withReferencePoint

      public Analyzer withReferencePoint(double... referencePoint)
      Sets the reference point used for computing the hypervolume metric.
      Parameters:
      referencePoint - the reference point
      Returns:
      a reference to this analyzer

    • addAll

      public Analyzer addAll(String name, Collection<NondominatedPopulation> approximationSets)
      Adds the collection of new samples with the specified name.
      Parameters:
      name - the name of these samples
      approximationSets - the approximation sets
      Returns:
      a reference to this analyzer

    • add

      public Analyzer add(String name, NondominatedPopulation approximationSet)
      Adds a new sample with the specified name. If multiple samples are added using the same name, each sample is treated as an individual seed. Analyses can be performed on both the individual seeds and aggregates of the seeds.
      Parameters:
      name - the name of this sample
      approximationSet - the approximation set
      Returns:
      a reference to this analyzer

    • saveData

      public Analyzer saveData(File directory, String prefix, String suffix) throws IOException
      Saves all data stored in this analyzer, which can subsequently be read using loadData(File, String, String) with matching arguments.
      Parameters:
      directory - the directory in which the data is stored
      prefix - the prefix for filenames
      suffix - the suffix (extension) for filenames
      Returns:
      a reference to this analyzer
      Throws:
      IOException - if an I/O error occurred

    • loadData

      public Analyzer loadData(File directory, String prefix, String suffix) throws IOException
      Loads data into this analyzer, which was previously saved using saveData(File, String, String) with matching arguments.
      Parameters:
      directory - the directory in which the data is stored
      prefix - the prefix for filenames
      suffix - the suffix (extension) for filenames
      Returns:
      a reference to this analyzer
      Throws:
      IOException - if an I/O error occurred

    • loadAs

      public Analyzer loadAs(String name, File resultFile) throws IOException
      Loads the samples stored in a result file using ResultFileReader.
      Parameters:
      name - the name of the samples
      resultFile - the result file to load
      Returns:
      a reference to this analyzer
      Throws:
      IOException - if an I/O error occurred

    • saveAs

      public Analyzer saveAs(String name, File resultFile) throws IOException
      Saves the samples to a result file using ResultFileWriter. If name is null, the reference set is saved. Otherwise, the approximation sets for the named entries are saved.
      Parameters:
      name - the name of the samples
      resultFile - the result file to which the data is saved
      Returns:
      a reference to this analyzer
      Throws:
      IOException - if an I/O error occurred

    • saveAnalysis

      public Analyzer saveAnalysis(File file) throws IOException
      Saves the analysis of all data recorded in this analyzer to the specified file.
      Parameters:
      file - the file to which the analysis is saved
      Returns:
      a reference to this analyzer
      Throws:
      IOException - if an I/O error occurred

    • printAnalysis

      public Analyzer printAnalysis()
      Prints the analysis of all data recorded in this analyzer to standard output.
      Returns:
      a reference to this analyzer

    • saveReferenceSet

      public Analyzer saveReferenceSet(File file) throws IOException
      Saves the reference set to the specified file.
      Parameters:
      file - the file to which the reference set is saved
      Returns:
      a reference to this analyzer
      Throws:
      IOException - if an I/O error occurred
      See Also:

    • getReferenceSet

      public NondominatedPopulation getReferenceSet()
      Returns the reference set used by this analyzer. The reference set is generated as follows:
      1. If withReferenceSet(File) has been set, the contents of the reference set file are returned;
      2. If the problem factory provides a reference set via the ProblemFactory.getReferenceSet(String) method, this reference set is returned;
      3. Otherwise, the reference set is aggregated from all individual approximation sets.
      Returns:
      the reference set used by this analyzer
      Throws:
      IllegalArgumentException - if the reference set could not be loaded

    • getAnalysis

      public Analyzer.AnalyzerResults getAnalysis()
      Generates the analysis of all data recorded in this analyzer.
      Returns:
      an object storing the results of the analysis

    • printAnalysis

      public Analyzer printAnalysis(PrintStream ps)
      Prints the analysis of all data recorded in this analyzer.
      Parameters:
      ps - the stream to which the analysis is written
      Returns:
      a reference to this analyzer

    • display

      public void display(PrintStream ps)
      Description copied from interface: Displayable
      Displays the contents of this object to the given output stream. This method does not close the underlying stream; the caller is responsible for disposing it.
      Specified by:
      display in interface Displayable
      Parameters:
      ps - the output stream

    • clear

      public Analyzer clear()
      Clears all data stored in this analyzer.
      Returns:
      a reference to this analyzer