MOEA Framework 2.12
API Specification

org.moeaframework
Class Analyzer

java.lang.Object
  extended by org.moeaframework.Analyzer

public class Analyzer
extends Object

Performs basic end-of-run analysis. For example, the following demonstrates its typical use. First construct and configure the analyzer:

   Analyzer analyzer = new Analyzer()
       .withProblem("DTLZ2_2")
       .includeGenerationalDistance()
       .includeInvertedGenerationalDistance()
       .includeAdditiveEpsilonIndicator()
       .includeContribution()
       .showAggregate()
       .showStatisticalSignificance();
 

The problem must always be specified. Next, add the data to be analyzed:

   Executor executor = new Executor().withProblem("DTLZ2_2");
   add("NSGAII", executor.withAlgorithm("NSGAII").run());
   add("eMOEA", executor.withAlgorithm("eMOEA").run());
 
Lastly, print the results of the analysis:
   analyzer.printAnalysis();
 
The output produced is compatible with the YAML format, and thus can be postprocessed easily with any YAML parser. The results can also be accessed programatically by calling getAnalysis().


Nested Class Summary
 class Analyzer.AlgorithmResult
          Stores the results for a single algorithm.
 class Analyzer.AnalyzerResults
          Stores the results produced by this analyzer.
 class Analyzer.IndicatorResult
          Inner class for storing the results for a single performance indicator.
 
Constructor Summary
Analyzer()
          Constructs a new analyzer initialized with default settings.
 
Method Summary
 Analyzer add(String name, NondominatedPopulation result)
          Adds a new sample with the specified name.
 Analyzer addAll(String name, Collection<NondominatedPopulation> results)
          Adds the collection of new samples with the specified name.
 Analyzer clear()
          Clears all data stored in this analyzer.
 Analyzer.AnalyzerResults getAnalysis()
          Generates the analysis of all data recorded in this analyzer.
 NondominatedPopulation getReferenceSet()
          Returns the reference set used by this analyzer.
 Analyzer includeAdditiveEpsilonIndicator()
          Enables the evaluation of the additive ε-indicator metric.
 Analyzer includeAllMetrics()
          Enables the evaluation of all metrics.
 Analyzer includeContribution()
          Enables the evaluation of the contribution metric.
 Analyzer includeGenerationalDistance()
          Enables the evaluation of the generational distance metric.
 Analyzer includeHypervolume()
          Enables the evaluation of the hypervolume metric.
 Analyzer includeInvertedGenerationalDistance()
          Enables the evaluation of the inverted generational distance metric.
 Analyzer includeMaximumParetoFrontError()
          Enables the evaluation of the maximum Pareto front error metric.
 Analyzer includeR1()
          Enables the evaluation of the R1 indicator.
 Analyzer includeR2()
          Enables the evaluation of the R2 indicator.
 Analyzer includeR3()
          Enables the evaluation of the R3 indicator.
 Analyzer includeSpacing()
          Enables the evaluation of the spacing metric.
 Analyzer loadAs(String name, File resultFile)
          Loads the samples stored in a result file using ResultFileReader.
 Analyzer loadData(File directory, String prefix, String suffix)
          Loads data into this analyzer, which was previously saved using saveData(File, String, String) with matching arguments.
 Analyzer printAnalysis()
          Prints the analysis of all data recorded in this analyzer to standard output.
 Analyzer printAnalysis(PrintStream ps)
          Prints the analysis of all data recorded in this analyzer.
 Analyzer saveAnalysis(File file)
          Saves the analysis of all data recorded in this analyzer to the specified file.
 Analyzer saveAs(String name, File resultFile)
          Saves the samples to a result file using ResultFileWriter.
 Analyzer saveData(File directory, String prefix, String suffix)
          Saves all data stored in this analyzer, which can subsequently be read using loadData(File, String, String) with matching arguments.
 Analyzer saveReferenceSet(File file)
          Saves the reference set to the specified file.
 Analyzer showAggregate()
          Enables the output of the metric value of the aggregate approximation set, produced by merging all individual seeds.
 Analyzer showAll()
          Enables the output of all analysis results.
 Analyzer showIndividualValues()
          Enables the output of individual metric values for each seed.
 Analyzer showStatistic(org.apache.commons.math3.stat.descriptive.UnivariateStatistic statistic)
          Specifies the UnivariateStatistics calculated during the analysis.
 Analyzer showStatisticalSignificance()
          Enables the output of statistical significance tests.
 Analyzer usingProblemFactory(ProblemFactory problemFactory)
          Sets the problem factory used by this builder.
 Analyzer withEpsilon(double... epsilon)
          Sets the ε values used by this builder, specifying the archive returned by newArchive().
 Analyzer withIdealPoint(double... idealPoint)
          Sets the ideal point used for computing the hypervolume metric.
 Analyzer withProblem(Problem problemInstance)
          Sets the problem instance used by this builder.
 Analyzer withProblem(String problemName)
          Sets the problem used by this builder.
 Analyzer withProblemClass(Class<?> problemClass, Object... problemArguments)
          Sets the problem used by this builder.
 Analyzer withProblemClass(String problemClassName, Object... problemArguments)
          Sets the problem used by this builder.
 Analyzer withReferencePoint(double... referencePoint)
          Sets the reference point used for computing the hypervolume metric.
 Analyzer withReferenceSet(File referenceSetFile)
          Sets the file containing the reference set to be used by this builder.
 Analyzer withSameProblemAs(org.moeaframework.ProblemBuilder builder)
          Configures this builder to use the same problem, reference set, ε values and ProblemFactory as the specified instance.
 Analyzer withSignifianceLevel(double significanceLevel)
          Sets the level of significance used when testing the statistical significance of observed differences in the medians.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

Analyzer

public Analyzer()
Constructs a new analyzer initialized with default settings.

Method Detail

withSameProblemAs

public Analyzer withSameProblemAs(org.moeaframework.ProblemBuilder builder)
Configures this builder to use the same problem, reference set, ε values and ProblemFactory as the specified instance.

Parameters:
builder - the instance whose problem, reference set, ε values and ProblemFactory are to be copied
Returns:
a reference to this builder

usingProblemFactory

public Analyzer usingProblemFactory(ProblemFactory problemFactory)
Sets the problem factory used by this builder.

Parameters:
problemFactory - the problem factory
Returns:
a reference to this builder

withProblem

public Analyzer withProblem(String problemName)
Sets the problem used by this builder.

Parameters:
problemName - the problem name
Returns:
a reference to this builder

withProblem

public Analyzer withProblem(Problem problemInstance)
Sets the problem instance used by this builder. Until the other withProblem methods, using a problem instance will not close the problem. It is the responsibility of the user to ensure any problems holding resources are properly closed.

Parameters:
problemInstance - the problem instance
Returns:
a reference to this builder

withProblemClass

public Analyzer withProblemClass(Class<?> problemClass,
                                 Object... problemArguments)
Sets the problem used by this builder. Instances of the problem are created using the Class.newInstance() method. Ensure the problem class satisfies all the conditions of newInstance(), otherwise exceptions may be thrown when attempting to create an instance of the problem.

Parameters:
problemClass - the problem class
problemArguments - the arguments passed to the problem constructor
Returns:
a reference to this builder

withProblemClass

public Analyzer withProblemClass(String problemClassName,
                                 Object... problemArguments)
                          throws ClassNotFoundException
Sets the problem used by this builder. Instances of the problem are created using the Class.newInstance() method. Ensure the problem class satisfies all the conditions of newInstance(), otherwise exceptions may be thrown when attempting to create an instance of the problem.

Parameters:
problemClassName - the problem class name
problemArguments - the arguments passed to the problem constructor
Returns:
a reference to this builder
Throws:
ClassNotFoundException - if the specified problem class name could not be found

withEpsilon

public Analyzer withEpsilon(double... epsilon)
Sets the ε values used by this builder, specifying the archive returned by newArchive().

Parameters:
epsilon - the ε values
Returns:
a reference to this builder

withReferenceSet

public Analyzer withReferenceSet(File referenceSetFile)
Sets the file containing the reference set to be used by this builder. If not specified, the reference set should be aggregated from all individual approximation sets.

Parameters:
referenceSetFile - the reference set file
Returns:
a reference to this builder

includeHypervolume

public Analyzer includeHypervolume()
Enables the evaluation of the hypervolume metric.

Returns:
a reference to this analyzer

includeGenerationalDistance

public Analyzer includeGenerationalDistance()
Enables the evaluation of the generational distance metric.

Returns:
a reference to this analyzer

includeInvertedGenerationalDistance

public Analyzer includeInvertedGenerationalDistance()
Enables the evaluation of the inverted generational distance metric.

Returns:
a reference to this analyzer

includeAdditiveEpsilonIndicator

public Analyzer includeAdditiveEpsilonIndicator()
Enables the evaluation of the additive ε-indicator metric.

Returns:
a reference to this analyzer

includeMaximumParetoFrontError

public Analyzer includeMaximumParetoFrontError()
Enables the evaluation of the maximum Pareto front error metric.

Returns:
a reference to this analyzer

includeSpacing

public Analyzer includeSpacing()
Enables the evaluation of the spacing metric.

Returns:
a reference to this analyzer

includeContribution

public Analyzer includeContribution()
Enables the evaluation of the contribution metric.

Returns:
a reference to this analyzer

includeR1

public Analyzer includeR1()
Enables the evaluation of the R1 indicator.

Returns:
a reference to this analyzer

includeR2

public Analyzer includeR2()
Enables the evaluation of the R2 indicator.

Returns:
a reference to this analyzer

includeR3

public Analyzer includeR3()
Enables the evaluation of the R3 indicator.

Returns:
a reference to this analyzer

includeAllMetrics

public Analyzer includeAllMetrics()
Enables the evaluation of all metrics.

Returns:
a reference to this analyzer

showAll

public Analyzer showAll()
Enables the output of all analysis results.

Returns:
a reference to this analyzer

showIndividualValues

public Analyzer showIndividualValues()
Enables the output of individual metric values for each seed.

Returns:
a reference to this analyzer

showAggregate

public Analyzer showAggregate()
Enables the output of the metric value of the aggregate approximation set, produced by merging all individual seeds.

Returns:
a reference to this analyzer

showStatisticalSignificance

public Analyzer showStatisticalSignificance()
Enables the output of statistical significance tests. If enabled, it is necessary to record multiple seeds for each entry.

Returns:
a reference to this analyzer

showStatistic

public Analyzer showStatistic(org.apache.commons.math3.stat.descriptive.UnivariateStatistic statistic)
Specifies the UnivariateStatistics calculated during the analysis. If none are specified by the user, then Min, Median and Max are used.

Parameters:
statistic - the statistic to calculate
Returns:
a reference to this analyzer

withSignifianceLevel

public Analyzer withSignifianceLevel(double significanceLevel)
Sets the level of significance used when testing the statistical significance of observed differences in the medians. Commonly used levels of significance are 0.05 and 0.01.

Parameters:
significanceLevel - the level of significance
Returns:
a reference to this analyzer

withIdealPoint

public Analyzer withIdealPoint(double... idealPoint)
Sets the ideal point used for computing the hypervolume metric.

Parameters:
idealPoint - the ideal point
Returns:
a reference to this analyzer

withReferencePoint

public Analyzer withReferencePoint(double... referencePoint)
Sets the reference point used for computing the hypervolume metric.

Parameters:
referencePoint - the reference point
Returns:
a reference to this analyzer

addAll

public Analyzer addAll(String name,
                       Collection<NondominatedPopulation> results)
Adds the collection of new samples with the specified name.

Parameters:
name - the name of these samples
results - the approximation sets
Returns:
a reference to this analyzer

add

public Analyzer add(String name,
                    NondominatedPopulation result)
Adds a new sample with the specified name. If multiple samples are added using the same name, each sample is treated as an individual seed. Analyses can be performed on both the individual seeds and aggregates of the seeds.

Parameters:
name - the name of this sample
result - the approximation set
Returns:
a reference to this analyzer

saveData

public Analyzer saveData(File directory,
                         String prefix,
                         String suffix)
                  throws IOException
Saves all data stored in this analyzer, which can subsequently be read using loadData(File, String, String) with matching arguments.

Parameters:
directory - the directory in which the data is stored
prefix - the prefix for filenames
suffix - the suffix (extension) for filenames
Returns:
a reference to this analyzer
Throws:
IOException - if an I/O error occurred

loadData

public Analyzer loadData(File directory,
                         String prefix,
                         String suffix)
                  throws IOException
Loads data into this analyzer, which was previously saved using saveData(File, String, String) with matching arguments.

Parameters:
directory - the directory in which the data is stored
prefix - the prefix for filenames
suffix - the suffix (extension) for filenames
Returns:
a reference to this analyzer
Throws:
IOException - if an I/O error occurred

loadAs

public Analyzer loadAs(String name,
                       File resultFile)
                throws IOException
Loads the samples stored in a result file using ResultFileReader.

Parameters:
name - the name of the samples
resultFile - the result file to load
Returns:
a reference to this analyzer
Throws:
IOException - if an I/O error occurred

saveAs

public Analyzer saveAs(String name,
                       File resultFile)
                throws IOException
Saves the samples to a result file using ResultFileWriter. If name is null, the reference set is saved. Otherwise, the approximation sets for the named entries are saved.

Parameters:
name - the name of the samples
resultFile - the result file to which the data is saved
Returns:
a reference to this analyzer
Throws:
IOException - if an I/O error occurred

saveAnalysis

public Analyzer saveAnalysis(File file)
                      throws IOException
Saves the analysis of all data recorded in this analyzer to the specified file.

Parameters:
file - the file to which the analysis is saved
Returns:
a reference to this analyzer
Throws:
IOException - if an I/O error occurred

printAnalysis

public Analyzer printAnalysis()
Prints the analysis of all data recorded in this analyzer to standard output.

Returns:
a reference to this analyzer

saveReferenceSet

public Analyzer saveReferenceSet(File file)
                          throws IOException
Saves the reference set to the specified file.

Parameters:
file - the file to which the reference set is saved
Returns:
a reference to this analyzer
Throws:
IOException - if an I/O error occurred
See Also:
getReferenceSet()

getReferenceSet

public NondominatedPopulation getReferenceSet()
Returns the reference set used by this analyzer. The reference set is generated as follows:
  1. If withReferenceSet(File) has been set, the contents of the reference set file are returned;
  2. If the problem factory provides a reference set via the ProblemFactory.getReferenceSet(String) method, this reference set is returned;
  3. Otherwise, the reference set is aggregated from all individual approximation sets.

Returns:
the reference set used by this analyzer
Throws:
IllegalArgumentException - if the reference set could not be loaded

getAnalysis

public Analyzer.AnalyzerResults getAnalysis()
Generates the analysis of all data recorded in this analyzer.

Returns:
an object storing the results of the analysis

printAnalysis

public Analyzer printAnalysis(PrintStream ps)
Prints the analysis of all data recorded in this analyzer.

Parameters:
ps - the stream to which the analysis is written
Returns:
a reference to this analyzer
Throws:
IOException - if an I/O error occurred

clear

public Analyzer clear()
Clears all data stored in this analyzer.

Returns:
a reference to this analyzer

MOEA Framework 2.12
API Specification

Copyright 2009-2016 MOEA Framework. All rights reserved.
Licensed under the GNU Lesser General Public License.
Return to the MOEA Framework homepage. Visit us on Github!