experiment package

Subpackages

• experiment.hooks package
  • Submodules
  • experiment.hooks.hook_skeleton module
  • experiment.hooks.ldst_filterstats_hook module
  • experiment.hooks.plot_hooks module
  • experiment.hooks.time_hook module
  • Module contents
• experiment.specification package
  • Submodules
  • experiment.specification.exp_chapelle module
  • experiment.specification.exp_cifar module
  • experiment.specification.exp_debug module
  • experiment.specification.exp_filter_LDST module
  • experiment.specification.specification_bits module
  • experiment.specification.specification_skeleton module
  • Module contents

Submodules

experiment.experiments module

Created on 27 de mar de 2019

@author: klaus

class experiment.experiments.Experiment(args)

Bases: object

Encapsulates an experiment, composed of the following steps.

1. Reading the input features and true labels.

2. Apply some noise process to the true labels, obtaining the corrupted labels.

3. Create the Affinity matrix from the input features (and, optionally, noisy labels).

4. Apply some filter to the corrupted labels, obtaining filtered labels.

5. Run an GSSL algorithm to obtain the classification

6. Get performance measures from the classification and filtered labels.

X

The calculated input matrix

Type

NDArray[float]shape[N,D]

W

The affinity matrix encoding the graph.

Type

NDArray[float]shape[N,N]

Y

Initial label matrix

Type

NDArray[float]shape[N,C]

__init__(args)

Initialize self. See help(type(self)) for accurate signature.

run(hook_list=[<Hook.INIT_LABELED: 2>, <Hook.INIT_ALL: 1>, <Hook.NOISE_AFTER: 3>, <Hook.ALG_RESULT: 10>, <Hook.ALG_ITER: 11>])

experiment.experiments.keys_multiplex(args)

experiment.experiments.main()

experiment.experiments.postprocess(mplex)

Performs some postprocessing on the multiplexed keys.

experiment.experiments.run_debug_example_all()

experiment.experiments.run_debug_example_one(hook_list=[])

experiment.prefixes module

Created on 28 de mar de 2019

@author: klaus

experiment.selector module

Created on 28 de mar de 2019

@author: klaus

class experiment.selector.Hook

Bases: enum.Enum

An enumeration.

ALG_ITER = 11

ALG_RESULT = 10

FILTER_AFTER = 9

FILTER_ITER = 8

GTAM_F = 13

GTAM_Q = 14

GTAM_Y = 12

INIT_ALL = 1

INIT_LABELED = 2

LDST_STATS_HOOK = 19

NOISE_AFTER = 3

T_AFFMAT = 18

T_ALG = 17

T_FILTER = 16

T_NOISE = 15

W_FILTER_AFTER = 7

W_INIT_ALL = 4

W_INIT_LABELED = 5

W_NOISE_AFTER = 6

class experiment.selector.HookTimes

Bases: enum.Enum

Information used to infer at which point of the execution a hook takes place.

AFTER_AFFMAT = 5

AFTER_CLASSIFIER = 11

AFTER_FILTER = 8

AFTER_NOISE = 2

BEFORE_AFFMAT = 3

BEFORE_CLASSIFIER = 9

BEFORE_FILTER = 6

BEFORE_NOISE = 0

DURING_AFFMAT = 4

DURING_CLASSIFIER = 10

DURING_FILTER = 7

DURING_NOISE = 1

experiment.selector.select_affmat(**kwargs)

Delegates to the appropriate Affinity Matrix generator.

Currently, we simply forward it to gssl.graph.gssl_affmat.AffMatGenerator. More extensive documentation on the affinity matrix specification can be found there, as well.

Args: **kwargs: configuration of affinity matrix

Returns

The appropriately configured affinity matrix generator

Return type

(AffMatGenerator)

experiment.selector.select_and_add_hook(hook_list, mplex, experiment=None)

Obtains the relevant hook for the classifier/filter.

Each hook is associated with a filename prefix, which is partially determined by the prefix t#, where # is given according to the enum experiment.selector.HookTimes Currently available hooks:

• Hook.INIT_ALL: At HookTimes.BEFORE_NOISE, plots true labels of all instances

• Hook.INIT_LABELED: At HookTimes.BEFORE_NOISE, plots true labels of instances marked as ‘labeled’

• Hook.NOISE_AFTER: At HookTimes.AFTER_NOISE, plots corrupted labels

• Hook.W_INIT_ALL : At HookTimes.BEFORE_FILTER, plots true labels of all instances, with the affinity matrix.

• Hook.W_INIT_LABELED : At HookTimes.BEFORE_FILTER, plots true labels of instances marked as ‘labeled’, with the affinity matrix.

• Hook.W_NOISE_AFTER : At HookTimes.BEFORE_FILTER, plots corrupted labels, with the affinity matrix.

• Hook.FILTER_AFTER: At HookTimes.AFTER_FILTER, plots filtered labels

• Hook.FILTER_ITER: At HookTimes.DURING_FILTER, plots the filtered labels at each step.

• Hook.ALG_ITER: At HookTimes.DURING_CLASSIFIER, plots the classification at each step.

• Hook.ALG_RESULT: At HookTimes.AFTER_CLASSIFIER, plots classification of algorithm.

• Hook.T_NOISE: At HookTimes.AFTER_NOISE, adds time taken by the noise process to the output dictionary.

• Hook.T_AFFMAT: At HookTimes.AFTER_AFFMAT, adds time taken by the affinity matrix generation to the output dictionary.

• Hook.T_FILTER: At HookTimes.AFTER_FILTER, adds time taken by filtering process to the output dictionary.

• Hook.T_ALG: At HookTimes.AFTER_CLASSIFIER, adds time taken by the classification to the output dictionary.

Parameters

• hook_mode (List[int]) – the identifier for the hook

• mplex (Dict[Dict]]) – The multiplexed configs. See experiment.experiments.keys_multiplex()

• experiment (Experiment) – The experiment calling this function. Could be used for callbacks for certain hooks.

Returns

The appropriately configured hook

Return type

(Dict[GSSLHook])

experiment.selector.select_classifier(**kwargs)

Delegates to the appropriate classifier.

Currently available classifiers:

• LGC (Local and Global Consistency) : gssl.classifiers.LGC.LGCClassifier

• GTAM (Graph Transduction via Alternating Minimization) : gssl.classifiers.GTAM.GTAMClassifier

• GFHF (Gaussian Fields and Harmonic Functions ) : gssl.classifiers.GFHF.GFHFlassifier

Parameters

**kwargs – configuration of classifier

Returns

The appropriately configured classifier

Return type

(GSSLClassifier)

experiment.selector.select_filter(**kwargs)

Delegates to the appropriate filter.

Currently available classifiers:

• LDST (Label Diagnosis through self tuning): gssl.filters.LDST.LDST

Parameters

**kwargs – configuration of classifier

Returns

The appropriately configured filter

Return type

(GSSLFilter)

experiment.selector.select_input(**kwargs)

Gets the input dataset, according to some specification.

Currently, the following parameters are required:

• dataset : identifies the dataset. Currently, this may be

  1. The name of any of the toy datasets.

  2. sk_gaussian to use sklearn’s make_blob command at runtime. requires dataset_sd config to determine the dispersion.

  3. sk_spiral to use sklearn’s make_moons command at runtime. requires dataset_sd config to determine the dispersion.

• seed : Specifies the seed for reproducibility purposes.

• labeled_percent : Specifies the percentage of instances to be marked as ‘labeled’.

Args:

**kwargs: Key-value pairs with the configuration options of the input.

Returns:

(tuple): tuple containing:

1. (NDArray[float].shape[N,D]) : An input matrix, describing N instances of dimension D.

2. (NDArray[float].shape[N,C]) : A belief matrix corresponding to the clean labels. Every row is one-hot, marking down the correct label.

3. (NDArray[bool].shape[N]): A boolean array, indicating which instances are to be interpreted as labeled.

Raises:

KeyError: If one of the required keys is not found.

experiment.selector.select_noise(**kwargs)

Delegates to the appropriate noise process.

Currently, we simply forward it to input.noise.noise_process.LabelNoiseProcess. More extensive documentation on the label noise process specification can be found there, as well.

Args: **kwargs: configuration of affinity matrix

Returns

The appropriately configured label noise process.

Return type

(LabelNoiseProcess)

Module contents