Active Baum-Welch for MDP

class jajapy.Active_BW_MDP

Class for the Active Baum-Welch algorithm on MDP. This algorithm is described here: https://arxiv.org/pdf/2110.03014.pdf

fit(traces, sul, lr, nb_iterations: int, nb_sequences: int, sequence_length: Optional[int] = None, epsilon_greedy: float = 0.9, initial_model=None, nb_states: Optional[int] = None, random_initial_state: bool = True, output_file: Optional[str] = None, output_file_prism: Optional[str] = None, epsilon: float = 0.01, max_it: int = inf, pp: str = '', verbose: int = 2, return_data: bool = False, stormpy_output: bool = True, processes: Optional[int] = None)

Fits the model according to traces.

Parameters

tracesSet, list or ndarray.: The training set.
sulMDP: The system under learning. Used to generate new sequences.
lrfloat, or 0, or "dynamic": Learning rate. If lr=0 the current hypothesis will be updated on traces plus all the active samples. If lr is a float, the current hypothesis will be merged with an updated version of it. The updated version is a version of the current hypothesis fitted on the last active sample. If lr=”dynamic” the learning rate decrease at each iteration.
nb_iterations: int: Number of active learning iterations.
nb_sequences: int: Size of one active sample. I.e: number of sequences generated at each active learning iteration.
sequence_length: int, optional.: Size of one sequence generated during an active learning iteration. If not set it will be equal to the size of the first sequence in traces.
epsilon_greedy: float, optional.: It will use the active learning scheduler with probability epsilon_greedy while genereting the active learning sample, and with probability 1-epsilon_greedy it will use an uniform scheduler. By default 0.9
initial_modelModel or stormpy.sparse model, optional.: The first hypothesis. If not set it will create a random Model with nb_states states.
nb_states: int, optional.: If initial_model is not set it will create a random Model with nb_states states. Must be set if initial_model is not set.
random_initial_state: bool, optional.: If initial_model is not set it will create a random Model with random initial state according to this sequence of probabilities. Must be set if initial_model is not set. Default is True.
output_filestr, optional: If set, the output model will be saved at this location. Otherwise the output model will not be saved.
output_file_prismstr, optional: If set, the output model will be saved in a prism file at this location. Otherwise the output model will not be saved. This parameter is ignored if the model under learning is a HMM or a GoHMM.
epsilonfloat, optional: The learning process stops when the difference between the loglikelihood of the training set under the two last hypothesis is lower than epsilon. The lower this value the better the output, but the longer the running time. Default is 0.01.
max_it: int, optional: Maximal number of iterations. The algorithm will stop after max_it iterations. Default is infinity.
pp: str, optional: Will be printed at each iteration. Default is an empty string.
verbose: int, optional.: Define the level of information that will print during the learning 0 - nothing (no warnings, no progress bar, no recap at the end) 1 - minimal (warnings only) 2 - default (warnings and progress bar, no recap at the end) 3 - maximal (warnings, progress bar and recap) Default is 2.
return_data: bool, optional.: If set to True, a dictionary containing following values will be returned alongside the output model once the learning is done. ‘learning_rounds’, ‘learning_time’, ‘training_set_loglikelihood’. Default is False.
stormpy_output: bool, optional.: If set to True the output model will be a Stormpy sparse model. Doesn’t work for HMM and GOHMM. Default is True.
processesint, optional: Number of processes used during the learning. Only for linux: for Windows and Mac OS it is 1. Default is cpu_count()-1.

Returns

MDP: fitted MDP.