pyopmnearwell.ml.nn module#

Transform ensemble data into datasets and train neural networks.

type ArrayLike = Tensor | ndarray#
get_FCNN(ninputs, noutputs, depth=5, hidden_dim=10, saved_model=None, activation='sigmoid', kernel_initializer='glorot_normal', normalization=False)[source]#

Return a fully connected neural network with the specified architecture.

Parameters:

  • ninputs (int) -- Number of inputs to the model.

  • noutputs (int) -- Number of outputs from the model.

  • depth (int, optional) -- Number of hidden layers in the model. Defaults to 5.

  • hidden_dim (int, optional) -- Number of neurons in each hidden layer. Defaults to 10.

  • saved_model (str, optional) -- Path to a saved model to load weights from. Defaults to None.

  • activation (Literal["sigmoid", "relu", "tanh"], optional) -- Activation function to use in the hidden layers. Defaults to "sigmoid".

  • kernel_initializer (Literal["glorot_normal", "glorot_uniform"], optional) -- Weight initialization method to use in the hidden layers. Defaults to "glorot_normal".

  • normalization (bool, optional) -- Whether to use batch normalization in the model. Defaults to False.

Returns:

A fully connected neural network.

Return type:

keras.Model

get_RNN(ninputs, noutputs, units=20, saved_model=None, activation='tanh', kernel_initializer='glorot_uniform')[source]#

Return a recurrent neural network with the specified architecture.

Parameters:

  • ninputs (int) -- Number of inputs to the model.

  • noutputs (int) -- Number of outputs from the model.

  • units (int, optional) -- Size of internal model state. Defaults to 20.

  • hidden_dim (int, optional) -- Number of neurons in each hidden layer. Defaults to 10.

  • saved_model (str, optional) -- Path to a saved model to load weights from. Defaults to None.

  • activation (Literal["sigmoid", "relu", "tanh"], optional) -- Activation function to use in the hidden layers. Defaults to "sigmoid".

  • kernel_initializer (Literal["glorot_normal", "glorot_uniform"], optional) -- Weight initialization method to use in the hidden layers. Defaults to "glorot_normal".

Returns:

A fully connected neural network.

Return type:

keras.Model

get_GRU(ninputs, noutputs, units=20, saved_model=None, activation='tanh', kernel_initializer='glorot_uniform')[source]#

Return a recurrent neural network with the specified architecture.

Parameters:

  • ninputs (int) -- Number of inputs to the model.

  • noutputs (int) -- Number of outputs from the model.

  • units (int, optional) -- Size of internal model state. Defaults to 20.

  • hidden_dim (int, optional) -- Number of neurons in each hidden layer. Defaults to 10.

  • saved_model (str, optional) -- Path to a saved model to load weights from. Defaults to None.

  • activation (Literal["sigmoid", "relu", "tanh"], optional) -- Activation function to use in the hidden layers. Defaults to "sigmoid".

  • kernel_initializer (Literal["glorot_normal", "glorot_uniform"], optional) -- Weight initialization method to use in the hidden layers. Defaults to "glorot_normal".

Returns:

A fully connected neural network.

Return type:

keras.Model

get_LSTM(ninputs, noutputs, units=20, saved_model=None, activation='tanh', kernel_initializer='glorot_uniform')[source]#

Return a recurrent neural network with the specified architecture.

Parameters:

  • ninputs (int) -- Number of inputs to the model.

  • noutputs (int) -- Number of outputs from the model.

  • units (int, optional) -- Size of internal model state. Defaults to 20.

  • hidden_dim (int, optional) -- Number of neurons in each hidden layer. Defaults to 10.

  • saved_model (str, optional) -- Path to a saved model to load weights from. Defaults to None.

  • activation (Literal["sigmoid", "relu", "tanh"], optional) -- Activation function to use in the hidden layers. Defaults to "sigmoid".

  • kernel_initializer (Literal["glorot_normal", "glorot_uniform"], optional) -- Weight initialization method to use in the hidden layers. Defaults to "glorot_normal".

Returns:

A fully connected neural network.

Return type:

keras.Model

scale_and_prepare_dataset(dsfile, feature_names, savepath, train_split=0.9, val_split=0.1, test_split=None, shuffle='first', feature_range=(-1, 1), target_range=(-1, 1), scale=True, **kwargs)[source]#

Scale, shuffle and split a dataset.

Parameters:

  • dsfile (str | pathlib.Path) -- Dataset file.

  • feature_names (list[str]) -- List of feature names.

  • savepath (pathlib.Path) -- Savepath for the scaling values.

  • train_split (float, optional) -- Train split. Defaults to 0.9.

  • val_split (float, optional) -- Val split. Defaults to 0.1.

  • test_split (float, optional) -- Test split. Defaults to None.

  • shuffle (Literal["first", "last", "false"], optional) --

    Options for shuffling the dataset:

    • "first": The dataset gets shuffled before the split.

    • "last": The dataset gets shuffled after the split.

    • "false": The dataset does not get shuffled.

    Defaults to "first".

  • feature_range (tuple[float, float], optional) -- Target range of feature scaling. Defaults to (-1, 1).

  • target_range (tuple[float, float], optional) -- Target range of target scaling. Defaults to (-1, 1)

  • scale (bool, optional) -- Whether to scale the dataset. Defaults to True.

Returns:

tuple[tuple[np.ndarray, np.ndarray], tuple[np.ndarray, np.ndarray]]
tuple[ tuple[np.ndarray, np.ndarray], tuple[np.ndarray, np.ndarray], tuple[np.ndarray, np.ndarray],
]: Tuple of scaled and split dataset. Includes test set only if

test_split > 0.

Return type:

tuple[tuple[ndarray, ndarray], tuple[ndarray, ndarray]] | tuple[tuple[ndarray, ndarray], tuple[ndarray, ndarray], tuple[ndarray, ndarray]]

train(model, train_data, val_data, savepath, lr=0.1, epochs=500, bs=64, patience=100, lr_patience=10, kerasify=True, loss_func='mse', recompile_model=True, **kwargs)[source]#

Train a tensorflow model on the provided training data and save the best model.

Parameters:

  • model (tf.Module) -- Model to be trained.

  • train_data (tuple[ArrayLike, ArrayLike]) -- Training features and targets.

  • val_data (tuple[ArrayLike, ArrayLike]) -- Validation features and targets.

  • savepath (pathlib.Path) -- Savepath for models and logging.

  • lr (float, optional) -- Initial learning rate. Defaults to 0.1.

  • epochs (_type_, optional) -- Training epochs. Defaults to 500.

  • bs (int, optional) -- Batch size. Defaults to 64.

  • patience (int, optional) -- Number of epochs without improvement before early stopping. Defaults to 100.

  • lr_patience (int, optional) -- Number of epochs without improvement before lr decay. Defaults to 10.

  • kerasify (bool, optional) -- Export the best model with kerasify after training. Defaults to True.

  • loss_func (Literal["mse", "MeanAbsolutePercentageError", "MeanSquaredLogarithmicError"], optional) -- Loss function. Defaults to "mse".

  • recompile_model (bool, optional) -- Whether to recompile the model before training. Can e.g., be set to false, if the model is built and compiled by a different function. Defaults to True.

  • **kwargs -- Get passed to the model.fit() method.

Return type:

None

build_model(hp, ninputs, noutputs, lr_tune=0.1)[source]#

Build and compile a FCNN with the given hyperparameters.

Parameters:

  • hp (keras_tuner.Hyperparameters) -- Hyperparameters object.

  • ninputs (int) -- Number of inputs.

  • noutputs (int) -- Number of outputs.

  • lr_tune (float)

Returns:

The built neural network model.

Return type:

tf.Module

tune(ninputs, noutputs, train_data, val_data, savepath, objective='val_loss', max_trials=5, executions_per_trial=1, sample_weight=None, lr_tune=0.1, **kwargs)[source]#

Tune the hyperparameters of a neural network model using random search.

Parameters:

  • ninputs (int) -- Number of input features to the model.

  • noutputs (int) -- Number of output features to the model.

  • train_data (tuple[ArrayLike, ArrayLike]) -- Tuple of training input and target data.

  • val_data (tuple[ArrayLike, ArrayLike],) -- Tuple of validation input and target data.

  • objective (Literal["loss", "val_loss"], optional) -- Objective for search. Defaults to "val_loss".

  • max_trials (int) -- Default is 5.

  • executions_per_trial (int) -- Default is 1.

  • sample_weight (ArrayLike | None) -- (ArrayLike): Default is np.array([1.0]).

  • **kwargs -- Get passed to the tuner's search method.

  • savepath (str | Path)

  • lr_tune (float)

Returns:

The model compiled with the best hyperparameters. keras_tuner.Tuner: The tuner.

Return type:

tf.Module

Raises:

ValueError -- If train_data or val_data is not a tuple of two tensors.

save_tune_results(tuner, savepath)[source]#

Save hyperparameter-tuning results as a CSV file.

Parameters:

  • tuner (keras_tuner.Tuner) -- Completed tuner whose trials are written in score order.

  • savepath (str | pathlib.Path) -- Directory in which tuner_results.csv is written.

Return type:

None

scale_and_evaluate(model, model_input, scalingsfile)[source]#

Scale the input, evaluate with the model and scale the output.

Parameters:

  • model (tf.keras.Model) -- A Keras model to evaluate the input with.

  • model_input (ArrayLike) -- Input tensor. Can be a batch.

  • scalingsfile (str | pathlib.Path) -- The path to the CSV file containing the scaling parameters for MinMaxScaling.

Returns:

The model's output, scaled back to the original range.

Return type:

tf.Tensor

Raises:
handle_zeros_in_scale(scale)[source]#

Set scales of near constant features to 1.

Note: This behavior is in line with sklearn.preprocessing.MinMaxScaler.

Parameters:

scale (ArrayLike) -- The scale array.

Returns:

The modified scale array.

Return type:

np.ndarray