Class FlexibleNN

class HeteroSymNN.Core.Nets.neural_nets.FlexibleNN(nodes_structure: list[int], activation_config: list[str | NodeConfig], initial_values: list[LayerValues] | None = None, initializer: Initializer | None = None, learning_rate: float = 0.001, learning_mode: str = 'Static', training_mode: Literal['batch', 'mini-batch', 'stochastic'] = 'stochastic', batch_size: int = 32, loss_function: Loss | None = None, optimizer: Optimizer | None = None, num_treaning_iter: int = 1000)[source]

Bases: ConfigurableNN

Intermediate class for creating dense neural networks with flexible activation configurations. Child class of ConfigurableNN.

Parameters:

nodes_structure (list[int]) – List containing the number of nodes in each layer including input and output layers.
activation_config (list[FlexibleNodeConfig]) – List containing the activation configuration for each layer. Each element can be a string (activation name) or a tuple (activation name, parameters dictionary).
initial_values (Optional[list[LayerValues]], optional) – List of initial values for weights and biases for each layer. If not provided, weights and biases will be initialized using the specified initializer., by default None
initializer (Optional[Initializer], optional) – Initializer to use for initializing weights and biases. If not provided, HeNormal will be used., by default None
learning_rate (float, optional) – Learning rate for the network., by default 0.001
learning_mode (str, optional) – Learning mode of the network. Currently only “Static” is supported., by default “Static”
training_mode (Literal["batch", "mini-batch", "stochastic"], optional) – Training mode to use during training. In case of “batch” or “stochastic” the batch size attribute will be ignored., by default “stochastic”
batch_size (int, optional) – Batch size to use during training. In the case of using “stochastic” or “batch” training mode this attribute will be ignored and in training time the batch size will be set to 1 or to the full dataset size respectively., by default 32
loss_function (Loss, optional) – Loss function to use for training. If not provided, MSELoss will be used., by default None
optimizer (Optimizer, optional) – Optimizer to use for training. If not provided, AdamOptimizer will be used., by default None
num_treaning_iter (int, optional) – Number of training iterations (epochs)., by default 1000

num_treaning_iterations

Number of training iterations (epochs) for the network.

Type:: int, read-write

learning_mode

Learning mode of the network. Currently only “Static” is supported.

Type:: str, read-write

training_mode

Training mode to use during training. When seting it to “mini-batch” from “stochastic” or “batch” the batch size that will be used is the one stored in the attribute batch_size.

Type:: Literal[“batch”, “mini-batch”, “stochastic”], read-write

batch_size

Batch size to use during training.

Type:: int, read-write

histogram_losses

List of loss values recorded at each epoch during training.

Type:: list[float], read-only

num_complited_train_iterations

Number of completed training steps.

Type:: int, read-only

num_completed_epochs

Number of completed training epochs.

Type:: int, read-only

Examples

>>> from HeteroSymNN.Core.Nets.neural_nets import FlexibleNN
>>> FNN = FlexibleNN(
...     nodes_structure=[3, 5, 2],
...     activation_config=[
...         "relu",
...         ("sigmoid", {})
...     ],
...     learning_rate=0.01,
...     batch_size=16,
...     training_mode="mini-batch"
... )

property BATCH_SIZE: int

Property to get the current batch size being used by the network in case of mini-batch training. Read-only.

Return type:: int

property COMPUTATIONAL_METHOD: Literal['GPU_CUDA', 'CPU_JIT', 'CPU_PYTHON']

Property to get the current computational method being used by the network. Read-only.

To change the computational method use the _change_COMPUTATIONAL_METHOD method.

Return type:: Literal[“GPU_CUDA”,”CPU_JIT”,”CPU_PYTHON”]

property CURRENT_DEVICE: Literal['CPU', 'GPU']

Property to get the current device where the network parameters are located. Read-only.

For changing location of the network parameters use the change_device method.

Return type:: Literal[“CPU”,”GPU”]

property GPU_ID: int

Property to get the current GPU ID being used by the network. Read-only.

For setting a new GPU ID, use the set_gpu_id method.

Return type:: int

property INICIALIZER: Initializer

Property to get the current initializer being used by the network. Read-only.

Return type:: Initializer

property LAYERS: list[Layer]

Property to get the layers of the network. Read-only.

Return type:: list[Layer]

property LOSS_FUNCTION: Loss

Property to get the current loss function being used by the network. Read-only.

Return type:: Loss

property NODE_STRUCTURE: list[int]

Property to get the number of nodes per layer of the network. Read-only.

Return type:: list[int]

property OPTIMIZER: Optimizer

Property to get the current optimizer being used by the network. Read-only.

Return type:: Optimizer

backward(error_values: BackendArray) → BackendArray

Internal method to perform a backward pass through the network.

Parameters:: error_values (BackendArray) – Error values to propagate back through the network.
Returns:: Error values propagated back to the input layer.
Return type:: BackendArray

change_constants(new_constants: dict[int, list[ConstantToUpdate] | ConstantToUpdate]) → None

Change the constants in the activation functions of the network layers.

Parameters:: new_constants (dict[int,Union[list[ConstantToUpdate], ConstantToUpdate]]) – Dictionary mapping layer indices to new constant values for the activation functions.

change_device(device: Literal['CPU', 'GPU']) → None

Change location of the network parameters to the specified device.

Parameters:: device (Literal["CPU","GPU"]) – Device to move the network parameters to.

get_config() → dict[str, Any]

Get the configuration of the neural network.

Returns:

Dictionary containing the configuration of the network.

Parameters include:

”nodes_structure” (list[int]): List of number of nodes per layer.
”detailed_activations” (list[list[NodeConfig]): Activation configuration for each node.
”learning_rate” (float): Learning rate of the network.
”learning_mode” (str): Learning mode of the network.
”training_mode” (str): Training mode (“batch”, “mini-batch”, “stochastic”).
”batch_size” (int): Batch size used during training.
”initializer_config” (dict[str, Any]): Configuration of the initializer.
”optimizer_config” (dict[str, Any]): Configuration of the optimizer.
”loss_config” (dict[str, Any]): Configuration of the loss function.
”num_treaning_iterations” (int): Number of training iterations (epochs).

Return type:

dict[str,Any]

get_parameters() → dict[str | int, dict[str, numpy.ndarray]]

Get the parameters (weights and biases) of the network.

Returns:: Dictionary containing the parameters of each layer.
Return type:: dict[Union[str,int],dict[str,np.ndarray]]

property learning_rate: float

Property to get the current learning rate of the network. Read-write.

Return type:: float

predict(input_values: list | list[list], to_cpu: bool = True) → numpy.ndarray | BackendArray

Make predictions using the neural network.

Parameters:

input_values (list or list[list]) – Input values for making predictions. In case of multiple samples, shape should be (num_samples, num_features) or (num_features, num_samples).
to_cpu (bool, optional) – Whether to return the predictions as a NumPy array on the CPU. If False, returns in the current backend array format., by default True

Returns:

Predicted output values.

Return type:

np.ndarray or BackendArray

set_gpu_id(new_id: int) → None

Method to set a new GPU ID for the network. This will change the device of the network parameters if currently on GPU.

Parameters:: new_id (int) – New GPU ID to set.
Raises:: ValueError – If the Id for the new GPU is greater than the number of available GPUs.

set_parameters(params: dict[str | int, dict[str, numpy.ndarray]]) → None

Set the parameters (weights and biases) of the network.

Parameters:: params (dict[Union[str,int],dict[str,np.ndarray]]) – Dictionary containing the parameters for each layer.

train(training_inputs: list[list[float]], training_targets: list[list[float]], num_iterations=None, training_mode: Literal['batch', 'mini-batch', 'stochastic'] | None = None, batch_size: int | None = None) → list[float]

Train the neural network using the provided training data.

Parameters:

training_inputs (list[list[float]]) – List of input samples for training. Shape should be (num_samples, num_features).
training_targets (list[list[float]]) – List of target output samples for training. Shape should be (num_samples, num_outputs).
num_iterations (int, optional) – Number of training iterations (epochs) to perform. If not provided, uses the value of the attribute num_treaning_iterations., by default None
training_mode (Literal["batch", "mini-batch", "stochastic"], optional) – Training mode to use during training. Options are “batch”, “mini-batch”, and “stochastic”. If not provided, uses the current value of the attribute training_mode., by default None
batch_size (int, optional) – Batch size to use during training if training_mode is “mini-batch”. If not provided, uses the current value of the attribute batch_size., by default None

Returns:

List of loss values recorded at each epoch during training.

Return type:

list[float]

train_step(x_input: BackendArray, y_target: BackendArray) → float

Perform a single training step (forward pass, loss computation, backward pass, and parameter update).

Parameters:

x_input (BackendArray) – Input values to train the network on.
y_target (BackendArray) – Target output values for the network.

Returns:

Computed loss for the training step.

Return type:

float

update_params(inputs: BackendArray) → None

Update the network parameters using the optimizer.

Parameters:: inputs (BackendArray) – Input values used for the parameter update.