Class ConfigurableNN

class HeteroSymNN.Core.Nets.neural_nets.ConfigurableNN(nodes_structure: list[int], detailed_activations: list[list[NodeConfig]], initial_values: list[LayerValues] | None = None, initializer: Initializer | None = None, learning_rate: float = 0.001, batch_size: int = 32, training_mode: Literal['batch', 'mini-batch', 'stochastic'] = 'mini-batch', learning_mode: str = 'Static', loss_function: Loss | None = None, optimizer: Optimizer | None = None, num_treaning_iter: int = 1000)[source]

Bases: object

Base class for creating a dense neural network with customizable architecture, activation functions, and training parameters.

Parameters:

nodes_structure (list[int]) – List with the number of nodes per layer including input and output layers.
detailed_activations (list[list[NodeConfig]]) – List of lists containing the activation configuration for each node in each layer.
initial_values (Optional[list[LayerValues]], optional) – Optional list of initial values for each layer. If not provided, weights and biases will be initialized using the specified initializer., by default None
initializer (Optional[Initializer], optional) – Initializer to use for weights and biases if initial_values is not provided. Most pass an instance of Initializer and the default uses HeNormal, value by default is None.
learning_rate (float, optional) – Learning rate for the network. In the case that a custom optimizer is provided with its own learning rate this value will be overwritten., by default 0.001
batch_size (int, optional) – Batch size to use during training, by default 32 if training_mode is “mini-batch”, 1 if “stochastic” and size of the dataset if “batch”.
training_mode (Literal["batch", "mini-batch", "stochastic"], optional) – Training mode to use during training. Options are “batch”, “mini-batch”, and “stochastic”. By default “mini-batch”.
learning_mode (str, optional) – Learning mode of the network. Currently only “Static” is supported., by default “Static”
loss_function (Loss, optional) – Loss function to use during training. Must be an instance of Loss. If not provided, :MSELoss will be used., value by default is None.
optimizer (Optional[Optimizer], optional) – Optimizer to use for updating the network parameters. Must be an instance of Optimizer. If not provided, AdamOptimizer will be used.,value by default is None.
num_treaning_iter (int, optional) – Number of Epochs to use during training, by default 1000

num_treaning_iterations

Number of training iterations (epochs) for the network.

Type:: int, read-write

learning_mode

Learning mode of the network. Currently only “Static” is supported.

Type:: str, read-write

training_mode

Training mode to use during training. When seting it to “mini-batch” from “stochastic” or “batch” the batch size that will be used is the one stored in the attribute batch_size.

Type:: Literal[“batch”, “mini-batch”, “stochastic”], read-write

batch_size

Batch size to use during training.

Type:: int, read-write

histogram_losses

List of loss values recorded at each epoch during training.

Type:: list[float], read-only

num_complited_train_iterations

Number of completed training steps.

Type:: int, read-only

num_completed_epochs

Number of completed training epochs.

Type:: int, read-only

Examples

>>> from HeteroSymNN.Core.Nets.neural_nets import ConfigurableNN
>>> CNN = ConfigurableNN(
...     nodes_structure=[3, 5, 2],
...     detailed_activations=[
...         [("relu", {}), ("relu", {}), ("relu", {}), ("relu", {}), ("relu", {})],
...         [("sigmoid", {}), ("sigmoid", {})]
...     ],
...     learning_rate=0.01,
...     batch_size=16,
...     training_mode="mini-batch"
... )

>>> from HeteroSymNN.Core.Nets import losses as lossC, optimizers as OptiC, initializers as InitC
>>> custom_loss = lossC.CrossEntropyLoss()
>>> custom_optimizer = OptiC.SGDOptimizer(learning_rate=0.01)
>>> custom_initializer = InitC.XavierUniform()
>>> CNN = ConfigurableNN(
...     nodes_structure=[4, 6, 3],
...     detailed_activations=[
...         [("tanh", {}), ("tanh", {}), ("tanh", {}), ("tanh", {}), ("tanh", {}), ("tanh", {})],
...         [("softmax", {}), ("softmax", {}), ("softmax", {})]
...     ],
...     initializer=custom_initializer,
...     loss_function=custom_loss,
...     optimizer=custom_optimizer,
...     training_mode="batch"
... )

property BATCH_SIZE: int

Property to get the current batch size being used by the network in case of mini-batch training. Read-only.

Return type:: int

property COMPUTATIONAL_METHOD: Literal['GPU_CUDA', 'CPU_JIT', 'CPU_PYTHON']

Property to get the current computational method being used by the network. Read-only.

To change the computational method use the _change_COMPUTATIONAL_METHOD method.

Return type:: Literal[“GPU_CUDA”,”CPU_JIT”,”CPU_PYTHON”]

property CURRENT_DEVICE: Literal['CPU', 'GPU']

Property to get the current device where the network parameters are located. Read-only.

For changing location of the network parameters use the change_device method.

Return type:: Literal[“CPU”,”GPU”]

property GPU_ID: int

Property to get the current GPU ID being used by the network. Read-only.

For setting a new GPU ID, use the set_gpu_id method.

Return type:: int

property INICIALIZER: Initializer

Property to get the current initializer being used by the network. Read-only.

Return type:: Initializer

property LAYERS: list[Layer]

Property to get the layers of the network. Read-only.

Return type:: list[Layer]

property LOSS_FUNCTION: Loss

Property to get the current loss function being used by the network. Read-only.

Return type:: Loss

property NODE_STRUCTURE: list[int]

Property to get the number of nodes per layer of the network. Read-only.

Return type:: list[int]

property OPTIMIZER: Optimizer

Property to get the current optimizer being used by the network. Read-only.

Return type:: Optimizer

backward(error_values: BackendArray) → BackendArray[source]

Internal method to perform a backward pass through the network.

Parameters:: error_values (BackendArray) – Error values to propagate back through the network.
Returns:: Error values propagated back to the input layer.
Return type:: BackendArray

change_constants(new_constants: dict[int, list[ConstantToUpdate] | ConstantToUpdate]) → None[source]

Change the constants in the activation functions of the network layers.

Parameters:: new_constants (dict[int,Union[list[ConstantToUpdate], ConstantToUpdate]]) – Dictionary mapping layer indices to new constant values for the activation functions.

change_device(device: Literal['CPU', 'GPU']) → None[source]

Change location of the network parameters to the specified device.

Parameters:: device (Literal["CPU","GPU"]) – Device to move the network parameters to.

get_config() → dict[str, Any][source]

Get the configuration of the neural network.

Returns:

Dictionary containing the configuration of the network.

Parameters include:

”nodes_structure” (list[int]): List of number of nodes per layer.
”detailed_activations” (list[list[NodeConfig]): Activation configuration for each node.
”learning_rate” (float): Learning rate of the network.
”learning_mode” (str): Learning mode of the network.
”training_mode” (str): Training mode (“batch”, “mini-batch”, “stochastic”).
”batch_size” (int): Batch size used during training.
”initializer_config” (dict[str, Any]): Configuration of the initializer.
”optimizer_config” (dict[str, Any]): Configuration of the optimizer.
”loss_config” (dict[str, Any]): Configuration of the loss function.
”num_treaning_iterations” (int): Number of training iterations (epochs).

Return type:

dict[str,Any]

get_parameters() → dict[str | int, dict[str, numpy.ndarray]][source]

Get the parameters (weights and biases) of the network.

Returns:: Dictionary containing the parameters of each layer.
Return type:: dict[Union[str,int],dict[str,np.ndarray]]

property learning_rate: float

Property to get the current learning rate of the network. Read-write.

Return type:: float

predict(input_values: list | list[list], to_cpu: bool = True) → numpy.ndarray | BackendArray[source]

Make predictions using the neural network.

Parameters:

input_values (list or list[list]) – Input values for making predictions. In case of multiple samples, shape should be (num_samples, num_features) or (num_features, num_samples).
to_cpu (bool, optional) – Whether to return the predictions as a NumPy array on the CPU. If False, returns in the current backend array format., by default True

Returns:

Predicted output values.

Return type:

np.ndarray or BackendArray

set_gpu_id(new_id: int) → None[source]

Method to set a new GPU ID for the network. This will change the device of the network parameters if currently on GPU.

Parameters:: new_id (int) – New GPU ID to set.
Raises:: ValueError – If the Id for the new GPU is greater than the number of available GPUs.

set_parameters(params: dict[str | int, dict[str, numpy.ndarray]]) → None[source]

Set the parameters (weights and biases) of the network.

Parameters:: params (dict[Union[str,int],dict[str,np.ndarray]]) – Dictionary containing the parameters for each layer.

train(training_inputs: list[list[float]], training_targets: list[list[float]], num_iterations=None, training_mode: Literal['batch', 'mini-batch', 'stochastic'] | None = None, batch_size: int | None = None) → list[float][source]

Train the neural network using the provided training data.

Parameters:

training_inputs (list[list[float]]) – List of input samples for training. Shape should be (num_samples, num_features).
training_targets (list[list[float]]) – List of target output samples for training. Shape should be (num_samples, num_outputs).
num_iterations (int, optional) – Number of training iterations (epochs) to perform. If not provided, uses the value of the attribute num_treaning_iterations., by default None
training_mode (Literal["batch", "mini-batch", "stochastic"], optional) – Training mode to use during training. Options are “batch”, “mini-batch”, and “stochastic”. If not provided, uses the current value of the attribute training_mode., by default None
batch_size (int, optional) – Batch size to use during training if training_mode is “mini-batch”. If not provided, uses the current value of the attribute batch_size., by default None

Returns:

List of loss values recorded at each epoch during training.

Return type:

list[float]

train_step(x_input: BackendArray, y_target: BackendArray) → float[source]

Perform a single training step (forward pass, loss computation, backward pass, and parameter update).

Parameters:

x_input (BackendArray) – Input values to train the network on.
y_target (BackendArray) – Target output values for the network.

Returns:

Computed loss for the training step.

Return type:

float

update_params(inputs: BackendArray) → None[source]

Update the network parameters using the optimizer.

Parameters:: inputs (BackendArray) – Input values used for the parameter update.