This software provide the possibility of computing the first and second derivatives with respect to the input, as well as the gradient with respect of all the parameters of an arbitrary deep feed-forward neural network.
In the header file `network.h' can be found the supported network structures as well as all the relevant functions, which are briefly summarized here.
Structures
multilayerD network structure for second order derivatives computations
multilayerD1 network structure for first order derivatives computations
multilayer network structure for just parameter gradients computations
multiL_act prototype for activation functions
Functions
We outline the functions for the multilayerD struct, being the functions for the other structures similar (mutatis mutandis)
The architecture of the network is provided as an array of integers, where each entries set the number of nodes in the specific layer.
multilD_getNpar assigned an architecture return the total number of network parameters, inputs: (number of layers, architecture array)
multilD_init_net initialize the network, returns a copy of the network, inputs: (number of layers, architecture array)
multilD_setMode set the mode to full Hessian calculation or diagonal Hessian calculation, inputs: (pointer to the network, mode -> true = diagonal Hessian)
multilD_set_act set all the activatin functions of the hidden layers to the supplied one, inputs: (pointer to the network, ptr to activation function, ptr to derivative of a.f., ptr to double der. of a.f., ptr to triple der of a.f.)
multilD_set_act_one set one a.f. to the supplied one, inputs: (ptr to the network, int indicating which layer has to be modified, ptr to activation function, ptr to derivative of a.f., ptr to double der. of a.f., ptr to triple der of a.f.)
multilD_(save/load)_net save or load, respectively, the network, inputs: (ptr to the network, file name)
multilD_free_net free the memory associated to the network, input: (ptr to the network)
multilD_Evaluate evaluate the output of the network, the input of the netwrok must be supplied separately, inputs: (ptr to the network)
multilD_EvaluateParGradient evaluate gradient w.r.t. the parameters of the network, the input of the netwrok must be supplied separately, inputs: (ptr to the network)
multilD_FullEvaluate evaluate the output of the network and the gradient w.r.t. the parameters of the network, inputs: (ptr to the network, input of the network); this function should be preferred over the previous two
multiD_get_grad get the partial derivative of the output w.r.t. a parameter, inputs: (index of output, index of the chosen parameter, ptr to the netwrok)
multiD_get_grad_d get partial derivative w.r.t. the input of the partial derivative of the output w.r.t. a parameter, inputs: (index of output, index of the input, index of the chosen parameter, ptr to the netwrok)
multiD_get_grad_d2 get double partial derivative w.r.t. the input of the partial derivative of the output w.r.t. a parameter, inputs: (index of output, index of the first input, indx of the second input, index of the chosen parameter, ptr to the netwrok)
multiD_get get the output of the network, inputs: (index of output, ptr to the netwrok)
multiD_get_d get the partial derivative of the output of the network w.r.t. the input, inputs: (index of output, index of the input, ptr to the netwrok)
multiD_get_d2 get the double partial derivative of the output of the network w.r.t. the input, inputs: (index of output, first index of the input, second index of the input, ptr to the netwrok)
Usage
In the parent directory a make file is provided.
Run make to compile both the main files generic_pot_D1.c and generic_pot_D2.c that are in the runs directory.
Run make d*der where *=1,2 to compile either generic_pot_D1.c or generic_pot_D2.c respectivley.