How to Reproduce the Results

Before the reproduction of the results in out benchmark, please follow the instructions in About the Dataset to download the datasets, and store them correctly under every subfolder. Then the structural inference methods in the benchmark and their results can be reproduced with following steps.

Methods based on Classical Statistics

ppcor

Requirements

To configure the environment, you can create a conda environment and install the environment.yml by:

$> conda env create -f environment.yml --name ppcor

Our environment included:

  • r-base=4.1.3

  • r-matrix=1.5_3

  • r-optparse=1.7.3

  • r-ppcor=1.1

  • r-reticulate=1.28

  • r-stringi=1.7.12

  • r-stringr=1.5.0

Reproduction Examples

Reproduce the results of ppcor in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 15 nodes, with the first repetition number:

$> Rscript run.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims15r1.npy" &

Reproduce the results of ppcor in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 30 nodes, with the second repetition number:

$> Rscript run.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" &

Reproduce the results of ppcor in the noisy trajectories generated by NetSims simulation, and by Brain Networks with 50 nodes, with the third repetition number, with two levels of added Gaussian noise:

$> Rscript run.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims50r3_n2.npy" &

Reproduce the results of ppcor in the noise-free trajectories generated by NetSims simulation, by Brain Networks with 30 nodes, with the second repetition number, and with 5 time steps:

$> Rscript run.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" --b-time-steps 5 &

TIGRESS

Requirements

To configure the environment, you can create a conda environment and install the environment.yml by:

$> conda env create -f environment.yml --name TIGRESS

Our environment included:

  • r-base=4.1.3

  • r-doparallel=1.0.17

  • r-foreach=1.5.2

  • r-matrix=1.5_4

  • r-optparse=1.7.3

  • r-reticulate=1.28

  • r-stringr=1.5.0

Reproduction Examples

Reproduce the results of TIGRESS in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 15 nodes, with the first repetition number:

$> Rscript run.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims15r1.npy" &

Reproduce the results of TIGRESS in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 30 nodes, with the second repetition number:

$> Rscript run.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" &

Reproduce the results of TIGRESS in the noisy trajectories generated by NetSims simulation, and by Brain Networks with 50 nodes, with the third repetition number, with two levels of added Gaussian noise:

$> Rscript run.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims50r3_n2.npy" &

Reproduce the results of TIGRESS in the noise-free trajectories generated by NetSims simulation, by Brain Networks with 30 nodes, with the second repetition number, and with 5 time steps:

$> Rscript run.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" --b-time-steps 5 &

Methods based on Information Theory

ARACNe

Requirements

To configure the environment, you can create a conda environment and install the environment.yml by:

$> conda env create -f environment.yml --name ARACNE

Our environment included:

  • r-base=4.2.3

  • r-biocmanager=1.30.20

  • r-optparse=1.7.3

  • r-reticulate=1.26

  • r-stringi=1.7.12

  • r-stringr=1.5.0

  • bioconductor-minet

Reproduction Examples

Reproduce the results of ARACNe in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 15 nodes, with the first repetition number:

$> Rscript test.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims15r1.npy" &

Reproduce the results of ARACNe in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 30 nodes, with the second repetition number:

$> Rscript test.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" &

Reproduce the results of ARACNe in the noisy trajectories generated by NetSims simulation, and by Brain Networks with 50 nodes, with the third repetition number, with two levels of added Gaussian noise:

$> Rscript test.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims50r3_n2.npy" &

Reproduce the results of ARACNe in the noise-free trajectories generated by NetSims simulation, by Brain Networks with 30 nodes, with the second repetition number, and with 5 time steps:

$> Rscript test.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" --b-time-steps=5 &

CLR

Requirements

To configure the environment, you can create a conda environment and install the environment.yml by:

$> conda env create -f environment.yml --name CLR

Our environment included:

  • r-base=4.2.3

  • r-biocmanager=1.30.20

  • r-matrix=1.5_3

  • r-optparse=1.7.3

  • r-reticulate=1.26

  • r-stringi=1.7.12

  • r-stringr=1.5.0

  • bioconductor-minet

Reproduction Examples

Reproduce the results of CLR in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 15 nodes, with the first repetition number:

$> Rscript test.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims15r1.npy" &

Reproduce the results of CLR in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 30 nodes, with the second repetition number:

$> Rscript test.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" &

Reproduce the results of CLR in the noisy trajectories generated by NetSims simulation, and by Brain Networks with 50 nodes, with the third repetition number, with two levels of added Gaussian noise:

$> Rscript test.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims50r3_n2.npy" &

Reproduce the results of CLR in the noise-free trajectories generated by NetSims simulation, by Brain Networks with 30 nodes, with the second repetition number, and with 5 time steps:

$> Rscript test.R --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" --b-time-steps 5 &

PIDC

Requirements

To configure the environment, you have to install a Julia executable.

Our environment included:

  • ArgParse

  • CSV

  • DataFrames

  • NPZ

  • NetworkInference

After installing Julia, you have to install packages in our project by:

1. Install it in Julia interactive session.

julia> using Pkg
julia> Pkg.instantiate()

2. Alternatively, install it in Julia REPL mode. On the shell:

$> julia --project=./PIDC/

On the Julia REPL mode:

(PIDC) pkg> instantiate

Reproduction Examples

Reproduce the results of PIDC in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 15 nodes, with the first repetition number:

$> julia --project=./PIDC/ -- run.jl --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims15r1.npy" &

Reproduce the results of PIDC in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 30 nodes, with the second repetition number:

$> julia --project=./PIDC/ -- run.jl --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" &

Reproduce the results of PIDC in the noisy trajectories generated by NetSims simulation, and by Brain Networks with 50 nodes, with the third repetition number, with two levels of added Gaussian noise:

$> julia --project=./PIDC/ -- run.jl --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims50r3_n2.npy" &

Reproduce the results of PIDC in the noise-free trajectories generated by NetSims simulation, by Brain Networks with 30 nodes, with the second repetition number, and with 5 time steps:

$> julia --project=./PIDC/ -- run.jl --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" --b-time-steps=5 &

Scribe

Requirements

To configure the environment, you can create a conda environment and install the environment.yml by:

$> conda env create -f environment.yml --name scribe

Our environment included:

  • numpy=1.23.5

  • pandas=1.5.2

  • python=3.9.16

  • pip: - scikit-learn==1.2.1 - scipy==1.10.0 - tqdm==4.64.1

Reproduction Examples

Reproduce the results of scribe in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 15 nodes, with the first repetition number:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims15r1.npy" &

Reproduce the results of scribe in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 30 nodes, with the second repetition number:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" &

Reproduce the results of scribe in the noisy trajectories generated by NetSims simulation, and by Brain Networks with 50 nodes, with the third repetition number, with two levels of added Gaussian noise:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims50r3_n2.npy" &

Reproduce the results of scribe in the noise-free trajectories generated by NetSims simulation, by Brain Networks with 30 nodes, with the second repetition number, and with 5 time steps:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" --b-time-steps=5 &

Methods based on Tree Algorithms

dynGENIE3

Requirements

To configure the environment, you can create a conda environment and install the environment.yml by:

$> conda env create -f environment.yml --name dynGENIE3

Our environment included:

Our environment included:

  • numpy=1.23.5

  • pandas=1.5.2

  • python=3.10.9

  • scikit-learn=1.2.0

  • scipy=1.10.0

Reproduction Examples

Reproduce the results of dynGENIE3 in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 15 nodes, with the first repetition number:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims15r1.npy" &

Reproduce the results of dynGENIE3 in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 30 nodes, with the second repetition number:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" &

Reproduce the results of dynGENIE3 in the noisy trajectories generated by NetSims simulation, and by Brain Networks with 50 nodes, with the third repetition number, with two levels of added Gaussian noise:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims50r3_n2.npy" &

Reproduce the results of dynGENIE3 in the noise-free trajectories generated by NetSims simulation, by Brain Networks with 30 nodes, with the second repetition number, and with 5 time steps:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" --b-time-steps=5 &

XGBGRN

Requirements

To configure the environment, you can create a conda environment and install the environment.yml by:

$> conda env create -f environment.yml --name GRNs_nonlinear_ODEs

Our environment included:

Our environment included:

  • numpy=1.23.5

  • pandas=1.5.2

  • py-xgboost-cpu=1.7.3

  • python=3.10.9

  • scikit-learn=1.2.0

Reproduction Examples

Reproduce the results of XGBGRN in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 15 nodes, with the first repetition number:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims15r1.npy" &

Reproduce the results of XGBGRN in the noise-free trajectories generated by NetSims simulation, and by Brain Networks with 30 nodes, with the second repetition number:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" &

Reproduce the results of XGBGRN in the noisy trajectories generated by NetSims simulation, and by Brain Networks with 50 nodes, with the third repetition number, with two levels of added Gaussian noise:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims50r3_n2.npy" &

Reproduce the results of XGBGRN in the noise-free trajectories generated by NetSims simulation, by Brain Networks with 30 nodes, with the second repetition number, and with 5 time steps:

$> python run.py --save-folder="./results" --b-network-type="brain_networks" --b-directed --b-simulation-type="netsims" --b-suffix="test_netsims30r2.npy" --b-time-steps=5 &

Methods based on VAEs

NRI

Please install the required packages first.

Requirements

  • Python >= 3.8

  • Numpy >= 1.23.4

  • pandas >= 1.5.1

  • matplotlib >= 3.6.2

  • sklearn >= 0.0.post1

  • torch >= 1.13.1

  • torchinfo >= 1.7.2

  • tqdm >= 4.64.1

Arguments

  • b-network-type: name of the graph type (in full name)

  • b-directed: if called, will load data from directed graphs

  • b-simulation-type: springs or netsims

  • b-suffix: choose graph with node X, the Y repetition and with noise level K with format “XrY_nK”. If use noise-free, omit “_nK”

Reproduction Examples

Run NRI with “chemical reaction networks in atmosphere (CRNA)”, “directed”, “15 nodes”, “springs simulation”, “noise-free”, and “the first repetition” :

$> cd /src/models/NRI/
$> python3 train.py --b-network-type 'chemical_reaction_networks_in_atmosphere' --b-directed --b-simulation-type 'springs' --b-suffix '15r1'

Run NRI with “brain networks (BN)”, “directed”, “netsims simulation”, “30 nodes”, “noise-free”, and “the second repetition”:

$> cd /src/models/NRI/
$> python3 train.py --b-network-type 'brain_networks' --b-directed --b-simulation-type 'netsims' --b-suffix '30r2'

Run NRI with “landscape networks (LN)”, “directed”, “netsims simulation”, “50 nodes”, “the third repetition”, and “noise level 2”:

$> cd /src/models/NRI/
$> python3 train.py --b-network-type 'landscape_networks' --b-simulation-type 'netsims' --b-suffix '50r3_n2'

ACD

Please install the required packages first.

Requirements

  • Python >= 3.8

  • Numpy >= 1.23.4

  • pandas >= 1.5.1

  • scipy >= 1.9.3

  • sklearn >= 0.0.post1

  • torch >= 1.13.1

  • torchinfo >= 1.7.2

  • tqdm >= 4.64.1

Arguments

  • b-network-type: name of the graph type (in full name)

  • b-directed: if called, will load data from directed graphs

  • b-simulation-type: springs or netsims

  • b-suffix: choose graph with node X, the Y repetition and with noise level K with format “XrY_nK”. If use noise-free, omit “_nK”

Reproduction Examples

Run ACD with “chemical reaction networks in atmosphere (CRNA)”, “directed”, “15 nodes”, “springs simulation”, “noise-free”, and “the first repetition” :

$> cd /src/models/ACD/
$> python3 train.py --b-network-type 'chemical_reaction_networks_in_atmosphere' --b-directed --b-simulation-type 'springs' --b-suffix '15r1'

Run ACD with “brain networks (BN)”, “directed”, “netsims simulation”, “30 nodes”, “noise-free”, and “the second repetition”:

$> cd /src/models/ACD/
$> python3 train.py --b-network-type 'brain_networks' --b-directed --b-simulation-type 'netsims' --b-suffix '30r2'

Run ACD with “landscape networks (LN)”, “directed”, “netsims simulation”, “50 nodes”, “the third repetition”, and “noise level 2”:

$> cd /src/models/ACD/
$> python3 train.py --b-network-type 'landscape_networks' --b-simulation-type 'netsims' --b-suffix '50r3_n2'

MPM

Please install the required packages first.

Requirements

  • Python >= 3.8

  • Numpy >= 1.23.4

  • scipy >= 1.9.3

  • sklearn >= 0.0.post1

  • torch >= 1.13.1

  • torch-geometric >= 2.2.0

  • torchinfo >= 1.7.2

  • tqdm >= 4.64.1

Arguments

  • b-network-type: name of the graph type (in full name)

  • b-directed: if called, will load data from directed graphs

  • b-simulation-type: springs or netsims

  • b-suffix: choose graph with node X, the Y repetition and with noise level K with format “XrY_nK”. If use noise-free, omit “_nK”

Reproduction Examples

Run ACD with “chemical reaction networks in atmosphere (CRNA)”, “directed”, “15 nodes”, “springs simulation”, “noise-free”, and “the first repetition” :

$> cd /src/models/MPM/
$> python3 run.py --b-network-type 'chemical_reaction_networks_in_atmosphere' --b-directed --b-simulation-type 'springs' --b-suffix '15r1'

Run ACD with “brain networks (BN)”, “directed”, “netsims simulation”, “30 nodes”, “noise-free”, and “the second repetition”:

$> cd /src/models/MPM/
$> python3 run.py --b-network-type 'brain_networks' --b-directed --b-simulation-type 'netsims' --b-suffix '30r2'

Run ACD with “landscape networks (LN)”, “directed”, “netsims simulation”, “50 nodes”, “the third repetition”, and “noise level 2”:

$> cd /src/models/ACD/
$> python3 run.py --b-network-type 'landscape_networks' --b-simulation-type 'netsims' --b-suffix '50r3_n2'

iSIDG

Please install the required packages first.

Requirements

  • Python >= 3.8

  • Numpy >= 1.23.4

  • pandas >= 1.5.1

  • matplotlib >= 3.6.2

  • sklearn >= 0.0.post1

  • torch >= 1.13.1

  • torchinfo >= 1.7.2

  • tqdm >= 4.64.1

Arguments

  • b-network-type: name of the graph type (in full name)

  • b-directed: if called, will load data from directed graphs

  • b-simulation-type: springs or netsims

  • b-suffix: choose graph with node X, the Y repetition and with noise level K with format “XrY_nK”. If use noise-free, omit “_nK”

Reproduction Examples

Run NRI with “chemical reaction networks in atmosphere (CRNA)”, “directed”, “15 nodes”, “springs simulation”, “noise-free”, and “the first repetition” :

$> cd /src/models/iSIDG/
$> python3 train.py --b-network-type 'chemical_reaction_networks_in_atmosphere' --b-directed --b-simulation-type 'springs' --b-suffix '15r1'

Run NRI with “brain networks (BN)”, “directed”, “netsims simulation”, “30 nodes”, “noise-free”, and “the second repetition”:

$> cd /src/models/iSIDG/
$> python3 train.py --b-network-type 'brain_networks' --b-directed --b-simulation-type 'netsims' --b-suffix '30r2'

Run NRI with “landscape networks (LN)”, “directed”, “netsims simulation”, “50 nodes”, “the third repetition”, and “noise level 2”:

$> cd /src/models/iSIDG/
$> python3 train.py --b-network-type 'landscape_networks' --b-simulation-type 'netsims' --b-suffix '50r3_n2'