Skip to content

Conversation

@laurenthuberdeau
Copy link
Contributor

@laurenthuberdeau laurenthuberdeau commented Feb 24, 2022

Results:

  • Significant improvements on benchmarks with lots of rules.
  • No regression on other benchmarks

Using Vector

Benchmark       bench-master  bench-vector    
clique/10       0.322e-3      0.039e-3 -87.94%
clique/25       2.201e-3      0.191e-3 -91.34%
clique/40       0.571e-2      0.051e-2 -91.01%
clique/5        0.910e-4      0.199e-4 -78.17%
line/10         0.369e-3      0.266e-3 -28.06%
line/25         3.156e-3      1.798e-3 -43.03%
line/40         1.056e-2      0.562e-2 -46.80%
line/5          0.883e-4      0.785e-4 -11.15%
loop/100        2.032e-3      1.381e-3 -32.03%
loop/1000       0.877e-1      0.842e-1  -3.93%
loop/50         0.505e-3      0.469e-3  -7.13%
loop/500        2.347e-2      2.144e-2  -8.66%
parse/100       1.269e-2      1.285e-2  +1.24%
parse/1000      1.327e-1      1.325e-1  -0.15%
parse/50        0.604e-2      0.602e-2  -0.33%
parse/500       0.662e-1      0.679e-1  +2.62%
tight/100       0.757e-3      0.729e-3  -3.73%
tight/1000      0.775e-1      0.721e-1  -6.89%
tight/50        2.109e-4      2.075e-4  -1.59%
tight/500       1.699e-2      1.655e-2  -2.61%
Geometric mean  0.358e-2      0.208e-2 -41.85%

Forcing the Vector in compilePred using force:

Benchmark       bench-master  bench-vector-strict
clique/10       0.322e-3      0.043e-3 -86.57%   
clique/25       2.201e-3      0.238e-3 -89.21%   
clique/40       0.571e-2      0.098e-2 -82.81%   
clique/5        0.910e-4      0.232e-4 -74.54%   
line/10         0.369e-3      0.267e-3 -27.58%   
line/25         3.156e-3      1.858e-3 -41.14%   
line/40         1.056e-2      0.583e-2 -44.83%   
line/5          0.883e-4      0.804e-4  -9.00%   
loop/100        2.032e-3      1.394e-3 -31.43%   
loop/1000       0.877e-1      0.852e-1  -2.82%   
loop/50         0.505e-3      0.470e-3  -6.93%   
loop/500        2.347e-2      2.140e-2  -8.82%   
parse/100       1.269e-2      1.286e-2  +1.30%   
parse/1000      1.327e-1      1.349e-1  +1.68%   
parse/50        0.604e-2      0.619e-2  +2.55%   
parse/500       0.662e-1      0.693e-1  +4.81%   
tight/100       0.757e-3      0.731e-3  -3.51%   
tight/1000      0.775e-1      0.736e-1  -5.01%   
tight/50        2.109e-4      2.076e-4  -1.54%   
tight/500       1.699e-2      1.700e-2  +0.03%   
Geometric mean  0.358e-2      0.223e-2 -37.73% 

Using Lists

Benchmark       bench-master  bench-list      
clique/10       0.322e-3      0.039e-3 -87.86%
clique/25       2.201e-3      0.192e-3 -91.29%
clique/40       0.571e-2      0.060e-2 -89.51%
clique/5        0.910e-4      0.200e-4 -78.05%
line/10         0.369e-3      0.275e-3 -25.63%
line/25         3.156e-3      1.974e-3 -37.46%
line/40         1.056e-2      0.580e-2 -45.14%
line/5          0.883e-4      0.781e-4 -11.57%
loop/100        2.032e-3      1.467e-3 -27.83%
loop/1000       0.877e-1      0.868e-1  -0.97%
loop/50         0.505e-3      0.480e-3  -4.91%
loop/500        2.347e-2      2.437e-2  +3.84%
parse/100       1.269e-2      1.291e-2  +1.66%
parse/1000      1.327e-1      1.355e-1  +2.13%
parse/50        0.604e-2      0.616e-2  +1.94%
parse/500       0.662e-1      0.713e-1  +7.71%
tight/100       0.757e-3      0.735e-3  -2.88%
tight/1000      0.775e-1      0.707e-1  -8.71%
tight/50        2.109e-4      2.084e-4  -1.19%
tight/500       1.699e-2      1.651e-2  -2.86%
Geometric mean  0.358e-2      0.215e-2 -39.91%

Using hashset:

Benchmark       bench-master  bench-hashset   
clique/10       0.322e-3      0.048e-3 -84.95%
clique/25       2.201e-3      0.297e-3 -86.52%
clique/40       0.571e-2      0.157e-2 -72.58%
clique/5        0.910e-4      0.214e-4 -76.45%
line/10         0.369e-3      0.259e-3 -29.82%
line/25         3.156e-3      1.776e-3 -43.75%
line/40         1.056e-2      0.554e-2 -47.56%
line/5          0.883e-4      0.769e-4 -12.93%
loop/100        2.032e-3      1.345e-3 -33.83%
loop/1000       0.877e-1      0.819e-1  -6.53%
loop/50         0.505e-3      0.456e-3  -9.66%
loop/500        2.347e-2      2.057e-2 -12.35%
parse/100       1.269e-2      1.230e-2  -3.08%
parse/1000      1.327e-1      1.327e-1  +0.04%
parse/50        0.604e-2      0.619e-2  +2.52%
parse/500       0.662e-1      0.663e-1  +0.29%
tight/100       0.757e-3      0.708e-3  -6.45%
tight/1000      0.775e-1      0.698e-1  -9.84%
tight/50        2.109e-4      2.004e-4  -4.96%
tight/500       1.699e-2      1.648e-2  -3.03%
Geometric mean  0.358e-2      0.225e-2 -37.25%

Data:
bench-hashset.csv
bench-list.csv
bench-master.csv
bench-vector-strict.csv
bench-vector.csv

@laurenthuberdeau
Copy link
Contributor Author

Benchmarks with inlining:

Benchmark       bench-inline-new  bench-optimizations-with-inline
clique/10       1.387e-4          0.303e-4 -78.18%               
clique/25       1.045e-3          0.183e-3 -82.46%               
clique/40       2.687e-3          0.803e-3 -70.12%               
clique/5        0.411e-4          0.119e-4 -70.94%               
line/10         1.528e-4          1.186e-4 -22.35%               
line/25         1.037e-3          0.631e-3 -39.10%               
line/40         2.973e-3          1.646e-3 -44.63%               
line/5          0.419e-4          0.390e-4  -6.93%               
loop/100        0.550e-3          0.540e-3  -1.88%               
loop/1000       1.968e-2          1.827e-2  -7.14%               
loop/50         2.178e-4          2.155e-4  -1.01%               
loop/500        0.614e-2          0.575e-2  -6.37%               
parse/100       1.292e-2          1.274e-2  -1.38%               
parse/1000      1.378e-1          1.361e-1  -1.22%               
parse/50        0.627e-2          0.616e-2  -1.85%               
parse/500       0.686e-1          0.676e-1  -1.48%               
tight/100       1.647e-4          1.464e-4 -11.10%               
tight/1000      1.122e-2          1.008e-2 -10.20%               
tight/50        0.565e-4          0.529e-4  -6.50%               
tight/500       3.024e-3          2.545e-3 -15.84%              
Geometric mean  1.420e-3          0.961e-3 -32.34%   

bench-inline-new.csv
bench-optimizations-with-inline.csv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant