Difference between revisions of "Version benchmarking"

From Heureka Wiki
Jump to navigation Jump to search
 
(48 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[Category:Release]]
 
[[Category:Release]]
  
−
Here we present a comparison between different versions for a benchmark problem run in PlanWise. We call this problem Hexagons1000, it can be downloaded from [https://www.heurekaslu.se/filerepository/PlanwiseBenchmark/ here]. The problem consists of one thousand simulated treatment units. The results refer to an optimization with simple even-flow constraints. The TPG-simulation includes nine forest domains and the somewhat time-consuming breeding model is activated. For each treatment unit managed with even-aged management a nature conservation retention patch is created resulting in a total of ? units. The forest is a simulated but realistic landscape with respect to age classes, site fertilities and tree species distribution.  
+
Here we present a comparison between different versions for a benchmark problem run in PlanWise. We call this problem Hexagons1000, it can be downloaded from [https://www.heurekaslu.se/filerepository/PlanwiseBenchmark/ here]. The problem consists of one thousand simulated treatment units. The results refer to an optimization with simple even-flow constraints. The TPG-simulation includes nine forest domains and the somewhat time-consuming breeding model is activated. For each treatment unit managed with even-aged management a nature conservation retention patch is created. The forest is a synthetic but is a realistic landscape with respect to age classes, site fertilities and tree species distribution.  
  
−
The comparison is intended to present to users a simple form of quality and status check and give some insigth into what can be expected. Net present value is an excellent indicator for whether there has been any model changes, since it basically depends on everything calculated by the growth-and-yield simulator.  
+
The comparison is intended to present to users a simple form of quality and status check and give some insight into what simulation times that can be expected. Please note that if more result variables are added (such as dead wood, carbon and nitrogen etc.) simulation times might be quite significantly longer.
 +
Net present value is an good indicator for whether there has been any large model changes, since it basically depends on everything calculated by the growth-and-yield simulator.  
  
 
This page was/is first published with version 2.11. We intend to run the benchmark problem for each major version released.
 
This page was/is first published with version 2.11. We intend to run the benchmark problem for each major version released.
Line 9: Line 10:
 
{| {{table}} border = 1 style="text-align: center; width: 100%; border-collapse: collapse; "
 
{| {{table}} border = 1 style="text-align: center; width: 100%; border-collapse: collapse; "
 
| style="background:#f0f0f0;width:8%;"|'''Version'''
 
| style="background:#f0f0f0;width:8%;"|'''Version'''
−
| style="background:#f0f0f0;width:12%;"|'''TPG-time, 1 CPU-core (s)'''
+
| style="background:#f0f0f0;width:5%;"|'''TPG-time, 1 CPU-thread(s)'''
−
| style="background:#f0f0f0;width:12%;"|'''TPG-time, 8 CPU-cores (s)'''
+
| style="background:#f0f0f0;width:5%;"|'''TPG-time, 16 CPU-threads(s)'''
−
| style="background:#f0f0f0;width:13%;;"|'''Total NPV'''
+
| style="background:#f0f0f0;width:5%;;"|'''Total NPV'''
−
| style="background:#f0f0f0;width:10%;;"|'''NPV/ha within managed area'''
+
| style="background:#f0f0f0;width:5%;;"|'''NPV/ha within managed area'''
−
| style="background:#f0f0f0;width:12%;;"|'''Final felling volume per year, all periods (m<sup>3</sup>sk/ha,yr)'''
+
| style="background:#f0f0f0;width:8%;;"|'''Final felling volume per year, all periods (m<sup>3</sup>sk/ha,yr)'''
−
| style="background:#f0f0f0;width:12%;;"|'''Final felling volume period 1 (m<sup>3</sup>sk/ha,yr)'''
+
| style="background:#f0f0f0;width:8%;;"|'''Final felling volume period 1 (m<sup>3</sup>sk/ha,yr)'''
−
| style="background:#f0f0f0;width:12%;;"|'''Final felling area per year, all periods (m<sup>3</sup>sk/ha,yr)'''
+
| style="background:#f0f0f0;width:8%;;"|'''Final felling area per year, all periods (m<sup>3</sup>sk/ha,yr)'''
−
| style="background:#f0f0f0;width:12%;;"|'''Thinning volume per year, all periods (m<sup>3</sup>sk/ha,yr)'''
+
| style="background:#f0f0f0;width:8%;;"|'''Thinning volume per year, all periods (m<sup>3</sup>sk/ha,yr)'''
−
| style="background:#f0f0f0;width:12%;;"|'''Thinning volume per year, period 1 (m<sup>3</sup>sk/ha,yr)'''
+
| style="background:#f0f0f0;width:8%;;"|'''Thinning volume per year, period 1 (m<sup>3</sup>sk/ha,yr)'''
−
| style="background:#f0f0f0;width:12%;;"|'''Thinning area per year, all periods (m<sup>3</sup>sk/ha,yr)'''
+
| style="background:#f0f0f0;width:8%;;"|'''Thinning area per year, all periods (m<sup>3</sup>sk/ha,yr)'''
−
| style="background:#f0f0f0;width:12%;;"|'''Total production (m<sup>3</sup>sk/ha,yr)'''
+
| style="background:#f0f0f0;width:5%;;"|'''Total production (m<sup>3</sup>sk/ha,yr)'''
−
| style="background:#f0f0f0;width:12%;;"|'''Release notes'''
+
| style="background:#f0f0f0;width:10%;;"|'''Comment'''
 +
| style="background:#f0f0f0;width:10%;;"|'''Release notes'''
 
|-style = "vertical-align: top; text-align: center;"
 
|-style = "vertical-align: top; text-align: center;"
−
||[[Version 2.11|2.11]]||2654s <br>(44m 14s)<sup>a,b</sup>||<sup>a,b</sup>||19 704 953||36 821||2 684||2 610||7.35||754||320||13.8||6.23||[[Version 2.11]]
+
|[[Version 2.19|2.19]]||||710<sup>d</sup> ||21 224 113 ||39 670 ||2 661 || 4 990 ||7.08||1 000 || 282||16.7||7.0||||[[Version 2.19]]
 
|-style = "vertical-align: top; text-align: center;"
 
|-style = "vertical-align: top; text-align: center;"
−
|[[Version 2.10|2.10]]||2532s <br>(42m 12s)<sup>a</sup>||-||19 606 195||36 624||2 711||2 652||7.40||697||403||11.5||6.20||[[Version 2.10]]
+
|[[Version 2.18.3|2.18.3]]||   ||2153<sup>d</sup> ||21 192 183 ||39 608 ||2 669 || 4 959 ||7.11||1 002 || 293||16.9||7.0||||[[Version 2.18.3]]
 
|-style = "vertical-align: top; text-align: center;"
 
|-style = "vertical-align: top; text-align: center;"
−
|[[Version 2.8.0|2.8.0]]||2360s <br>(39m 20s)<sup>a</sup>||-||19 652 164||36 712||2 724||2 652||7.40||713||389||11.7||6.29||[[Version 2.8.0]]
+
|[[Version 2.18.2|2.18.2]]||     ||2067<sup>d</sup> ||21 192 183 ||39 608 ||2 669 || 4 959 ||7.11||1 002 || 293||16.9||7.0||||[[Version 2.18.2]]
 
|-style = "vertical-align: top; text-align: center;"
 
|-style = "vertical-align: top; text-align: center;"
−
|[[Version 2.6.0|2.6.0]]||2370s <br>(40m 30s)<sup>a</sup>||-||19 634 926||36 679||2 724||2 652||7.40||713||389||11.7||6.29||[[Version 2.6.0]]
+
|[[Version 2.18|2.18]]||    ||1425<sup>d</sup> ||21 298 040 ||39 814 ||2 659 || 4 932 ||7.12||1 003 || 293||16.9||7.0||||[[Version 2.18]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.17|2.17]]||    ||1297<sup>d</sup> ||20 308 979 ||37 886 ||2 542 || 4 574 ||6.80|| 799 || 266||14.7||6.5||||
 +
[[Version 2.17]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.16|2.16]]||    ||1336<sup>d</sup> ||20 307 509 ||37 883 ||2 528 || 4 574 ||6.76|| 799 || 266||14.7||6.5||||[[Version 2.16]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.16|2.16 RC]]||                                ||1228<sup>a</sup>||20 114 643||37 515||2 612||2 607||6.92||765||302||13.7||6.31||||[[Version 2.16]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.15|2.15.4]]||2961 <br>(49m 21s)<sup>a</sup>||1247<sup>a</sup>||20 114 643||37 515||2 612||2 607||6.92||765||302||13.7||6.31||||[[Version 2.15]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.14|2.14]]||2861 <br>(48m 41s)<sup>a</sup>||1132<sup>a</sup>||20 060 196||37 412||2 685||2 633||7.33||748||340||13.7||6.25||||[[Version 2.14]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.13|2.13]]||2894 <br>(48m 14s)<sup>a</sup>||1038<sup>a</sup>||19 997 049||37 365||2 685||2 633||7.33||748||340||13.7||6.25||||[[Version 2.13]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.12|2.12.3]]            || ||1010<sup>a</sup>||20 081 186||37 527||2 681||2 622||7.35||748||331||13.6||6.23||Increased npv: New bucking algorithm, bug fixed ||[[Version 2.12]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.12|2.12.3<sup>c</sup>]]|| ||983<sup>a</sup>||19 915 687||37 207||2 713||2 637||7.36||718||376||13.3||6.25||Understorey cleaning inactivated||[[Version 2.12]]
 +
<!--
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.12|2.12.0]]||2729 <br>(45m 29s)<sup>a</sup>|| ||19 592 678||36 614||2 694||2 622||7.35||745||331||13.6||6.24||||[[Version 2.12]]
 +
-->
 +
<!--||[[Version 2.11|2.11<sup>b</sup>]]||2654 <br>(44m 14s)<sup>a</sup>||1674<sup>a</sup>||19 704 953||36 821||2 684||2 610||7.35||754||320||13.8||6.23||[[Version 2.11]]-->
 +
<!--|[[Version 2.11|2.11<sup>b</sup>]]||983 <br>(43m 25s)<sup>a</sup>||<sup>a</sup>||19 719 149||36 849||2 675||2 648||7.36||744||331||12.6||6.23||[[Version 2.11]]-->
 +
<!--|[[Version 2.11|2.11<sup>b</sup>]]||2287 <br>(38m 7s)<sup>a</sup>||<sup>a</sup>||19 600 016||36 623||2 705||2 688||7.44||719||859||12.9||6.22||[[Version 2.11]]-->
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.11|2.11.1<sup>b</sup>]]||2483s <br>(41m 23s)<sup>a</sup>|| ||19 707 063||36 824||2 680||2 610||7.34||756||326||13.8||6.24||||[[Version 2.11]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
<!--|[[Version 2.11|2.11.0<sup>b</sup>]]||2583s <br>(43m 3s)<sup>a</sup>||1674<sup>a</sup>||19 606 876||36 636||2 689||2 688||7.38||728||859||13.0||6.23||Note larger thinning volume in period 1, see [[Version_2.11#bugg211|bug 211]] ||[[Version 2.11]]
 +
|-style = "vertical-align: top; text-align: center;"-->
 +
|[[Version 2.10|2.10]]||2532s <br>(42m 12s)<sup>a</sup>||-||19 606 195||36 624||2 711||2 652||7.40||697||403||11.5||6.20|| ||[[Version 2.10]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.8.0|2.8.0]]||2360s <br>(39m 20s)<sup>a</sup>||-||19 652 164||36 712||2 724||2 652||7.40||713||389||11.7||6.29|| ||[[Version 2.8.0]]
 +
|-style = "vertical-align: top; text-align: center;"
 +
|[[Version 2.6.0|2.6.0]]||2370s <br>(40m 30s)<sup>a</sup>||-||19 634 926||36 679||2 724||2 652||7.40||713||389||11.7||6.29|| ||[[Version 2.6.0]]
 
|}
 
|}
  
−
<sup>a</sup>Windows 7, Processor Intel I7-4770, 8-core, 32GB RAM. Samsung SSD 840 EVO. SQL Server 2012 Prof. <br>
+
<sup>a</sup>Windows 7, Processor Intel I7-4770, 4-core, 32GB RAM. Samsung SSD 840 EVO. SQL Server 2012 Prof. <br>
−
<sup>b</sup>With Thinning Period Delay Max = 1 and 0 in generation 1 and 2, respectively.
+
<!--<sup>b</sup>With Thinning Period Delay Max = 1 and 0 in generation 1 and 2, respectively.-->
 +
<!--<sup>b</sup>With Thinning Period Delay Max = 2 and 1 in generation 1 and 2, respectively.-->
 +
<sup>b</sup>With Thinning Period Delay Max = 2 and 0 in generation 1 and 2, respectively. <br>
 +
<sup>c</sup>With understorey cleaning inactivated. <br>
 +
<sup>d</sup>Windows 10, AMD Ryzen 7 4750U, 8-core, 16GB RAM. <br>

Latest revision as of 15:10, 15 September 2022


Here we present a comparison between different versions for a benchmark problem run in PlanWise. We call this problem Hexagons1000, it can be downloaded from here. The problem consists of one thousand simulated treatment units. The results refer to an optimization with simple even-flow constraints. The TPG-simulation includes nine forest domains and the somewhat time-consuming breeding model is activated. For each treatment unit managed with even-aged management a nature conservation retention patch is created. The forest is a synthetic but is a realistic landscape with respect to age classes, site fertilities and tree species distribution.

The comparison is intended to present to users a simple form of quality and status check and give some insight into what simulation times that can be expected. Please note that if more result variables are added (such as dead wood, carbon and nitrogen etc.) simulation times might be quite significantly longer. Net present value is an good indicator for whether there has been any large model changes, since it basically depends on everything calculated by the growth-and-yield simulator.

This page was/is first published with version 2.11. We intend to run the benchmark problem for each major version released.

Version TPG-time, 1 CPU-thread(s) TPG-time, 16 CPU-threads(s) Total NPV NPV/ha within managed area Final felling volume per year, all periods (m3sk/ha,yr) Final felling volume period 1 (m3sk/ha,yr) Final felling area per year, all periods (m3sk/ha,yr) Thinning volume per year, all periods (m3sk/ha,yr) Thinning volume per year, period 1 (m3sk/ha,yr) Thinning area per year, all periods (m3sk/ha,yr) Total production (m3sk/ha,yr) Comment Release notes
2.19 710d 21 224 113 39 670 2 661 4 990 7.08 1 000 282 16.7 7.0 Version 2.19
2.18.3 2153d 21 192 183 39 608 2 669 4 959 7.11 1 002 293 16.9 7.0 Version 2.18.3
2.18.2 2067d 21 192 183 39 608 2 669 4 959 7.11 1 002 293 16.9 7.0 Version 2.18.2
2.18 1425d 21 298 040 39 814 2 659 4 932 7.12 1 003 293 16.9 7.0 Version 2.18
2.17 1297d 20 308 979 37 886 2 542 4 574 6.80 799 266 14.7 6.5

Version 2.17

2.16 1336d 20 307 509 37 883 2 528 4 574 6.76 799 266 14.7 6.5 Version 2.16
2.16 RC 1228a 20 114 643 37 515 2 612 2 607 6.92 765 302 13.7 6.31 Version 2.16
2.15.4 2961
(49m 21s)a		 1247a 20 114 643 37 515 2 612 2 607 6.92 765 302 13.7 6.31 Version 2.15
2.14 2861
(48m 41s)a		 1132a 20 060 196 37 412 2 685 2 633 7.33 748 340 13.7 6.25 Version 2.14
2.13 2894
(48m 14s)a		 1038a 19 997 049 37 365 2 685 2 633 7.33 748 340 13.7 6.25 Version 2.13
2.12.3 1010a 20 081 186 37 527 2 681 2 622 7.35 748 331 13.6 6.23 Increased npv: New bucking algorithm, bug fixed Version 2.12
2.12.3c 983a 19 915 687 37 207 2 713 2 637 7.36 718 376 13.3 6.25 Understorey cleaning inactivated Version 2.12
2.11.1b 2483s
(41m 23s)a		 19 707 063 36 824 2 680 2 610 7.34 756 326 13.8 6.24 Version 2.11
2.10 2532s
(42m 12s)a		 - 19 606 195 36 624 2 711 2 652 7.40 697 403 11.5 6.20 Version 2.10
2.8.0 2360s
(39m 20s)a		 - 19 652 164 36 712 2 724 2 652 7.40 713 389 11.7 6.29 Version 2.8.0
2.6.0 2370s
(40m 30s)a		 - 19 634 926 36 679 2 724 2 652 7.40 713 389 11.7 6.29 Version 2.6.0

aWindows 7, Processor Intel I7-4770, 4-core, 32GB RAM. Samsung SSD 840 EVO. SQL Server 2012 Prof.
bWith Thinning Period Delay Max = 2 and 0 in generation 1 and 2, respectively.
cWith understorey cleaning inactivated.
dWindows 10, AMD Ryzen 7 4750U, 8-core, 16GB RAM.

Retrieved from "https://www.heurekaslu.se/w/index.php?title=Version_benchmarking&oldid=10553"