Document component.fit and component.evaluate

docs/customization/index.html

@@ -144,6 +144,7 @@
                 
             <li class="second-level"><a href="#adjusting-component-aggresiveness">Adjusting component aggresiveness</a></li>
                 
+                <li class="third-level"><a href="#evaluating-component-performance">Evaluating component performance</a></li>
     </ul>
 </div></div>
         <div class="col-md-9" role="main">
@@ -198,6 +199,70 @@ to achieve a minimum desired true positive rate (also known as precision). The e
 a <code>PrimalSolutionComponent</code> which achieves 95% precision, possibly at the cost of a lower recall. To make the component
 more aggressive, this precision may be lowered.</p>
 <pre><code class="python">PrimalSolutionComponent(threshold=MinPrecisionThreshold(0.95))
+</code></pre>
+
+<h3 id="evaluating-component-performance">Evaluating component performance</h3>
+<p>MIPLearn allows solver components to be modified and evaluated in isolation. In the following example, we build and
+fit <code>PrimalSolutionComponent</code> outside a solver, then evaluate its performance.</p>
+<pre><code class="python">from miplearn import PrimalSolutionComponent
+
+# User-provided set os solved training instances
+train_instances = [...]
+
+# Construct and fit component on a subset of the training set
+comp = PrimalSolutionComponent()
+comp.fit(train_instances[:100])
+
+# Evaluate performance on an additional set of training instances
+ev = comp.evaluate(train_instances[100:150])
+</code></pre>
+
+<p>The method <code>evaluate</code> returns a dictionary with performance evaluation statistics for each training instance provided,
+and for each type of prediction the component makes. To obtain a summary across all instances, pandas may be used, as below:</p>
+<pre><code class="python">import pandas as pd
+pd.DataFrame(ev[&quot;Fix one&quot;]).mean(axis=1)
+</code></pre>
+
+<pre><code>Predicted positive          3.120000
+Predicted negative        196.880000
+Condition positive         62.500000
+Condition negative        137.500000
+True positive               3.060000
+True negative             137.440000
+False positive              0.060000
+False negative             59.440000
+Accuracy                    0.702500
+F1 score                    0.093050
+Recall                      0.048921
+Precision                   0.981667
+Predicted positive (%)      1.560000
+Predicted negative (%)     98.440000
+Condition positive (%)     31.250000
+Condition negative (%)     68.750000
+True positive (%)           1.530000
+True negative (%)          68.720000
+False positive (%)          0.030000
+False negative (%)         29.720000
+dtype: float64
+</code></pre>
+
+<p>Regression components (such as <code>ObjectiveValueComponent</code>) can also be used similarly, as shown in the next example:</p>
+<pre><code class="python">from miplearn import ObjectiveValueComponent
+comp = ObjectiveValueComponent()
+comp.fit(train_instances[:100])
+ev = comp.evaluate(train_instances[100:150])
+
+import pandas as pd
+pd.DataFrame(ev).mean(axis=1)
+</code></pre>
+
+<pre><code>Mean squared error       7001.977827
+Explained variance          0.519790
+Max error                 242.375804
+Mean absolute error        65.843924
+R2                          0.517612
+Median absolute error      65.843924
+dtype: float64
 </code></pre></div>
         
         

docs/index.html

@@ -273,5 +273,5 @@
 
 <!--
 MkDocs version : 1.1
-Build Date UTC : 2020-05-05 18:12:23
+Build Date UTC : 2020-05-05 18:30:25
 -->

src/docs/customization.md

@@ -61,3 +61,74 @@ more aggressive, this precision may be lowered.
 ```python
 PrimalSolutionComponent(threshold=MinPrecisionThreshold(0.95))
 ```
+
+### Evaluating component performance
+
+MIPLearn allows solver components to be modified and evaluated in isolation. In the following example, we build and
+fit `PrimalSolutionComponent` outside a solver, then evaluate its performance.
+
+```python
+from miplearn import PrimalSolutionComponent
+
+# User-provided set os solved training instances
+train_instances = [...]
+
+# Construct and fit component on a subset of the training set
+comp = PrimalSolutionComponent()
+comp.fit(train_instances[:100])
+
+# Evaluate performance on an additional set of training instances
+ev = comp.evaluate(train_instances[100:150])
+``` 
+
+The method `evaluate` returns a dictionary with performance evaluation statistics for each training instance provided,
+and for each type of prediction the component makes. To obtain a summary across all instances, pandas may be used, as below:
+
+```python
+import pandas as pd
+pd.DataFrame(ev["Fix one"]).mean(axis=1)
+```
+```
+Predicted positive          3.120000
+Predicted negative        196.880000
+Condition positive         62.500000
+Condition negative        137.500000
+True positive               3.060000
+True negative             137.440000
+False positive              0.060000
+False negative             59.440000
+Accuracy                    0.702500
+F1 score                    0.093050
+Recall                      0.048921
+Precision                   0.981667
+Predicted positive (%)      1.560000
+Predicted negative (%)     98.440000
+Condition positive (%)     31.250000
+Condition negative (%)     68.750000
+True positive (%)           1.530000
+True negative (%)          68.720000
+False positive (%)          0.030000
+False negative (%)         29.720000
+dtype: float64
+```
+
+Regression components (such as `ObjectiveValueComponent`) can also be used similarly, as shown in the next example:
+
+```python
+from miplearn import ObjectiveValueComponent
+comp = ObjectiveValueComponent()
+comp.fit(train_instances[:100])
+ev = comp.evaluate(train_instances[100:150])
+
+import pandas as pd
+pd.DataFrame(ev).mean(axis=1)
+```
+```
+Mean squared error       7001.977827
+Explained variance          0.519790
+Max error                 242.375804
+Mean absolute error        65.843924
+R2                          0.517612
+Median absolute error      65.843924
+dtype: float64
+```