Update docs

This commit is contained in:
2024-12-10 12:14:36 -06:00
parent 184af2a869
commit 112963c178
37 changed files with 15544 additions and 3662 deletions

View File

@@ -1,10 +1,10 @@
<!DOCTYPE html>
<html lang="en" data-content_root="../../">
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>7. Feature Extractors &#8212; MIPLearn 0.4</title>
<link href="../../_static/css/theme.css" rel="stylesheet" />
@@ -22,21 +22,22 @@
<link rel="stylesheet" type="text/css" href="../../_static/pygments.css?v=362ab14a" />
<link rel="stylesheet" type="text/css" href="../../_static/sphinx-book-theme.acff12b8f9c144ce68a297486a2fa670.css?v=b0dfe17c" />
<link rel="stylesheet" type="text/css" href="../../_static/nbsphinx-code-cells.css?v=2aa19091" />
<link rel="stylesheet" type="text/css" href="../../_static/custom.css?v=f8244a84" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../_static/sphinx-book-theme.acff12b8f9c144ce68a297486a2fa670.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="../../_static/nbsphinx-code-cells.css" />
<link rel="stylesheet" type="text/css" href="../../_static/nbsphinx-code-cells.css" />
<link rel="stylesheet" type="text/css" href="../../_static/custom.css" />
<link rel="preload" as="script" href="../../_static/js/index.1c5a1a01449ed65a7b51.js">
<script src="../../_static/documentation_options.js?v=751a5dd3"></script>
<script src="../../_static/doctools.js?v=888ff710"></script>
<script src="../../_static/sphinx_highlight.js?v=dc90522c"></script>
<script id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
<script src="../../_static/jquery.js"></script>
<script src="../../_static/underscore.js"></script>
<script src="../../_static/doctools.js"></script>
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
<script src="../../_static/sphinx-book-theme.12a9622fbb08dcb3a2a40b2c02b83a57.js?v=7c4c3336"></script>
<script src="../../_static/sphinx-book-theme.12a9622fbb08dcb3a2a40b2c02b83a57.js"></script>
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/x-mathjax-config">MathJax.Hub.Config({"tex2jax": {"inlineMath": [["\\(", "\\)"]], "displayMath": [["\\[", "\\]"]], "processRefs": false, "processEnvironments": false}})</script>
<script>window.MathJax = {"tex": {"inlineMath": [["$", "$"], ["\\(", "\\)"]], "processEscapes": true}, "options": {"ignoreHtmlClass": "tex2jax_ignore|mathjax_ignore|document", "processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script>
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
<link rel="index" title="Index" href="../../genindex/" />
<link rel="search" title="Search" href="../../search/" />
<link rel="next" title="8. Primal Components" href="../primal/" />
@@ -68,7 +69,7 @@
<input type="search" class="form-control" name="q" id="search-input" placeholder="Search the docs ..." aria-label="Search the docs ..." autocomplete="off" >
</form><nav class="bd-links" id="bd-docs-nav" aria-label="Main navigation">
<div class="bd-toc-item active">
<p class="caption" role="heading">
<p class="caption">
<span class="caption-text">
Tutorials
</span>
@@ -95,7 +96,7 @@
</a>
</li>
</ul>
<p class="caption" role="heading">
<p class="caption">
<span class="caption-text">
User Guide
</span>
@@ -127,7 +128,7 @@
</a>
</li>
</ul>
<p class="caption" role="heading">
<p class="caption">
<span class="caption-text">
Python API Reference
</span>
@@ -265,105 +266,105 @@
<div>
<section id="Feature-Extractors">
<h1><span class="section-number">7. </span>Feature Extractors<a class="headerlink" href="#Feature-Extractors" title="Link to this heading"></a></h1>
<div class="section" id="Feature-Extractors">
<h1><span class="section-number">7. </span>Feature Extractors<a class="headerlink" href="#Feature-Extractors" title="Permalink to this headline"></a></h1>
<p>In the previous page, we introduced <em>training data collectors</em>, which solve the optimization problem and collect raw training data, such as the optimal solution. In this page, we introduce <strong>feature extractors</strong>, which take the raw training data, stored in HDF5 files, and extract relevant information in order to train a machine learning model.</p>
<section id="Overview">
<h2><span class="section-number">7.1. </span>Overview<a class="headerlink" href="#Overview" title="Link to this heading"></a></h2>
<div class="section" id="Overview">
<h2><span class="section-number">7.1. </span>Overview<a class="headerlink" href="#Overview" title="Permalink to this headline"></a></h2>
<p>Feature extraction is an important step of the process of building a machine learning model because it helps to reduce the complexity of the data and convert it into a format that is more easily processed. Previous research has proposed converting absolute variable coefficients, for example, into relative values which are invariant to various transformations, such as problem scaling, making them more amenable to learning. Various other transformations have also been described.</p>
<p>In the framework, we treat data collection and feature extraction as two separate steps to accelerate the model development cycle. Specifically, collectors are typically time-consuming, as they often need to solve the problem to optimality, and therefore focus on collecting and storing all data that may or may not be relevant, in its raw format. Feature extractors, on the other hand, focus entirely on filtering the data and improving its representation, and are therefore much faster to run.
Experimenting with new data representations, therefore, can be done without resolving the instances.</p>
<p>In MIPLearn, extractors implement the abstract class <a class="reference external" href="../../api/collectors/#miplearn.features.fields.FeaturesExtractor">FeatureExtractor</a>, which has methods that take as input an <a class="reference external" href="../../api/helpers/#miplearn.h5.H5File">H5File</a> and produce either: (i) instance features, which describe the entire instances; (ii) variable features, which describe a particular decision variables; or (iii) constraint features, which describe a particular constraint. The extractor is free to implement only a
subset of these methods, if it is known that it will not be used with a machine learning component that requires the other types of features.</p>
</section>
<section id="H5FieldsExtractor">
<h2><span class="section-number">7.2. </span>H5FieldsExtractor<a class="headerlink" href="#H5FieldsExtractor" title="Link to this heading"></a></h2>
</div>
<div class="section" id="H5FieldsExtractor">
<h2><span class="section-number">7.2. </span>H5FieldsExtractor<a class="headerlink" href="#H5FieldsExtractor" title="Permalink to this headline"></a></h2>
<p><a class="reference internal" href="#H5FieldsExtractor"><span class="std std-ref">H5FieldsExtractor</span></a>, the most simple extractor in MIPLearn, simple extracts data that is already available in the HDF5 file, assembles it into a matrix and returns it as-is. The fields used to build instance, variable and constraint features are user-specified. The class also performs checks to ensure that the shapes of the returned matrices make sense.</p>
<section id="Example">
<h3>Example<a class="headerlink" href="#Example" title="Link to this heading"></a></h3>
<div class="section" id="Example">
<h3>Example<a class="headerlink" href="#Example" title="Permalink to this headline"></a></h3>
<p>The example below demonstrates the usage of H5FieldsExtractor in a randomly generated instance of the multi-dimensional knapsack problem.</p>
<div class="nbinput docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[1]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">glob</span> <span class="kn">import</span> <span class="n">glob</span>
<span class="kn">from</span> <span class="nn">shutil</span> <span class="kn">import</span> <span class="n">rmtree</span>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span>from glob import glob
from shutil import rmtree
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">scipy.stats</span> <span class="kn">import</span> <span class="n">uniform</span><span class="p">,</span> <span class="n">randint</span>
import numpy as np
from scipy.stats import uniform, randint
<span class="kn">from</span> <span class="nn">miplearn.collectors.basic</span> <span class="kn">import</span> <span class="n">BasicCollector</span>
<span class="kn">from</span> <span class="nn">miplearn.extractors.fields</span> <span class="kn">import</span> <span class="n">H5FieldsExtractor</span>
<span class="kn">from</span> <span class="nn">miplearn.h5</span> <span class="kn">import</span> <span class="n">H5File</span>
<span class="kn">from</span> <span class="nn">miplearn.io</span> <span class="kn">import</span> <span class="n">write_pkl_gz</span>
<span class="kn">from</span> <span class="nn">miplearn.problems.multiknapsack</span> <span class="kn">import</span> <span class="p">(</span>
<span class="n">MultiKnapsackGenerator</span><span class="p">,</span>
<span class="n">build_multiknapsack_model_gurobipy</span><span class="p">,</span>
<span class="p">)</span>
from miplearn.collectors.basic import BasicCollector
from miplearn.extractors.fields import H5FieldsExtractor
from miplearn.h5 import H5File
from miplearn.io import write_pkl_gz
from miplearn.problems.multiknapsack import (
MultiKnapsackGenerator,
build_multiknapsack_model_gurobipy,
)
<span class="c1"># Set random seed to make example reproducible</span>
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
# Set random seed to make example reproducible
np.random.seed(42)
<span class="c1"># Generate some random multiknapsack instances</span>
<span class="n">rmtree</span><span class="p">(</span><span class="s2">&quot;data/multiknapsack/&quot;</span><span class="p">,</span> <span class="n">ignore_errors</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">write_pkl_gz</span><span class="p">(</span>
<span class="n">MultiKnapsackGenerator</span><span class="p">(</span>
<span class="n">n</span><span class="o">=</span><span class="n">randint</span><span class="p">(</span><span class="n">low</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">high</span><span class="o">=</span><span class="mi">11</span><span class="p">),</span>
<span class="n">m</span><span class="o">=</span><span class="n">randint</span><span class="p">(</span><span class="n">low</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">high</span><span class="o">=</span><span class="mi">6</span><span class="p">),</span>
<span class="n">w</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mi">1000</span><span class="p">),</span>
<span class="n">K</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span>
<span class="n">u</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span>
<span class="n">alpha</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.25</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span>
<span class="n">w_jitter</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.95</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">0.1</span><span class="p">),</span>
<span class="n">p_jitter</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.75</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">0.5</span><span class="p">),</span>
<span class="n">fix_w</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
<span class="p">)</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span>
<span class="s2">&quot;data/multiknapsack&quot;</span><span class="p">,</span>
<span class="p">)</span>
# Generate some random multiknapsack instances
rmtree(&quot;data/multiknapsack/&quot;, ignore_errors=True)
write_pkl_gz(
MultiKnapsackGenerator(
n=randint(low=10, high=11),
m=randint(low=5, high=6),
w=uniform(loc=0, scale=1000),
K=uniform(loc=100, scale=0),
u=uniform(loc=1, scale=0),
alpha=uniform(loc=0.25, scale=0),
w_jitter=uniform(loc=0.95, scale=0.1),
p_jitter=uniform(loc=0.75, scale=0.5),
fix_w=True,
).generate(10),
&quot;data/multiknapsack&quot;,
)
<span class="c1"># Run the basic collector</span>
<span class="n">BasicCollector</span><span class="p">()</span><span class="o">.</span><span class="n">collect</span><span class="p">(</span>
<span class="n">glob</span><span class="p">(</span><span class="s2">&quot;data/multiknapsack/*&quot;</span><span class="p">),</span>
<span class="n">build_multiknapsack_model_gurobipy</span><span class="p">,</span>
<span class="n">n_jobs</span><span class="o">=</span><span class="mi">4</span><span class="p">,</span>
<span class="p">)</span>
# Run the basic collector
BasicCollector().collect(
glob(&quot;data/multiknapsack/*&quot;),
build_multiknapsack_model_gurobipy,
n_jobs=4,
)
<span class="n">ext</span> <span class="o">=</span> <span class="n">H5FieldsExtractor</span><span class="p">(</span>
<span class="c1"># Use as instance features the value of the LP relaxation and the</span>
<span class="c1"># vector of objective coefficients.</span>
<span class="n">instance_fields</span><span class="o">=</span><span class="p">[</span>
<span class="s2">&quot;lp_obj_value&quot;</span><span class="p">,</span>
<span class="s2">&quot;static_var_obj_coeffs&quot;</span><span class="p">,</span>
<span class="p">],</span>
<span class="c1"># For each variable, use as features the optimal value of the LP</span>
<span class="c1"># relaxation, the variable objective coefficient, the variable&#39;s</span>
<span class="c1"># value its reduced cost.</span>
<span class="n">var_fields</span><span class="o">=</span><span class="p">[</span>
<span class="s2">&quot;lp_obj_value&quot;</span><span class="p">,</span>
<span class="s2">&quot;static_var_obj_coeffs&quot;</span><span class="p">,</span>
<span class="s2">&quot;lp_var_values&quot;</span><span class="p">,</span>
<span class="s2">&quot;lp_var_reduced_costs&quot;</span><span class="p">,</span>
<span class="p">],</span>
<span class="c1"># For each constraint, use as features the RHS, dual value and slack.</span>
<span class="n">constr_fields</span><span class="o">=</span><span class="p">[</span>
<span class="s2">&quot;static_constr_rhs&quot;</span><span class="p">,</span>
<span class="s2">&quot;lp_constr_dual_values&quot;</span><span class="p">,</span>
<span class="s2">&quot;lp_constr_slacks&quot;</span><span class="p">,</span>
<span class="p">],</span>
<span class="p">)</span>
ext = H5FieldsExtractor(
# Use as instance features the value of the LP relaxation and the
# vector of objective coefficients.
instance_fields=[
&quot;lp_obj_value&quot;,
&quot;static_var_obj_coeffs&quot;,
],
# For each variable, use as features the optimal value of the LP
# relaxation, the variable objective coefficient, the variable&#39;s
# value its reduced cost.
var_fields=[
&quot;lp_obj_value&quot;,
&quot;static_var_obj_coeffs&quot;,
&quot;lp_var_values&quot;,
&quot;lp_var_reduced_costs&quot;,
],
# For each constraint, use as features the RHS, dual value and slack.
constr_fields=[
&quot;static_constr_rhs&quot;,
&quot;lp_constr_dual_values&quot;,
&quot;lp_constr_slacks&quot;,
],
)
<span class="k">with</span> <span class="n">H5File</span><span class="p">(</span><span class="s2">&quot;data/multiknapsack/00000.h5&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">h5</span><span class="p">:</span>
<span class="c1"># Extract and print instance features</span>
<span class="n">x1</span> <span class="o">=</span> <span class="n">ext</span><span class="o">.</span><span class="n">get_instance_features</span><span class="p">(</span><span class="n">h5</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;instance features&quot;</span><span class="p">,</span> <span class="n">x1</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">x1</span><span class="p">)</span>
with H5File(&quot;data/multiknapsack/00000.h5&quot;) as h5:
# Extract and print instance features
x1 = ext.get_instance_features(h5)
print(&quot;instance features&quot;, x1.shape, &quot;\n&quot;, x1)
<span class="c1"># Extract and print variable features</span>
<span class="n">x2</span> <span class="o">=</span> <span class="n">ext</span><span class="o">.</span><span class="n">get_var_features</span><span class="p">(</span><span class="n">h5</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;variable features&quot;</span><span class="p">,</span> <span class="n">x2</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">x2</span><span class="p">)</span>
# Extract and print variable features
x2 = ext.get_var_features(h5)
print(&quot;variable features&quot;, x2.shape, &quot;\n&quot;, x2)
<span class="c1"># Extract and print constraint features</span>
<span class="n">x3</span> <span class="o">=</span> <span class="n">ext</span><span class="o">.</span><span class="n">get_constr_features</span><span class="p">(</span><span class="n">h5</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;constraint features&quot;</span><span class="p">,</span> <span class="n">x3</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">x3</span><span class="p">)</span>
# Extract and print constraint features
x3 = ext.get_constr_features(h5)
print(&quot;constraint features&quot;, x3.shape, &quot;\n&quot;, x3)
</pre></div>
</div>
</div>
@@ -399,29 +400,29 @@ constraint features (5, 3)
<p class="admonition-title">Warning</p>
<p>You should ensure that the number of features remains the same for all relevant HDF5 files. In the previous example, to illustrate this issue, we used variable objective coefficients as instance features. While this is allowed, note that this requires all problem instances to have the same number of variables; otherwise the number of features would vary from instance to instance and MIPLearn would be unable to concatenate the matrices.</p>
</div>
</section>
</section>
<section id="AlvLouWeh2017Extractor">
<h2><span class="section-number">7.3. </span>AlvLouWeh2017Extractor<a class="headerlink" href="#AlvLouWeh2017Extractor" title="Link to this heading"></a></h2>
</div>
</div>
<div class="section" id="AlvLouWeh2017Extractor">
<h2><span class="section-number">7.3. </span>AlvLouWeh2017Extractor<a class="headerlink" href="#AlvLouWeh2017Extractor" title="Permalink to this headline"></a></h2>
<p>Alvarez, Louveaux and Wehenkel (2017) proposed a set features to describe a particular decision variable in a given node of the branch-and-bound tree, and applied it to the problem of mimicking strong branching decisions. The class <a class="reference internal" href="#AlvLouWeh2017Extractor"><span class="std std-ref">AlvLouWeh2017Extractor</span></a> implements a subset of these features (40 out of 64), which are available outside of the branch-and-bound tree. Some features are derived from the static defintion of the problem (i.e. from objective function and
constraint data), while some features are derived from the solution to the LP relaxation. The features have been designed to be: (i) independent of the size of the problem; (ii) invariant with respect to irrelevant problem transformations, such as row and column permutation; and (iii) independent of the scale of the problem. We refer to the paper for a more complete description.</p>
<section id="id1">
<h3>Example<a class="headerlink" href="#id1" title="Link to this heading"></a></h3>
<div class="section" id="id1">
<h3>Example<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h3>
<div class="nbinput docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[2]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">miplearn.extractors.AlvLouWeh2017</span> <span class="kn">import</span> <span class="n">AlvLouWeh2017Extractor</span>
<span class="kn">from</span> <span class="nn">miplearn.h5</span> <span class="kn">import</span> <span class="n">H5File</span>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span>from miplearn.extractors.AlvLouWeh2017 import AlvLouWeh2017Extractor
from miplearn.h5 import H5File
<span class="c1"># Build the extractor</span>
<span class="n">ext</span> <span class="o">=</span> <span class="n">AlvLouWeh2017Extractor</span><span class="p">()</span>
# Build the extractor
ext = AlvLouWeh2017Extractor()
<span class="c1"># Open previously-created multiknapsack training data</span>
<span class="k">with</span> <span class="n">H5File</span><span class="p">(</span><span class="s2">&quot;data/multiknapsack/00000.h5&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">h5</span><span class="p">:</span>
<span class="c1"># Extract and print variable features</span>
<span class="n">x1</span> <span class="o">=</span> <span class="n">ext</span><span class="o">.</span><span class="n">get_var_features</span><span class="p">(</span><span class="n">h5</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;x1&quot;</span><span class="p">,</span> <span class="n">x1</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">x1</span><span class="o">.</span><span class="n">round</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
# Open previously-created multiknapsack training data
with H5File(&quot;data/multiknapsack/00000.h5&quot;) as h5:
# Extract and print variable features
x1 = ext.get_var_features(h5)
print(&quot;x1&quot;, x1.shape, &quot;\n&quot;, x1.round(1))
</pre></div>
</div>
</div>
@@ -500,9 +501,9 @@ x1 (10, 40)
<li><p><strong>Alvarez, Alejandro Marcos, Quentin Louveaux, and Louis Wehenkel.</strong> <em>A machine learning-based approximation of strong branching.</em> INFORMS Journal on Computing 29.1 (2017): 185-195.</p></li>
</ul>
</div>
</section>
</section>
</section>
</div>
</div>
</div>
</div>