mirror of
https://github.com/ANL-CEEESA/MIPLearn.git
synced 2025-12-06 09:28:51 -06:00
Add 0.3 docs
This commit is contained in:
285
0.3/guide/collectors.ipynb
Normal file
285
0.3/guide/collectors.ipynb
Normal file
@@ -0,0 +1,285 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "505cea0b-5f5d-478a-9107-42bb5515937d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Training Data Collectors\n",
|
||||
"The first step in solving mixed-integer optimization problems with the assistance of supervised machine learning methods is solving a large set of training instances and collecting the raw training data. In this section, we describe the various training data collectors included in MIPLearn. Additionally, the framework follows the convention of storing all training data in files with a specific data format (namely, HDF5). In this section, we briefly describe this format and the rationale for choosing it.\n",
|
||||
"\n",
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"In MIPLearn, a **collector** is a class that solves or analyzes the problem and collects raw data which may be later useful for machine learning methods. Collectors, by convention, take as input: (i) a list of problem data filenames, in gzipped pickle format, ending with `.pkl.gz`; (ii) a function that builds the optimization model, such as `build_tsp_model`. After processing is done, collectors store the training data in a HDF5 file located alongside with the problem data. For example, if the problem data is stored in file `problem.pkl.gz`, then the collector writes to `problem.h5`. Collectors are, in general, very time consuming, as they may need to solve the problem to optimality, potentially multiple times.\n",
|
||||
"\n",
|
||||
"## HDF5 Format\n",
|
||||
"\n",
|
||||
"MIPLearn stores all training data in [HDF5](HDF5) (Hierarchical Data Format, Version 5) files. The HDF format was originally developed by the [National Center for Supercomputing Applications][NCSA] (NCSA) for storing and organizing large amounts of data, and supports a variety of data types, including integers, floating-point numbers, strings, and arrays. Compared to other formats, such as CSV, JSON or SQLite, the HDF5 format provides several advantages for MIPLearn, including:\n",
|
||||
"\n",
|
||||
"- *Storage of multiple scalars, vectors and matrices in a single file* --- This allows MIPLearn to store all training data related to a given problem instance in a single file, which makes training data easier to store, organize and transfer.\n",
|
||||
"- *High-performance partial I/O* --- Partial I/O allows MIPLearn to read a single element from the training data (e.g. value of the optimal solution) without loading the entire file to memory or reading it from beginning to end, which dramatically improves performance and reduces memory requirements. This is especially important when processing a large number of training data files.\n",
|
||||
"- *On-the-fly compression* --- HDF5 files can be transparently compressed, using the gzip method, which reduces storage requirements and accelerates network transfers.\n",
|
||||
"- *Stable, portable and well-supported data format* --- Training data files are typically expensive to generate. Having a stable and well supported data format ensures that these files remain usable in the future, potentially even by other non-Python MIP/ML frameworks.\n",
|
||||
"\n",
|
||||
"MIPLearn currently uses HDF5 as simple key-value storage for numerical data; more advanced features of the format, such as metadata, are not currently used. Although files generated by MIPLearn can be read with any HDF5 library, such as [h5py][h5py], some convenience functions are provided to make the access more simple and less error-prone. Specifically, the class [H5File][H5File], which is built on top of h5py, provides the methods [put_scalar][put_scalar], [put_array][put_array], [put_sparse][put_sparse], [put_bytes][put_bytes] to store, respectively, scalar values, dense multi-dimensional arrays, sparse multi-dimensional arrays and arbitrary binary data. The corresponding *get* methods are also provided. Compared to pure h5py methods, these methods automatically perform type-checking and gzip compression. The example below shows their usage.\n",
|
||||
"\n",
|
||||
"[HDF5]: https://en.wikipedia.org/wiki/Hierarchical_Data_Format\n",
|
||||
"[NCSA]: https://en.wikipedia.org/wiki/National_Center_for_Supercomputing_Applications\n",
|
||||
"[h5py]: https://www.h5py.org/\n",
|
||||
"[H5File]: ../../api/helpers/#miplearn.h5.H5File\n",
|
||||
"[put_scalar]: ../../api/helpers/#miplearn.h5.H5File.put_scalar\n",
|
||||
"[put_array]: ../../api/helpers/#miplearn.h5.H5File.put_scalar\n",
|
||||
"[put_sparse]: ../../api/helpers/#miplearn.h5.H5File.put_scalar\n",
|
||||
"[put_bytes]: ../../api/helpers/#miplearn.h5.H5File.put_scalar\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"### Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "3d9cc182",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"x1 = 1\n",
|
||||
"x2 = hello world\n",
|
||||
"x3 = [1 2 3]\n",
|
||||
"x4 = [[0.37454012 0.9507143 0.7319939 ]\n",
|
||||
" [0.5986585 0.15601864 0.15599452]\n",
|
||||
" [0.05808361 0.8661761 0.601115 ]]\n",
|
||||
"x5 = (2, 3)\t0.68030757\n",
|
||||
" (3, 2)\t0.45049927\n",
|
||||
" (4, 0)\t0.013264962\n",
|
||||
" (0, 2)\t0.94220173\n",
|
||||
" (4, 2)\t0.5632882\n",
|
||||
" (2, 1)\t0.3854165\n",
|
||||
" (1, 1)\t0.015966251\n",
|
||||
" (3, 0)\t0.23089382\n",
|
||||
" (4, 4)\t0.24102546\n",
|
||||
" (1, 3)\t0.68326354\n",
|
||||
" (3, 1)\t0.6099967\n",
|
||||
" (0, 3)\t0.8331949\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import scipy.sparse\n",
|
||||
"\n",
|
||||
"from miplearn.h5 import H5File\n",
|
||||
"\n",
|
||||
"# Set random seed to make example reproducible\n",
|
||||
"np.random.seed(42)\n",
|
||||
"\n",
|
||||
"# Create a new empty HDF5 file\n",
|
||||
"with H5File(\"test.h5\", \"w\") as h5:\n",
|
||||
" # Store a scalar\n",
|
||||
" h5.put_scalar(\"x1\", 1)\n",
|
||||
" h5.put_scalar(\"x2\", \"hello world\")\n",
|
||||
"\n",
|
||||
" # Store a dense array and a dense matrix\n",
|
||||
" h5.put_array(\"x3\", np.array([1, 2, 3]))\n",
|
||||
" h5.put_array(\"x4\", np.random.rand(3, 3))\n",
|
||||
"\n",
|
||||
" # Store a sparse matrix\n",
|
||||
" h5.put_sparse(\"x5\", scipy.sparse.random(5, 5, 0.5))\n",
|
||||
"\n",
|
||||
"# Re-open the file we just created and print\n",
|
||||
"# previously-stored data\n",
|
||||
"with H5File(\"test.h5\", \"r\") as h5:\n",
|
||||
" print(\"x1 =\", h5.get_scalar(\"x1\"))\n",
|
||||
" print(\"x2 =\", h5.get_scalar(\"x2\"))\n",
|
||||
" print(\"x3 =\", h5.get_array(\"x3\"))\n",
|
||||
" print(\"x4 =\", h5.get_array(\"x4\"))\n",
|
||||
" print(\"x5 =\", h5.get_sparse(\"x5\"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e7175752",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e993aa6f",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"## Basic collector\n",
|
||||
"\n",
|
||||
"[BasicCollector][BasicCollector] is the most fundamental collector, and performs the following steps:\n",
|
||||
"\n",
|
||||
"1. Extracts all model data, such as objective function and constraint right-hand sides into numpy arrays, which can later be easily and efficiently accessed without rebuilding the model or invoking the solver;\n",
|
||||
"2. Solves the linear relaxation of the problem and stores its optimal solution, basis status and sensitivity information, among other information;\n",
|
||||
"3. Solves the original mixed-integer optimization problem to optimality and stores its optimal solution, along with solve statistics, such as number of explored nodes and wallclock time.\n",
|
||||
"\n",
|
||||
"Data extracted in Phases 1, 2 and 3 above are prefixed, respectively as `static_`, `lp_` and `mip_`. The entire set of fields is shown in the table below.\n",
|
||||
"\n",
|
||||
"[BasicCollector]: ../../api/collectors/#miplearn.collectors.basic.BasicCollector\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "711639bd",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"### Data fields\n",
|
||||
"\n",
|
||||
"| Field | Type | Description |\n",
|
||||
"|-----------------------------------|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------|\n",
|
||||
"| `static_constr_lhs` | `(nconstrs, nvars)` | Constraint left-hand sides, in sparse matrix format |\n",
|
||||
"| `static_constr_names` | `(nconstrs,)` | Constraint names |\n",
|
||||
"| `static_constr_rhs` | `(nconstrs,)` | Constraint right-hand sides |\n",
|
||||
"| `static_constr_sense` | `(nconstrs,)` | Constraint senses (`\"<\"`, `\">\"` or `\"=\"`) |\n",
|
||||
"| `static_obj_offset` | `float` | Constant value added to the objective function |\n",
|
||||
"| `static_sense` | `str` | `\"min\"` if minimization problem or `\"max\"` otherwise |\n",
|
||||
"| `static_var_lower_bounds` | `(nvars,)` | Variable lower bounds |\n",
|
||||
"| `static_var_names` | `(nvars,)` | Variable names |\n",
|
||||
"| `static_var_obj_coeffs` | `(nvars,)` | Objective coefficients |\n",
|
||||
"| `static_var_types` | `(nvars,)` | Types of the decision variables (`\"C\"`, `\"B\"` and `\"I\"` for continuous, binary and integer, respectively) |\n",
|
||||
"| `static_var_upper_bounds` | `(nvars,)` | Variable upper bounds |\n",
|
||||
"| `lp_constr_basis_status` | `(nconstr,)` | Constraint basis status (`0` for basic, `-1` for non-basic) |\n",
|
||||
"| `lp_constr_dual_values` | `(nconstr,)` | Constraint dual value (or shadow price) |\n",
|
||||
"| `lp_constr_sa_rhs_{up,down}` | `(nconstr,)` | Sensitivity information for the constraint RHS |\n",
|
||||
"| `lp_constr_slacks` | `(nconstr,)` | Constraint slack in the solution to the LP relaxation |\n",
|
||||
"| `lp_obj_value` | `float` | Optimal value of the LP relaxation |\n",
|
||||
"| `lp_var_basis_status` | `(nvars,)` | Variable basis status (`0`, `-1`, `-2` or `-3` for basic, non-basic at lower bound, non-basic at upper bound, and superbasic, respectively) |\n",
|
||||
"| `lp_var_reduced_costs` | `(nvars,)` | Variable reduced costs |\n",
|
||||
"| `lp_var_sa_{obj,ub,lb}_{up,down}` | `(nvars,)` | Sensitivity information for the variable objective coefficient, lower and upper bound. |\n",
|
||||
"| `lp_var_values` | `(nvars,)` | Optimal solution to the LP relaxation |\n",
|
||||
"| `lp_wallclock_time` | `float` | Time taken to solve the LP relaxation (in seconds) |\n",
|
||||
"| `mip_constr_slacks` | `(nconstrs,)` | Constraint slacks in the best MIP solution |\n",
|
||||
"| `mip_gap` | `float` | Relative MIP optimality gap |\n",
|
||||
"| `mip_node_count` | `float` | Number of explored branch-and-bound nodes |\n",
|
||||
"| `mip_obj_bound` | `float` | Dual bound |\n",
|
||||
"| `mip_obj_value` | `float` | Value of the best MIP solution |\n",
|
||||
"| `mip_var_values` | `(nvars,)` | Best MIP solution |\n",
|
||||
"| `mip_wallclock_time` | `float` | Time taken to solve the MIP (in seconds) |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9b35d7f7",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
"\n",
|
||||
"The example below shows how to generate a few random instances of the traveling salesman problem, store its problem data, run the collector and print some of the training data to screen."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "7ea6b715",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/home/axavier/Software/anaconda3/envs/miplearn/lib/python3.8/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
|
||||
" from .autonotebook import tqdm as notebook_tqdm\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Removing empty/corrupted h5 file: data/tsp/00000.h5\n",
|
||||
"lp_obj_value = 2909.0\n",
|
||||
"mip_obj_value = 2921.0\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"import numpy as np\n",
|
||||
"from scipy.stats import uniform, randint\n",
|
||||
"from glob import glob\n",
|
||||
"\n",
|
||||
"from miplearn.problems.tsp import (\n",
|
||||
" TravelingSalesmanGenerator,\n",
|
||||
" build_tsp_model,\n",
|
||||
")\n",
|
||||
"from miplearn.io import save\n",
|
||||
"from miplearn.h5 import H5File\n",
|
||||
"from miplearn.collectors.basic import BasicCollector\n",
|
||||
"\n",
|
||||
"# Set random seed to make example reproducible.\n",
|
||||
"random.seed(42)\n",
|
||||
"np.random.seed(42)\n",
|
||||
"\n",
|
||||
"# Generate a few instances of the traveling salesman problem.\n",
|
||||
"data = TravelingSalesmanGenerator(\n",
|
||||
" n=randint(low=10, high=11),\n",
|
||||
" x=uniform(loc=0.0, scale=1000.0),\n",
|
||||
" y=uniform(loc=0.0, scale=1000.0),\n",
|
||||
" gamma=uniform(loc=0.90, scale=0.20),\n",
|
||||
" fix_cities=True,\n",
|
||||
" round=True,\n",
|
||||
").generate(10)\n",
|
||||
"\n",
|
||||
"# Save instance data to data/tsp/00000.pkl.gz, data/tsp/00001.pkl.gz, ...\n",
|
||||
"save(data, \"data/tsp\")\n",
|
||||
"\n",
|
||||
"# Solve all instances and collect basic solution information. Process at most four\n",
|
||||
"# instances in parallel, with a per-instance time limit of one hour.\n",
|
||||
"bc = BasicCollector(time_limit_sec=3600)\n",
|
||||
"bc.collect(glob(\"data/tsp/*.pkl.gz\"), build_tsp_model, n_jobs=4)\n",
|
||||
"\n",
|
||||
"# Read and print some training data for the first instance.\n",
|
||||
"with H5File(\"data/tsp/00000.h5\", \"r\") as h5:\n",
|
||||
" print(\"lp_obj_value = \", h5.get_scalar(\"lp_obj_value\"))\n",
|
||||
" print(\"mip_obj_value = \", h5.get_scalar(\"mip_obj_value\"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "97ac9136",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.13"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
839
0.3/guide/collectors/index.html
Normal file
839
0.3/guide/collectors/index.html
Normal file
@@ -0,0 +1,839 @@
|
||||
|
||||
<!DOCTYPE html>
|
||||
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>2. Training Data Collectors — MIPLearn 0.3</title>
|
||||
|
||||
<link href="../../_static/css/theme.css" rel="stylesheet" />
|
||||
<link href="../../_static/css/index.c5995385ac14fb8791e8eb36b4908be2.css" rel="stylesheet" />
|
||||
|
||||
|
||||
<link rel="stylesheet"
|
||||
href="../../_static/vendor/fontawesome/5.13.0/css/all.min.css">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="../../_static/vendor/fontawesome/5.13.0/webfonts/fa-solid-900.woff2">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="../../_static/vendor/fontawesome/5.13.0/webfonts/fa-brands-400.woff2">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
|
||||
<link rel="stylesheet" href="../../_static/sphinx-book-theme.acff12b8f9c144ce68a297486a2fa670.css" type="text/css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/custom.css" />
|
||||
|
||||
<link rel="preload" as="script" href="../../_static/js/index.1c5a1a01449ed65a7b51.js">
|
||||
|
||||
<script id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
|
||||
<script src="../../_static/jquery.js"></script>
|
||||
<script src="../../_static/underscore.js"></script>
|
||||
<script src="../../_static/doctools.js"></script>
|
||||
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
|
||||
<script src="../../_static/sphinx-book-theme.12a9622fbb08dcb3a2a40b2c02b83a57.js"></script>
|
||||
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
|
||||
<script type="text/x-mathjax-config">MathJax.Hub.Config({"tex2jax": {"inlineMath": [["\\(", "\\)"]], "displayMath": [["\\[", "\\]"]], "processRefs": false, "processEnvironments": false}})</script>
|
||||
<link rel="index" title="Index" href="../../genindex/" />
|
||||
<link rel="search" title="Search" href="../../search/" />
|
||||
<link rel="next" title="3. Feature Extractors" href="../features/" />
|
||||
<link rel="prev" title="1. Benchmark Problems" href="../problems/" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="en" />
|
||||
|
||||
</head>
|
||||
<body data-spy="scroll" data-target="#bd-toc-nav" data-offset="80">
|
||||
|
||||
<div class="container-fluid" id="banner"></div>
|
||||
|
||||
|
||||
|
||||
<div class="container-xl">
|
||||
<div class="row">
|
||||
|
||||
<div class="col-12 col-md-3 bd-sidebar site-navigation show" id="site-navigation">
|
||||
|
||||
<div class="navbar-brand-box">
|
||||
<a class="navbar-brand text-wrap" href="../../">
|
||||
|
||||
|
||||
<h1 class="site-logo" id="site-title">MIPLearn 0.3</h1>
|
||||
|
||||
</a>
|
||||
</div><form class="bd-search d-flex align-items-center" action="../../search/" method="get">
|
||||
<i class="icon fas fa-search"></i>
|
||||
<input type="search" class="form-control" name="q" id="search-input" placeholder="Search the docs ..." aria-label="Search the docs ..." autocomplete="off" >
|
||||
</form><nav class="bd-links" id="bd-docs-nav" aria-label="Main navigation">
|
||||
<div class="bd-toc-item active">
|
||||
<p class="caption">
|
||||
<span class="caption-text">
|
||||
User Guide
|
||||
</span>
|
||||
</p>
|
||||
<ul class="current nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../problems/">
|
||||
1. Benchmark Problems
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1 current active">
|
||||
<a class="current reference internal" href="#">
|
||||
2. Training Data Collectors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../features/">
|
||||
3. Feature Extractors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../primal/">
|
||||
4. Primal Components
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../solvers/">
|
||||
5. Solvers
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p class="caption">
|
||||
<span class="caption-text">
|
||||
API Reference
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/problems/">
|
||||
6. Benchmark Problems
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/collectors/">
|
||||
7. Collectors & Extractors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/components/">
|
||||
8. Components
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/solvers/">
|
||||
9. Solvers
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/helpers/">
|
||||
10. Helpers
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav> <!-- To handle the deprecated key -->
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<main class="col py-md-3 pl-md-4 bd-content overflow-auto" role="main">
|
||||
|
||||
<div class="topbar container-xl fixed-top">
|
||||
<div class="topbar-contents row">
|
||||
<div class="col-12 col-md-3 bd-topbar-whitespace site-navigation show"></div>
|
||||
<div class="col pl-md-4 topbar-main">
|
||||
|
||||
<button id="navbar-toggler" class="navbar-toggler ml-0" type="button" data-toggle="collapse"
|
||||
data-toggle="tooltip" data-placement="bottom" data-target=".site-navigation" aria-controls="navbar-menu"
|
||||
aria-expanded="true" aria-label="Toggle navigation" aria-controls="site-navigation"
|
||||
title="Toggle navigation" data-toggle="tooltip" data-placement="left">
|
||||
<i class="fas fa-bars"></i>
|
||||
<i class="fas fa-arrow-left"></i>
|
||||
<i class="fas fa-arrow-up"></i>
|
||||
</button>
|
||||
|
||||
|
||||
<div class="dropdown-buttons-trigger">
|
||||
<button id="dropdown-buttons-trigger" class="btn btn-secondary topbarbtn" aria-label="Download this page"><i
|
||||
class="fas fa-download"></i></button>
|
||||
|
||||
<div class="dropdown-buttons">
|
||||
<!-- ipynb file if we had a myst markdown file -->
|
||||
|
||||
<!-- Download raw file -->
|
||||
<a class="dropdown-buttons" href="../../_sources/guide/collectors.ipynb.txt"><button type="button"
|
||||
class="btn btn-secondary topbarbtn" title="Download source file" data-toggle="tooltip"
|
||||
data-placement="left">.ipynb</button></a>
|
||||
<!-- Download PDF via print -->
|
||||
<button type="button" id="download-print" class="btn btn-secondary topbarbtn" title="Print to PDF"
|
||||
onClick="window.print()" data-toggle="tooltip" data-placement="left">.pdf</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Source interaction buttons -->
|
||||
|
||||
<!-- Full screen (wrap in <a> to have style consistency -->
|
||||
|
||||
<a class="full-screen-button"><button type="button" class="btn btn-secondary topbarbtn" data-toggle="tooltip"
|
||||
data-placement="bottom" onclick="toggleFullScreen()" aria-label="Fullscreen mode"
|
||||
title="Fullscreen mode"><i
|
||||
class="fas fa-expand"></i></button></a>
|
||||
|
||||
<!-- Launch buttons -->
|
||||
|
||||
</div>
|
||||
|
||||
<!-- Table of contents -->
|
||||
<div class="d-none d-md-block col-md-2 bd-toc show">
|
||||
|
||||
<div class="tocsection onthispage pt-5 pb-3">
|
||||
<i class="fas fa-list"></i> Contents
|
||||
</div>
|
||||
<nav id="bd-toc-nav">
|
||||
<ul class="visible nav section-nav flex-column">
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Overview">
|
||||
2.1. Overview
|
||||
</a>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#HDF5-Format">
|
||||
2.2. HDF5 Format
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Example">
|
||||
Example
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Basic-collector">
|
||||
2.3. Basic collector
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Data-fields">
|
||||
Data fields
|
||||
</a>
|
||||
</li>
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#id1">
|
||||
Example
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="main-content" class="row">
|
||||
<div class="col-12 col-md-9 pl-md-3 pr-md-0">
|
||||
|
||||
<div>
|
||||
|
||||
|
||||
<style>
|
||||
/* CSS for nbsphinx extension */
|
||||
|
||||
/* remove conflicting styling from Sphinx themes */
|
||||
div.nbinput.container div.prompt *,
|
||||
div.nboutput.container div.prompt *,
|
||||
div.nbinput.container div.input_area pre,
|
||||
div.nboutput.container div.output_area pre,
|
||||
div.nbinput.container div.input_area .highlight,
|
||||
div.nboutput.container div.output_area .highlight {
|
||||
border: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
box-shadow: none;
|
||||
}
|
||||
|
||||
div.nbinput.container > div[class*=highlight],
|
||||
div.nboutput.container > div[class*=highlight] {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt *,
|
||||
div.nboutput.container div.prompt * {
|
||||
background: none;
|
||||
}
|
||||
|
||||
div.nboutput.container div.output_area .highlight,
|
||||
div.nboutput.container div.output_area pre {
|
||||
background: unset;
|
||||
}
|
||||
|
||||
div.nboutput.container div.output_area div.highlight {
|
||||
color: unset; /* override Pygments text color */
|
||||
}
|
||||
|
||||
/* avoid gaps between output lines */
|
||||
div.nboutput.container div[class*=highlight] pre {
|
||||
line-height: normal;
|
||||
}
|
||||
|
||||
/* input/output containers */
|
||||
div.nbinput.container,
|
||||
div.nboutput.container {
|
||||
display: -webkit-flex;
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
margin: 0;
|
||||
width: 100%;
|
||||
}
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container,
|
||||
div.nboutput.container {
|
||||
flex-direction: column;
|
||||
}
|
||||
}
|
||||
|
||||
/* input container */
|
||||
div.nbinput.container {
|
||||
padding-top: 5px;
|
||||
}
|
||||
|
||||
/* last container */
|
||||
div.nblast.container {
|
||||
padding-bottom: 5px;
|
||||
}
|
||||
|
||||
/* input prompt */
|
||||
div.nbinput.container div.prompt pre {
|
||||
color: #307FC1;
|
||||
}
|
||||
|
||||
/* output prompt */
|
||||
div.nboutput.container div.prompt pre {
|
||||
color: #BF5B3D;
|
||||
}
|
||||
|
||||
/* all prompts */
|
||||
div.nbinput.container div.prompt,
|
||||
div.nboutput.container div.prompt {
|
||||
width: 4.5ex;
|
||||
padding-top: 5px;
|
||||
position: relative;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt > div,
|
||||
div.nboutput.container div.prompt > div {
|
||||
position: absolute;
|
||||
right: 0;
|
||||
margin-right: 0.3ex;
|
||||
}
|
||||
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container div.prompt,
|
||||
div.nboutput.container div.prompt {
|
||||
width: unset;
|
||||
text-align: left;
|
||||
padding: 0.4em;
|
||||
}
|
||||
div.nboutput.container div.prompt.empty {
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt > div,
|
||||
div.nboutput.container div.prompt > div {
|
||||
position: unset;
|
||||
}
|
||||
}
|
||||
|
||||
/* disable scrollbars and line breaks on prompts */
|
||||
div.nbinput.container div.prompt pre,
|
||||
div.nboutput.container div.prompt pre {
|
||||
overflow: hidden;
|
||||
white-space: pre;
|
||||
}
|
||||
|
||||
/* input/output area */
|
||||
div.nbinput.container div.input_area,
|
||||
div.nboutput.container div.output_area {
|
||||
-webkit-flex: 1;
|
||||
flex: 1;
|
||||
overflow: auto;
|
||||
}
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container div.input_area,
|
||||
div.nboutput.container div.output_area {
|
||||
width: 100%;
|
||||
}
|
||||
}
|
||||
|
||||
/* input area */
|
||||
div.nbinput.container div.input_area {
|
||||
border: 1px solid #e0e0e0;
|
||||
border-radius: 2px;
|
||||
/*background: #f5f5f5;*/
|
||||
}
|
||||
|
||||
/* override MathJax center alignment in output cells */
|
||||
div.nboutput.container div[class*=MathJax] {
|
||||
text-align: left !important;
|
||||
}
|
||||
|
||||
/* override sphinx.ext.imgmath center alignment in output cells */
|
||||
div.nboutput.container div.math p {
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
/* standard error */
|
||||
div.nboutput.container div.output_area.stderr {
|
||||
background: #fdd;
|
||||
}
|
||||
|
||||
/* ANSI colors */
|
||||
.ansi-black-fg { color: #3E424D; }
|
||||
.ansi-black-bg { background-color: #3E424D; }
|
||||
.ansi-black-intense-fg { color: #282C36; }
|
||||
.ansi-black-intense-bg { background-color: #282C36; }
|
||||
.ansi-red-fg { color: #E75C58; }
|
||||
.ansi-red-bg { background-color: #E75C58; }
|
||||
.ansi-red-intense-fg { color: #B22B31; }
|
||||
.ansi-red-intense-bg { background-color: #B22B31; }
|
||||
.ansi-green-fg { color: #00A250; }
|
||||
.ansi-green-bg { background-color: #00A250; }
|
||||
.ansi-green-intense-fg { color: #007427; }
|
||||
.ansi-green-intense-bg { background-color: #007427; }
|
||||
.ansi-yellow-fg { color: #DDB62B; }
|
||||
.ansi-yellow-bg { background-color: #DDB62B; }
|
||||
.ansi-yellow-intense-fg { color: #B27D12; }
|
||||
.ansi-yellow-intense-bg { background-color: #B27D12; }
|
||||
.ansi-blue-fg { color: #208FFB; }
|
||||
.ansi-blue-bg { background-color: #208FFB; }
|
||||
.ansi-blue-intense-fg { color: #0065CA; }
|
||||
.ansi-blue-intense-bg { background-color: #0065CA; }
|
||||
.ansi-magenta-fg { color: #D160C4; }
|
||||
.ansi-magenta-bg { background-color: #D160C4; }
|
||||
.ansi-magenta-intense-fg { color: #A03196; }
|
||||
.ansi-magenta-intense-bg { background-color: #A03196; }
|
||||
.ansi-cyan-fg { color: #60C6C8; }
|
||||
.ansi-cyan-bg { background-color: #60C6C8; }
|
||||
.ansi-cyan-intense-fg { color: #258F8F; }
|
||||
.ansi-cyan-intense-bg { background-color: #258F8F; }
|
||||
.ansi-white-fg { color: #C5C1B4; }
|
||||
.ansi-white-bg { background-color: #C5C1B4; }
|
||||
.ansi-white-intense-fg { color: #A1A6B2; }
|
||||
.ansi-white-intense-bg { background-color: #A1A6B2; }
|
||||
|
||||
.ansi-default-inverse-fg { color: #FFFFFF; }
|
||||
.ansi-default-inverse-bg { background-color: #000000; }
|
||||
|
||||
.ansi-bold { font-weight: bold; }
|
||||
.ansi-underline { text-decoration: underline; }
|
||||
|
||||
|
||||
div.nbinput.container div.input_area div[class*=highlight] > pre,
|
||||
div.nboutput.container div.output_area div[class*=highlight] > pre,
|
||||
div.nboutput.container div.output_area div[class*=highlight].math,
|
||||
div.nboutput.container div.output_area.rendered_html,
|
||||
div.nboutput.container div.output_area > div.output_javascript,
|
||||
div.nboutput.container div.output_area:not(.rendered_html) > img{
|
||||
padding: 5px;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
/* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
|
||||
div.nbinput.container div.input_area > div[class^='highlight'],
|
||||
div.nboutput.container div.output_area > div[class^='highlight']{
|
||||
overflow-y: hidden;
|
||||
}
|
||||
|
||||
/* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
|
||||
.prompt .copybtn {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* Some additional styling taken form the Jupyter notebook CSS */
|
||||
.jp-RenderedHTMLCommon table,
|
||||
div.rendered_html table {
|
||||
border: none;
|
||||
border-collapse: collapse;
|
||||
border-spacing: 0;
|
||||
color: black;
|
||||
font-size: 12px;
|
||||
table-layout: fixed;
|
||||
}
|
||||
.jp-RenderedHTMLCommon thead,
|
||||
div.rendered_html thead {
|
||||
border-bottom: 1px solid black;
|
||||
vertical-align: bottom;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tr,
|
||||
.jp-RenderedHTMLCommon th,
|
||||
.jp-RenderedHTMLCommon td,
|
||||
div.rendered_html tr,
|
||||
div.rendered_html th,
|
||||
div.rendered_html td {
|
||||
text-align: right;
|
||||
vertical-align: middle;
|
||||
padding: 0.5em 0.5em;
|
||||
line-height: normal;
|
||||
white-space: normal;
|
||||
max-width: none;
|
||||
border: none;
|
||||
}
|
||||
.jp-RenderedHTMLCommon th,
|
||||
div.rendered_html th {
|
||||
font-weight: bold;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tbody tr:nth-child(odd),
|
||||
div.rendered_html tbody tr:nth-child(odd) {
|
||||
background: #f5f5f5;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tbody tr:hover,
|
||||
div.rendered_html tbody tr:hover {
|
||||
background: rgba(66, 165, 245, 0.2);
|
||||
}
|
||||
</style>
|
||||
<div class="section" id="Training-Data-Collectors">
|
||||
<h1><span class="section-number">2. </span>Training Data Collectors<a class="headerlink" href="#Training-Data-Collectors" title="Permalink to this headline">¶</a></h1>
|
||||
<p>The first step in solving mixed-integer optimization problems with the assistance of supervised machine learning methods is solving a large set of training instances and collecting the raw training data. In this section, we describe the various training data collectors included in MIPLearn. Additionally, the framework follows the convention of storing all training data in files with a specific data format (namely, HDF5). In this section, we briefly describe this format and the rationale for
|
||||
choosing it.</p>
|
||||
<div class="section" id="Overview">
|
||||
<h2><span class="section-number">2.1. </span>Overview<a class="headerlink" href="#Overview" title="Permalink to this headline">¶</a></h2>
|
||||
<p>In MIPLearn, a <strong>collector</strong> is a class that solves or analyzes the problem and collects raw data which may be later useful for machine learning methods. Collectors, by convention, take as input: (i) a list of problem data filenames, in gzipped pickle format, ending with <code class="docutils literal notranslate"><span class="pre">.pkl.gz</span></code>; (ii) a function that builds the optimization model, such as <code class="docutils literal notranslate"><span class="pre">build_tsp_model</span></code>. After processing is done, collectors store the training data in a HDF5 file located alongside with the problem data. For example, if
|
||||
the problem data is stored in file <code class="docutils literal notranslate"><span class="pre">problem.pkl.gz</span></code>, then the collector writes to <code class="docutils literal notranslate"><span class="pre">problem.h5</span></code>. Collectors are, in general, very time consuming, as they may need to solve the problem to optimality, potentially multiple times.</p>
|
||||
</div>
|
||||
<div class="section" id="HDF5-Format">
|
||||
<h2><span class="section-number">2.2. </span>HDF5 Format<a class="headerlink" href="#HDF5-Format" title="Permalink to this headline">¶</a></h2>
|
||||
<p>MIPLearn stores all training data in <a class="reference external" href="HDF5">HDF5</a> (Hierarchical Data Format, Version 5) files. The HDF format was originally developed by the <a class="reference external" href="https://en.wikipedia.org/wiki/National_Center_for_Supercomputing_Applications">National Center for Supercomputing Applications</a> (NCSA) for storing and organizing large amounts of data, and supports a variety of data types, including integers, floating-point numbers, strings, and arrays. Compared to other formats, such as CSV, JSON or SQLite, the
|
||||
HDF5 format provides several advantages for MIPLearn, including:</p>
|
||||
<ul class="simple">
|
||||
<li><p><em>Storage of multiple scalars, vectors and matrices in a single file</em> — This allows MIPLearn to store all training data related to a given problem instance in a single file, which makes training data easier to store, organize and transfer.</p></li>
|
||||
<li><p><em>High-performance partial I/O</em> — Partial I/O allows MIPLearn to read a single element from the training data (e.g. value of the optimal solution) without loading the entire file to memory or reading it from beginning to end, which dramatically improves performance and reduces memory requirements. This is especially important when processing a large number of training data files.</p></li>
|
||||
<li><p><em>On-the-fly compression</em> — HDF5 files can be transparently compressed, using the gzip method, which reduces storage requirements and accelerates network transfers.</p></li>
|
||||
<li><p><em>Stable, portable and well-supported data format</em> — Training data files are typically expensive to generate. Having a stable and well supported data format ensures that these files remain usable in the future, potentially even by other non-Python MIP/ML frameworks.</p></li>
|
||||
</ul>
|
||||
<p>MIPLearn currently uses HDF5 as simple key-value storage for numerical data; more advanced features of the format, such as metadata, are not currently used. Although files generated by MIPLearn can be read with any HDF5 library, such as <a class="reference external" href="https://www.h5py.org/">h5py</a>, some convenience functions are provided to make the access more simple and less error-prone. Specifically, the class <a class="reference external" href="../../api/helpers/#miplearn.h5.H5File">H5File</a>, which is built on top of h5py, provides the methods
|
||||
<a class="reference external" href="../../api/helpers/#miplearn.h5.H5File.put_scalar">put_scalar</a>, <a class="reference external" href="../../api/helpers/#miplearn.h5.H5File.put_scalar">put_array</a>, <a class="reference external" href="../../api/helpers/#miplearn.h5.H5File.put_scalar">put_sparse</a>, <a class="reference external" href="../../api/helpers/#miplearn.h5.H5File.put_scalar">put_bytes</a> to store, respectively, scalar values, dense multi-dimensional arrays, sparse multi-dimensional arrays and arbitrary binary data. The corresponding <em>get</em> methods are also provided. Compared to pure h5py methods, these methods
|
||||
automatically perform type-checking and gzip compression. The example below shows their usage.</p>
|
||||
<div class="section" id="Example">
|
||||
<h3>Example<a class="headerlink" href="#Example" title="Permalink to this headline">¶</a></h3>
|
||||
<div class="nbinput docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[1]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">scipy.sparse</span>
|
||||
|
||||
<span class="kn">from</span> <span class="nn">miplearn.h5</span> <span class="kn">import</span> <span class="n">H5File</span>
|
||||
|
||||
<span class="c1"># Set random seed to make example reproducible</span>
|
||||
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Create a new empty HDF5 file</span>
|
||||
<span class="k">with</span> <span class="n">H5File</span><span class="p">(</span><span class="s2">"test.h5"</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">h5</span><span class="p">:</span>
|
||||
<span class="c1"># Store a scalar</span>
|
||||
<span class="n">h5</span><span class="o">.</span><span class="n">put_scalar</span><span class="p">(</span><span class="s2">"x1"</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">h5</span><span class="o">.</span><span class="n">put_scalar</span><span class="p">(</span><span class="s2">"x2"</span><span class="p">,</span> <span class="s2">"hello world"</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Store a dense array and a dense matrix</span>
|
||||
<span class="n">h5</span><span class="o">.</span><span class="n">put_array</span><span class="p">(</span><span class="s2">"x3"</span><span class="p">,</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]))</span>
|
||||
<span class="n">h5</span><span class="o">.</span><span class="n">put_array</span><span class="p">(</span><span class="s2">"x4"</span><span class="p">,</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">))</span>
|
||||
|
||||
<span class="c1"># Store a sparse matrix</span>
|
||||
<span class="n">h5</span><span class="o">.</span><span class="n">put_sparse</span><span class="p">(</span><span class="s2">"x5"</span><span class="p">,</span> <span class="n">scipy</span><span class="o">.</span><span class="n">sparse</span><span class="o">.</span><span class="n">random</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">))</span>
|
||||
|
||||
<span class="c1"># Re-open the file we just created and print</span>
|
||||
<span class="c1"># previously-stored data</span>
|
||||
<span class="k">with</span> <span class="n">H5File</span><span class="p">(</span><span class="s2">"test.h5"</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">h5</span><span class="p">:</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"x1 ="</span><span class="p">,</span> <span class="n">h5</span><span class="o">.</span><span class="n">get_scalar</span><span class="p">(</span><span class="s2">"x1"</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"x2 ="</span><span class="p">,</span> <span class="n">h5</span><span class="o">.</span><span class="n">get_scalar</span><span class="p">(</span><span class="s2">"x2"</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"x3 ="</span><span class="p">,</span> <span class="n">h5</span><span class="o">.</span><span class="n">get_array</span><span class="p">(</span><span class="s2">"x3"</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"x4 ="</span><span class="p">,</span> <span class="n">h5</span><span class="o">.</span><span class="n">get_array</span><span class="p">(</span><span class="s2">"x4"</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"x5 ="</span><span class="p">,</span> <span class="n">h5</span><span class="o">.</span><span class="n">get_sparse</span><span class="p">(</span><span class="s2">"x5"</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="nboutput nblast docutils container">
|
||||
<div class="prompt empty docutils container">
|
||||
</div>
|
||||
<div class="output_area docutils container">
|
||||
<div class="highlight"><pre>
|
||||
x1 = 1
|
||||
x2 = hello world
|
||||
x3 = [1 2 3]
|
||||
x4 = [[0.37454012 0.9507143 0.7319939 ]
|
||||
[0.5986585 0.15601864 0.15599452]
|
||||
[0.05808361 0.8661761 0.601115 ]]
|
||||
x5 = (2, 3) 0.68030757
|
||||
(3, 2) 0.45049927
|
||||
(4, 0) 0.013264962
|
||||
(0, 2) 0.94220173
|
||||
(4, 2) 0.5632882
|
||||
(2, 1) 0.3854165
|
||||
(1, 1) 0.015966251
|
||||
(3, 0) 0.23089382
|
||||
(4, 4) 0.24102546
|
||||
(1, 3) 0.68326354
|
||||
(3, 1) 0.6099967
|
||||
(0, 3) 0.8331949
|
||||
</pre></div></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="Basic-collector">
|
||||
<h2><span class="section-number">2.3. </span>Basic collector<a class="headerlink" href="#Basic-collector" title="Permalink to this headline">¶</a></h2>
|
||||
<p><a class="reference external" href="../../api/collectors/#miplearn.collectors.basic.BasicCollector">BasicCollector</a> is the most fundamental collector, and performs the following steps:</p>
|
||||
<ol class="arabic simple">
|
||||
<li><p>Extracts all model data, such as objective function and constraint right-hand sides into numpy arrays, which can later be easily and efficiently accessed without rebuilding the model or invoking the solver;</p></li>
|
||||
<li><p>Solves the linear relaxation of the problem and stores its optimal solution, basis status and sensitivity information, among other information;</p></li>
|
||||
<li><p>Solves the original mixed-integer optimization problem to optimality and stores its optimal solution, along with solve statistics, such as number of explored nodes and wallclock time.</p></li>
|
||||
</ol>
|
||||
<p>Data extracted in Phases 1, 2 and 3 above are prefixed, respectively as <code class="docutils literal notranslate"><span class="pre">static_</span></code>, <code class="docutils literal notranslate"><span class="pre">lp_</span></code> and <code class="docutils literal notranslate"><span class="pre">mip_</span></code>. The entire set of fields is shown in the table below.</p>
|
||||
<div class="section" id="Data-fields">
|
||||
<h3>Data fields<a class="headerlink" href="#Data-fields" title="Permalink to this headline">¶</a></h3>
|
||||
<table class="docutils align-default">
|
||||
<colgroup>
|
||||
<col style="width: 18%" />
|
||||
<col style="width: 11%" />
|
||||
<col style="width: 72%" />
|
||||
</colgroup>
|
||||
<thead>
|
||||
<tr class="row-odd"><th class="head"><p>Field</p></th>
|
||||
<th class="head"><p>Type</p></th>
|
||||
<th class="head"><p>Description</p></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">static_constr_lhs</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstrs,</span> <span class="pre">nvars)</span></code></p></td>
|
||||
<td><p>Constraint left-hand sides, in sparse matrix format</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">static_constr_names</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstrs,)</span></code></p></td>
|
||||
<td><p>Constraint names</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">static_constr_rhs</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstrs,)</span></code></p></td>
|
||||
<td><p>Constraint right-hand sides</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">static_constr_sense</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstrs,)</span></code></p></td>
|
||||
<td><p>Constraint senses (<code class="docutils literal notranslate"><span class="pre">"<"</span></code>, <code class="docutils literal notranslate"><span class="pre">">"</span></code> or <code class="docutils literal notranslate"><span class="pre">"="</span></code>)</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">static_obj_offset</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">float</span></code></p></td>
|
||||
<td><p>Constant value added to the objective function</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">static_sense</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">str</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">"min"</span></code> if minimization problem or <code class="docutils literal notranslate"><span class="pre">"max"</span></code> otherwise</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">static_var_lower_bounds</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Variable lower bounds</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">static_var_names</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Variable names</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">static_var_obj_coeffs</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Objective coefficients</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">static_var_types</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Types of the decision variables (<code class="docutils literal notranslate"><span class="pre">"C"</span></code>, <code class="docutils literal notranslate"><span class="pre">"B"</span></code> and <code class="docutils literal notranslate"><span class="pre">"I"</span></code> for continuous, binary and integer, respectively)</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">static_var_upper_bounds</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Variable upper bounds</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">lp_constr_basis_status</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstr,)</span></code></p></td>
|
||||
<td><p>Constraint basis status (<code class="docutils literal notranslate"><span class="pre">0</span></code> for basic, <code class="docutils literal notranslate"><span class="pre">-1</span></code> for non-basic)</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">lp_constr_dual_values</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstr,)</span></code></p></td>
|
||||
<td><p>Constraint dual value (or shadow price)</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">lp_constr_sa_rhs_{up,down}</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstr,)</span></code></p></td>
|
||||
<td><p>Sensitivity information for the constraint RHS</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">lp_constr_slacks</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstr,)</span></code></p></td>
|
||||
<td><p>Constraint slack in the solution to the LP relaxation</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">lp_obj_value</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">float</span></code></p></td>
|
||||
<td><p>Optimal value of the LP relaxation</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">lp_var_basis_status</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Variable basis status (<code class="docutils literal notranslate"><span class="pre">0</span></code>, <code class="docutils literal notranslate"><span class="pre">-1</span></code>, <code class="docutils literal notranslate"><span class="pre">-2</span></code> or <code class="docutils literal notranslate"><span class="pre">-3</span></code> for basic, non-basic at lower bound, non-basic at upper bound, and superbasic, respectively)</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">lp_var_reduced_costs</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Variable reduced costs</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">lp_var_sa_{obj,ub,lb}_{up,down}</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Sensitivity information for the variable objective coefficient, lower and upper bound.</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">lp_var_values</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Optimal solution to the LP relaxation</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">lp_wallclock_time</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">float</span></code></p></td>
|
||||
<td><p>Time taken to solve the LP relaxation (in seconds)</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">mip_constr_slacks</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nconstrs,)</span></code></p></td>
|
||||
<td><p>Constraint slacks in the best MIP solution</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">mip_gap</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">float</span></code></p></td>
|
||||
<td><p>Relative MIP optimality gap</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">mip_node_count</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">float</span></code></p></td>
|
||||
<td><p>Number of explored branch-and-bound nodes</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">mip_obj_bound</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">float</span></code></p></td>
|
||||
<td><p>Dual bound</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">mip_obj_value</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">float</span></code></p></td>
|
||||
<td><p>Value of the best MIP solution</p></td>
|
||||
</tr>
|
||||
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">mip_var_values</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">(nvars,)</span></code></p></td>
|
||||
<td><p>Best MIP solution</p></td>
|
||||
</tr>
|
||||
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">mip_wallclock_time</span></code></p></td>
|
||||
<td><p><code class="docutils literal notranslate"><span class="pre">float</span></code></p></td>
|
||||
<td><p>Time taken to solve the MIP (in seconds)</p></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<div class="section" id="id1">
|
||||
<h3>Example<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3>
|
||||
<p>The example below shows how to generate a few random instances of the traveling salesman problem, store its problem data, run the collector and print some of the training data to screen.</p>
|
||||
<div class="nbinput docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[2]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">random</span>
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">from</span> <span class="nn">scipy.stats</span> <span class="kn">import</span> <span class="n">uniform</span><span class="p">,</span> <span class="n">randint</span>
|
||||
<span class="kn">from</span> <span class="nn">glob</span> <span class="kn">import</span> <span class="n">glob</span>
|
||||
|
||||
<span class="kn">from</span> <span class="nn">miplearn.problems.tsp</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">TravelingSalesmanGenerator</span><span class="p">,</span>
|
||||
<span class="n">build_tsp_model</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.io</span> <span class="kn">import</span> <span class="n">save</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.h5</span> <span class="kn">import</span> <span class="n">H5File</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.collectors.basic</span> <span class="kn">import</span> <span class="n">BasicCollector</span>
|
||||
|
||||
<span class="c1"># Set random seed to make example reproducible.</span>
|
||||
<span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
|
||||
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Generate a few instances of the traveling salesman problem.</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">TravelingSalesmanGenerator</span><span class="p">(</span>
|
||||
<span class="n">n</span><span class="o">=</span><span class="n">randint</span><span class="p">(</span><span class="n">low</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">high</span><span class="o">=</span><span class="mi">11</span><span class="p">),</span>
|
||||
<span class="n">x</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.0</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">1000.0</span><span class="p">),</span>
|
||||
<span class="n">y</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.0</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">1000.0</span><span class="p">),</span>
|
||||
<span class="n">gamma</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.90</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">0.20</span><span class="p">),</span>
|
||||
<span class="n">fix_cities</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
<span class="nb">round</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
<span class="p">)</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Save instance data to data/tsp/00000.pkl.gz, data/tsp/00001.pkl.gz, ...</span>
|
||||
<span class="n">save</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="s2">"data/tsp"</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Solve all instances and collect basic solution information. Process at most four</span>
|
||||
<span class="c1"># instances in parallel, with a per-instance time limit of one hour.</span>
|
||||
<span class="n">bc</span> <span class="o">=</span> <span class="n">BasicCollector</span><span class="p">(</span><span class="n">time_limit_sec</span><span class="o">=</span><span class="mi">3600</span><span class="p">)</span>
|
||||
<span class="n">bc</span><span class="o">.</span><span class="n">collect</span><span class="p">(</span><span class="n">glob</span><span class="p">(</span><span class="s2">"data/tsp/*.pkl.gz"</span><span class="p">),</span> <span class="n">build_tsp_model</span><span class="p">,</span> <span class="n">n_jobs</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Read and print some training data for the first instance.</span>
|
||||
<span class="k">with</span> <span class="n">H5File</span><span class="p">(</span><span class="s2">"data/tsp/00000.h5"</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">h5</span><span class="p">:</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"lp_obj_value = "</span><span class="p">,</span> <span class="n">h5</span><span class="o">.</span><span class="n">get_scalar</span><span class="p">(</span><span class="s2">"lp_obj_value"</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"mip_obj_value = "</span><span class="p">,</span> <span class="n">h5</span><span class="o">.</span><span class="n">get_scalar</span><span class="p">(</span><span class="s2">"mip_obj_value"</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="nboutput docutils container">
|
||||
<div class="prompt empty docutils container">
|
||||
</div>
|
||||
<div class="output_area stderr docutils container">
|
||||
<div class="highlight"><pre>
|
||||
/home/axavier/Software/anaconda3/envs/miplearn/lib/python3.8/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
|
||||
from .autonotebook import tqdm as notebook_tqdm
|
||||
</pre></div></div>
|
||||
</div>
|
||||
<div class="nboutput nblast docutils container">
|
||||
<div class="prompt empty docutils container">
|
||||
</div>
|
||||
<div class="output_area docutils container">
|
||||
<div class="highlight"><pre>
|
||||
Removing empty/corrupted h5 file: data/tsp/00000.h5
|
||||
lp_obj_value = 2909.0
|
||||
mip_obj_value = 2921.0
|
||||
</pre></div></div>
|
||||
</div>
|
||||
<div class="nbinput nblast docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[2]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<div class='prev-next-bottom'>
|
||||
|
||||
<a class='left-prev' id="prev-link" href="../problems/" title="previous page"><span class="section-number">1. </span>Benchmark Problems</a>
|
||||
<a class='right-next' id="next-link" href="../features/" title="next page"><span class="section-number">3. </span>Feature Extractors</a>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<footer class="footer mt-5 mt-md-0">
|
||||
<div class="container">
|
||||
<p>
|
||||
|
||||
© Copyright 2020-2022, UChicago Argonne, LLC.<br/>
|
||||
</p>
|
||||
</div>
|
||||
</footer>
|
||||
</main>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="../../_static/js/index.1c5a1a01449ed65a7b51.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
343
0.3/guide/features.ipynb
Normal file
343
0.3/guide/features.ipynb
Normal file
@@ -0,0 +1,343 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cdc6ebe9-d1d4-4de1-9b5a-4fc8ef57b11b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Feature Extractors\n",
|
||||
"\n",
|
||||
"In the previous page, we introduced *training data collectors*, which solve the optimization problem and collect raw training data, such as the optimal solution. In this page, we introduce **feature extractors**, which take the raw training data, stored in HDF5 files, and extract relevant information in order to train a machine learning model. We describe the extractors readily available in MIPLearn."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "599ac2f7",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"\n",
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"Feature extraction is an important step of the process of building a machine learning model because it helps to reduce the complexity of the data and convert it into a format that is more easily processed. Previous research has proposed converting absolute variable coefficients, for example, into relative values which are invariant to various transformations, such as problem scaling, making them more amenable to learning. Various other transformations have also been described.\n",
|
||||
"\n",
|
||||
"In the framework, we treat data collection and feature extraction as two separate steps to accelerate the model development cycle. Specifically, collectors are typically time-consuming, as they often need to solve the problem to optimality, and therefore focus on collecting and storing all data that may or may not be relevant, in its raw format. Feature extractors, on the other hand, focus entirely on filtering the data and improving its representation, and are therefore much faster to run. Experimenting with new data representations, therefore, can be done without resolving the instances.\n",
|
||||
"\n",
|
||||
"In MIPLearn, extractors implement the abstract class [FeatureExtractor][FeatureExtractor], which has methods that take as input an [H5File][H5File] and produce either: (i) instance features, which describe the entire instances; (ii) variable features, which describe a particular decision variables; or (iii) constraint features, which describe a particular constraint. The extractor is free to implement only a subset of these methods, if it is known that it will not be used with a machine learning component that requires the other types of features.\n",
|
||||
"\n",
|
||||
"[FeatureExtractor]: ../../api/collectors/#miplearn.features.fields.FeaturesExtractor\n",
|
||||
"[H5File]: ../../api/helpers/#miplearn.h5.H5File"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "60efe8f5",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"\n",
|
||||
"## H5FieldsExtractor\n",
|
||||
"\n",
|
||||
"[H5FieldsExtractor][H5FieldsExtractor], the most simple extractor in MIPLearn, simple extracts data that is already available in the HDF5 file, assembles it into a matrix and returns it as-is. The fields used to build instance, variable and constraint features are user-specified. The class also performs checks to ensure that the shapes of the returned matrices make sense."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "dfc32e59",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
"\n",
|
||||
"The example below demonstrates the usage of H5FieldsExtractor in a randomly generated instance of the multi-dimensional knapsack problem."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "ed2f5006",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"instance features (11,) \n",
|
||||
" [-1531.24308771 -350. -692. -454.\n",
|
||||
" -709. -605. -543. -321.\n",
|
||||
" -674. -571. -341. ]\n",
|
||||
"variable features (10, 4) \n",
|
||||
" [[-1.53124309e+03 -3.50000000e+02 0.00000000e+00 9.43468018e+01]\n",
|
||||
" [-1.53124309e+03 -6.92000000e+02 2.51703322e-01 0.00000000e+00]\n",
|
||||
" [-1.53124309e+03 -4.54000000e+02 0.00000000e+00 8.25504150e+01]\n",
|
||||
" [-1.53124309e+03 -7.09000000e+02 1.11373022e-01 0.00000000e+00]\n",
|
||||
" [-1.53124309e+03 -6.05000000e+02 1.00000000e+00 -1.26055283e+02]\n",
|
||||
" [-1.53124309e+03 -5.43000000e+02 0.00000000e+00 1.68693771e+02]\n",
|
||||
" [-1.53124309e+03 -3.21000000e+02 1.07488781e-01 0.00000000e+00]\n",
|
||||
" [-1.53124309e+03 -6.74000000e+02 8.82293701e-01 0.00000000e+00]\n",
|
||||
" [-1.53124309e+03 -5.71000000e+02 0.00000000e+00 1.41129074e+02]\n",
|
||||
" [-1.53124309e+03 -3.41000000e+02 1.28830120e-01 0.00000000e+00]]\n",
|
||||
"constraint features (5, 3) \n",
|
||||
" [[ 1.3100000e+03 -1.5978307e-01 0.0000000e+00]\n",
|
||||
" [ 9.8800000e+02 -3.2881632e-01 0.0000000e+00]\n",
|
||||
" [ 1.0040000e+03 -4.0601316e-01 0.0000000e+00]\n",
|
||||
" [ 1.2690000e+03 -1.3659772e-01 0.0000000e+00]\n",
|
||||
" [ 1.0070000e+03 -2.8800571e-01 0.0000000e+00]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from glob import glob\n",
|
||||
"from shutil import rmtree\n",
|
||||
"\n",
|
||||
"import numpy as np\n",
|
||||
"from scipy.stats import uniform, randint\n",
|
||||
"\n",
|
||||
"from miplearn.collectors.basic import BasicCollector\n",
|
||||
"from miplearn.features.fields import H5FieldsExtractor\n",
|
||||
"from miplearn.h5 import H5File\n",
|
||||
"from miplearn.io import save\n",
|
||||
"from miplearn.problems.multiknapsack import (\n",
|
||||
" MultiKnapsackGenerator,\n",
|
||||
" build_multiknapsack_model\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Set random seed to make example reproducible\n",
|
||||
"np.random.seed(42)\n",
|
||||
"\n",
|
||||
"# Generate some random multiknapsack instances\n",
|
||||
"rmtree(\"data/multiknapsack/\", ignore_errors=True)\n",
|
||||
"save(\n",
|
||||
" MultiKnapsackGenerator(\n",
|
||||
" n=randint(low=10, high=11),\n",
|
||||
" m=randint(low=5, high=6),\n",
|
||||
" w=uniform(loc=0, scale=1000),\n",
|
||||
" K=uniform(loc=100, scale=0),\n",
|
||||
" u=uniform(loc=1, scale=0),\n",
|
||||
" alpha=uniform(loc=0.25, scale=0),\n",
|
||||
" w_jitter=uniform(loc=0.95, scale=0.1),\n",
|
||||
" p_jitter=uniform(loc=0.75, scale=0.5),\n",
|
||||
" fix_w=True,\n",
|
||||
" ).generate(10),\n",
|
||||
" \"data/multiknapsack\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Run the basic collector\n",
|
||||
"BasicCollector().collect(\n",
|
||||
" glob(\"data/multiknapsack/*\"),\n",
|
||||
" build_multiknapsack_model,\n",
|
||||
" n_jobs=4,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"ext = H5FieldsExtractor(\n",
|
||||
" # Use as instance features the value of the LP relaxation and the\n",
|
||||
" # vector of objective coefficients.\n",
|
||||
" instance_fields=[\n",
|
||||
" \"lp_obj_value\",\n",
|
||||
" \"static_var_obj_coeffs\",\n",
|
||||
" ],\n",
|
||||
" # For each variable, use as features the optimal value of the LP\n",
|
||||
" # relaxation, the variable objective coefficient, the variable's\n",
|
||||
" # value its reduced cost.\n",
|
||||
" var_fields=[\n",
|
||||
" \"lp_obj_value\",\n",
|
||||
" \"static_var_obj_coeffs\",\n",
|
||||
" \"lp_var_values\",\n",
|
||||
" \"lp_var_reduced_costs\",\n",
|
||||
" ],\n",
|
||||
" # For each constraint, use as features the RHS, dual value and slack.\n",
|
||||
" constr_fields=[\n",
|
||||
" \"static_constr_rhs\",\n",
|
||||
" \"lp_constr_dual_values\",\n",
|
||||
" \"lp_constr_slacks\",\n",
|
||||
" ]\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"with H5File(\"data/multiknapsack/00000.h5\") as h5:\n",
|
||||
" # Extract and print instance features\n",
|
||||
" x1 = ext.get_instance_features(h5)\n",
|
||||
" print(\"instance features\", x1.shape, \"\\n\", x1)\n",
|
||||
"\n",
|
||||
" # Extract and print variable features\n",
|
||||
" x2 = ext.get_var_features(h5)\n",
|
||||
" print(\"variable features\", x2.shape, \"\\n\", x2)\n",
|
||||
"\n",
|
||||
" # Extract and print constraint features\n",
|
||||
" x3 = ext.get_constr_features(h5)\n",
|
||||
" print(\"constraint features\", x3.shape, \"\\n\", x3)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "35dc7ce3",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"\n",
|
||||
"[H5FieldsExtractor]: ../../api/collectors/#miplearn.features.fields.H5FieldsExtractor"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cc32efd6",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"<div class=\"alert alert-warning\">\n",
|
||||
"Warning\n",
|
||||
"\n",
|
||||
"You should ensure that the number of features remains the same for all relevant HDF5 files. In the previous example, to illustrate this issue, we used variable objective coefficients as instance features. While this is allowed, note that this requires all problem instances to have the same number of variables; otherwise the number of features would vary from instance to instance and MIPLearn would be unable to concatenate the matrices.\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2e3ba4fe",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"## AlvLouWeh2017Extractor\n",
|
||||
"\n",
|
||||
"Alvarez, Louveaux and Wehenkel (2017) proposed a set features to describe a particular decision variable in a given node of the branch-and-bound tree, and applied it to the problem of mimicking strong branching decisions. The class [AlvLouWeh2017Extractor][] implements a subset of these features (40 out of 64), which are available outside of the branch-and-bound tree. Some features are derived from the static defintion of the problem (i.e. from objective function and constraint data), while some features are derived from the solution to the LP relaxation. The features have been designed to be: (i) independent of the size of the problem; (ii) invariant with respect to irrelevant problem transformations, such as row and column permutation; and (iii) independent of the scale of the problem. We refer to the paper for a more complete description.\n",
|
||||
"\n",
|
||||
"### Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "4944a4a4",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"x1 (10, 40) \n",
|
||||
" [[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 0.00e+00 1.00e+00 6.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 6.00e-01 1.00e+00 1.75e+01 1.00e+00 2.00e-01\n",
|
||||
" 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 -1.00e+00 0.00e+00 1.00e+20]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 1.00e-01 1.00e+00 1.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 7.00e-01 1.00e+00 5.10e+00 1.00e+00 2.00e-01\n",
|
||||
" 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 3.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 0.00e+00 1.00e+00 9.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 5.00e-01 1.00e+00 1.30e+01 1.00e+00 2.00e-01\n",
|
||||
" 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 -1.00e+00 0.00e+00 1.00e+20]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 2.00e-01 1.00e+00 9.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 8.00e-01 1.00e+00 3.40e+00 1.00e+00 2.00e-01\n",
|
||||
" 1.00e+00 1.00e-01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 1.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 1.00e-01 1.00e+00 7.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 6.00e-01 1.00e+00 3.80e+00 1.00e+00 2.00e-01\n",
|
||||
" 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 1.00e-01 1.00e+00 8.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 7.00e-01 1.00e+00 3.30e+00 1.00e+00 2.00e-01\n",
|
||||
" 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 -1.00e+00 0.00e+00 1.00e+20]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 0.00e+00 1.00e+00 3.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 1.00e+00 1.00e+00 5.70e+00 1.00e+00 1.00e-01\n",
|
||||
" 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 1.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 1.00e-01 1.00e+00 6.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 8.00e-01 1.00e+00 6.80e+00 1.00e+00 2.00e-01\n",
|
||||
" 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 1.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 4.00e-01 1.00e+00 6.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 8.00e-01 1.00e+00 1.40e+00 1.00e+00 1.00e-01\n",
|
||||
" 1.00e+00 1.00e-01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 -1.00e+00 0.00e+00 1.00e+20]\n",
|
||||
" [-1.00e+00 1.00e+20 1.00e-01 1.00e+00 0.00e+00 1.00e+00 5.00e-01\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 1.00e+00 5.00e-01 1.00e+00 7.60e+00 1.00e+00 1.00e-01\n",
|
||||
" 1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00\n",
|
||||
" 1.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from miplearn.features.AlvLouWeh2017 import AlvLouWeh2017Extractor\n",
|
||||
"from miplearn.h5 import H5File\n",
|
||||
"\n",
|
||||
"# Build the extractor\n",
|
||||
"ext = AlvLouWeh2017Extractor()\n",
|
||||
"\n",
|
||||
"# Open previously-created multiknapsack training data\n",
|
||||
"with H5File(\"data/multiknapsack/00000.h5\") as h5:\n",
|
||||
"\n",
|
||||
" # Extract and print variable features\n",
|
||||
" x1 = ext.get_var_features(h5)\n",
|
||||
" print(\"x1\", x1.shape, \"\\n\", x1.round(1))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "04eab813",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"<div class=\"alert alert-info\">\n",
|
||||
"References\n",
|
||||
"\n",
|
||||
"* **Alvarez, Alejandro Marcos.** *Computational and theoretical synergies between linear optimization and supervised machine learning.* (2016). University of Liège.\n",
|
||||
"* **Alvarez, Alejandro Marcos, Quentin Louveaux, and Louis Wehenkel.** *A machine learning-based approximation of strong branching.* INFORMS Journal on Computing 29.1 (2017): 185-195.\n",
|
||||
"\n",
|
||||
"</div>"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.13"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
768
0.3/guide/features/index.html
Normal file
768
0.3/guide/features/index.html
Normal file
@@ -0,0 +1,768 @@
|
||||
|
||||
<!DOCTYPE html>
|
||||
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>3. Feature Extractors — MIPLearn 0.3</title>
|
||||
|
||||
<link href="../../_static/css/theme.css" rel="stylesheet" />
|
||||
<link href="../../_static/css/index.c5995385ac14fb8791e8eb36b4908be2.css" rel="stylesheet" />
|
||||
|
||||
|
||||
<link rel="stylesheet"
|
||||
href="../../_static/vendor/fontawesome/5.13.0/css/all.min.css">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="../../_static/vendor/fontawesome/5.13.0/webfonts/fa-solid-900.woff2">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="../../_static/vendor/fontawesome/5.13.0/webfonts/fa-brands-400.woff2">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
|
||||
<link rel="stylesheet" href="../../_static/sphinx-book-theme.acff12b8f9c144ce68a297486a2fa670.css" type="text/css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/custom.css" />
|
||||
|
||||
<link rel="preload" as="script" href="../../_static/js/index.1c5a1a01449ed65a7b51.js">
|
||||
|
||||
<script id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
|
||||
<script src="../../_static/jquery.js"></script>
|
||||
<script src="../../_static/underscore.js"></script>
|
||||
<script src="../../_static/doctools.js"></script>
|
||||
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
|
||||
<script src="../../_static/sphinx-book-theme.12a9622fbb08dcb3a2a40b2c02b83a57.js"></script>
|
||||
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
|
||||
<script type="text/x-mathjax-config">MathJax.Hub.Config({"tex2jax": {"inlineMath": [["\\(", "\\)"]], "displayMath": [["\\[", "\\]"]], "processRefs": false, "processEnvironments": false}})</script>
|
||||
<link rel="index" title="Index" href="../../genindex/" />
|
||||
<link rel="search" title="Search" href="../../search/" />
|
||||
<link rel="next" title="4. Primal Components" href="../primal/" />
|
||||
<link rel="prev" title="2. Training Data Collectors" href="../collectors/" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="en" />
|
||||
|
||||
</head>
|
||||
<body data-spy="scroll" data-target="#bd-toc-nav" data-offset="80">
|
||||
|
||||
<div class="container-fluid" id="banner"></div>
|
||||
|
||||
|
||||
|
||||
<div class="container-xl">
|
||||
<div class="row">
|
||||
|
||||
<div class="col-12 col-md-3 bd-sidebar site-navigation show" id="site-navigation">
|
||||
|
||||
<div class="navbar-brand-box">
|
||||
<a class="navbar-brand text-wrap" href="../../">
|
||||
|
||||
|
||||
<h1 class="site-logo" id="site-title">MIPLearn 0.3</h1>
|
||||
|
||||
</a>
|
||||
</div><form class="bd-search d-flex align-items-center" action="../../search/" method="get">
|
||||
<i class="icon fas fa-search"></i>
|
||||
<input type="search" class="form-control" name="q" id="search-input" placeholder="Search the docs ..." aria-label="Search the docs ..." autocomplete="off" >
|
||||
</form><nav class="bd-links" id="bd-docs-nav" aria-label="Main navigation">
|
||||
<div class="bd-toc-item active">
|
||||
<p class="caption">
|
||||
<span class="caption-text">
|
||||
User Guide
|
||||
</span>
|
||||
</p>
|
||||
<ul class="current nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../problems/">
|
||||
1. Benchmark Problems
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../collectors/">
|
||||
2. Training Data Collectors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1 current active">
|
||||
<a class="current reference internal" href="#">
|
||||
3. Feature Extractors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../primal/">
|
||||
4. Primal Components
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../solvers/">
|
||||
5. Solvers
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p class="caption">
|
||||
<span class="caption-text">
|
||||
API Reference
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/problems/">
|
||||
6. Benchmark Problems
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/collectors/">
|
||||
7. Collectors & Extractors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/components/">
|
||||
8. Components
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/solvers/">
|
||||
9. Solvers
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/helpers/">
|
||||
10. Helpers
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav> <!-- To handle the deprecated key -->
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<main class="col py-md-3 pl-md-4 bd-content overflow-auto" role="main">
|
||||
|
||||
<div class="topbar container-xl fixed-top">
|
||||
<div class="topbar-contents row">
|
||||
<div class="col-12 col-md-3 bd-topbar-whitespace site-navigation show"></div>
|
||||
<div class="col pl-md-4 topbar-main">
|
||||
|
||||
<button id="navbar-toggler" class="navbar-toggler ml-0" type="button" data-toggle="collapse"
|
||||
data-toggle="tooltip" data-placement="bottom" data-target=".site-navigation" aria-controls="navbar-menu"
|
||||
aria-expanded="true" aria-label="Toggle navigation" aria-controls="site-navigation"
|
||||
title="Toggle navigation" data-toggle="tooltip" data-placement="left">
|
||||
<i class="fas fa-bars"></i>
|
||||
<i class="fas fa-arrow-left"></i>
|
||||
<i class="fas fa-arrow-up"></i>
|
||||
</button>
|
||||
|
||||
|
||||
<div class="dropdown-buttons-trigger">
|
||||
<button id="dropdown-buttons-trigger" class="btn btn-secondary topbarbtn" aria-label="Download this page"><i
|
||||
class="fas fa-download"></i></button>
|
||||
|
||||
<div class="dropdown-buttons">
|
||||
<!-- ipynb file if we had a myst markdown file -->
|
||||
|
||||
<!-- Download raw file -->
|
||||
<a class="dropdown-buttons" href="../../_sources/guide/features.ipynb.txt"><button type="button"
|
||||
class="btn btn-secondary topbarbtn" title="Download source file" data-toggle="tooltip"
|
||||
data-placement="left">.ipynb</button></a>
|
||||
<!-- Download PDF via print -->
|
||||
<button type="button" id="download-print" class="btn btn-secondary topbarbtn" title="Print to PDF"
|
||||
onClick="window.print()" data-toggle="tooltip" data-placement="left">.pdf</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Source interaction buttons -->
|
||||
|
||||
<!-- Full screen (wrap in <a> to have style consistency -->
|
||||
|
||||
<a class="full-screen-button"><button type="button" class="btn btn-secondary topbarbtn" data-toggle="tooltip"
|
||||
data-placement="bottom" onclick="toggleFullScreen()" aria-label="Fullscreen mode"
|
||||
title="Fullscreen mode"><i
|
||||
class="fas fa-expand"></i></button></a>
|
||||
|
||||
<!-- Launch buttons -->
|
||||
|
||||
</div>
|
||||
|
||||
<!-- Table of contents -->
|
||||
<div class="d-none d-md-block col-md-2 bd-toc show">
|
||||
|
||||
<div class="tocsection onthispage pt-5 pb-3">
|
||||
<i class="fas fa-list"></i> Contents
|
||||
</div>
|
||||
<nav id="bd-toc-nav">
|
||||
<ul class="visible nav section-nav flex-column">
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Overview">
|
||||
3.1. Overview
|
||||
</a>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#H5FieldsExtractor">
|
||||
3.2. H5FieldsExtractor
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Example">
|
||||
Example
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#AlvLouWeh2017Extractor">
|
||||
3.3. AlvLouWeh2017Extractor
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#id1">
|
||||
Example
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="main-content" class="row">
|
||||
<div class="col-12 col-md-9 pl-md-3 pr-md-0">
|
||||
|
||||
<div>
|
||||
|
||||
|
||||
<style>
|
||||
/* CSS for nbsphinx extension */
|
||||
|
||||
/* remove conflicting styling from Sphinx themes */
|
||||
div.nbinput.container div.prompt *,
|
||||
div.nboutput.container div.prompt *,
|
||||
div.nbinput.container div.input_area pre,
|
||||
div.nboutput.container div.output_area pre,
|
||||
div.nbinput.container div.input_area .highlight,
|
||||
div.nboutput.container div.output_area .highlight {
|
||||
border: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
box-shadow: none;
|
||||
}
|
||||
|
||||
div.nbinput.container > div[class*=highlight],
|
||||
div.nboutput.container > div[class*=highlight] {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt *,
|
||||
div.nboutput.container div.prompt * {
|
||||
background: none;
|
||||
}
|
||||
|
||||
div.nboutput.container div.output_area .highlight,
|
||||
div.nboutput.container div.output_area pre {
|
||||
background: unset;
|
||||
}
|
||||
|
||||
div.nboutput.container div.output_area div.highlight {
|
||||
color: unset; /* override Pygments text color */
|
||||
}
|
||||
|
||||
/* avoid gaps between output lines */
|
||||
div.nboutput.container div[class*=highlight] pre {
|
||||
line-height: normal;
|
||||
}
|
||||
|
||||
/* input/output containers */
|
||||
div.nbinput.container,
|
||||
div.nboutput.container {
|
||||
display: -webkit-flex;
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
margin: 0;
|
||||
width: 100%;
|
||||
}
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container,
|
||||
div.nboutput.container {
|
||||
flex-direction: column;
|
||||
}
|
||||
}
|
||||
|
||||
/* input container */
|
||||
div.nbinput.container {
|
||||
padding-top: 5px;
|
||||
}
|
||||
|
||||
/* last container */
|
||||
div.nblast.container {
|
||||
padding-bottom: 5px;
|
||||
}
|
||||
|
||||
/* input prompt */
|
||||
div.nbinput.container div.prompt pre {
|
||||
color: #307FC1;
|
||||
}
|
||||
|
||||
/* output prompt */
|
||||
div.nboutput.container div.prompt pre {
|
||||
color: #BF5B3D;
|
||||
}
|
||||
|
||||
/* all prompts */
|
||||
div.nbinput.container div.prompt,
|
||||
div.nboutput.container div.prompt {
|
||||
width: 4.5ex;
|
||||
padding-top: 5px;
|
||||
position: relative;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt > div,
|
||||
div.nboutput.container div.prompt > div {
|
||||
position: absolute;
|
||||
right: 0;
|
||||
margin-right: 0.3ex;
|
||||
}
|
||||
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container div.prompt,
|
||||
div.nboutput.container div.prompt {
|
||||
width: unset;
|
||||
text-align: left;
|
||||
padding: 0.4em;
|
||||
}
|
||||
div.nboutput.container div.prompt.empty {
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt > div,
|
||||
div.nboutput.container div.prompt > div {
|
||||
position: unset;
|
||||
}
|
||||
}
|
||||
|
||||
/* disable scrollbars and line breaks on prompts */
|
||||
div.nbinput.container div.prompt pre,
|
||||
div.nboutput.container div.prompt pre {
|
||||
overflow: hidden;
|
||||
white-space: pre;
|
||||
}
|
||||
|
||||
/* input/output area */
|
||||
div.nbinput.container div.input_area,
|
||||
div.nboutput.container div.output_area {
|
||||
-webkit-flex: 1;
|
||||
flex: 1;
|
||||
overflow: auto;
|
||||
}
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container div.input_area,
|
||||
div.nboutput.container div.output_area {
|
||||
width: 100%;
|
||||
}
|
||||
}
|
||||
|
||||
/* input area */
|
||||
div.nbinput.container div.input_area {
|
||||
border: 1px solid #e0e0e0;
|
||||
border-radius: 2px;
|
||||
/*background: #f5f5f5;*/
|
||||
}
|
||||
|
||||
/* override MathJax center alignment in output cells */
|
||||
div.nboutput.container div[class*=MathJax] {
|
||||
text-align: left !important;
|
||||
}
|
||||
|
||||
/* override sphinx.ext.imgmath center alignment in output cells */
|
||||
div.nboutput.container div.math p {
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
/* standard error */
|
||||
div.nboutput.container div.output_area.stderr {
|
||||
background: #fdd;
|
||||
}
|
||||
|
||||
/* ANSI colors */
|
||||
.ansi-black-fg { color: #3E424D; }
|
||||
.ansi-black-bg { background-color: #3E424D; }
|
||||
.ansi-black-intense-fg { color: #282C36; }
|
||||
.ansi-black-intense-bg { background-color: #282C36; }
|
||||
.ansi-red-fg { color: #E75C58; }
|
||||
.ansi-red-bg { background-color: #E75C58; }
|
||||
.ansi-red-intense-fg { color: #B22B31; }
|
||||
.ansi-red-intense-bg { background-color: #B22B31; }
|
||||
.ansi-green-fg { color: #00A250; }
|
||||
.ansi-green-bg { background-color: #00A250; }
|
||||
.ansi-green-intense-fg { color: #007427; }
|
||||
.ansi-green-intense-bg { background-color: #007427; }
|
||||
.ansi-yellow-fg { color: #DDB62B; }
|
||||
.ansi-yellow-bg { background-color: #DDB62B; }
|
||||
.ansi-yellow-intense-fg { color: #B27D12; }
|
||||
.ansi-yellow-intense-bg { background-color: #B27D12; }
|
||||
.ansi-blue-fg { color: #208FFB; }
|
||||
.ansi-blue-bg { background-color: #208FFB; }
|
||||
.ansi-blue-intense-fg { color: #0065CA; }
|
||||
.ansi-blue-intense-bg { background-color: #0065CA; }
|
||||
.ansi-magenta-fg { color: #D160C4; }
|
||||
.ansi-magenta-bg { background-color: #D160C4; }
|
||||
.ansi-magenta-intense-fg { color: #A03196; }
|
||||
.ansi-magenta-intense-bg { background-color: #A03196; }
|
||||
.ansi-cyan-fg { color: #60C6C8; }
|
||||
.ansi-cyan-bg { background-color: #60C6C8; }
|
||||
.ansi-cyan-intense-fg { color: #258F8F; }
|
||||
.ansi-cyan-intense-bg { background-color: #258F8F; }
|
||||
.ansi-white-fg { color: #C5C1B4; }
|
||||
.ansi-white-bg { background-color: #C5C1B4; }
|
||||
.ansi-white-intense-fg { color: #A1A6B2; }
|
||||
.ansi-white-intense-bg { background-color: #A1A6B2; }
|
||||
|
||||
.ansi-default-inverse-fg { color: #FFFFFF; }
|
||||
.ansi-default-inverse-bg { background-color: #000000; }
|
||||
|
||||
.ansi-bold { font-weight: bold; }
|
||||
.ansi-underline { text-decoration: underline; }
|
||||
|
||||
|
||||
div.nbinput.container div.input_area div[class*=highlight] > pre,
|
||||
div.nboutput.container div.output_area div[class*=highlight] > pre,
|
||||
div.nboutput.container div.output_area div[class*=highlight].math,
|
||||
div.nboutput.container div.output_area.rendered_html,
|
||||
div.nboutput.container div.output_area > div.output_javascript,
|
||||
div.nboutput.container div.output_area:not(.rendered_html) > img{
|
||||
padding: 5px;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
/* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
|
||||
div.nbinput.container div.input_area > div[class^='highlight'],
|
||||
div.nboutput.container div.output_area > div[class^='highlight']{
|
||||
overflow-y: hidden;
|
||||
}
|
||||
|
||||
/* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
|
||||
.prompt .copybtn {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* Some additional styling taken form the Jupyter notebook CSS */
|
||||
.jp-RenderedHTMLCommon table,
|
||||
div.rendered_html table {
|
||||
border: none;
|
||||
border-collapse: collapse;
|
||||
border-spacing: 0;
|
||||
color: black;
|
||||
font-size: 12px;
|
||||
table-layout: fixed;
|
||||
}
|
||||
.jp-RenderedHTMLCommon thead,
|
||||
div.rendered_html thead {
|
||||
border-bottom: 1px solid black;
|
||||
vertical-align: bottom;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tr,
|
||||
.jp-RenderedHTMLCommon th,
|
||||
.jp-RenderedHTMLCommon td,
|
||||
div.rendered_html tr,
|
||||
div.rendered_html th,
|
||||
div.rendered_html td {
|
||||
text-align: right;
|
||||
vertical-align: middle;
|
||||
padding: 0.5em 0.5em;
|
||||
line-height: normal;
|
||||
white-space: normal;
|
||||
max-width: none;
|
||||
border: none;
|
||||
}
|
||||
.jp-RenderedHTMLCommon th,
|
||||
div.rendered_html th {
|
||||
font-weight: bold;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tbody tr:nth-child(odd),
|
||||
div.rendered_html tbody tr:nth-child(odd) {
|
||||
background: #f5f5f5;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tbody tr:hover,
|
||||
div.rendered_html tbody tr:hover {
|
||||
background: rgba(66, 165, 245, 0.2);
|
||||
}
|
||||
</style>
|
||||
<div class="section" id="Feature-Extractors">
|
||||
<h1><span class="section-number">3. </span>Feature Extractors<a class="headerlink" href="#Feature-Extractors" title="Permalink to this headline">¶</a></h1>
|
||||
<p>In the previous page, we introduced <em>training data collectors</em>, which solve the optimization problem and collect raw training data, such as the optimal solution. In this page, we introduce <strong>feature extractors</strong>, which take the raw training data, stored in HDF5 files, and extract relevant information in order to train a machine learning model. We describe the extractors readily available in MIPLearn.</p>
|
||||
<div class="section" id="Overview">
|
||||
<h2><span class="section-number">3.1. </span>Overview<a class="headerlink" href="#Overview" title="Permalink to this headline">¶</a></h2>
|
||||
<p>Feature extraction is an important step of the process of building a machine learning model because it helps to reduce the complexity of the data and convert it into a format that is more easily processed. Previous research has proposed converting absolute variable coefficients, for example, into relative values which are invariant to various transformations, such as problem scaling, making them more amenable to learning. Various other transformations have also been described.</p>
|
||||
<p>In the framework, we treat data collection and feature extraction as two separate steps to accelerate the model development cycle. Specifically, collectors are typically time-consuming, as they often need to solve the problem to optimality, and therefore focus on collecting and storing all data that may or may not be relevant, in its raw format. Feature extractors, on the other hand, focus entirely on filtering the data and improving its representation, and are therefore much faster to run.
|
||||
Experimenting with new data representations, therefore, can be done without resolving the instances.</p>
|
||||
<p>In MIPLearn, extractors implement the abstract class <a class="reference external" href="../../api/collectors/#miplearn.features.fields.FeaturesExtractor">FeatureExtractor</a>, which has methods that take as input an <a class="reference external" href="../../api/helpers/#miplearn.h5.H5File">H5File</a> and produce either: (i) instance features, which describe the entire instances; (ii) variable features, which describe a particular decision variables; or (iii) constraint features, which describe a particular constraint. The extractor is free to implement only a
|
||||
subset of these methods, if it is known that it will not be used with a machine learning component that requires the other types of features.</p>
|
||||
</div>
|
||||
<div class="section" id="H5FieldsExtractor">
|
||||
<h2><span class="section-number">3.2. </span>H5FieldsExtractor<a class="headerlink" href="#H5FieldsExtractor" title="Permalink to this headline">¶</a></h2>
|
||||
<p><a class="reference external" href="#h5fieldsextractor">H5FieldsExtractor</a>, the most simple extractor in MIPLearn, simple extracts data that is already available in the HDF5 file, assembles it into a matrix and returns it as-is. The fields used to build instance, variable and constraint features are user-specified. The class also performs checks to ensure that the shapes of the returned matrices make sense.</p>
|
||||
<div class="section" id="Example">
|
||||
<h3>Example<a class="headerlink" href="#Example" title="Permalink to this headline">¶</a></h3>
|
||||
<p>The example below demonstrates the usage of H5FieldsExtractor in a randomly generated instance of the multi-dimensional knapsack problem.</p>
|
||||
<div class="nbinput docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[1]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">glob</span> <span class="kn">import</span> <span class="n">glob</span>
|
||||
<span class="kn">from</span> <span class="nn">shutil</span> <span class="kn">import</span> <span class="n">rmtree</span>
|
||||
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">from</span> <span class="nn">scipy.stats</span> <span class="kn">import</span> <span class="n">uniform</span><span class="p">,</span> <span class="n">randint</span>
|
||||
|
||||
<span class="kn">from</span> <span class="nn">miplearn.collectors.basic</span> <span class="kn">import</span> <span class="n">BasicCollector</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.features.fields</span> <span class="kn">import</span> <span class="n">H5FieldsExtractor</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.h5</span> <span class="kn">import</span> <span class="n">H5File</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.io</span> <span class="kn">import</span> <span class="n">save</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.problems.multiknapsack</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">MultiKnapsackGenerator</span><span class="p">,</span>
|
||||
<span class="n">build_multiknapsack_model</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># Set random seed to make example reproducible</span>
|
||||
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Generate some random multiknapsack instances</span>
|
||||
<span class="n">rmtree</span><span class="p">(</span><span class="s2">"data/multiknapsack/"</span><span class="p">,</span> <span class="n">ignore_errors</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
<span class="n">save</span><span class="p">(</span>
|
||||
<span class="n">MultiKnapsackGenerator</span><span class="p">(</span>
|
||||
<span class="n">n</span><span class="o">=</span><span class="n">randint</span><span class="p">(</span><span class="n">low</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">high</span><span class="o">=</span><span class="mi">11</span><span class="p">),</span>
|
||||
<span class="n">m</span><span class="o">=</span><span class="n">randint</span><span class="p">(</span><span class="n">low</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">high</span><span class="o">=</span><span class="mi">6</span><span class="p">),</span>
|
||||
<span class="n">w</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mi">1000</span><span class="p">),</span>
|
||||
<span class="n">K</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span>
|
||||
<span class="n">u</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span>
|
||||
<span class="n">alpha</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.25</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span>
|
||||
<span class="n">w_jitter</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.95</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">0.1</span><span class="p">),</span>
|
||||
<span class="n">p_jitter</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.75</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">0.5</span><span class="p">),</span>
|
||||
<span class="n">fix_w</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
<span class="p">)</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span>
|
||||
<span class="s2">"data/multiknapsack"</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># Run the basic collector</span>
|
||||
<span class="n">BasicCollector</span><span class="p">()</span><span class="o">.</span><span class="n">collect</span><span class="p">(</span>
|
||||
<span class="n">glob</span><span class="p">(</span><span class="s2">"data/multiknapsack/*"</span><span class="p">),</span>
|
||||
<span class="n">build_multiknapsack_model</span><span class="p">,</span>
|
||||
<span class="n">n_jobs</span><span class="o">=</span><span class="mi">4</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="n">ext</span> <span class="o">=</span> <span class="n">H5FieldsExtractor</span><span class="p">(</span>
|
||||
<span class="c1"># Use as instance features the value of the LP relaxation and the</span>
|
||||
<span class="c1"># vector of objective coefficients.</span>
|
||||
<span class="n">instance_fields</span><span class="o">=</span><span class="p">[</span>
|
||||
<span class="s2">"lp_obj_value"</span><span class="p">,</span>
|
||||
<span class="s2">"static_var_obj_coeffs"</span><span class="p">,</span>
|
||||
<span class="p">],</span>
|
||||
<span class="c1"># For each variable, use as features the optimal value of the LP</span>
|
||||
<span class="c1"># relaxation, the variable objective coefficient, the variable's</span>
|
||||
<span class="c1"># value its reduced cost.</span>
|
||||
<span class="n">var_fields</span><span class="o">=</span><span class="p">[</span>
|
||||
<span class="s2">"lp_obj_value"</span><span class="p">,</span>
|
||||
<span class="s2">"static_var_obj_coeffs"</span><span class="p">,</span>
|
||||
<span class="s2">"lp_var_values"</span><span class="p">,</span>
|
||||
<span class="s2">"lp_var_reduced_costs"</span><span class="p">,</span>
|
||||
<span class="p">],</span>
|
||||
<span class="c1"># For each constraint, use as features the RHS, dual value and slack.</span>
|
||||
<span class="n">constr_fields</span><span class="o">=</span><span class="p">[</span>
|
||||
<span class="s2">"static_constr_rhs"</span><span class="p">,</span>
|
||||
<span class="s2">"lp_constr_dual_values"</span><span class="p">,</span>
|
||||
<span class="s2">"lp_constr_slacks"</span><span class="p">,</span>
|
||||
<span class="p">]</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="k">with</span> <span class="n">H5File</span><span class="p">(</span><span class="s2">"data/multiknapsack/00000.h5"</span><span class="p">)</span> <span class="k">as</span> <span class="n">h5</span><span class="p">:</span>
|
||||
<span class="c1"># Extract and print instance features</span>
|
||||
<span class="n">x1</span> <span class="o">=</span> <span class="n">ext</span><span class="o">.</span><span class="n">get_instance_features</span><span class="p">(</span><span class="n">h5</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"instance features"</span><span class="p">,</span> <span class="n">x1</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="n">x1</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Extract and print variable features</span>
|
||||
<span class="n">x2</span> <span class="o">=</span> <span class="n">ext</span><span class="o">.</span><span class="n">get_var_features</span><span class="p">(</span><span class="n">h5</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"variable features"</span><span class="p">,</span> <span class="n">x2</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="n">x2</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Extract and print constraint features</span>
|
||||
<span class="n">x3</span> <span class="o">=</span> <span class="n">ext</span><span class="o">.</span><span class="n">get_constr_features</span><span class="p">(</span><span class="n">h5</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"constraint features"</span><span class="p">,</span> <span class="n">x3</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="n">x3</span><span class="p">)</span>
|
||||
<br/></pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="nboutput nblast docutils container">
|
||||
<div class="prompt empty docutils container">
|
||||
</div>
|
||||
<div class="output_area docutils container">
|
||||
<div class="highlight"><pre>
|
||||
instance features (11,)
|
||||
[-1531.24308771 -350. -692. -454.
|
||||
-709. -605. -543. -321.
|
||||
-674. -571. -341. ]
|
||||
variable features (10, 4)
|
||||
[[-1.53124309e+03 -3.50000000e+02 0.00000000e+00 9.43468018e+01]
|
||||
[-1.53124309e+03 -6.92000000e+02 2.51703322e-01 0.00000000e+00]
|
||||
[-1.53124309e+03 -4.54000000e+02 0.00000000e+00 8.25504150e+01]
|
||||
[-1.53124309e+03 -7.09000000e+02 1.11373022e-01 0.00000000e+00]
|
||||
[-1.53124309e+03 -6.05000000e+02 1.00000000e+00 -1.26055283e+02]
|
||||
[-1.53124309e+03 -5.43000000e+02 0.00000000e+00 1.68693771e+02]
|
||||
[-1.53124309e+03 -3.21000000e+02 1.07488781e-01 0.00000000e+00]
|
||||
[-1.53124309e+03 -6.74000000e+02 8.82293701e-01 0.00000000e+00]
|
||||
[-1.53124309e+03 -5.71000000e+02 0.00000000e+00 1.41129074e+02]
|
||||
[-1.53124309e+03 -3.41000000e+02 1.28830120e-01 0.00000000e+00]]
|
||||
constraint features (5, 3)
|
||||
[[ 1.3100000e+03 -1.5978307e-01 0.0000000e+00]
|
||||
[ 9.8800000e+02 -3.2881632e-01 0.0000000e+00]
|
||||
[ 1.0040000e+03 -4.0601316e-01 0.0000000e+00]
|
||||
[ 1.2690000e+03 -1.3659772e-01 0.0000000e+00]
|
||||
[ 1.0070000e+03 -2.8800571e-01 0.0000000e+00]]
|
||||
</pre></div></div>
|
||||
</div>
|
||||
<div class="admonition warning">
|
||||
<p class="admonition-title">Warning</p>
|
||||
<p>You should ensure that the number of features remains the same for all relevant HDF5 files. In the previous example, to illustrate this issue, we used variable objective coefficients as instance features. While this is allowed, note that this requires all problem instances to have the same number of variables; otherwise the number of features would vary from instance to instance and MIPLearn would be unable to concatenate the matrices.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="AlvLouWeh2017Extractor">
|
||||
<h2><span class="section-number">3.3. </span>AlvLouWeh2017Extractor<a class="headerlink" href="#AlvLouWeh2017Extractor" title="Permalink to this headline">¶</a></h2>
|
||||
<p>Alvarez, Louveaux and Wehenkel (2017) proposed a set features to describe a particular decision variable in a given node of the branch-and-bound tree, and applied it to the problem of mimicking strong branching decisions. The class <a class="reference external" href="#alvlouweh2017extractor">AlvLouWeh2017Extractor</a> implements a subset of these features (40 out of 64), which are available outside of the branch-and-bound tree. Some features are derived from the static defintion of the problem (i.e. from objective function and
|
||||
constraint data), while some features are derived from the solution to the LP relaxation. The features have been designed to be: (i) independent of the size of the problem; (ii) invariant with respect to irrelevant problem transformations, such as row and column permutation; and (iii) independent of the scale of the problem. We refer to the paper for a more complete description.</p>
|
||||
<div class="section" id="id1">
|
||||
<h3>Example<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3>
|
||||
<div class="nbinput docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[5]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">miplearn.features.AlvLouWeh2017</span> <span class="kn">import</span> <span class="n">AlvLouWeh2017Extractor</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.h5</span> <span class="kn">import</span> <span class="n">H5File</span>
|
||||
|
||||
<span class="c1"># Build the extractor</span>
|
||||
<span class="n">ext</span> <span class="o">=</span> <span class="n">AlvLouWeh2017Extractor</span><span class="p">()</span>
|
||||
|
||||
<span class="c1"># Open previously-created multiknapsack training data</span>
|
||||
<span class="k">with</span> <span class="n">H5File</span><span class="p">(</span><span class="s2">"data/multiknapsack/00000.h5"</span><span class="p">)</span> <span class="k">as</span> <span class="n">h5</span><span class="p">:</span>
|
||||
|
||||
<span class="c1"># Extract and print variable features</span>
|
||||
<span class="n">x1</span> <span class="o">=</span> <span class="n">ext</span><span class="o">.</span><span class="n">get_var_features</span><span class="p">(</span><span class="n">h5</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"x1"</span><span class="p">,</span> <span class="n">x1</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="n">x1</span><span class="o">.</span><span class="n">round</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="nboutput nblast docutils container">
|
||||
<div class="prompt empty docutils container">
|
||||
</div>
|
||||
<div class="output_area docutils container">
|
||||
<div class="highlight"><pre>
|
||||
x1 (10, 40)
|
||||
[[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 0.00e+00 1.00e+00 6.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 6.00e-01 1.00e+00 1.75e+01 1.00e+00 2.00e-01
|
||||
1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 -1.00e+00 0.00e+00 1.00e+20]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 1.00e-01 1.00e+00 1.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 7.00e-01 1.00e+00 5.10e+00 1.00e+00 2.00e-01
|
||||
1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
3.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 0.00e+00 1.00e+00 9.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 5.00e-01 1.00e+00 1.30e+01 1.00e+00 2.00e-01
|
||||
1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 -1.00e+00 0.00e+00 1.00e+20]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 2.00e-01 1.00e+00 9.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 8.00e-01 1.00e+00 3.40e+00 1.00e+00 2.00e-01
|
||||
1.00e+00 1.00e-01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
1.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 1.00e-01 1.00e+00 7.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 6.00e-01 1.00e+00 3.80e+00 1.00e+00 2.00e-01
|
||||
1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 1.00e-01 1.00e+00 8.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 7.00e-01 1.00e+00 3.30e+00 1.00e+00 2.00e-01
|
||||
1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 -1.00e+00 0.00e+00 1.00e+20]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 0.00e+00 1.00e+00 3.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 1.00e+00 1.00e+00 5.70e+00 1.00e+00 1.00e-01
|
||||
1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
1.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 1.00e-01 1.00e+00 6.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 8.00e-01 1.00e+00 6.80e+00 1.00e+00 2.00e-01
|
||||
1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
1.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 4.00e-01 1.00e+00 6.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 8.00e-01 1.00e+00 1.40e+00 1.00e+00 1.00e-01
|
||||
1.00e+00 1.00e-01 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 -1.00e+00 0.00e+00 1.00e+20]
|
||||
[-1.00e+00 1.00e+20 1.00e-01 1.00e+00 0.00e+00 1.00e+00 5.00e-01
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 1.00e+00 5.00e-01 1.00e+00 7.60e+00 1.00e+00 1.00e-01
|
||||
1.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
|
||||
1.00e-01 -1.00e+00 -1.00e+00 0.00e+00 0.00e+00]]
|
||||
</pre></div></div>
|
||||
</div>
|
||||
<div class="admonition note">
|
||||
<p class="admonition-title">References</p>
|
||||
<ul class="simple">
|
||||
<li><p><strong>Alvarez, Alejandro Marcos.</strong> <em>Computational and theoretical synergies between linear optimization and supervised machine learning.</em> (2016). University of Liège.</p></li>
|
||||
<li><p><strong>Alvarez, Alejandro Marcos, Quentin Louveaux, and Louis Wehenkel.</strong> <em>A machine learning-based approximation of strong branching.</em> INFORMS Journal on Computing 29.1 (2017): 185-195.</p></li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<div class='prev-next-bottom'>
|
||||
|
||||
<a class='left-prev' id="prev-link" href="../collectors/" title="previous page"><span class="section-number">2. </span>Training Data Collectors</a>
|
||||
<a class='right-next' id="next-link" href="../primal/" title="next page"><span class="section-number">4. </span>Primal Components</a>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<footer class="footer mt-5 mt-md-0">
|
||||
<div class="container">
|
||||
<p>
|
||||
|
||||
© Copyright 2020-2022, UChicago Argonne, LLC.<br/>
|
||||
</p>
|
||||
</div>
|
||||
</footer>
|
||||
</main>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="../../_static/js/index.1c5a1a01449ed65a7b51.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
290
0.3/guide/primal.ipynb
Normal file
290
0.3/guide/primal.ipynb
Normal file
@@ -0,0 +1,290 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "880cf4c7-d3c4-4b92-85c7-04a32264cdae",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Primal Components\n",
|
||||
"\n",
|
||||
"In MIPLearn, a **primal component** is class that uses machine learning to predict a (potentially partial) assignment of values to the decision variables of the problem. Predicting high-quality primal solutions may be beneficial, as they allow the MIP solver to prune potentially large portions of the search space. Alternatively, if proof of optimality is not required, the MIP solver can be used to complete the partial solution generated by the machine learning model and and double-check its feasibility. MIPLearn allows both of these usage patterns.\n",
|
||||
"\n",
|
||||
"In this page, we describe the four primal components currently included in MIPLearn, which employ machine learning in different ways. Each component is highly configurable, and accepts an user-provided machine learning model, which it uses for all predictions. Each component can also be configured to provide the solution to the solver in multiple ways, depending on whether proof of optimality is required.\n",
|
||||
"\n",
|
||||
"## Primal component actions\n",
|
||||
"\n",
|
||||
"Before presenting the primal components themselves, we briefly discuss the three ways a solution may be provided to the solver. Each approach has benefits and limitations, which we also discuss in this section. All primal components can be configured to use any of the following approaches.\n",
|
||||
"\n",
|
||||
"The first approach is to provide the solution to the solver as a **warm start**. This is implemented by the class [SetWarmStart](SetWarmStart). The main advantage is that this method maintains all optimality and feasibility guarantees of the MIP solver, while still providing significant performance benefits for various classes of problems. If the machine learning model is able to predict multiple solutions, it is also possible to set multiple warm starts. In this case, the solver evaluates each warm start, discards the infeasible ones, then proceeds with the one that has the best objective value. The main disadvantage of this approach, compared to the next two, is that it provides relatively modest speedups for most problem classes, and no speedup at all for many others, even when the machine learning predictions are 100% accurate.\n",
|
||||
"\n",
|
||||
"[SetWarmStart]: ../../api/components/#miplearn.components.primal.actions.SetWarmStart\n",
|
||||
"\n",
|
||||
"The second approach is to **fix the decision variables** to their predicted values, then solve a restricted optimization problem on the remaining variables. This approach is implemented by the class `FixVariables`. The main advantage is its potential speedup: if machine learning can accurately predict values for a significant portion of the decision variables, then the MIP solver can typically complete the solution in a small fraction of the time it would take to find the same solution from scratch. The main disadvantage of this approach is that it loses optimality guarantees; that is, the complete solution found by the MIP solver may no longer be globally optimal. Also, if the machine learning predictions are not sufficiently accurate, there might not even be a feasible assignment for the variables that were left free.\n",
|
||||
"\n",
|
||||
"Finally, the third approach, which tries to strike a balance between the two previous ones, is to **enforce proximity** to a given solution. This strategy is implemented by the class `EnforceProximity`. More precisely, given values $\\bar{x}_1,\\ldots,\\bar{x}_n$ for a subset of binary decision variables $x_1,\\ldots,x_n$, this approach adds the constraint\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\sum_{i : \\bar{x}_i=0} x_i + \\sum_{i : \\bar{x}_i=1} \\left(1 - x_i\\right) \\leq k,\n",
|
||||
"$$\n",
|
||||
"to the problem, where $k$ is a user-defined parameter, which indicates how many of the predicted variables are allowed to deviate from the machine learning suggestion. The main advantage of this approach, compared to fixing variables, is its tolerance to lower-quality machine learning predictions. Its main disadvantage is that it typically leads to smaller speedups, especially for larger values of $k$. This approach also loses optimality guarantees.\n",
|
||||
"\n",
|
||||
"## Memorizing primal component\n",
|
||||
"\n",
|
||||
"A simple machine learning strategy for the prediction of primal solutions is to memorize all distinct solutions seen during training, then try to predict, during inference time, which of those memorized solutions are most likely to be feasible and to provide a good objective value for the current instance. The most promising solutions may alternatively be combined into a single partial solution, which is then provided to the MIP solver. Both variations of this strategy are implemented by the `MemorizingPrimalComponent` class. Note that it is only applicable if the problem size, and in fact if the meaning of the decision variables, remains the same across problem instances.\n",
|
||||
"\n",
|
||||
"More precisely, let $I_1,\\ldots,I_n$ be the training instances, and let $\\bar{x}^1,\\ldots,\\bar{x}^n$ be their respective optimal solutions. Given a new instance $I_{n+1}$, `MemorizingPrimalComponent` expects a user-provided binary classifier that assigns (through the `predict_proba` method, following scikit-learn's conventions) a score $\\delta_i$ to each solution $\\bar{x}^i$, such that solutions with higher score are more likely to be good solutions for $I_{n+1}$. The features provided to the classifier are the instance features computed by an user-provided extractor. Given these scores, the component then performs one of the following to actions, as decided by the user:\n",
|
||||
"\n",
|
||||
"1. Selects the top $k$ solutions with the highest scores and provides them to the solver; this is implemented by `SelectTopSolutions`, and it is typically used with the `SetWarmStart` action.\n",
|
||||
"\n",
|
||||
"2. Merges the top $k$ solutions into a single partial solution, then provides it to the solver. This is implemented by `MergeTopSolutions`. More precisely, suppose that the machine learning regressor ordered the solutions in the sequence $\\bar{x}^{i_1},\\ldots,\\bar{x}^{i_n}$, with the most promising solutions appearing first, and with ties being broken arbitrarily. The component starts by keeping only the $k$ most promising solutions $\\bar{x}^{i_1},\\ldots,\\bar{x}^{i_k}$. Then it computes, for each binary decision variable $x_l$, its average assigned value $\\tilde{x}_l$:\n",
|
||||
"$$\n",
|
||||
" \\tilde{x}_l = \\frac{1}{k} \\sum_{j=1}^k \\bar{x}^{i_j}_l.\n",
|
||||
"$$\n",
|
||||
" Finally, the component constructs a merged solution $y$, defined as:\n",
|
||||
"$$\n",
|
||||
" y_j = \\begin{cases}\n",
|
||||
" 0 & \\text{ if } \\tilde{x}_l \\le \\theta_0 \\\\\n",
|
||||
" 1 & \\text{ if } \\tilde{x}_l \\ge \\theta_1 \\\\\n",
|
||||
" \\square & \\text{otherwise,}\n",
|
||||
" \\end{cases}\n",
|
||||
"$$\n",
|
||||
" where $\\theta_0$ and $\\theta_1$ are user-specified parameters, and where $\\square$ indicates that the variable is left undefined. The solution $y$ is then provided by the solver using any of the three approaches defined in the previous section.\n",
|
||||
"\n",
|
||||
"The above specification of `MemorizingPrimalComponent` is meant to be as general as possible. Simpler strategies can be implemented by configuring this component in specific ways. For example, a simpler approach employed in the literature is to collect all optimal solutions, then provide the entire list of solutions to the solver as warm starts, without any filtering or post-processing. This strategy can be implemented with `MemorizingPrimalComponent` by using a model that returns a constant value for all solutions (e.g. [scikit-learn's DummyClassifier][DummyClassifier]), then selecting the top $n$ (instead of $k$) solutions. See example below. Another simple approach is taking the solution to the most similar instance, and using it, by itself, as a warm start. This can be implemented by using a model that computes distances between the current instance and the training ones (e.g. [scikit-learn's KNeighborsClassifier][KNeighborsClassifier]), then select the solution to the nearest one. See also example below. More complex strategies, of course, can also be configured.\n",
|
||||
"\n",
|
||||
"[DummyClassifier]: https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyClassifier.html\n",
|
||||
"[KNeighborsClassifier]: https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html\n",
|
||||
"\n",
|
||||
"### Examples"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "94907996",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\n",
|
||||
"from sklearn.dummy import DummyClassifier\n",
|
||||
"from sklearn.neighbors import KNeighborsClassifier\n",
|
||||
"\n",
|
||||
"from miplearn.components.primal.actions import (\n",
|
||||
" SetWarmStart,\n",
|
||||
" FixVariables,\n",
|
||||
" EnforceProximity,\n",
|
||||
")\n",
|
||||
"from miplearn.components.primal.mem import (\n",
|
||||
" MemorizingPrimalComponent,\n",
|
||||
" SelectTopSolutions,\n",
|
||||
" MergeTopSolutions,\n",
|
||||
")\n",
|
||||
"from miplearn.extractors.dummy import DummyExtractor\n",
|
||||
"from miplearn.extractors.fields import H5FieldsExtractor\n",
|
||||
"\n",
|
||||
"# Configures a memorizing primal component that collects\n",
|
||||
"# all distinct solutions seen during training and provides\n",
|
||||
"# them to the solver without any filtering or post-processing.\n",
|
||||
"comp1 = MemorizingPrimalComponent(\n",
|
||||
" clf=DummyClassifier(),\n",
|
||||
" extractor=DummyExtractor(),\n",
|
||||
" constructor=SelectTopSolutions(1_000_000),\n",
|
||||
" action=SetWarmStart(),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Configures a memorizing primal component that finds the\n",
|
||||
"# training instance with the closest objective function, then\n",
|
||||
"# fixes the decision variables to the values they assumed\n",
|
||||
"# at the optimal solution for that instance.\n",
|
||||
"comp2 = MemorizingPrimalComponent(\n",
|
||||
" clf=KNeighborsClassifier(n_neighbors=1),\n",
|
||||
" extractor=H5FieldsExtractor(\n",
|
||||
" instance_fields=[\"static_var_obj_coeffs\"],\n",
|
||||
" ),\n",
|
||||
" constructor=SelectTopSolutions(1),\n",
|
||||
" action=FixVariables(),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Configures a memorizing primal component that finds the distinct\n",
|
||||
"# solutions to the 10 most similar training problem instances,\n",
|
||||
"# selects the 3 solutions that were most often optimal to these\n",
|
||||
"# training instances, combines them into a single partial solution,\n",
|
||||
"# then enforces proximity, allowing at most 3 variables to deviate\n",
|
||||
"# from the machine learning suggestion.\n",
|
||||
"comp3 = MemorizingPrimalComponent(\n",
|
||||
" clf=KNeighborsClassifier(n_neighbors=10),\n",
|
||||
" extractor=H5FieldsExtractor(instance_fields=[\"static_var_obj_coeffs\"]),\n",
|
||||
" constructor=MergeTopSolutions(k=3, thresholds=[0.25, 0.75]),\n",
|
||||
" action=EnforceProximity(3),\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6b854305",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"## Independent vars primal component\n",
|
||||
"\n",
|
||||
"Instead of memorizing previously-seen primal solutions, it is also natural to use machine learning models to directly predict the values of the decision variables, constructing a solution from scratch. This approach has the benefit of potentially constructing novel high-quality solutions, never observed in the training data. Two variations of this strategy are supported by MIPLearn: (i) predicting the values of the decision variables independently, using multiple ML models; or (ii) predicting the values jointly, with a single model. We describe the first variation in this section, and the second variation in the next section.\n",
|
||||
"\n",
|
||||
"Let $I_1,\\ldots,I_n$ be the training instances, and let $\\bar{x}^1,\\ldots,\\bar{x}^n$ be their respective optimal solutions. For each binary decision variable $x_j$, the component `IndependentVarsPrimalComponent` creates a copy of a user-provided binary classifier and trains it to predict the optimal value of $x_j$, given $\\bar{x}^1_j,\\ldots,\\bar{x}^n_j$ as training labels. The features provided to the model are the variable features computed by an user-provided extractor. During inference time, the component uses these $n$ binary classifiers to construct a solution and provides it to the solver using one of the available actions.\n",
|
||||
"\n",
|
||||
"Three issues often arise in practice when using this approach:\n",
|
||||
"\n",
|
||||
" 1. For certain binary variables $x_j$, it is frequently the case that its optimal value is either always zero or always one in the training dataset, which poses problems to some standard scikit-learn classifiers, since they do not expect a single class. The wrapper `SingleClassFix` can be used to fix this issue (see example below).\n",
|
||||
"2. It is also frequently the case that machine learning classifier can only reliably predict the values of some variables with high accuracy, not all of them. In this situation, instead of computing a complete primal solution, it may be more beneficial to construct a partial solution containing values only for the variables for which the ML made a high-confidence prediction. The meta-classifier `MinProbabilityClassifier` can be used for this purpose. It asks the base classifier for the probability of the value being zero or one (using the `predict_proba` method) and erases from the primal solution all values whose probabilities are below a given threshold.\n",
|
||||
"3. To make multiple copies of the provided ML classifier, MIPLearn uses the standard `sklearn.base.clone` method, which may not be suitable for classifiers from other frameworks. To handle this, it is possible to override the clone function using the `clone_fn` constructor argument.\n",
|
||||
"\n",
|
||||
"### Examples"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "f5024983",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.linear_model import LogisticRegression\n",
|
||||
"from miplearn.classifiers.minprob import MinProbabilityClassifier\n",
|
||||
"from miplearn.classifiers.singleclass import SingleClassFix\n",
|
||||
"from miplearn.components.primal.indep import IndependentVarsPrimalComponent\n",
|
||||
"from miplearn.extractors.AlvLouWeh2017 import AlvLouWeh2017Extractor\n",
|
||||
"from miplearn.solvers.learning import LearningSolver\n",
|
||||
"from miplearn.components.primal.actions import SetWarmStart\n",
|
||||
"\n",
|
||||
"# Configures a primal component that independently predicts the value of each\n",
|
||||
"# binary variable using logistic regression and provides it to the solver as\n",
|
||||
"# warm start. Erases predictions with probability less than 99%; applies\n",
|
||||
"# single-class fix; and uses AlvLouWeh2017 features.\n",
|
||||
"comp = IndependentVarsPrimalComponent(\n",
|
||||
" base_clf=SingleClassFix(\n",
|
||||
" MinProbabilityClassifier(\n",
|
||||
" base_clf=LogisticRegression(),\n",
|
||||
" thresholds=[0.99, 0.99],\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
" extractor=AlvLouWeh2017Extractor(),\n",
|
||||
" action=SetWarmStart(),\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "dcb10079",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"## Joint vars primal component\n",
|
||||
"In the previous subsection, we used multiple machine learning models to independently predict the values of the binary decision variables. When these values are correlated, an alternative approach is to jointly predict the values of all binary variables using a single machine learning model. This strategy is implemented by `JointVarsPrimalComponent`. Compared to the previous ones, this component is much more straightforwad. It simply extracts instance features, using the user-provided feature extractor, then directly trains the user-provided binary classifier (using the `fit` method), without making any copies. The trained classifier is then used to predict entire solutions (using the `predict` method), which are given to the solver using one of the previously discussed methods. In the example below, we illustrate the usage of this component with a simple feed-forward neural network.\n",
|
||||
"\n",
|
||||
"`JointVarsPrimalComponent` can also be used to implement strategies that use multiple machine learning models, but not indepedently. For example, a common strategy in multioutput prediction is building a *classifier chain*. In this approach, the first decision variable is predicted using the instance features alone; but the $n$-th decision variable is predicted using the instance features plus the predicted values of the $n-1$ previous variables. This can be easily implemented using scikit-learn's `ClassifierChain` estimator, as shown in the example below.\n",
|
||||
"\n",
|
||||
"### Examples"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "c98f7eb8",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.multioutput import ClassifierChain\n",
|
||||
"from sklearn.neural_network import MLPClassifier\n",
|
||||
"from miplearn.components.primal.joint import JointVarsPrimalComponent\n",
|
||||
"from miplearn.extractors.fields import H5FieldsExtractor\n",
|
||||
"from miplearn.solvers.learning import LearningSolver\n",
|
||||
"from miplearn.components.primal.actions import SetWarmStart\n",
|
||||
"\n",
|
||||
"# Configures a primal component that uses a feedforward neural network\n",
|
||||
"# to jointly predict the values of the binary variables, based on the\n",
|
||||
"# objective cost function, and provides the solution to the solver as\n",
|
||||
"# a warm start.\n",
|
||||
"comp = JointVarsPrimalComponent(\n",
|
||||
" clf=MLPClassifier(),\n",
|
||||
" extractor=H5FieldsExtractor(\n",
|
||||
" instance_fields=['static_var_obj_coeffs'],\n",
|
||||
" ),\n",
|
||||
" action=SetWarmStart(),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Configures a primal component that uses a chain of logistic regression\n",
|
||||
"# models to jointly predict the values of the binary variables, based on\n",
|
||||
"# the objective function.\n",
|
||||
"comp = JointVarsPrimalComponent(\n",
|
||||
" clf=ClassifierChain(SingleClassFix(LogisticRegression())),\n",
|
||||
" extractor=H5FieldsExtractor(\n",
|
||||
" instance_fields=['static_var_obj_coeffs'],\n",
|
||||
" ),\n",
|
||||
" action=SetWarmStart(),\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "92461394",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"source": [
|
||||
"## Expert primal component\n",
|
||||
"\n",
|
||||
"Before spending time and effort choosing a machine learning strategy and tweaking its parameters, it is usually a good idea to evaluate what would be the performance impact of the model if its predictions were 100% accurate. This is especially important for the prediction of warm starts, since they are not always very beneficial. To simplify this task, MIPLearn provides `ExpertPrimalComponent`, a component which simply loads the optimal solution from the HDF5 file, assuming that it has already been computed, then directly provides it to the solver using one of the available methods. This component is useful in benchmarks, to evaluate how close to the best theoretical performance the machine learning components are.\n",
|
||||
"\n",
|
||||
"### Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "fb7dbdd0",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from miplearn.components.primal.expert import ExpertPrimalComponent\n",
|
||||
"from miplearn.components.primal.actions import SetWarmStart\n",
|
||||
"\n",
|
||||
"# Configures an expert primal component, which reads a pre-computed\n",
|
||||
"# optimal solution from the HDF5 file and provides it to the solver\n",
|
||||
"# as warm start.\n",
|
||||
"comp = ExpertPrimalComponent(\n",
|
||||
" action=SetWarmStart()\n",
|
||||
")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.13"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
774
0.3/guide/primal/index.html
Normal file
774
0.3/guide/primal/index.html
Normal file
@@ -0,0 +1,774 @@
|
||||
|
||||
<!DOCTYPE html>
|
||||
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>4. Primal Components — MIPLearn 0.3</title>
|
||||
|
||||
<link href="../../_static/css/theme.css" rel="stylesheet" />
|
||||
<link href="../../_static/css/index.c5995385ac14fb8791e8eb36b4908be2.css" rel="stylesheet" />
|
||||
|
||||
|
||||
<link rel="stylesheet"
|
||||
href="../../_static/vendor/fontawesome/5.13.0/css/all.min.css">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="../../_static/vendor/fontawesome/5.13.0/webfonts/fa-solid-900.woff2">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="../../_static/vendor/fontawesome/5.13.0/webfonts/fa-brands-400.woff2">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
|
||||
<link rel="stylesheet" href="../../_static/sphinx-book-theme.acff12b8f9c144ce68a297486a2fa670.css" type="text/css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/custom.css" />
|
||||
|
||||
<link rel="preload" as="script" href="../../_static/js/index.1c5a1a01449ed65a7b51.js">
|
||||
|
||||
<script id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
|
||||
<script src="../../_static/jquery.js"></script>
|
||||
<script src="../../_static/underscore.js"></script>
|
||||
<script src="../../_static/doctools.js"></script>
|
||||
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
|
||||
<script src="../../_static/sphinx-book-theme.12a9622fbb08dcb3a2a40b2c02b83a57.js"></script>
|
||||
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
|
||||
<script type="text/x-mathjax-config">MathJax.Hub.Config({"tex2jax": {"inlineMath": [["\\(", "\\)"]], "displayMath": [["\\[", "\\]"]], "processRefs": false, "processEnvironments": false}})</script>
|
||||
<link rel="index" title="Index" href="../../genindex/" />
|
||||
<link rel="search" title="Search" href="../../search/" />
|
||||
<link rel="next" title="5. Solvers" href="../solvers/" />
|
||||
<link rel="prev" title="3. Feature Extractors" href="../features/" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="en" />
|
||||
|
||||
</head>
|
||||
<body data-spy="scroll" data-target="#bd-toc-nav" data-offset="80">
|
||||
|
||||
<div class="container-fluid" id="banner"></div>
|
||||
|
||||
|
||||
|
||||
<div class="container-xl">
|
||||
<div class="row">
|
||||
|
||||
<div class="col-12 col-md-3 bd-sidebar site-navigation show" id="site-navigation">
|
||||
|
||||
<div class="navbar-brand-box">
|
||||
<a class="navbar-brand text-wrap" href="../../">
|
||||
|
||||
|
||||
<h1 class="site-logo" id="site-title">MIPLearn 0.3</h1>
|
||||
|
||||
</a>
|
||||
</div><form class="bd-search d-flex align-items-center" action="../../search/" method="get">
|
||||
<i class="icon fas fa-search"></i>
|
||||
<input type="search" class="form-control" name="q" id="search-input" placeholder="Search the docs ..." aria-label="Search the docs ..." autocomplete="off" >
|
||||
</form><nav class="bd-links" id="bd-docs-nav" aria-label="Main navigation">
|
||||
<div class="bd-toc-item active">
|
||||
<p class="caption">
|
||||
<span class="caption-text">
|
||||
User Guide
|
||||
</span>
|
||||
</p>
|
||||
<ul class="current nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../problems/">
|
||||
1. Benchmark Problems
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../collectors/">
|
||||
2. Training Data Collectors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../features/">
|
||||
3. Feature Extractors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1 current active">
|
||||
<a class="current reference internal" href="#">
|
||||
4. Primal Components
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../solvers/">
|
||||
5. Solvers
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p class="caption">
|
||||
<span class="caption-text">
|
||||
API Reference
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/problems/">
|
||||
6. Benchmark Problems
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/collectors/">
|
||||
7. Collectors & Extractors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/components/">
|
||||
8. Components
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/solvers/">
|
||||
9. Solvers
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/helpers/">
|
||||
10. Helpers
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav> <!-- To handle the deprecated key -->
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<main class="col py-md-3 pl-md-4 bd-content overflow-auto" role="main">
|
||||
|
||||
<div class="topbar container-xl fixed-top">
|
||||
<div class="topbar-contents row">
|
||||
<div class="col-12 col-md-3 bd-topbar-whitespace site-navigation show"></div>
|
||||
<div class="col pl-md-4 topbar-main">
|
||||
|
||||
<button id="navbar-toggler" class="navbar-toggler ml-0" type="button" data-toggle="collapse"
|
||||
data-toggle="tooltip" data-placement="bottom" data-target=".site-navigation" aria-controls="navbar-menu"
|
||||
aria-expanded="true" aria-label="Toggle navigation" aria-controls="site-navigation"
|
||||
title="Toggle navigation" data-toggle="tooltip" data-placement="left">
|
||||
<i class="fas fa-bars"></i>
|
||||
<i class="fas fa-arrow-left"></i>
|
||||
<i class="fas fa-arrow-up"></i>
|
||||
</button>
|
||||
|
||||
|
||||
<div class="dropdown-buttons-trigger">
|
||||
<button id="dropdown-buttons-trigger" class="btn btn-secondary topbarbtn" aria-label="Download this page"><i
|
||||
class="fas fa-download"></i></button>
|
||||
|
||||
<div class="dropdown-buttons">
|
||||
<!-- ipynb file if we had a myst markdown file -->
|
||||
|
||||
<!-- Download raw file -->
|
||||
<a class="dropdown-buttons" href="../../_sources/guide/primal.ipynb.txt"><button type="button"
|
||||
class="btn btn-secondary topbarbtn" title="Download source file" data-toggle="tooltip"
|
||||
data-placement="left">.ipynb</button></a>
|
||||
<!-- Download PDF via print -->
|
||||
<button type="button" id="download-print" class="btn btn-secondary topbarbtn" title="Print to PDF"
|
||||
onClick="window.print()" data-toggle="tooltip" data-placement="left">.pdf</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Source interaction buttons -->
|
||||
|
||||
<!-- Full screen (wrap in <a> to have style consistency -->
|
||||
|
||||
<a class="full-screen-button"><button type="button" class="btn btn-secondary topbarbtn" data-toggle="tooltip"
|
||||
data-placement="bottom" onclick="toggleFullScreen()" aria-label="Fullscreen mode"
|
||||
title="Fullscreen mode"><i
|
||||
class="fas fa-expand"></i></button></a>
|
||||
|
||||
<!-- Launch buttons -->
|
||||
|
||||
</div>
|
||||
|
||||
<!-- Table of contents -->
|
||||
<div class="d-none d-md-block col-md-2 bd-toc show">
|
||||
|
||||
<div class="tocsection onthispage pt-5 pb-3">
|
||||
<i class="fas fa-list"></i> Contents
|
||||
</div>
|
||||
<nav id="bd-toc-nav">
|
||||
<ul class="visible nav section-nav flex-column">
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Primal-component-actions">
|
||||
4.1. Primal component actions
|
||||
</a>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Memorizing-primal-component">
|
||||
4.2. Memorizing primal component
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Examples">
|
||||
Examples
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Independent-vars-primal-component">
|
||||
4.3. Independent vars primal component
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#id1">
|
||||
Examples
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Joint-vars-primal-component">
|
||||
4.4. Joint vars primal component
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#id2">
|
||||
Examples
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Expert-primal-component">
|
||||
4.5. Expert primal component
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Example">
|
||||
Example
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="main-content" class="row">
|
||||
<div class="col-12 col-md-9 pl-md-3 pr-md-0">
|
||||
|
||||
<div>
|
||||
|
||||
|
||||
<style>
|
||||
/* CSS for nbsphinx extension */
|
||||
|
||||
/* remove conflicting styling from Sphinx themes */
|
||||
div.nbinput.container div.prompt *,
|
||||
div.nboutput.container div.prompt *,
|
||||
div.nbinput.container div.input_area pre,
|
||||
div.nboutput.container div.output_area pre,
|
||||
div.nbinput.container div.input_area .highlight,
|
||||
div.nboutput.container div.output_area .highlight {
|
||||
border: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
box-shadow: none;
|
||||
}
|
||||
|
||||
div.nbinput.container > div[class*=highlight],
|
||||
div.nboutput.container > div[class*=highlight] {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt *,
|
||||
div.nboutput.container div.prompt * {
|
||||
background: none;
|
||||
}
|
||||
|
||||
div.nboutput.container div.output_area .highlight,
|
||||
div.nboutput.container div.output_area pre {
|
||||
background: unset;
|
||||
}
|
||||
|
||||
div.nboutput.container div.output_area div.highlight {
|
||||
color: unset; /* override Pygments text color */
|
||||
}
|
||||
|
||||
/* avoid gaps between output lines */
|
||||
div.nboutput.container div[class*=highlight] pre {
|
||||
line-height: normal;
|
||||
}
|
||||
|
||||
/* input/output containers */
|
||||
div.nbinput.container,
|
||||
div.nboutput.container {
|
||||
display: -webkit-flex;
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
margin: 0;
|
||||
width: 100%;
|
||||
}
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container,
|
||||
div.nboutput.container {
|
||||
flex-direction: column;
|
||||
}
|
||||
}
|
||||
|
||||
/* input container */
|
||||
div.nbinput.container {
|
||||
padding-top: 5px;
|
||||
}
|
||||
|
||||
/* last container */
|
||||
div.nblast.container {
|
||||
padding-bottom: 5px;
|
||||
}
|
||||
|
||||
/* input prompt */
|
||||
div.nbinput.container div.prompt pre {
|
||||
color: #307FC1;
|
||||
}
|
||||
|
||||
/* output prompt */
|
||||
div.nboutput.container div.prompt pre {
|
||||
color: #BF5B3D;
|
||||
}
|
||||
|
||||
/* all prompts */
|
||||
div.nbinput.container div.prompt,
|
||||
div.nboutput.container div.prompt {
|
||||
width: 4.5ex;
|
||||
padding-top: 5px;
|
||||
position: relative;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt > div,
|
||||
div.nboutput.container div.prompt > div {
|
||||
position: absolute;
|
||||
right: 0;
|
||||
margin-right: 0.3ex;
|
||||
}
|
||||
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container div.prompt,
|
||||
div.nboutput.container div.prompt {
|
||||
width: unset;
|
||||
text-align: left;
|
||||
padding: 0.4em;
|
||||
}
|
||||
div.nboutput.container div.prompt.empty {
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt > div,
|
||||
div.nboutput.container div.prompt > div {
|
||||
position: unset;
|
||||
}
|
||||
}
|
||||
|
||||
/* disable scrollbars and line breaks on prompts */
|
||||
div.nbinput.container div.prompt pre,
|
||||
div.nboutput.container div.prompt pre {
|
||||
overflow: hidden;
|
||||
white-space: pre;
|
||||
}
|
||||
|
||||
/* input/output area */
|
||||
div.nbinput.container div.input_area,
|
||||
div.nboutput.container div.output_area {
|
||||
-webkit-flex: 1;
|
||||
flex: 1;
|
||||
overflow: auto;
|
||||
}
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container div.input_area,
|
||||
div.nboutput.container div.output_area {
|
||||
width: 100%;
|
||||
}
|
||||
}
|
||||
|
||||
/* input area */
|
||||
div.nbinput.container div.input_area {
|
||||
border: 1px solid #e0e0e0;
|
||||
border-radius: 2px;
|
||||
/*background: #f5f5f5;*/
|
||||
}
|
||||
|
||||
/* override MathJax center alignment in output cells */
|
||||
div.nboutput.container div[class*=MathJax] {
|
||||
text-align: left !important;
|
||||
}
|
||||
|
||||
/* override sphinx.ext.imgmath center alignment in output cells */
|
||||
div.nboutput.container div.math p {
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
/* standard error */
|
||||
div.nboutput.container div.output_area.stderr {
|
||||
background: #fdd;
|
||||
}
|
||||
|
||||
/* ANSI colors */
|
||||
.ansi-black-fg { color: #3E424D; }
|
||||
.ansi-black-bg { background-color: #3E424D; }
|
||||
.ansi-black-intense-fg { color: #282C36; }
|
||||
.ansi-black-intense-bg { background-color: #282C36; }
|
||||
.ansi-red-fg { color: #E75C58; }
|
||||
.ansi-red-bg { background-color: #E75C58; }
|
||||
.ansi-red-intense-fg { color: #B22B31; }
|
||||
.ansi-red-intense-bg { background-color: #B22B31; }
|
||||
.ansi-green-fg { color: #00A250; }
|
||||
.ansi-green-bg { background-color: #00A250; }
|
||||
.ansi-green-intense-fg { color: #007427; }
|
||||
.ansi-green-intense-bg { background-color: #007427; }
|
||||
.ansi-yellow-fg { color: #DDB62B; }
|
||||
.ansi-yellow-bg { background-color: #DDB62B; }
|
||||
.ansi-yellow-intense-fg { color: #B27D12; }
|
||||
.ansi-yellow-intense-bg { background-color: #B27D12; }
|
||||
.ansi-blue-fg { color: #208FFB; }
|
||||
.ansi-blue-bg { background-color: #208FFB; }
|
||||
.ansi-blue-intense-fg { color: #0065CA; }
|
||||
.ansi-blue-intense-bg { background-color: #0065CA; }
|
||||
.ansi-magenta-fg { color: #D160C4; }
|
||||
.ansi-magenta-bg { background-color: #D160C4; }
|
||||
.ansi-magenta-intense-fg { color: #A03196; }
|
||||
.ansi-magenta-intense-bg { background-color: #A03196; }
|
||||
.ansi-cyan-fg { color: #60C6C8; }
|
||||
.ansi-cyan-bg { background-color: #60C6C8; }
|
||||
.ansi-cyan-intense-fg { color: #258F8F; }
|
||||
.ansi-cyan-intense-bg { background-color: #258F8F; }
|
||||
.ansi-white-fg { color: #C5C1B4; }
|
||||
.ansi-white-bg { background-color: #C5C1B4; }
|
||||
.ansi-white-intense-fg { color: #A1A6B2; }
|
||||
.ansi-white-intense-bg { background-color: #A1A6B2; }
|
||||
|
||||
.ansi-default-inverse-fg { color: #FFFFFF; }
|
||||
.ansi-default-inverse-bg { background-color: #000000; }
|
||||
|
||||
.ansi-bold { font-weight: bold; }
|
||||
.ansi-underline { text-decoration: underline; }
|
||||
|
||||
|
||||
div.nbinput.container div.input_area div[class*=highlight] > pre,
|
||||
div.nboutput.container div.output_area div[class*=highlight] > pre,
|
||||
div.nboutput.container div.output_area div[class*=highlight].math,
|
||||
div.nboutput.container div.output_area.rendered_html,
|
||||
div.nboutput.container div.output_area > div.output_javascript,
|
||||
div.nboutput.container div.output_area:not(.rendered_html) > img{
|
||||
padding: 5px;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
/* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
|
||||
div.nbinput.container div.input_area > div[class^='highlight'],
|
||||
div.nboutput.container div.output_area > div[class^='highlight']{
|
||||
overflow-y: hidden;
|
||||
}
|
||||
|
||||
/* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
|
||||
.prompt .copybtn {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* Some additional styling taken form the Jupyter notebook CSS */
|
||||
.jp-RenderedHTMLCommon table,
|
||||
div.rendered_html table {
|
||||
border: none;
|
||||
border-collapse: collapse;
|
||||
border-spacing: 0;
|
||||
color: black;
|
||||
font-size: 12px;
|
||||
table-layout: fixed;
|
||||
}
|
||||
.jp-RenderedHTMLCommon thead,
|
||||
div.rendered_html thead {
|
||||
border-bottom: 1px solid black;
|
||||
vertical-align: bottom;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tr,
|
||||
.jp-RenderedHTMLCommon th,
|
||||
.jp-RenderedHTMLCommon td,
|
||||
div.rendered_html tr,
|
||||
div.rendered_html th,
|
||||
div.rendered_html td {
|
||||
text-align: right;
|
||||
vertical-align: middle;
|
||||
padding: 0.5em 0.5em;
|
||||
line-height: normal;
|
||||
white-space: normal;
|
||||
max-width: none;
|
||||
border: none;
|
||||
}
|
||||
.jp-RenderedHTMLCommon th,
|
||||
div.rendered_html th {
|
||||
font-weight: bold;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tbody tr:nth-child(odd),
|
||||
div.rendered_html tbody tr:nth-child(odd) {
|
||||
background: #f5f5f5;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tbody tr:hover,
|
||||
div.rendered_html tbody tr:hover {
|
||||
background: rgba(66, 165, 245, 0.2);
|
||||
}
|
||||
</style>
|
||||
<div class="section" id="Primal-Components">
|
||||
<h1><span class="section-number">4. </span>Primal Components<a class="headerlink" href="#Primal-Components" title="Permalink to this headline">¶</a></h1>
|
||||
<p>In MIPLearn, a <strong>primal component</strong> is class that uses machine learning to predict a (potentially partial) assignment of values to the decision variables of the problem. Predicting high-quality primal solutions may be beneficial, as they allow the MIP solver to prune potentially large portions of the search space. Alternatively, if proof of optimality is not required, the MIP solver can be used to complete the partial solution generated by the machine learning model and and double-check its
|
||||
feasibility. MIPLearn allows both of these usage patterns.</p>
|
||||
<p>In this page, we describe the four primal components currently included in MIPLearn, which employ machine learning in different ways. Each component is highly configurable, and accepts an user-provided machine learning model, which it uses for all predictions. Each component can also be configured to provide the solution to the solver in multiple ways, depending on whether proof of optimality is required.</p>
|
||||
<div class="section" id="Primal-component-actions">
|
||||
<h2><span class="section-number">4.1. </span>Primal component actions<a class="headerlink" href="#Primal-component-actions" title="Permalink to this headline">¶</a></h2>
|
||||
<p>Before presenting the primal components themselves, we briefly discuss the three ways a solution may be provided to the solver. Each approach has benefits and limitations, which we also discuss in this section. All primal components can be configured to use any of the following approaches.</p>
|
||||
<p>The first approach is to provide the solution to the solver as a <strong>warm start</strong>. This is implemented by the class <a class="reference external" href="SetWarmStart">SetWarmStart</a>. The main advantage is that this method maintains all optimality and feasibility guarantees of the MIP solver, while still providing significant performance benefits for various classes of problems. If the machine learning model is able to predict multiple solutions, it is also possible to set multiple warm starts. In this case, the solver evaluates
|
||||
each warm start, discards the infeasible ones, then proceeds with the one that has the best objective value. The main disadvantage of this approach, compared to the next two, is that it provides relatively modest speedups for most problem classes, and no speedup at all for many others, even when the machine learning predictions are 100% accurate.</p>
|
||||
<p>The second approach is to <strong>fix the decision variables</strong> to their predicted values, then solve a restricted optimization problem on the remaining variables. This approach is implemented by the class <code class="docutils literal notranslate"><span class="pre">FixVariables</span></code>. The main advantage is its potential speedup: if machine learning can accurately predict values for a significant portion of the decision variables, then the MIP solver can typically complete the solution in a small fraction of the time it would take to find the same solution from
|
||||
scratch. The main disadvantage of this approach is that it loses optimality guarantees; that is, the complete solution found by the MIP solver may no longer be globally optimal. Also, if the machine learning predictions are not sufficiently accurate, there might not even be a feasible assignment for the variables that were left free.</p>
|
||||
<p>Finally, the third approach, which tries to strike a balance between the two previous ones, is to <strong>enforce proximity</strong> to a given solution. This strategy is implemented by the class <code class="docutils literal notranslate"><span class="pre">EnforceProximity</span></code>. More precisely, given values <span class="math notranslate nohighlight">\(\bar{x}_1,\ldots,\bar{x}_n\)</span> for a subset of binary decision variables <span class="math notranslate nohighlight">\(x_1,\ldots,x_n\)</span>, this approach adds the constraint</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[\sum_{i : \bar{x}_i=0} x_i + \sum_{i : \bar{x}_i=1} \left(1 - x_i\right) \leq k,\]</div>
|
||||
<p>to the problem, where <span class="math notranslate nohighlight">\(k\)</span> is a user-defined parameter, which indicates how many of the predicted variables are allowed to deviate from the machine learning suggestion. The main advantage of this approach, compared to fixing variables, is its tolerance to lower-quality machine learning predictions. Its main disadvantage is that it typically leads to smaller speedups, especially for larger values of <span class="math notranslate nohighlight">\(k\)</span>. This approach also loses optimality guarantees.</p>
|
||||
</div>
|
||||
<div class="section" id="Memorizing-primal-component">
|
||||
<h2><span class="section-number">4.2. </span>Memorizing primal component<a class="headerlink" href="#Memorizing-primal-component" title="Permalink to this headline">¶</a></h2>
|
||||
<p>A simple machine learning strategy for the prediction of primal solutions is to memorize all distinct solutions seen during training, then try to predict, during inference time, which of those memorized solutions are most likely to be feasible and to provide a good objective value for the current instance. The most promising solutions may alternatively be combined into a single partial solution, which is then provided to the MIP solver. Both variations of this strategy are implemented by the
|
||||
<code class="docutils literal notranslate"><span class="pre">MemorizingPrimalComponent</span></code> class. Note that it is only applicable if the problem size, and in fact if the meaning of the decision variables, remains the same across problem instances.</p>
|
||||
<p>More precisely, let <span class="math notranslate nohighlight">\(I_1,\ldots,I_n\)</span> be the training instances, and let <span class="math notranslate nohighlight">\(\bar{x}^1,\ldots,\bar{x}^n\)</span> be their respective optimal solutions. Given a new instance <span class="math notranslate nohighlight">\(I_{n+1}\)</span>, <code class="docutils literal notranslate"><span class="pre">MemorizingPrimalComponent</span></code> expects a user-provided binary classifier that assigns (through the <code class="docutils literal notranslate"><span class="pre">predict_proba</span></code> method, following scikit-learn’s conventions) a score <span class="math notranslate nohighlight">\(\delta_i\)</span> to each solution <span class="math notranslate nohighlight">\(\bar{x}^i\)</span>, such that solutions with higher score are more likely to be good solutions for
|
||||
<span class="math notranslate nohighlight">\(I_{n+1}\)</span>. The features provided to the classifier are the instance features computed by an user-provided extractor. Given these scores, the component then performs one of the following to actions, as decided by the user:</p>
|
||||
<ol class="arabic">
|
||||
<li><p>Selects the top <span class="math notranslate nohighlight">\(k\)</span> solutions with the highest scores and provides them to the solver; this is implemented by <code class="docutils literal notranslate"><span class="pre">SelectTopSolutions</span></code>, and it is typically used with the <code class="docutils literal notranslate"><span class="pre">SetWarmStart</span></code> action.</p></li>
|
||||
<li><p>Merges the top <span class="math notranslate nohighlight">\(k\)</span> solutions into a single partial solution, then provides it to the solver. This is implemented by <code class="docutils literal notranslate"><span class="pre">MergeTopSolutions</span></code>. More precisely, suppose that the machine learning regressor ordered the solutions in the sequence <span class="math notranslate nohighlight">\(\bar{x}^{i_1},\ldots,\bar{x}^{i_n}\)</span>, with the most promising solutions appearing first, and with ties being broken arbitrarily. The component starts by keeping only the <span class="math notranslate nohighlight">\(k\)</span> most promising solutions <span class="math notranslate nohighlight">\(\bar{x}^{i_1},\ldots,\bar{x}^{i_k}\)</span>.
|
||||
Then it computes, for each binary decision variable <span class="math notranslate nohighlight">\(x_l\)</span>, its average assigned value <span class="math notranslate nohighlight">\(\tilde{x}_l\)</span>:</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[\tilde{x}_l = \frac{1}{k} \sum_{j=1}^k \bar{x}^{i_j}_l.\]</div>
|
||||
<p>Finally, the component constructs a merged solution <span class="math notranslate nohighlight">\(y\)</span>, defined as:</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[\begin{split}y_j = \begin{cases}
|
||||
0 & \text{ if } \tilde{x}_l \le \theta_0 \\
|
||||
1 & \text{ if } \tilde{x}_l \ge \theta_1 \\
|
||||
\square & \text{otherwise,}
|
||||
\end{cases}\end{split}\]</div>
|
||||
<p>where <span class="math notranslate nohighlight">\(\theta_0\)</span> and <span class="math notranslate nohighlight">\(\theta_1\)</span> are user-specified parameters, and where <span class="math notranslate nohighlight">\(\square\)</span> indicates that the variable is left undefined. The solution <span class="math notranslate nohighlight">\(y\)</span> is then provided by the solver using any of the three approaches defined in the previous section.</p>
|
||||
</li>
|
||||
</ol>
|
||||
<p>The above specification of <code class="docutils literal notranslate"><span class="pre">MemorizingPrimalComponent</span></code> is meant to be as general as possible. Simpler strategies can be implemented by configuring this component in specific ways. For example, a simpler approach employed in the literature is to collect all optimal solutions, then provide the entire list of solutions to the solver as warm starts, without any filtering or post-processing. This strategy can be implemented with <code class="docutils literal notranslate"><span class="pre">MemorizingPrimalComponent</span></code> by using a model that returns a constant
|
||||
value for all solutions (e.g. <a class="reference external" href="https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyClassifier.html">scikit-learn’s DummyClassifier</a>), then selecting the top <span class="math notranslate nohighlight">\(n\)</span> (instead of <span class="math notranslate nohighlight">\(k\)</span>) solutions. See example below. Another simple approach is taking the solution to the most similar instance, and using it, by itself, as a warm start. This can be implemented by using a model that computes distances between the current instance and the training ones (e.g. <a class="reference external" href="https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html">scikit-learn’s
|
||||
KNeighborsClassifier</a>), then select the solution to the nearest one. See also example below. More complex strategies, of course, can also be configured.</p>
|
||||
<div class="section" id="Examples">
|
||||
<h3>Examples<a class="headerlink" href="#Examples" title="Permalink to this headline">¶</a></h3>
|
||||
<div class="nbinput nblast docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[3]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><br/><span></span><span class="kn">from</span> <span class="nn">sklearn.dummy</span> <span class="kn">import</span> <span class="n">DummyClassifier</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.neighbors</span> <span class="kn">import</span> <span class="n">KNeighborsClassifier</span>
|
||||
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.actions</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">SetWarmStart</span><span class="p">,</span>
|
||||
<span class="n">FixVariables</span><span class="p">,</span>
|
||||
<span class="n">EnforceProximity</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.mem</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">MemorizingPrimalComponent</span><span class="p">,</span>
|
||||
<span class="n">SelectTopSolutions</span><span class="p">,</span>
|
||||
<span class="n">MergeTopSolutions</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.extractors.dummy</span> <span class="kn">import</span> <span class="n">DummyExtractor</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.extractors.fields</span> <span class="kn">import</span> <span class="n">H5FieldsExtractor</span>
|
||||
|
||||
<span class="c1"># Configures a memorizing primal component that collects</span>
|
||||
<span class="c1"># all distinct solutions seen during training and provides</span>
|
||||
<span class="c1"># them to the solver without any filtering or post-processing.</span>
|
||||
<span class="n">comp1</span> <span class="o">=</span> <span class="n">MemorizingPrimalComponent</span><span class="p">(</span>
|
||||
<span class="n">clf</span><span class="o">=</span><span class="n">DummyClassifier</span><span class="p">(),</span>
|
||||
<span class="n">extractor</span><span class="o">=</span><span class="n">DummyExtractor</span><span class="p">(),</span>
|
||||
<span class="n">constructor</span><span class="o">=</span><span class="n">SelectTopSolutions</span><span class="p">(</span><span class="mi">1_000_000</span><span class="p">),</span>
|
||||
<span class="n">action</span><span class="o">=</span><span class="n">SetWarmStart</span><span class="p">(),</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># Configures a memorizing primal component that finds the</span>
|
||||
<span class="c1"># training instance with the closest objective function, then</span>
|
||||
<span class="c1"># fixes the decision variables to the values they assumed</span>
|
||||
<span class="c1"># at the optimal solution for that instance.</span>
|
||||
<span class="n">comp2</span> <span class="o">=</span> <span class="n">MemorizingPrimalComponent</span><span class="p">(</span>
|
||||
<span class="n">clf</span><span class="o">=</span><span class="n">KNeighborsClassifier</span><span class="p">(</span><span class="n">n_neighbors</span><span class="o">=</span><span class="mi">1</span><span class="p">),</span>
|
||||
<span class="n">extractor</span><span class="o">=</span><span class="n">H5FieldsExtractor</span><span class="p">(</span>
|
||||
<span class="n">instance_fields</span><span class="o">=</span><span class="p">[</span><span class="s2">"static_var_obj_coeffs"</span><span class="p">],</span>
|
||||
<span class="p">),</span>
|
||||
<span class="n">constructor</span><span class="o">=</span><span class="n">SelectTopSolutions</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span>
|
||||
<span class="n">action</span><span class="o">=</span><span class="n">FixVariables</span><span class="p">(),</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># Configures a memorizing primal component that finds the distinct</span>
|
||||
<span class="c1"># solutions to the 10 most similar training problem instances,</span>
|
||||
<span class="c1"># selects the 3 solutions that were most often optimal to these</span>
|
||||
<span class="c1"># training instances, combines them into a single partial solution,</span>
|
||||
<span class="c1"># then enforces proximity, allowing at most 3 variables to deviate</span>
|
||||
<span class="c1"># from the machine learning suggestion.</span>
|
||||
<span class="n">comp3</span> <span class="o">=</span> <span class="n">MemorizingPrimalComponent</span><span class="p">(</span>
|
||||
<span class="n">clf</span><span class="o">=</span><span class="n">KNeighborsClassifier</span><span class="p">(</span><span class="n">n_neighbors</span><span class="o">=</span><span class="mi">10</span><span class="p">),</span>
|
||||
<span class="n">extractor</span><span class="o">=</span><span class="n">H5FieldsExtractor</span><span class="p">(</span><span class="n">instance_fields</span><span class="o">=</span><span class="p">[</span><span class="s2">"static_var_obj_coeffs"</span><span class="p">]),</span>
|
||||
<span class="n">constructor</span><span class="o">=</span><span class="n">MergeTopSolutions</span><span class="p">(</span><span class="n">k</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span> <span class="n">thresholds</span><span class="o">=</span><span class="p">[</span><span class="mf">0.25</span><span class="p">,</span> <span class="mf">0.75</span><span class="p">]),</span>
|
||||
<span class="n">action</span><span class="o">=</span><span class="n">EnforceProximity</span><span class="p">(</span><span class="mi">3</span><span class="p">),</span>
|
||||
<span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="Independent-vars-primal-component">
|
||||
<h2><span class="section-number">4.3. </span>Independent vars primal component<a class="headerlink" href="#Independent-vars-primal-component" title="Permalink to this headline">¶</a></h2>
|
||||
<p>Instead of memorizing previously-seen primal solutions, it is also natural to use machine learning models to directly predict the values of the decision variables, constructing a solution from scratch. This approach has the benefit of potentially constructing novel high-quality solutions, never observed in the training data. Two variations of this strategy are supported by MIPLearn: (i) predicting the values of the decision variables independently, using multiple ML models; or (ii) predicting
|
||||
the values jointly, with a single model. We describe the first variation in this section, and the second variation in the next section.</p>
|
||||
<p>Let <span class="math notranslate nohighlight">\(I_1,\ldots,I_n\)</span> be the training instances, and let <span class="math notranslate nohighlight">\(\bar{x}^1,\ldots,\bar{x}^n\)</span> be their respective optimal solutions. For each binary decision variable <span class="math notranslate nohighlight">\(x_j\)</span>, the component <code class="docutils literal notranslate"><span class="pre">IndependentVarsPrimalComponent</span></code> creates a copy of a user-provided binary classifier and trains it to predict the optimal value of <span class="math notranslate nohighlight">\(x_j\)</span>, given <span class="math notranslate nohighlight">\(\bar{x}^1_j,\ldots,\bar{x}^n_j\)</span> as training labels. The features provided to the model are the variable features computed by an user-provided
|
||||
extractor. During inference time, the component uses these <span class="math notranslate nohighlight">\(n\)</span> binary classifiers to construct a solution and provides it to the solver using one of the available actions.</p>
|
||||
<p>Three issues often arise in practice when using this approach:</p>
|
||||
<ol class="arabic simple">
|
||||
<li><p>For certain binary variables <span class="math notranslate nohighlight">\(x_j\)</span>, it is frequently the case that its optimal value is either always zero or always one in the training dataset, which poses problems to some standard scikit-learn classifiers, since they do not expect a single class. The wrapper <code class="docutils literal notranslate"><span class="pre">SingleClassFix</span></code> can be used to fix this issue (see example below).</p></li>
|
||||
<li><p>It is also frequently the case that machine learning classifier can only reliably predict the values of some variables with high accuracy, not all of them. In this situation, instead of computing a complete primal solution, it may be more beneficial to construct a partial solution containing values only for the variables for which the ML made a high-confidence prediction. The meta-classifier <code class="docutils literal notranslate"><span class="pre">MinProbabilityClassifier</span></code> can be used for this purpose. It asks the base classifier for the
|
||||
probability of the value being zero or one (using the <code class="docutils literal notranslate"><span class="pre">predict_proba</span></code> method) and erases from the primal solution all values whose probabilities are below a given threshold.</p></li>
|
||||
<li><p>To make multiple copies of the provided ML classifier, MIPLearn uses the standard <code class="docutils literal notranslate"><span class="pre">sklearn.base.clone</span></code> method, which may not be suitable for classifiers from other frameworks. To handle this, it is possible to override the clone function using the <code class="docutils literal notranslate"><span class="pre">clone_fn</span></code> constructor argument.</p></li>
|
||||
</ol>
|
||||
<div class="section" id="id1">
|
||||
<h3>Examples<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3>
|
||||
<div class="nbinput nblast docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[4]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="kn">import</span> <span class="n">LogisticRegression</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.classifiers.minprob</span> <span class="kn">import</span> <span class="n">MinProbabilityClassifier</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.classifiers.singleclass</span> <span class="kn">import</span> <span class="n">SingleClassFix</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.indep</span> <span class="kn">import</span> <span class="n">IndependentVarsPrimalComponent</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.extractors.AlvLouWeh2017</span> <span class="kn">import</span> <span class="n">AlvLouWeh2017Extractor</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.solvers.learning</span> <span class="kn">import</span> <span class="n">LearningSolver</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.actions</span> <span class="kn">import</span> <span class="n">SetWarmStart</span>
|
||||
|
||||
<span class="c1"># Configures a primal component that independently predicts the value of each</span>
|
||||
<span class="c1"># binary variable using logistic regression and provides it to the solver as</span>
|
||||
<span class="c1"># warm start. Erases predictions with probability less than 99%; applies</span>
|
||||
<span class="c1"># single-class fix; and uses AlvLouWeh2017 features.</span>
|
||||
<span class="n">comp</span> <span class="o">=</span> <span class="n">IndependentVarsPrimalComponent</span><span class="p">(</span>
|
||||
<span class="n">base_clf</span><span class="o">=</span><span class="n">SingleClassFix</span><span class="p">(</span>
|
||||
<span class="n">MinProbabilityClassifier</span><span class="p">(</span>
|
||||
<span class="n">base_clf</span><span class="o">=</span><span class="n">LogisticRegression</span><span class="p">(),</span>
|
||||
<span class="n">thresholds</span><span class="o">=</span><span class="p">[</span><span class="mf">0.99</span><span class="p">,</span> <span class="mf">0.99</span><span class="p">],</span>
|
||||
<span class="p">),</span>
|
||||
<span class="p">),</span>
|
||||
<span class="n">extractor</span><span class="o">=</span><span class="n">AlvLouWeh2017Extractor</span><span class="p">(),</span>
|
||||
<span class="n">action</span><span class="o">=</span><span class="n">SetWarmStart</span><span class="p">(),</span>
|
||||
<span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="Joint-vars-primal-component">
|
||||
<h2><span class="section-number">4.4. </span>Joint vars primal component<a class="headerlink" href="#Joint-vars-primal-component" title="Permalink to this headline">¶</a></h2>
|
||||
<p>In the previous subsection, we used multiple machine learning models to independently predict the values of the binary decision variables. When these values are correlated, an alternative approach is to jointly predict the values of all binary variables using a single machine learning model. This strategy is implemented by <code class="docutils literal notranslate"><span class="pre">JointVarsPrimalComponent</span></code>. Compared to the previous ones, this component is much more straightforwad. It simply extracts instance features, using the user-provided feature
|
||||
extractor, then directly trains the user-provided binary classifier (using the <code class="docutils literal notranslate"><span class="pre">fit</span></code> method), without making any copies. The trained classifier is then used to predict entire solutions (using the <code class="docutils literal notranslate"><span class="pre">predict</span></code> method), which are given to the solver using one of the previously discussed methods. In the example below, we illustrate the usage of this component with a simple feed-forward neural network.</p>
|
||||
<p><code class="docutils literal notranslate"><span class="pre">JointVarsPrimalComponent</span></code> can also be used to implement strategies that use multiple machine learning models, but not indepedently. For example, a common strategy in multioutput prediction is building a <em>classifier chain</em>. In this approach, the first decision variable is predicted using the instance features alone; but the <span class="math notranslate nohighlight">\(n\)</span>-th decision variable is predicted using the instance features plus the predicted values of the <span class="math notranslate nohighlight">\(n-1\)</span> previous variables. This can be easily implemented
|
||||
using scikit-learn’s <code class="docutils literal notranslate"><span class="pre">ClassifierChain</span></code> estimator, as shown in the example below.</p>
|
||||
<div class="section" id="id2">
|
||||
<h3>Examples<a class="headerlink" href="#id2" title="Permalink to this headline">¶</a></h3>
|
||||
<div class="nbinput nblast docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[5]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.multioutput</span> <span class="kn">import</span> <span class="n">ClassifierChain</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.neural_network</span> <span class="kn">import</span> <span class="n">MLPClassifier</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.joint</span> <span class="kn">import</span> <span class="n">JointVarsPrimalComponent</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.extractors.fields</span> <span class="kn">import</span> <span class="n">H5FieldsExtractor</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.solvers.learning</span> <span class="kn">import</span> <span class="n">LearningSolver</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.actions</span> <span class="kn">import</span> <span class="n">SetWarmStart</span>
|
||||
|
||||
<span class="c1"># Configures a primal component that uses a feedforward neural network</span>
|
||||
<span class="c1"># to jointly predict the values of the binary variables, based on the</span>
|
||||
<span class="c1"># objective cost function, and provides the solution to the solver as</span>
|
||||
<span class="c1"># a warm start.</span>
|
||||
<span class="n">comp</span> <span class="o">=</span> <span class="n">JointVarsPrimalComponent</span><span class="p">(</span>
|
||||
<span class="n">clf</span><span class="o">=</span><span class="n">MLPClassifier</span><span class="p">(),</span>
|
||||
<span class="n">extractor</span><span class="o">=</span><span class="n">H5FieldsExtractor</span><span class="p">(</span>
|
||||
<span class="n">instance_fields</span><span class="o">=</span><span class="p">[</span><span class="s1">'static_var_obj_coeffs'</span><span class="p">],</span>
|
||||
<span class="p">),</span>
|
||||
<span class="n">action</span><span class="o">=</span><span class="n">SetWarmStart</span><span class="p">(),</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># Configures a primal component that uses a chain of logistic regression</span>
|
||||
<span class="c1"># models to jointly predict the values of the binary variables, based on</span>
|
||||
<span class="c1"># the objective function.</span>
|
||||
<span class="n">comp</span> <span class="o">=</span> <span class="n">JointVarsPrimalComponent</span><span class="p">(</span>
|
||||
<span class="n">clf</span><span class="o">=</span><span class="n">ClassifierChain</span><span class="p">(</span><span class="n">SingleClassFix</span><span class="p">(</span><span class="n">LogisticRegression</span><span class="p">())),</span>
|
||||
<span class="n">extractor</span><span class="o">=</span><span class="n">H5FieldsExtractor</span><span class="p">(</span>
|
||||
<span class="n">instance_fields</span><span class="o">=</span><span class="p">[</span><span class="s1">'static_var_obj_coeffs'</span><span class="p">],</span>
|
||||
<span class="p">),</span>
|
||||
<span class="n">action</span><span class="o">=</span><span class="n">SetWarmStart</span><span class="p">(),</span>
|
||||
<span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="Expert-primal-component">
|
||||
<h2><span class="section-number">4.5. </span>Expert primal component<a class="headerlink" href="#Expert-primal-component" title="Permalink to this headline">¶</a></h2>
|
||||
<p>Before spending time and effort choosing a machine learning strategy and tweaking its parameters, it is usually a good idea to evaluate what would be the performance impact of the model if its predictions were 100% accurate. This is especially important for the prediction of warm starts, since they are not always very beneficial. To simplify this task, MIPLearn provides <code class="docutils literal notranslate"><span class="pre">ExpertPrimalComponent</span></code>, a component which simply loads the optimal solution from the HDF5 file, assuming that it has already
|
||||
been computed, then directly provides it to the solver using one of the available methods. This component is useful in benchmarks, to evaluate how close to the best theoretical performance the machine learning components are.</p>
|
||||
<div class="section" id="Example">
|
||||
<h3>Example<a class="headerlink" href="#Example" title="Permalink to this headline">¶</a></h3>
|
||||
<div class="nbinput nblast docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[4]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">miplearn.components.primal.expert</span> <span class="kn">import</span> <span class="n">ExpertPrimalComponent</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.actions</span> <span class="kn">import</span> <span class="n">SetWarmStart</span>
|
||||
|
||||
<span class="c1"># Configures an expert primal component, which reads a pre-computed</span>
|
||||
<span class="c1"># optimal solution from the HDF5 file and provides it to the solver</span>
|
||||
<span class="c1"># as warm start.</span>
|
||||
<span class="n">comp</span> <span class="o">=</span> <span class="n">ExpertPrimalComponent</span><span class="p">(</span>
|
||||
<span class="n">action</span><span class="o">=</span><span class="n">SetWarmStart</span><span class="p">()</span>
|
||||
<span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<div class='prev-next-bottom'>
|
||||
|
||||
<a class='left-prev' id="prev-link" href="../features/" title="previous page"><span class="section-number">3. </span>Feature Extractors</a>
|
||||
<a class='right-next' id="next-link" href="../solvers/" title="next page"><span class="section-number">5. </span>Solvers</a>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<footer class="footer mt-5 mt-md-0">
|
||||
<div class="container">
|
||||
<p>
|
||||
|
||||
© Copyright 2020-2022, UChicago Argonne, LLC.<br/>
|
||||
</p>
|
||||
</div>
|
||||
</footer>
|
||||
</main>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="../../_static/js/index.1c5a1a01449ed65a7b51.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
1565
0.3/guide/problems.ipynb
Normal file
1565
0.3/guide/problems.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
1831
0.3/guide/problems/index.html
Normal file
1831
0.3/guide/problems/index.html
Normal file
File diff suppressed because it is too large
Load Diff
211
0.3/guide/solvers.ipynb
Normal file
211
0.3/guide/solvers.ipynb
Normal file
@@ -0,0 +1,211 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3371f072-be1e-4c47-b765-b5d30fdbfae6",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Solvers\n",
|
||||
"\n",
|
||||
"## LearningSolver\n",
|
||||
"\n",
|
||||
"### Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "dac704b5",
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Gurobi Optimizer version 9.5.2 build v9.5.2rc0 (linux64)\n",
|
||||
"Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
|
||||
"Optimize a model with 10 rows, 45 columns and 90 nonzeros\n",
|
||||
"Model fingerprint: 0x6ddcd141\n",
|
||||
"Coefficient statistics:\n",
|
||||
" Matrix range [1e+00, 1e+00]\n",
|
||||
" Objective range [4e+01, 1e+03]\n",
|
||||
" Bounds range [1e+00, 1e+00]\n",
|
||||
" RHS range [2e+00, 2e+00]\n",
|
||||
"Presolve time: 0.00s\n",
|
||||
"Presolved: 10 rows, 45 columns, 90 nonzeros\n",
|
||||
"\n",
|
||||
"Iteration Objective Primal Inf. Dual Inf. Time\n",
|
||||
" 0 6.3600000e+02 1.700000e+01 0.000000e+00 0s\n",
|
||||
" 15 2.7610000e+03 0.000000e+00 0.000000e+00 0s\n",
|
||||
"\n",
|
||||
"Solved in 15 iterations and 0.01 seconds (0.00 work units)\n",
|
||||
"Optimal objective 2.761000000e+03\n",
|
||||
"Set parameter LazyConstraints to value 1\n",
|
||||
"Gurobi Optimizer version 9.5.2 build v9.5.2rc0 (linux64)\n",
|
||||
"Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
|
||||
"Optimize a model with 10 rows, 45 columns and 90 nonzeros\n",
|
||||
"Model fingerprint: 0x74ca3d0a\n",
|
||||
"Variable types: 0 continuous, 45 integer (45 binary)\n",
|
||||
"Coefficient statistics:\n",
|
||||
" Matrix range [1e+00, 1e+00]\n",
|
||||
" Objective range [4e+01, 1e+03]\n",
|
||||
" Bounds range [1e+00, 1e+00]\n",
|
||||
" RHS range [2e+00, 2e+00]\n",
|
||||
"\n",
|
||||
"User MIP start produced solution with objective 2796 (0.01s)\n",
|
||||
"Loaded user MIP start with objective 2796\n",
|
||||
"\n",
|
||||
"Presolve time: 0.00s\n",
|
||||
"Presolved: 10 rows, 45 columns, 90 nonzeros\n",
|
||||
"Variable types: 0 continuous, 45 integer (45 binary)\n",
|
||||
"\n",
|
||||
"Root relaxation: objective 2.761000e+03, 14 iterations, 0.00 seconds (0.00 work units)\n",
|
||||
"\n",
|
||||
" Nodes | Current Node | Objective Bounds | Work\n",
|
||||
" Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time\n",
|
||||
"\n",
|
||||
" 0 0 cutoff 0 2796.00000 2796.00000 0.00% - 0s\n",
|
||||
"\n",
|
||||
"Cutting planes:\n",
|
||||
" Lazy constraints: 3\n",
|
||||
"\n",
|
||||
"Explored 1 nodes (15 simplex iterations) in 0.02 seconds (0.00 work units)\n",
|
||||
"Thread count was 12 (of 12 available processors)\n",
|
||||
"\n",
|
||||
"Solution count 1: 2796 \n",
|
||||
"\n",
|
||||
"Optimal solution found (tolerance 1.00e-04)\n",
|
||||
"Best objective 2.796000000000e+03, best bound 2.796000000000e+03, gap 0.0000%\n",
|
||||
"\n",
|
||||
"User-callback calls 103, time in user-callback 0.00 sec\n",
|
||||
"Gurobi Optimizer version 9.5.2 build v9.5.2rc0 (linux64)\n",
|
||||
"Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
|
||||
"Optimize a model with 10 rows, 45 columns and 90 nonzeros\n",
|
||||
"Model fingerprint: 0x74ca3d0a\n",
|
||||
"Variable types: 0 continuous, 45 integer (45 binary)\n",
|
||||
"Coefficient statistics:\n",
|
||||
" Matrix range [1e+00, 1e+00]\n",
|
||||
" Objective range [4e+01, 1e+03]\n",
|
||||
" Bounds range [1e+00, 1e+00]\n",
|
||||
" RHS range [2e+00, 2e+00]\n",
|
||||
"Presolved: 10 rows, 45 columns, 90 nonzeros\n",
|
||||
"\n",
|
||||
"Continuing optimization...\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Cutting planes:\n",
|
||||
" Lazy constraints: 3\n",
|
||||
"\n",
|
||||
"Explored 1 nodes (15 simplex iterations) in 0.01 seconds (0.00 work units)\n",
|
||||
"Thread count was 12 (of 12 available processors)\n",
|
||||
"\n",
|
||||
"Solution count 1: 2796 \n",
|
||||
"\n",
|
||||
"Optimal solution found (tolerance 1.00e-04)\n",
|
||||
"Best objective 2.796000000000e+03, best bound 2.796000000000e+03, gap 0.0000%\n",
|
||||
"\n",
|
||||
"User-callback calls 27, time in user-callback 0.00 sec\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"import numpy as np\n",
|
||||
"from scipy.stats import uniform, randint\n",
|
||||
"from sklearn.linear_model import LogisticRegression\n",
|
||||
"\n",
|
||||
"from miplearn.classifiers.minprob import MinProbabilityClassifier\n",
|
||||
"from miplearn.classifiers.singleclass import SingleClassFix\n",
|
||||
"from miplearn.collectors.basic import BasicCollector\n",
|
||||
"from miplearn.components.primal.actions import SetWarmStart\n",
|
||||
"from miplearn.components.primal.indep import IndependentVarsPrimalComponent\n",
|
||||
"from miplearn.extractors.AlvLouWeh2017 import AlvLouWeh2017Extractor\n",
|
||||
"from miplearn.io import save\n",
|
||||
"from miplearn.problems.tsp import (\n",
|
||||
" TravelingSalesmanGenerator,\n",
|
||||
" build_tsp_model,\n",
|
||||
")\n",
|
||||
"from miplearn.solvers.learning import LearningSolver\n",
|
||||
"\n",
|
||||
"# Set random seed to make example reproducible.\n",
|
||||
"random.seed(42)\n",
|
||||
"np.random.seed(42)\n",
|
||||
"\n",
|
||||
"# Generate a few instances of the traveling salesman problem.\n",
|
||||
"data = TravelingSalesmanGenerator(\n",
|
||||
" n=randint(low=10, high=11),\n",
|
||||
" x=uniform(loc=0.0, scale=1000.0),\n",
|
||||
" y=uniform(loc=0.0, scale=1000.0),\n",
|
||||
" gamma=uniform(loc=0.90, scale=0.20),\n",
|
||||
" fix_cities=True,\n",
|
||||
" round=True,\n",
|
||||
").generate(50)\n",
|
||||
"\n",
|
||||
"# Save instance data to data/tsp/00000.pkl.gz, data/tsp/00001.pkl.gz, ...\n",
|
||||
"all_data = save(data, \"data/tsp\")\n",
|
||||
"\n",
|
||||
"# Split train/test data\n",
|
||||
"train_data = all_data[:40]\n",
|
||||
"test_data = all_data[40:]\n",
|
||||
"\n",
|
||||
"# Collect training data\n",
|
||||
"bc = BasicCollector(time_limit_sec=3600)\n",
|
||||
"bc.collect(train_data, build_tsp_model, n_jobs=4)\n",
|
||||
"\n",
|
||||
"# Build learning solver\n",
|
||||
"solver = LearningSolver(\n",
|
||||
" components=[\n",
|
||||
" IndependentVarsPrimalComponent(\n",
|
||||
" base_clf=SingleClassFix(\n",
|
||||
" MinProbabilityClassifier(\n",
|
||||
" base_clf=LogisticRegression(),\n",
|
||||
" thresholds=[0.95, 0.95],\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
" extractor=AlvLouWeh2017Extractor(),\n",
|
||||
" action=SetWarmStart(),\n",
|
||||
" )\n",
|
||||
" ]\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Train ML models\n",
|
||||
"solver.fit(train_data)\n",
|
||||
"\n",
|
||||
"# Solve a test instance\n",
|
||||
"solver.optimize(test_data[0], build_tsp_model);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "e27d2cbd-5341-461d-bbc1-8131aee8d949",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.13"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
691
0.3/guide/solvers/index.html
Normal file
691
0.3/guide/solvers/index.html
Normal file
@@ -0,0 +1,691 @@
|
||||
|
||||
<!DOCTYPE html>
|
||||
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>5. Solvers — MIPLearn 0.3</title>
|
||||
|
||||
<link href="../../_static/css/theme.css" rel="stylesheet" />
|
||||
<link href="../../_static/css/index.c5995385ac14fb8791e8eb36b4908be2.css" rel="stylesheet" />
|
||||
|
||||
|
||||
<link rel="stylesheet"
|
||||
href="../../_static/vendor/fontawesome/5.13.0/css/all.min.css">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="../../_static/vendor/fontawesome/5.13.0/webfonts/fa-solid-900.woff2">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="../../_static/vendor/fontawesome/5.13.0/webfonts/fa-brands-400.woff2">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
|
||||
<link rel="stylesheet" href="../../_static/sphinx-book-theme.acff12b8f9c144ce68a297486a2fa670.css" type="text/css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../_static/custom.css" />
|
||||
|
||||
<link rel="preload" as="script" href="../../_static/js/index.1c5a1a01449ed65a7b51.js">
|
||||
|
||||
<script id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
|
||||
<script src="../../_static/jquery.js"></script>
|
||||
<script src="../../_static/underscore.js"></script>
|
||||
<script src="../../_static/doctools.js"></script>
|
||||
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
|
||||
<script src="../../_static/sphinx-book-theme.12a9622fbb08dcb3a2a40b2c02b83a57.js"></script>
|
||||
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
|
||||
<script type="text/x-mathjax-config">MathJax.Hub.Config({"tex2jax": {"inlineMath": [["\\(", "\\)"]], "displayMath": [["\\[", "\\]"]], "processRefs": false, "processEnvironments": false}})</script>
|
||||
<link rel="index" title="Index" href="../../genindex/" />
|
||||
<link rel="search" title="Search" href="../../search/" />
|
||||
<link rel="next" title="6. Benchmark Problems" href="../../api/problems/" />
|
||||
<link rel="prev" title="4. Primal Components" href="../primal/" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="en" />
|
||||
|
||||
</head>
|
||||
<body data-spy="scroll" data-target="#bd-toc-nav" data-offset="80">
|
||||
|
||||
<div class="container-fluid" id="banner"></div>
|
||||
|
||||
|
||||
|
||||
<div class="container-xl">
|
||||
<div class="row">
|
||||
|
||||
<div class="col-12 col-md-3 bd-sidebar site-navigation show" id="site-navigation">
|
||||
|
||||
<div class="navbar-brand-box">
|
||||
<a class="navbar-brand text-wrap" href="../../">
|
||||
|
||||
|
||||
<h1 class="site-logo" id="site-title">MIPLearn 0.3</h1>
|
||||
|
||||
</a>
|
||||
</div><form class="bd-search d-flex align-items-center" action="../../search/" method="get">
|
||||
<i class="icon fas fa-search"></i>
|
||||
<input type="search" class="form-control" name="q" id="search-input" placeholder="Search the docs ..." aria-label="Search the docs ..." autocomplete="off" >
|
||||
</form><nav class="bd-links" id="bd-docs-nav" aria-label="Main navigation">
|
||||
<div class="bd-toc-item active">
|
||||
<p class="caption">
|
||||
<span class="caption-text">
|
||||
User Guide
|
||||
</span>
|
||||
</p>
|
||||
<ul class="current nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../problems/">
|
||||
1. Benchmark Problems
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../collectors/">
|
||||
2. Training Data Collectors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../features/">
|
||||
3. Feature Extractors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../primal/">
|
||||
4. Primal Components
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1 current active">
|
||||
<a class="current reference internal" href="#">
|
||||
5. Solvers
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p class="caption">
|
||||
<span class="caption-text">
|
||||
API Reference
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/problems/">
|
||||
6. Benchmark Problems
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/collectors/">
|
||||
7. Collectors & Extractors
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/components/">
|
||||
8. Components
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/solvers/">
|
||||
9. Solvers
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="../../api/helpers/">
|
||||
10. Helpers
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav> <!-- To handle the deprecated key -->
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<main class="col py-md-3 pl-md-4 bd-content overflow-auto" role="main">
|
||||
|
||||
<div class="topbar container-xl fixed-top">
|
||||
<div class="topbar-contents row">
|
||||
<div class="col-12 col-md-3 bd-topbar-whitespace site-navigation show"></div>
|
||||
<div class="col pl-md-4 topbar-main">
|
||||
|
||||
<button id="navbar-toggler" class="navbar-toggler ml-0" type="button" data-toggle="collapse"
|
||||
data-toggle="tooltip" data-placement="bottom" data-target=".site-navigation" aria-controls="navbar-menu"
|
||||
aria-expanded="true" aria-label="Toggle navigation" aria-controls="site-navigation"
|
||||
title="Toggle navigation" data-toggle="tooltip" data-placement="left">
|
||||
<i class="fas fa-bars"></i>
|
||||
<i class="fas fa-arrow-left"></i>
|
||||
<i class="fas fa-arrow-up"></i>
|
||||
</button>
|
||||
|
||||
|
||||
<div class="dropdown-buttons-trigger">
|
||||
<button id="dropdown-buttons-trigger" class="btn btn-secondary topbarbtn" aria-label="Download this page"><i
|
||||
class="fas fa-download"></i></button>
|
||||
|
||||
<div class="dropdown-buttons">
|
||||
<!-- ipynb file if we had a myst markdown file -->
|
||||
|
||||
<!-- Download raw file -->
|
||||
<a class="dropdown-buttons" href="../../_sources/guide/solvers.ipynb.txt"><button type="button"
|
||||
class="btn btn-secondary topbarbtn" title="Download source file" data-toggle="tooltip"
|
||||
data-placement="left">.ipynb</button></a>
|
||||
<!-- Download PDF via print -->
|
||||
<button type="button" id="download-print" class="btn btn-secondary topbarbtn" title="Print to PDF"
|
||||
onClick="window.print()" data-toggle="tooltip" data-placement="left">.pdf</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Source interaction buttons -->
|
||||
|
||||
<!-- Full screen (wrap in <a> to have style consistency -->
|
||||
|
||||
<a class="full-screen-button"><button type="button" class="btn btn-secondary topbarbtn" data-toggle="tooltip"
|
||||
data-placement="bottom" onclick="toggleFullScreen()" aria-label="Fullscreen mode"
|
||||
title="Fullscreen mode"><i
|
||||
class="fas fa-expand"></i></button></a>
|
||||
|
||||
<!-- Launch buttons -->
|
||||
|
||||
</div>
|
||||
|
||||
<!-- Table of contents -->
|
||||
<div class="d-none d-md-block col-md-2 bd-toc show">
|
||||
|
||||
<div class="tocsection onthispage pt-5 pb-3">
|
||||
<i class="fas fa-list"></i> Contents
|
||||
</div>
|
||||
<nav id="bd-toc-nav">
|
||||
<ul class="visible nav section-nav flex-column">
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#LearningSolver">
|
||||
5.1. LearningSolver
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#Example">
|
||||
Example
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="main-content" class="row">
|
||||
<div class="col-12 col-md-9 pl-md-3 pr-md-0">
|
||||
|
||||
<div>
|
||||
|
||||
|
||||
<style>
|
||||
/* CSS for nbsphinx extension */
|
||||
|
||||
/* remove conflicting styling from Sphinx themes */
|
||||
div.nbinput.container div.prompt *,
|
||||
div.nboutput.container div.prompt *,
|
||||
div.nbinput.container div.input_area pre,
|
||||
div.nboutput.container div.output_area pre,
|
||||
div.nbinput.container div.input_area .highlight,
|
||||
div.nboutput.container div.output_area .highlight {
|
||||
border: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
box-shadow: none;
|
||||
}
|
||||
|
||||
div.nbinput.container > div[class*=highlight],
|
||||
div.nboutput.container > div[class*=highlight] {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt *,
|
||||
div.nboutput.container div.prompt * {
|
||||
background: none;
|
||||
}
|
||||
|
||||
div.nboutput.container div.output_area .highlight,
|
||||
div.nboutput.container div.output_area pre {
|
||||
background: unset;
|
||||
}
|
||||
|
||||
div.nboutput.container div.output_area div.highlight {
|
||||
color: unset; /* override Pygments text color */
|
||||
}
|
||||
|
||||
/* avoid gaps between output lines */
|
||||
div.nboutput.container div[class*=highlight] pre {
|
||||
line-height: normal;
|
||||
}
|
||||
|
||||
/* input/output containers */
|
||||
div.nbinput.container,
|
||||
div.nboutput.container {
|
||||
display: -webkit-flex;
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
margin: 0;
|
||||
width: 100%;
|
||||
}
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container,
|
||||
div.nboutput.container {
|
||||
flex-direction: column;
|
||||
}
|
||||
}
|
||||
|
||||
/* input container */
|
||||
div.nbinput.container {
|
||||
padding-top: 5px;
|
||||
}
|
||||
|
||||
/* last container */
|
||||
div.nblast.container {
|
||||
padding-bottom: 5px;
|
||||
}
|
||||
|
||||
/* input prompt */
|
||||
div.nbinput.container div.prompt pre {
|
||||
color: #307FC1;
|
||||
}
|
||||
|
||||
/* output prompt */
|
||||
div.nboutput.container div.prompt pre {
|
||||
color: #BF5B3D;
|
||||
}
|
||||
|
||||
/* all prompts */
|
||||
div.nbinput.container div.prompt,
|
||||
div.nboutput.container div.prompt {
|
||||
width: 4.5ex;
|
||||
padding-top: 5px;
|
||||
position: relative;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt > div,
|
||||
div.nboutput.container div.prompt > div {
|
||||
position: absolute;
|
||||
right: 0;
|
||||
margin-right: 0.3ex;
|
||||
}
|
||||
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container div.prompt,
|
||||
div.nboutput.container div.prompt {
|
||||
width: unset;
|
||||
text-align: left;
|
||||
padding: 0.4em;
|
||||
}
|
||||
div.nboutput.container div.prompt.empty {
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
div.nbinput.container div.prompt > div,
|
||||
div.nboutput.container div.prompt > div {
|
||||
position: unset;
|
||||
}
|
||||
}
|
||||
|
||||
/* disable scrollbars and line breaks on prompts */
|
||||
div.nbinput.container div.prompt pre,
|
||||
div.nboutput.container div.prompt pre {
|
||||
overflow: hidden;
|
||||
white-space: pre;
|
||||
}
|
||||
|
||||
/* input/output area */
|
||||
div.nbinput.container div.input_area,
|
||||
div.nboutput.container div.output_area {
|
||||
-webkit-flex: 1;
|
||||
flex: 1;
|
||||
overflow: auto;
|
||||
}
|
||||
@media (max-width: 540px) {
|
||||
div.nbinput.container div.input_area,
|
||||
div.nboutput.container div.output_area {
|
||||
width: 100%;
|
||||
}
|
||||
}
|
||||
|
||||
/* input area */
|
||||
div.nbinput.container div.input_area {
|
||||
border: 1px solid #e0e0e0;
|
||||
border-radius: 2px;
|
||||
/*background: #f5f5f5;*/
|
||||
}
|
||||
|
||||
/* override MathJax center alignment in output cells */
|
||||
div.nboutput.container div[class*=MathJax] {
|
||||
text-align: left !important;
|
||||
}
|
||||
|
||||
/* override sphinx.ext.imgmath center alignment in output cells */
|
||||
div.nboutput.container div.math p {
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
/* standard error */
|
||||
div.nboutput.container div.output_area.stderr {
|
||||
background: #fdd;
|
||||
}
|
||||
|
||||
/* ANSI colors */
|
||||
.ansi-black-fg { color: #3E424D; }
|
||||
.ansi-black-bg { background-color: #3E424D; }
|
||||
.ansi-black-intense-fg { color: #282C36; }
|
||||
.ansi-black-intense-bg { background-color: #282C36; }
|
||||
.ansi-red-fg { color: #E75C58; }
|
||||
.ansi-red-bg { background-color: #E75C58; }
|
||||
.ansi-red-intense-fg { color: #B22B31; }
|
||||
.ansi-red-intense-bg { background-color: #B22B31; }
|
||||
.ansi-green-fg { color: #00A250; }
|
||||
.ansi-green-bg { background-color: #00A250; }
|
||||
.ansi-green-intense-fg { color: #007427; }
|
||||
.ansi-green-intense-bg { background-color: #007427; }
|
||||
.ansi-yellow-fg { color: #DDB62B; }
|
||||
.ansi-yellow-bg { background-color: #DDB62B; }
|
||||
.ansi-yellow-intense-fg { color: #B27D12; }
|
||||
.ansi-yellow-intense-bg { background-color: #B27D12; }
|
||||
.ansi-blue-fg { color: #208FFB; }
|
||||
.ansi-blue-bg { background-color: #208FFB; }
|
||||
.ansi-blue-intense-fg { color: #0065CA; }
|
||||
.ansi-blue-intense-bg { background-color: #0065CA; }
|
||||
.ansi-magenta-fg { color: #D160C4; }
|
||||
.ansi-magenta-bg { background-color: #D160C4; }
|
||||
.ansi-magenta-intense-fg { color: #A03196; }
|
||||
.ansi-magenta-intense-bg { background-color: #A03196; }
|
||||
.ansi-cyan-fg { color: #60C6C8; }
|
||||
.ansi-cyan-bg { background-color: #60C6C8; }
|
||||
.ansi-cyan-intense-fg { color: #258F8F; }
|
||||
.ansi-cyan-intense-bg { background-color: #258F8F; }
|
||||
.ansi-white-fg { color: #C5C1B4; }
|
||||
.ansi-white-bg { background-color: #C5C1B4; }
|
||||
.ansi-white-intense-fg { color: #A1A6B2; }
|
||||
.ansi-white-intense-bg { background-color: #A1A6B2; }
|
||||
|
||||
.ansi-default-inverse-fg { color: #FFFFFF; }
|
||||
.ansi-default-inverse-bg { background-color: #000000; }
|
||||
|
||||
.ansi-bold { font-weight: bold; }
|
||||
.ansi-underline { text-decoration: underline; }
|
||||
|
||||
|
||||
div.nbinput.container div.input_area div[class*=highlight] > pre,
|
||||
div.nboutput.container div.output_area div[class*=highlight] > pre,
|
||||
div.nboutput.container div.output_area div[class*=highlight].math,
|
||||
div.nboutput.container div.output_area.rendered_html,
|
||||
div.nboutput.container div.output_area > div.output_javascript,
|
||||
div.nboutput.container div.output_area:not(.rendered_html) > img{
|
||||
padding: 5px;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
/* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
|
||||
div.nbinput.container div.input_area > div[class^='highlight'],
|
||||
div.nboutput.container div.output_area > div[class^='highlight']{
|
||||
overflow-y: hidden;
|
||||
}
|
||||
|
||||
/* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
|
||||
.prompt .copybtn {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* Some additional styling taken form the Jupyter notebook CSS */
|
||||
.jp-RenderedHTMLCommon table,
|
||||
div.rendered_html table {
|
||||
border: none;
|
||||
border-collapse: collapse;
|
||||
border-spacing: 0;
|
||||
color: black;
|
||||
font-size: 12px;
|
||||
table-layout: fixed;
|
||||
}
|
||||
.jp-RenderedHTMLCommon thead,
|
||||
div.rendered_html thead {
|
||||
border-bottom: 1px solid black;
|
||||
vertical-align: bottom;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tr,
|
||||
.jp-RenderedHTMLCommon th,
|
||||
.jp-RenderedHTMLCommon td,
|
||||
div.rendered_html tr,
|
||||
div.rendered_html th,
|
||||
div.rendered_html td {
|
||||
text-align: right;
|
||||
vertical-align: middle;
|
||||
padding: 0.5em 0.5em;
|
||||
line-height: normal;
|
||||
white-space: normal;
|
||||
max-width: none;
|
||||
border: none;
|
||||
}
|
||||
.jp-RenderedHTMLCommon th,
|
||||
div.rendered_html th {
|
||||
font-weight: bold;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tbody tr:nth-child(odd),
|
||||
div.rendered_html tbody tr:nth-child(odd) {
|
||||
background: #f5f5f5;
|
||||
}
|
||||
.jp-RenderedHTMLCommon tbody tr:hover,
|
||||
div.rendered_html tbody tr:hover {
|
||||
background: rgba(66, 165, 245, 0.2);
|
||||
}
|
||||
</style>
|
||||
<div class="section" id="Solvers">
|
||||
<h1><span class="section-number">5. </span>Solvers<a class="headerlink" href="#Solvers" title="Permalink to this headline">¶</a></h1>
|
||||
<div class="section" id="LearningSolver">
|
||||
<h2><span class="section-number">5.1. </span>LearningSolver<a class="headerlink" href="#LearningSolver" title="Permalink to this headline">¶</a></h2>
|
||||
<div class="section" id="Example">
|
||||
<h3>Example<a class="headerlink" href="#Example" title="Permalink to this headline">¶</a></h3>
|
||||
<div class="nbinput docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[3]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">random</span>
|
||||
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">from</span> <span class="nn">scipy.stats</span> <span class="kn">import</span> <span class="n">uniform</span><span class="p">,</span> <span class="n">randint</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="kn">import</span> <span class="n">LogisticRegression</span>
|
||||
|
||||
<span class="kn">from</span> <span class="nn">miplearn.classifiers.minprob</span> <span class="kn">import</span> <span class="n">MinProbabilityClassifier</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.classifiers.singleclass</span> <span class="kn">import</span> <span class="n">SingleClassFix</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.collectors.basic</span> <span class="kn">import</span> <span class="n">BasicCollector</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.actions</span> <span class="kn">import</span> <span class="n">SetWarmStart</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.components.primal.indep</span> <span class="kn">import</span> <span class="n">IndependentVarsPrimalComponent</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.extractors.AlvLouWeh2017</span> <span class="kn">import</span> <span class="n">AlvLouWeh2017Extractor</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.io</span> <span class="kn">import</span> <span class="n">save</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.problems.tsp</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">TravelingSalesmanGenerator</span><span class="p">,</span>
|
||||
<span class="n">build_tsp_model</span><span class="p">,</span>
|
||||
<span class="p">)</span>
|
||||
<span class="kn">from</span> <span class="nn">miplearn.solvers.learning</span> <span class="kn">import</span> <span class="n">LearningSolver</span>
|
||||
|
||||
<span class="c1"># Set random seed to make example reproducible.</span>
|
||||
<span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
|
||||
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Generate a few instances of the traveling salesman problem.</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">TravelingSalesmanGenerator</span><span class="p">(</span>
|
||||
<span class="n">n</span><span class="o">=</span><span class="n">randint</span><span class="p">(</span><span class="n">low</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">high</span><span class="o">=</span><span class="mi">11</span><span class="p">),</span>
|
||||
<span class="n">x</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.0</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">1000.0</span><span class="p">),</span>
|
||||
<span class="n">y</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.0</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">1000.0</span><span class="p">),</span>
|
||||
<span class="n">gamma</span><span class="o">=</span><span class="n">uniform</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mf">0.90</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">0.20</span><span class="p">),</span>
|
||||
<span class="n">fix_cities</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
<span class="nb">round</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
<span class="p">)</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="mi">50</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Save instance data to data/tsp/00000.pkl.gz, data/tsp/00001.pkl.gz, ...</span>
|
||||
<span class="n">all_data</span> <span class="o">=</span> <span class="n">save</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="s2">"data/tsp"</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Split train/test data</span>
|
||||
<span class="n">train_data</span> <span class="o">=</span> <span class="n">all_data</span><span class="p">[:</span><span class="mi">40</span><span class="p">]</span>
|
||||
<span class="n">test_data</span> <span class="o">=</span> <span class="n">all_data</span><span class="p">[</span><span class="mi">40</span><span class="p">:]</span>
|
||||
|
||||
<span class="c1"># Collect training data</span>
|
||||
<span class="n">bc</span> <span class="o">=</span> <span class="n">BasicCollector</span><span class="p">(</span><span class="n">time_limit_sec</span><span class="o">=</span><span class="mi">3600</span><span class="p">)</span>
|
||||
<span class="n">bc</span><span class="o">.</span><span class="n">collect</span><span class="p">(</span><span class="n">train_data</span><span class="p">,</span> <span class="n">build_tsp_model</span><span class="p">,</span> <span class="n">n_jobs</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Build learning solver</span>
|
||||
<span class="n">solver</span> <span class="o">=</span> <span class="n">LearningSolver</span><span class="p">(</span>
|
||||
<span class="n">components</span><span class="o">=</span><span class="p">[</span>
|
||||
<span class="n">IndependentVarsPrimalComponent</span><span class="p">(</span>
|
||||
<span class="n">base_clf</span><span class="o">=</span><span class="n">SingleClassFix</span><span class="p">(</span>
|
||||
<span class="n">MinProbabilityClassifier</span><span class="p">(</span>
|
||||
<span class="n">base_clf</span><span class="o">=</span><span class="n">LogisticRegression</span><span class="p">(),</span>
|
||||
<span class="n">thresholds</span><span class="o">=</span><span class="p">[</span><span class="mf">0.95</span><span class="p">,</span> <span class="mf">0.95</span><span class="p">],</span>
|
||||
<span class="p">),</span>
|
||||
<span class="p">),</span>
|
||||
<span class="n">extractor</span><span class="o">=</span><span class="n">AlvLouWeh2017Extractor</span><span class="p">(),</span>
|
||||
<span class="n">action</span><span class="o">=</span><span class="n">SetWarmStart</span><span class="p">(),</span>
|
||||
<span class="p">)</span>
|
||||
<span class="p">]</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="c1"># Train ML models</span>
|
||||
<span class="n">solver</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">train_data</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Solve a test instance</span>
|
||||
<span class="n">solver</span><span class="o">.</span><span class="n">optimize</span><span class="p">(</span><span class="n">test_data</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">build_tsp_model</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="nboutput nblast docutils container">
|
||||
<div class="prompt empty docutils container">
|
||||
</div>
|
||||
<div class="output_area docutils container">
|
||||
<div class="highlight"><pre>
|
||||
Gurobi Optimizer version 9.5.2 build v9.5.2rc0 (linux64)
|
||||
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
|
||||
Optimize a model with 10 rows, 45 columns and 90 nonzeros
|
||||
Model fingerprint: 0x6ddcd141
|
||||
Coefficient statistics:
|
||||
Matrix range [1e+00, 1e+00]
|
||||
Objective range [4e+01, 1e+03]
|
||||
Bounds range [1e+00, 1e+00]
|
||||
RHS range [2e+00, 2e+00]
|
||||
Presolve time: 0.00s
|
||||
Presolved: 10 rows, 45 columns, 90 nonzeros
|
||||
|
||||
Iteration Objective Primal Inf. Dual Inf. Time
|
||||
0 6.3600000e+02 1.700000e+01 0.000000e+00 0s
|
||||
15 2.7610000e+03 0.000000e+00 0.000000e+00 0s
|
||||
|
||||
Solved in 15 iterations and 0.01 seconds (0.00 work units)
|
||||
Optimal objective 2.761000000e+03
|
||||
Set parameter LazyConstraints to value 1
|
||||
Gurobi Optimizer version 9.5.2 build v9.5.2rc0 (linux64)
|
||||
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
|
||||
Optimize a model with 10 rows, 45 columns and 90 nonzeros
|
||||
Model fingerprint: 0x74ca3d0a
|
||||
Variable types: 0 continuous, 45 integer (45 binary)
|
||||
Coefficient statistics:
|
||||
Matrix range [1e+00, 1e+00]
|
||||
Objective range [4e+01, 1e+03]
|
||||
Bounds range [1e+00, 1e+00]
|
||||
RHS range [2e+00, 2e+00]
|
||||
|
||||
User MIP start produced solution with objective 2796 (0.01s)
|
||||
Loaded user MIP start with objective 2796
|
||||
|
||||
Presolve time: 0.00s
|
||||
Presolved: 10 rows, 45 columns, 90 nonzeros
|
||||
Variable types: 0 continuous, 45 integer (45 binary)
|
||||
|
||||
Root relaxation: objective 2.761000e+03, 14 iterations, 0.00 seconds (0.00 work units)
|
||||
|
||||
Nodes | Current Node | Objective Bounds | Work
|
||||
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time
|
||||
|
||||
0 0 cutoff 0 2796.00000 2796.00000 0.00% - 0s
|
||||
|
||||
Cutting planes:
|
||||
Lazy constraints: 3
|
||||
|
||||
Explored 1 nodes (15 simplex iterations) in 0.02 seconds (0.00 work units)
|
||||
Thread count was 12 (of 12 available processors)
|
||||
|
||||
Solution count 1: 2796
|
||||
|
||||
Optimal solution found (tolerance 1.00e-04)
|
||||
Best objective 2.796000000000e+03, best bound 2.796000000000e+03, gap 0.0000%
|
||||
|
||||
User-callback calls 103, time in user-callback 0.00 sec
|
||||
Gurobi Optimizer version 9.5.2 build v9.5.2rc0 (linux64)
|
||||
Thread count: 6 physical cores, 12 logical processors, using up to 12 threads
|
||||
Optimize a model with 10 rows, 45 columns and 90 nonzeros
|
||||
Model fingerprint: 0x74ca3d0a
|
||||
Variable types: 0 continuous, 45 integer (45 binary)
|
||||
Coefficient statistics:
|
||||
Matrix range [1e+00, 1e+00]
|
||||
Objective range [4e+01, 1e+03]
|
||||
Bounds range [1e+00, 1e+00]
|
||||
RHS range [2e+00, 2e+00]
|
||||
Presolved: 10 rows, 45 columns, 90 nonzeros
|
||||
|
||||
Continuing optimization...
|
||||
|
||||
|
||||
Cutting planes:
|
||||
Lazy constraints: 3
|
||||
|
||||
Explored 1 nodes (15 simplex iterations) in 0.01 seconds (0.00 work units)
|
||||
Thread count was 12 (of 12 available processors)
|
||||
|
||||
Solution count 1: 2796
|
||||
|
||||
Optimal solution found (tolerance 1.00e-04)
|
||||
Best objective 2.796000000000e+03, best bound 2.796000000000e+03, gap 0.0000%
|
||||
|
||||
User-callback calls 27, time in user-callback 0.00 sec
|
||||
</pre></div></div>
|
||||
</div>
|
||||
<div class="nbinput nblast docutils container">
|
||||
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[2]:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<div class='prev-next-bottom'>
|
||||
|
||||
<a class='left-prev' id="prev-link" href="../primal/" title="previous page"><span class="section-number">4. </span>Primal Components</a>
|
||||
<a class='right-next' id="next-link" href="../../api/problems/" title="next page"><span class="section-number">6. </span>Benchmark Problems</a>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<footer class="footer mt-5 mt-md-0">
|
||||
<div class="container">
|
||||
<p>
|
||||
|
||||
© Copyright 2020-2022, UChicago Argonne, LLC.<br/>
|
||||
</p>
|
||||
</div>
|
||||
</footer>
|
||||
</main>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="../../_static/js/index.1c5a1a01449ed65a7b51.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
Reference in New Issue
Block a user