MIPLearn v0.3

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,26 +0,0 @@
----
-name: Bug report
-about: Something is broken in the package
-title: ''
-labels: ''
-assignees: ''
-
----
-
-## Description
-
-A clear and concise description of what the bug is.
-
-## Steps to Reproduce
-
-Please describe how can the developers reproduce the problem in their own computers. Code snippets and sample input files are specially helpful. For example:
-
-1. Install the package
-2. Run the code below with the attached input file...
-3. The following error appears...
-
-## System Information
- - Operating System: [e.g. Ubuntu 20.04]
- - Python version: [e.g. 3.6]
- - Solver: [e.g. Gurobi 9.0]
- - Package version: [e.g. 0.1.0]

.github/ISSUE_TEMPLATE/config.yml

@@ -1,8 +0,0 @@
-blank_issues_enabled: false
-contact_links:
-  - name: Feature Request
-    url: https://github.com/ANL-CEEESA/MIPLearn/discussions/categories/feature-requests
-    about:  Submit ideas for new features and small enhancements
-  - name: Help & FAQ
-    url: https://github.com/ANL-CEEESA/MIPLearn/discussions/categories/help-faq
-    about: Ask questions about the package and get help from the community

.github/workflows/lint.yml

@@ -1,11 +0,0 @@
-name: Lint
-
-on: [push, pull_request]
-
-jobs:
-  lint:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v2
-      - uses: actions/setup-python@v2
-      - uses: psf/black@20.8b1

.github/workflows/test.yml

@@ -1,27 +0,0 @@
-name: Test
-on:
-  push:
-  pull_request:
-  schedule:
-    - cron: '45 10 * * *'
-jobs:
-  test:
-    runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        python-version: [3.7, 3.8]
-
-    steps:
-    - name: Check out source code
-      uses: actions/checkout@v2
-
-    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v2
-      with:
-        python-version: ${{ matrix.python-version }}
-
-    - name: Install dependencies
-      run: make install-deps
-
-    - name: Test
-      run: make test

.gitignore

@@ -78,6 +78,8 @@ wheels/
 notebooks/
 .vscode
 tmp
-benchmark/tsp
+benchmark/data
-benchmark/stab
+benchmark/results
-benchmark/knapsack
+**/*.xz
+**/*.h5
+**/*.jld2

.pre-commit-config.yaml

@@ -1,6 +0,0 @@
-repos:
-  - repo: https://github.com/ambv/black
-    rev: 20.8b1
-    hooks:
-    - id: black
-      args: ["--check"]

.zenodo.json

@@ -0,0 +1,27 @@
+{
+    "creators": [
+        {
+            "orcid": "0000-0002-5022-9802",
+            "affiliation": "Argonne National Laboratory",
+            "name": "Santos Xavier, Alinson"
+        },
+        {
+            "affiliation": "Argonne National Laboratory",
+            "name": "Qiu, Feng"
+        },
+        {
+            "affiliation": "Georgia Institute of Technology",
+            "name": "Gu, Xiaoyi"
+        },
+        {
+            "affiliation": "Georgia Institute of Technology",
+            "name": "Becu, Berkay"
+        },
+        {
+            "affiliation": "Georgia Institute of Technology",
+            "name": "Dey, Santanu S."
+        }
+    ],
+    "title": "MIPLearn: An Extensible Framework for Learning-Enhanced Optimization",
+    "description": "<b>MIPLearn</b> is an extensible framework for solving discrete optimization problems using a combination of Mixed-Integer Linear Programming (MIP) and Machine Learning (ML). MIPLearn uses ML methods to automatically identify patterns in previously solved instances of the problem, then uses these patterns to accelerate the performance of conventional state-of-the-art MIP solvers such as CPLEX, Gurobi or XPRESS."
+}

CHANGELOG.md

@@ -1,44 +1,33 @@
-# MIPLearn: Changelog
+# Changelog
 
-## [0.2.0] - [Unreleased]
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.3.0] - 2023-06-08
+
+This is a complete rewrite of the original prototype package, with an entirely new API, focused on performance, scalability and flexibility.
 
 ### Added
 
-- **Added two new machine learning components:**
+- Add support for Python/Gurobipy and Julia/JuMP, in addition to the existing Python/Pyomo interface.
-  - Added `StaticLazyConstraintComponent`, which allows the user to mark some constraints in the formulation as lazy, instead of constructing them in a callback. ML predicts which static lazy constraints should be kept in the formulation, and which should be removed. 
+- Add six new random instance generators (bin packing, capacitated p-median, set cover, set packing, unit commitment, vertex cover), in addition to the three existing generators (multiknapsack, stable set, tsp).
-  - Added `UserCutComponents`, which predicts which user cuts should be generated and added to the formulation as constraints ahead-of-time, before solving the MIP.
+- Collect some additional raw training data (e.g. basis status, reduced costs, etc)
-- **Added support to additional MILP solvers:**
+- Add new primal solution ML strategies (memorizing, independent vars and joint vars)
-  - Added support for CPLEX and XPRESS, through the Pyomo modeling language, in addition to (existing) Gurobi. The solver classes are named `CplexPyomoSolver`, `XpressPyomoSolver` and `GurobiPyomoSolver`.
+- Add new primal solution actions (set warm start, fix variables, enforce proximity)
-  - Added support for Gurobi without any modeling language. The solver class is named `GurobiSolver`. In this case, `instance.to_model` should return ` gp.Model` object.
+- Add runnable tutorials and user guides to the documentation.
-  - Added support to direct MPS files, produced externally, through the `GurobiSolver` class mentioned above.
-- **Added dynamic thresholds:** 
-  - In previous versions of the package, it was necessary to manually adjust component aggressiveness to reach a desired precision/recall. This can now be done automatically with `MinProbabilityThreshold`, `MinPrecisionThreshold` and `MinRecallThreshold`.
-- **Reduced memory requirements:**
-  - Previous versions of the package required all training instances to be kept in memory at all times, which was prohibitive for large-scale problems. It is now possible to store instances in file until they are needed, using `PickledGzInstance`.
-- **Refactoring:**
-  - Added static types to all classes (with mypy).
 
 ### Changed
 
-- Variables are now referenced by their names, instead of tuples `(var_name, index)`. This change was required to improve the compatibility with modeling languages other than Pyomo, which do not follow this convention. For performance reasons, the functions `get_variable_features` and `get_variable_categories` should now return a dictionary containing categories and features for all relevant variables. Previously, MIPLearn had to perform two function calls per variable, which was too slow for very large models.
+- To support large-scale problems and datasets, switch from an in-memory architecture to a file-based architecture, using HDF5 files.
-- Internal solvers must now be specified as objects, instead of strings. For example,
+- To accelerate development cycle, split training data collection from feature extraction.
-  ```python
-  solver = LearningSolver(
-      solver=GurobiPyomoSolver(
-          params={
-              "TimeLimit": 300,
-              "Threads": 4,
-          }      
-      )
-  )
-  ```
-- `LazyConstraintComponent` has been renamed to `DynamicLazyConstraintsComponent`.
-- Categories, lazy constraints and cutting plane identifiers must now be strings, instead `Hashable`. This change was required for compatibility with HDF5 data format.
 
 ### Removed
 
-- Temporarily removed the experimental `BranchPriorityComponent`. This component will be re-added in the Julia version of the package.
+- Temporarily remove ML strategies for lazy constraints
-- Removed `solver.add` method, previously used to add components to an existing solver. Use the constructor `LearningSolver(components=[...])` instead.
+- Remove benchmarks from documentation. These will be published in a separate paper.
+
 
 ## [0.1.0] - 2020-11-23
 

Makefile

@@ -3,7 +3,7 @@ PYTEST      := pytest
 PIP         := $(PYTHON) -m pip
 MYPY        := $(PYTHON) -m mypy
 PYTEST_ARGS := -W ignore::DeprecationWarning -vv --log-level=DEBUG
-VERSION     := 0.2
+VERSION     := 0.3
 
 all: docs test
 
@@ -24,11 +24,8 @@ docs:
 	cd docs; make clean; make dirhtml
 	rsync -avP --delete-after docs/_build/dirhtml/ ../docs/$(VERSION)
 
-
 install-deps:
 	$(PIP) install --upgrade pip
-	$(PIP) install --upgrade -i https://pypi.gurobi.com gurobipy
-	$(PIP) install --upgrade xpress
 	$(PIP) install --upgrade -r requirements.txt
 
 install:
@@ -41,10 +38,11 @@ reformat:
 	$(PYTHON) -m black .
 
 test:
-	rm -rf .mypy_cache
+	# pyflakes miplearn tests
+	black --check .
+	# rm -rf .mypy_cache
 	$(MYPY) -p miplearn
 	$(MYPY) -p tests
-	$(MYPY) -p benchmark
 	$(PYTEST) $(PYTEST_ARGS) 
 
 .PHONY: test test-watch docs install dist

README.md

@@ -14,36 +14,51 @@
   </a>
 </p>
 
-**MIPLearn** is an extensible framework for solving discrete optimization problems using a combination of Mixed-Integer Linear Programming (MIP) and Machine Learning (ML).
+**MIPLearn** is an extensible framework for solving discrete optimization problems using a combination of Mixed-Integer Linear Programming (MIP) and Machine Learning (ML). MIPLearn uses ML methods to automatically identify patterns in previously solved instances of the problem, then uses these patterns to accelerate the performance of conventional state-of-the-art MIP solvers such as CPLEX, Gurobi or XPRESS.
 
-MIPLearn uses ML methods to automatically identify patterns in previously solved instances of the problem, then uses these patterns to accelerate the performance of conventional state-of-the-art MIP solvers such as CPLEX, Gurobi or XPRESS. Unlike pure ML methods, MIPLearn is not only able to find high-quality solutions to discrete optimization problems, but it can also prove the optimality and feasibility of these solutions. Unlike conventional MIP solvers, MIPLearn can take full advantage of very specific observations that happen to be true in a particular family of instances (such as the observation that a particular constraint is typically redundant, or that a particular variable typically assumes a certain value). For certain classes of problems, this approach has been shown to provide significant performance benefits (see [benchmarks](https://anl-ceeesa.github.io/MIPLearn/0.1/problems/) and [references](https://anl-ceeesa.github.io/MIPLearn/0.1/about/)).
+Unlike pure ML methods, MIPLearn is not only able to find high-quality solutions to discrete optimization problems, but it can also prove the optimality and feasibility of these solutions. Unlike conventional MIP solvers, MIPLearn can take full advantage of very specific observations that happen to be true in a particular family of instances (such as the observation that a particular constraint is typically redundant, or that a particular variable typically assumes a certain value). For certain classes of problems, this approach may provide significant performance benefits.
-
-Features
---------
-* **MIPLearn proposes a flexible problem specification format,** which allows users to describe their particular optimization problems to a Learning-Enhanced MIP solver, both from the MIP perspective and from the ML perspective, without making any assumptions on the problem being modeled, the mathematical formulation of the problem, or ML encoding.
-
-* **MIPLearn provides a reference implementation of a *Learning-Enhanced Solver*,** which can use the above problem specification format to automatically predict, based on previously solved instances, a number of hints to accelerate MIP performance. 
-
-* **MIPLearn provides a set of benchmark problems and random instance generators,** covering applications from different domains, which can be used to quickly evaluate new learning-enhanced MIP techniques in a measurable and reproducible way.
-
-* **MIPLearn is customizable and extensible**. For MIP and ML researchers exploring new techniques to accelerate MIP performance based on historical data, each component of the reference solver can be individually replaced, extended or customized.
 
 Documentation
 -------------
 
-For installation instructions, basic usage and benchmarks results, see the [official documentation](https://anl-ceeesa.github.io/MIPLearn/).
+- Tutorials:
+    1. [Getting started (Pyomo)](https://anl-ceeesa.github.io/MIPLearn/0.3/tutorials/getting-started-pyomo/)
+    2. [Getting started (Gurobipy)](https://anl-ceeesa.github.io/MIPLearn/0.3/tutorials/getting-started-gurobipy/)
+    3. [Getting started (JuMP)](https://anl-ceeesa.github.io/MIPLearn/0.3/tutorials/getting-started-jump/)
+- User Guide
+    1. [Benchmark problems](https://anl-ceeesa.github.io/MIPLearn/0.3/guide/problems/)
+    2. [Training data collectors](https://anl-ceeesa.github.io/MIPLearn/0.3/guide/collectors/)
+    3. [Feature extractors](https://anl-ceeesa.github.io/MIPLearn/0.3/guide/features/)
+    4. [Primal components](https://anl-ceeesa.github.io/MIPLearn/0.3/guide/primal/)
+    5. [Learning solver](https://anl-ceeesa.github.io/MIPLearn/0.3/guide/solvers/)
+- Python API Reference
+    1. [Benchmark problems](https://anl-ceeesa.github.io/MIPLearn/0.3/api/problems/)
+    2. [Collectors & extractors](https://anl-ceeesa.github.io/MIPLearn/0.3/api/collectors/)
+    3. [Components](https://anl-ceeesa.github.io/MIPLearn/0.3/api/components/)
+    4. [Solvers](https://anl-ceeesa.github.io/MIPLearn/0.3/api/solvers/)
+    5. [Helpers](https://anl-ceeesa.github.io/MIPLearn/0.3/api/helpers/)
+
+Authors
+-------
+
+- **Alinson S. Xavier** (Argonne National Laboratory)
+- **Feng Qiu** (Argonne National Laboratory)
+- **Xiaoyi Gu** (Georgia Institute of Technology)
+- **Berkay Becu** (Georgia Institute of Technology)
+- **Santanu S. Dey**  (Georgia Institute of Technology)
+
 
 Acknowledgments
 ---------------
-* Based upon work supported by **Laboratory Directed Research and Development** (LDRD) funding from Argonne National Laboratory, provided by the Director, Office of Science, of the U.S. Department of Energy under Contract No. DE-AC02-06CH11357.
+* Based upon work supported by **Laboratory Directed Research and Development** (LDRD) funding from Argonne National Laboratory, provided by the Director, Office of Science, of the U.S. Department of Energy.
-* Based upon work supported by the **U.S. Department of Energy Advanced Grid Modeling Program** under Grant DE-OE0000875.
+* Based upon work supported by the **U.S. Department of Energy Advanced Grid Modeling Program**.
 
 Citing MIPLearn
 ---------------
 
 If you use MIPLearn in your research (either the solver or the included problem generators), we kindly request that you cite the package as follows:
 
-* **Alinson S. Xavier, Feng Qiu.** *MIPLearn: An Extensible Framework for Learning-Enhanced Optimization*. Zenodo (2020). DOI: [10.5281/zenodo.4287567](https://doi.org/10.5281/zenodo.4287567)
+* **Alinson S. Xavier, Feng Qiu, Xiaoyi Gu, Berkay Becu, Santanu S. Dey.** *MIPLearn: An Extensible Framework for Learning-Enhanced Optimization (Version 0.3)*. Zenodo (2023). DOI: [10.5281/zenodo.4287567](https://doi.org/10.5281/zenodo.4287567)
 
 If you use MIPLearn in the field of power systems optimization, we kindly request that you cite the reference below, in which the main techniques implemented in MIPLearn were first developed:
 

benchmark/Makefile

@@ -1,31 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-#  Written by Alinson S. Xavier <axavier@anl.gov>
-
-CHALLENGES := \
-    stab/ChallengeA \
-    knapsack/ChallengeA \
-    tsp/ChallengeA
-
-test: $(addsuffix /performance.png, $(CHALLENGES))
-
-train: $(addsuffix /train/done, $(CHALLENGES))
-
-%/train/done:
-	python benchmark.py train $*
-
-%/benchmark_baseline.csv: %/train/done
-	python benchmark.py test-baseline $*
-
-%/benchmark_ml.csv: %/benchmark_baseline.csv
-	python benchmark.py test-ml $*
-
-%/performance.png: %/benchmark_ml.csv
-	python benchmark.py charts $*
-
-clean:
-	rm -rvf $(CHALLENGES)
-
-.PHONY: clean
-.SECONDARY:

benchmark/benchmark.py

@@ -1,268 +0,0 @@
-#!/usr/bin/env python
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-"""MIPLearn Benchmark Scripts
-
-Usage:
-    benchmark.py train [options] <challenge>
-    benchmark.py test-baseline [options] <challenge>
-    benchmark.py test-ml [options] <challenge>
-    benchmark.py charts <challenge>
-    
-Options:
-    -h --help               Show this screen
-    --train-jobs=<n>        Number of instances to solve in parallel during training [default: 10]
-    --train-time-limit=<n>  Solver time limit during training in seconds [default: 900]
-    --test-jobs=<n>         Number of instances to solve in parallel during test [default: 5]
-    --test-time-limit=<n>   Solver time limit during test in seconds [default: 900]
-    --solver-threads=<n>    Number of threads the solver is allowed to use [default: 4]
-"""
-import glob
-import importlib
-import logging
-import os
-from pathlib import Path
-from typing import Dict, List
-
-import matplotlib.pyplot as plt
-import pandas as pd
-import seaborn as sns
-from docopt import docopt
-from numpy import median
-
-from miplearn import (
-    LearningSolver,
-    BenchmarkRunner,
-    GurobiPyomoSolver,
-    setup_logger,
-    PickleGzInstance,
-    write_pickle_gz_multiple,
-    Instance,
-)
-
-setup_logger()
-logging.getLogger("gurobipy").setLevel(logging.ERROR)
-logging.getLogger("pyomo.core").setLevel(logging.ERROR)
-logger = logging.getLogger("benchmark")
-
-
-def train(args: Dict) -> None:
-    basepath = args["<challenge>"]
-    problem_name, challenge_name = args["<challenge>"].split("/")
-    pkg = importlib.import_module(f"miplearn.problems.{problem_name}")
-    challenge = getattr(pkg, challenge_name)()
-
-    if not os.path.isdir(f"{basepath}/train"):
-        write_pickle_gz_multiple(challenge.training_instances, f"{basepath}/train")
-        write_pickle_gz_multiple(challenge.test_instances, f"{basepath}/test")
-
-    done_filename = f"{basepath}/train/done"
-    if not os.path.isfile(done_filename):
-        train_instances: List[Instance] = [
-            PickleGzInstance(f) for f in glob.glob(f"{basepath}/train/*.gz")
-        ]
-        solver = LearningSolver(
-            solver=GurobiPyomoSolver(
-                params={
-                    "TimeLimit": int(args["--train-time-limit"]),
-                    "Threads": int(args["--solver-threads"]),
-                }
-            ),
-        )
-        solver.parallel_solve(
-            train_instances,
-            n_jobs=int(args["--train-jobs"]),
-        )
-        Path(done_filename).touch(exist_ok=True)
-
-
-def test_baseline(args: Dict) -> None:
-    basepath = args["<challenge>"]
-    test_instances: List[Instance] = [
-        PickleGzInstance(f) for f in glob.glob(f"{basepath}/test/*.gz")
-    ]
-    csv_filename = f"{basepath}/benchmark_baseline.csv"
-    if not os.path.isfile(csv_filename):
-        solvers = {
-            "baseline": LearningSolver(
-                solver=GurobiPyomoSolver(
-                    params={
-                        "TimeLimit": int(args["--test-time-limit"]),
-                        "Threads": int(args["--solver-threads"]),
-                    }
-                ),
-            ),
-        }
-        benchmark = BenchmarkRunner(solvers)
-        benchmark.parallel_solve(
-            test_instances,
-            n_jobs=int(args["--test-jobs"]),
-        )
-        benchmark.write_csv(csv_filename)
-
-
-def test_ml(args: Dict) -> None:
-    basepath = args["<challenge>"]
-    test_instances: List[Instance] = [
-        PickleGzInstance(f) for f in glob.glob(f"{basepath}/test/*.gz")
-    ]
-    train_instances: List[Instance] = [
-        PickleGzInstance(f) for f in glob.glob(f"{basepath}/train/*.gz")
-    ]
-    csv_filename = f"{basepath}/benchmark_ml.csv"
-    if not os.path.isfile(csv_filename):
-        solvers = {
-            "ml-exact": LearningSolver(
-                solver=GurobiPyomoSolver(
-                    params={
-                        "TimeLimit": int(args["--test-time-limit"]),
-                        "Threads": int(args["--solver-threads"]),
-                    }
-                ),
-            ),
-            "ml-heuristic": LearningSolver(
-                solver=GurobiPyomoSolver(
-                    params={
-                        "TimeLimit": int(args["--test-time-limit"]),
-                        "Threads": int(args["--solver-threads"]),
-                    }
-                ),
-                mode="heuristic",
-            ),
-        }
-        benchmark = BenchmarkRunner(solvers)
-        benchmark.fit(train_instances)
-        benchmark.parallel_solve(
-            test_instances,
-            n_jobs=int(args["--test-jobs"]),
-        )
-        benchmark.write_csv(csv_filename)
-
-
-def charts(args: Dict) -> None:
-    basepath = args["<challenge>"]
-    sns.set_style("whitegrid")
-    sns.set_palette("Blues_r")
-
-    csv_files = [
-        f"{basepath}/benchmark_baseline.csv",
-        f"{basepath}/benchmark_ml.csv",
-    ]
-    results = pd.concat(map(pd.read_csv, csv_files))
-    groups = results.groupby("Instance")
-    best_lower_bound = groups["Lower bound"].transform("max")
-    best_upper_bound = groups["Upper bound"].transform("min")
-    results["Relative lower bound"] = results["Lower bound"] / best_lower_bound
-    results["Relative upper bound"] = results["Upper bound"] / best_upper_bound
-
-    sense = results.loc[0, "Sense"]
-    if (sense == "min").any():
-        primal_column = "Relative upper bound"
-        obj_column = "Upper bound"
-        predicted_obj_column = "Objective: Predicted upper bound"
-    else:
-        primal_column = "Relative lower bound"
-        obj_column = "Lower bound"
-        predicted_obj_column = "Objective: Predicted lower bound"
-
-    palette = {"baseline": "#9b59b6", "ml-exact": "#3498db", "ml-heuristic": "#95a5a6"}
-    fig, (ax1, ax2, ax3, ax4) = plt.subplots(
-        nrows=1,
-        ncols=4,
-        figsize=(12, 4),
-        gridspec_kw={"width_ratios": [2, 1, 1, 2]},
-    )
-
-    # Wallclock time
-    sns.stripplot(
-        x="Solver",
-        y="Wallclock time",
-        data=results,
-        ax=ax1,
-        jitter=0.25,
-        palette=palette,
-        size=4.0,
-    )
-    sns.barplot(
-        x="Solver",
-        y="Wallclock time",
-        data=results,
-        ax=ax1,
-        errwidth=0.0,
-        alpha=0.4,
-        palette=palette,
-        estimator=median,
-    )
-    ax1.set(ylabel="Wallclock time (s)")
-
-    # Gap
-    ax2.set_ylim(-0.5, 5.5)
-    sns.stripplot(
-        x="Solver",
-        y="Gap",
-        jitter=0.25,
-        data=results[results["Solver"] != "ml-heuristic"],
-        ax=ax2,
-        palette=palette,
-        size=4.0,
-    )
-
-    # Relative primal bound
-    ax3.set_ylim(0.95, 1.05)
-    sns.stripplot(
-        x="Solver",
-        y=primal_column,
-        jitter=0.25,
-        data=results[results["Solver"] == "ml-heuristic"],
-        ax=ax3,
-        palette=palette,
-    )
-    sns.scatterplot(
-        x=obj_column,
-        y=predicted_obj_column,
-        hue="Solver",
-        data=results[results["Solver"] == "ml-exact"],
-        ax=ax4,
-        palette=palette,
-    )
-
-    # Predicted vs actual primal bound
-    xlim, ylim = ax4.get_xlim(), ax4.get_ylim()
-    ax4.plot(
-        [-1e10, 1e10],
-        [-1e10, 1e10],
-        ls="-",
-        color="#cccccc",
-    )
-    ax4.set_xlim(xlim)
-    ax4.set_ylim(ylim)
-    ax4.get_legend().remove()
-    ax4.set(
-        ylabel="Predicted value",
-        xlabel="Actual value",
-    )
-
-    fig.tight_layout()
-    plt.savefig(
-        f"{basepath}/performance.png",
-        bbox_inches="tight",
-        dpi=150,
-    )
-
-
-def main() -> None:
-    args = docopt(__doc__)
-    if args["train"]:
-        train(args)
-    if args["test-baseline"]:
-        test_baseline(args)
-    if args["test-ml"]:
-        test_ml(args)
-    if args["charts"]:
-        charts(args)
-
-
-if __name__ == "__main__":
-    main()

docs/Makefile

@@ -1,8 +1,14 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
 SPHINXOPTS    ?=
 SPHINXBUILD   ?= sphinx-build
 SOURCEDIR     = .
 BUILDDIR      = _build
 
+# Put it first so that "make" without argument is like "make help".
 help:
 	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
 

docs/_static/custom.css

@@ -5,3 +5,116 @@ h1.site-logo {
 h1.site-logo small {
     font-size: 20px !important;
 }
+
+code {
+    display: inline-block;
+    border-radius: 4px;
+    padding: 0 4px;
+    background-color: #eee;
+    color: rgb(232, 62, 140);
+}
+
+.right-next, .left-prev {
+    border-radius: 8px;
+    border-width: 0px !important;
+    box-shadow: 2px 2px 6px rgba(0, 0, 0, 0.2);
+}
+
+.right-next:hover, .left-prev:hover {
+    text-decoration: none;
+}
+
+.admonition {
+    border-radius: 8px;
+    border-width: 0;
+    box-shadow: 0 0 0 !important;
+}
+
+.note { background-color: rgba(0, 123, 255, 0.1); }
+.note * { color: rgb(69 94 121); }
+
+.warning { background-color: rgb(220 150 40 / 10%); }
+.warning * { color: rgb(105 72 28); }
+
+.input_area, .output_area, .output_area img {
+    border-radius: 8px !important;
+    border-width: 0 !important;
+    margin: 8px 0 8px 0;
+}
+
+.output_area {
+    padding: 4px;
+    background-color: hsl(227 60% 11% / 0.7) !important;
+}
+
+.output_area pre {
+    color: #fff;
+    line-height: 20px !important;
+}
+
+.input_area pre {
+    background-color: rgba(0 0 0 / 3%) !important;
+    padding: 12px !important;
+    line-height: 20px;
+}
+
+.ansi-green-intense-fg {
+    color: #64d88b !important;
+}
+
+#site-navigation {
+    background-color: #fafafa;
+}
+
+.container, .container-lg, .container-md, .container-sm, .container-xl {
+    max-width: inherit !important;
+}
+
+h1, h2 {
+    font-weight: bold !important;
+}
+
+#main-content .section {
+    max-width: 900px !important;
+    margin: 0 auto !important;
+    font-size: 16px;
+}
+
+p.caption {
+    font-weight: bold;
+}
+
+h2 {
+    padding-bottom: 5px;
+    border-bottom: 1px solid #ccc;
+}
+
+h3 {
+    margin-top: 1.5rem;
+}
+
+tbody, thead, pre {
+    border: 1px solid rgba(0, 0, 0, 0.25);
+}
+
+table td, th {
+    padding: 8px;
+}
+
+table p {
+    margin-bottom: 0;
+}
+
+table td code {
+    white-space: nowrap;
+}
+
+table tr,
+table th {
+    border-bottom: 1px solid rgba(0, 0, 0, 0.1);
+}
+
+table tr:last-child {
+    border-bottom: 0;
+}
+

docs/about.md

@@ -1,58 +0,0 @@
-```{sectnum}
----
-start: 4
-depth: 2
-suffix: .
----
-```
-
-# About
-
-## Authors
-
-* **Alinson S. Xavier,** Argonne National Laboratory <<axavier@anl.gov>>
-* **Feng Qiu,** Argonne National Laboratory <<fqiu@anl.gov>>
-
-## Acknowledgments
-
-* Based upon work supported by **Laboratory Directed Research and Development** (LDRD) funding from Argonne National Laboratory, provided by the Director, Office of Science, of the U.S. Department of Energy under Contract No. DE-AC02-06CH11357, and the **U.S. Department of Energy Advanced Grid Modeling Program** under Grant DE-OE0000875.
-
-## References
-
-
-If you use MIPLearn in your research, or the included problem generators, we kindly request that you cite the package as follows:
-
-* **Alinson S. Xavier, Feng Qiu.** *MIPLearn: An Extensible Framework for Learning-Enhanced Optimization*. Zenodo (2020). DOI: [10.5281/zenodo.4287567](https://doi.org/10.5281/zenodo.4287567)
-
-If you use MIPLearn in the field of power systems optimization, we kindly request that you cite the reference below, in which the main techniques implemented in MIPLearn were first developed:
-
-* **Alinson S. Xavier, Feng Qiu, Shabbir Ahmed.** *Learning to Solve Large-Scale Unit Commitment Problems.* INFORMS Journal on Computing (2020). DOI: [10.1287/ijoc.2020.0976](https://doi.org/10.1287/ijoc.2020.0976)
-
-## License
-
-```text
-MIPLearn, an extensible framework for Learning-Enhanced Mixed-Integer Optimization
-Copyright © 2020, UChicago Argonne, LLC. All Rights Reserved.
-
-Redistribution and use in source and binary forms, with or without modification, are permitted
-provided that the following conditions are met:
-
-1. Redistributions of source code must retain the above copyright notice, this list of
-   conditions and the following disclaimer.
-2. Redistributions in binary form must reproduce the above copyright notice, this list of
-   conditions and the following disclaimer in the documentation and/or other materials provided
-   with the distribution.
-3. Neither the name of the copyright holder nor the names of its contributors may be used to
-   endorse or promote products derived from this software without specific prior written
-   permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
-IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
-AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
-CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-POSSIBILITY OF SUCH DAMAGE.
-```

docs/api/collectors.rst

@@ -0,0 +1,42 @@
+Collectors & Extractors
+=======================
+
+miplearn.classifiers.minprob
+----------------------------
+
+.. automodule:: miplearn.classifiers.minprob
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.classifiers.singleclass
+--------------------------------
+
+.. automodule:: miplearn.classifiers.singleclass
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.collectors.basic
+-------------------------
+
+.. automodule:: miplearn.collectors.basic
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.extractors.fields
+--------------------------
+
+.. automodule:: miplearn.extractors.fields
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.extractors.AlvLouWeh2017
+---------------------------------
+
+.. automodule:: miplearn.extractors.AlvLouWeh2017
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/api/components.rst

@@ -0,0 +1,44 @@
+Components
+==========
+
+miplearn.components.primal.actions
+----------------------------------
+
+.. automodule:: miplearn.components.primal.actions
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.components.primal.expert
+----------------------------------
+
+.. automodule:: miplearn.components.primal.expert
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.components.primal.indep
+----------------------------------
+
+.. automodule:: miplearn.components.primal.indep
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.components.primal.joint
+----------------------------------
+
+.. automodule:: miplearn.components.primal.joint
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.components.primal.mem
+----------------------------------
+
+.. automodule:: miplearn.components.primal.mem
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+   

docs/api/helpers.rst

@@ -0,0 +1,18 @@
+Helpers
+=======
+
+miplearn.io
+-----------
+
+.. automodule:: miplearn.io
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.h5
+-----------
+
+.. automodule:: miplearn.h5
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/api/problems.rst

@@ -0,0 +1,57 @@
+Benchmark Problems
+==================
+
+miplearn.problems.binpack
+-------------------------
+
+.. automodule:: miplearn.problems.binpack
+   :members:
+
+miplearn.problems.multiknapsack
+-------------------------------
+
+.. automodule:: miplearn.problems.multiknapsack
+   :members:
+
+miplearn.problems.pmedian
+-------------------------
+
+.. automodule:: miplearn.problems.pmedian
+   :members:
+
+miplearn.problems.setcover
+--------------------------
+
+.. automodule:: miplearn.problems.setcover
+   :members:
+
+miplearn.problems.setpack
+-------------------------
+
+.. automodule:: miplearn.problems.setpack
+   :members:
+
+miplearn.problems.stab
+----------------------
+
+.. automodule:: miplearn.problems.stab
+   :members:
+
+miplearn.problems.tsp
+---------------------
+
+.. automodule:: miplearn.problems.tsp
+   :members:
+
+miplearn.problems.uc
+--------------------
+
+.. automodule:: miplearn.problems.uc
+   :members:
+
+miplearn.problems.vertexcover
+-----------------------------
+
+.. automodule:: miplearn.problems.vertexcover
+   :members:
+

docs/api/solvers.rst

@@ -0,0 +1,26 @@
+Solvers
+=======
+
+miplearn.solvers.abstract
+-------------------------
+
+.. automodule:: miplearn.solvers.abstract
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.solvers.gurobi
+-------------------------
+
+.. automodule:: miplearn.solvers.gurobi
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+miplearn.solvers.learning
+-------------------------
+
+.. automodule:: miplearn.solvers.learning
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/benchmark.md

@@ -1,177 +0,0 @@
-```{sectnum}
----
-start: 2
-depth: 2
-suffix: .
----
-```
-
-# Benchmarks
-
-MIPLearn provides a selection of benchmark problems and random instance generators, covering applications from different fields, that can be used to evaluate new learning-enhanced MIP techniques in a measurable and reproducible way. In this page, we describe these problems, the included instance generators, and we present some benchmark results for  `LearningSolver` with default parameters.
-
-## Preliminaries
-
-### Benchmark challenges
-
-When evaluating the performance of a conventional MIP solver, *benchmark sets*, such as MIPLIB and TSPLIB, are typically used. The performance of newly proposed solvers or solution techniques are typically measured as the average (or total) running time the solver takes to solve the entire benchmark set. For Learning-Enhanced MIP solvers, it is also necessary to specify what instances should the solver be trained on (the *training instances*) before solving the actual set of instances we are interested in (the *test instances*). If the training instances are very similar to the test instances, we would expect a Learning-Enhanced Solver to present stronger perfomance benefits.
-
-In MIPLearn, each optimization problem comes with a set of **benchmark challenges**, which specify how should the training and test instances be generated. The first challenges are typically easier, in the sense that training and test instances are very similar. Later challenges gradually make the sets more distinct, and therefore harder to learn from.
-
-### Baseline results
-
-To illustrate the performance of `LearningSolver`, and to set a baseline for newly proposed techniques, we present in this page, for each benchmark challenge, a small set of computational results measuring the solution speed of the solver and the solution quality with default parameters. For more detailed computational studies, see [references](about.md#references). We compare three solvers:
-
-* **baseline:** Gurobi 9.0 with default settings (a conventional state-of-the-art MIP solver)
-* **ml-exact:** `LearningSolver` with default settings, using Gurobi 9.0 as internal MIP solver
-* **ml-heuristic:** Same as above, but with `mode="heuristic"`
-
-All experiments presented here were performed on a Linux server (Ubuntu Linux 18.04 LTS) with Intel Xeon Gold 6230s (2 processors, 40 cores, 80 threads) and 256 GB RAM (DDR4, 2933 MHz). All solvers were restricted to use 4 threads, with no time limits, and 10 instances were solved simultaneously at a time.
-
-
-
-## Maximum Weight Stable Set Problem
-
-### Problem definition
-
-Given a simple undirected graph $G=(V,E)$ and weights $w \in \mathbb{R}^V$, the problem is to find a stable set $S \subseteq V$ that maximizes $ \sum_{v \in V} w_v$. We recall that a subset $S \subseteq V$ is a *stable set* if no two vertices of $S$ are adjacent. This is one of Karp's 21 NP-complete problems.
-
-### Random instance generator
-
-The class `MaxWeightStableSetGenerator` can be used to generate random instances of this problem, with user-specified probability distributions. When the constructor parameter `fix_graph=True` is provided, one random Erdős-Rényi graph $G_{n,p}$ is generated during the constructor, where $n$ and $p$ are sampled from user-provided probability distributions `n` and `p`. To generate each instance, the generator independently samples each $w_v$ from the user-provided probability distribution `w`. When `fix_graph=False`, a new random graph is generated for each instance, while the remaining parameters are sampled in the same way.
-
-### Challenge A
-
-* Fixed random Erdős-Rényi graph $G_{n,p}$ with $n=200$ and $p=5\%$
-* Random vertex weights $w_v \sim U(100, 150)$
-* 500 training instances, 50 test instances
-
-```python
-MaxWeightStableSetGenerator(w=uniform(loc=100., scale=50.),
-                            n=randint(low=200, high=201),
-                            p=uniform(loc=0.05, scale=0.0),
-                            fix_graph=True)
-```
-
-![alt](figures/benchmark_stab_a.png)
-
-
-## Traveling Salesman Problem
-
-### Problem definition
-
-Given a list of cities and the distance between each pair of cities, the problem asks for the
-shortest route starting at the first city, visiting each other city exactly once, then returning
-to the first city. This problem is a generalization of the Hamiltonian path problem, one of Karp's
-21 NP-complete problems.
-
-### Random problem generator
-
-The class `TravelingSalesmanGenerator` can be used to generate random instances of this
-problem. Initially, the generator creates $n$ cities $(x_1,y_1),\ldots,(x_n,y_n) \in \mathbb{R}^2$,
-where $n, x_i$ and $y_i$ are sampled independently from the provided probability distributions `n`,
-`x` and `y`. For each pair of cities $(i,j)$, the distance $d_{i,j}$ between them is set to:
-$$
-    d_{i,j} = \gamma_{i,j} \sqrt{(x_i-x_j)^2 + (y_i - y_j)^2}
-$$
-where $\gamma_{i,j}$ is sampled from the distribution `gamma`.
-
-If `fix_cities=True` is provided, the list of cities is kept the same for all generated instances.
-The $gamma$ values, and therefore also the distances, are still different.
-
-By default, all distances $d_{i,j}$ are rounded to the nearest integer.  If `round=False`
-is provided, this rounding will be disabled.
-
-### Challenge A
-
-* Fixed list of 350 cities in the $[0, 1000]^2$ square
-* $\gamma_{i,j} \sim U(0.95, 1.05)$
-* 500 training instances, 50 test instances
-
-
-```python
-TravelingSalesmanGenerator(x=uniform(loc=0.0, scale=1000.0),
-                           y=uniform(loc=0.0, scale=1000.0),
-                           n=randint(low=350, high=351),
-                           gamma=uniform(loc=0.95, scale=0.1),
-                           fix_cities=True,
-                           round=True,
-                          )
-```
-
-![alt](figures/benchmark_tsp_a.png)
-
-
-## Multidimensional 0-1 Knapsack Problem
-
-### Problem definition
-
-Given a set of $n$ items and $m$ types of resources (also called *knapsacks*), the problem is to find a subset of items that maximizes profit without consuming more resources than it is available. More precisely, the problem is:
-
-$$
-\begin{align*}
-    \text{maximize}
-        & \sum_{j=1}^n p_j x_j
-        \\
-    \text{subject to}
-        & \sum_{j=1}^n w_{ij} x_j \leq b_i
-        & \forall i=1,\ldots,m \\
-    & x_j \in \{0,1\}
-        & \forall j=1,\ldots,n
-\end{align*}
-$$
-
-### Random instance generator
-
-The class `MultiKnapsackGenerator` can be used to generate random instances of this problem. The number of items $n$ and knapsacks $m$ are sampled from the user-provided probability distributions `n` and `m`. The weights $w_{ij}$ are sampled independently from the provided distribution `w`. The capacity of knapsack $i$ is set to
-
-$$
-    b_i = \alpha_i \sum_{j=1}^n w_{ij}
-$$
-
-where $\alpha_i$, the tightness ratio, is sampled from the provided probability
-distribution `alpha`. To make the instances more challenging, the costs of the items
-are linearly correlated to their average weights. More specifically, the price of each
-item $j$ is set to:
-
-$$
-    p_j = \sum_{i=1}^m \frac{w_{ij}}{m} + K  u_j,
-$$
-
-where $K$, the correlation coefficient, and $u_j$, the correlation multiplier, are sampled
-from the provided probability distributions `K` and `u`.
-
-If `fix_w=True` is provided, then $w_{ij}$ are kept the same in all generated instances. This also implies that $n$ and $m$ are kept fixed. Although the prices and capacities are derived from $w_{ij}$, as long as `u` and `K` are not constants, the generated instances will still not be completely identical.
-
-
-If a probability distribution `w_jitter` is provided, then item weights will be set to $w_{ij} \gamma_{ij}$ where $\gamma_{ij}$ is sampled from `w_jitter`. When combined with `fix_w=True`, this argument may be used to generate instances where the weight of each item is roughly the same, but not exactly identical, across all instances. The prices of the items and the capacities of the knapsacks will be calculated as above, but using these perturbed weights instead.
-
-By default, all generated prices, weights and capacities are rounded to the nearest integer number. If `round=False` is provided, this rounding will be disabled.
-
-
-!!! note "References"
-    * Freville, Arnaud, and Gérard Plateau. *An efficient preprocessing procedure for the multidimensional 0–1 knapsack problem.* Discrete applied mathematics 49.1-3 (1994): 189-212.
-    * Fréville, Arnaud. *The multidimensional 0–1 knapsack problem: An overview.* European Journal of Operational Research 155.1 (2004): 1-21.
-    
-### Challenge A
-
-* 250 variables, 10 constraints, fixed weights
-* $w \sim U(0, 1000), \gamma \sim U(0.95, 1.05)$
-* $K = 500, u \sim U(0, 1), \alpha = 0.25$
-* 500 training instances, 50 test instances
-
-
-```python
-MultiKnapsackGenerator(n=randint(low=250, high=251),
-                       m=randint(low=10, high=11),
-                       w=uniform(loc=0.0, scale=1000.0),
-                       K=uniform(loc=500.0, scale=0.0),
-                       u=uniform(loc=0.0, scale=1.0),
-                       alpha=uniform(loc=0.25, scale=0.0),
-                       fix_w=True,
-                       w_jitter=uniform(loc=0.95, scale=0.1),
-                      )
-```
-
-![alt](figures/benchmark_knapsack_a.png)
-

docs/conf.py

@@ -1,16 +1,25 @@
 project = "MIPLearn"
-copyright = "2020-2021, UChicago Argonne, LLC"
+copyright = "2020-2023, UChicago Argonne, LLC"
 author = ""
-release = "0.2.0"
+release = "0.3"
-extensions = ["myst_parser"]
+extensions = [
+    "myst_parser",
+    "nbsphinx",
+    "sphinx_multitoc_numbering",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.napoleon",
+]
 templates_path = ["_templates"]
 exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 html_theme = "sphinx_book_theme"
 html_static_path = ["_static"]
-html_css_files = ["custom.css"]
+html_css_files = [
+    "custom.css",
+]
 html_theme_options = {
     "repository_url": "https://github.com/ANL-CEEESA/MIPLearn/",
-    "use_repository_button": True,
+    "use_repository_button": False,
     "extra_navbar": "",
 }
-html_title = f"MIPLearn<br/><small>{release}</small>"
+html_title = f"MIPLearn {release}"
+nbsphinx_execute = "never"

docs/customization.md

@@ -1,182 +0,0 @@
-```{sectnum}
----
-start: 3
-depth: 2
-suffix: .
----
-```
-
-# Customization
-
-## Customizing solver parameters
-
-### Selecting the internal MIP solver
-
-By default, `LearningSolver` uses [Gurobi](https://www.gurobi.com/) as its internal MIP solver, and expects models to be provided using the Pyomo modeling language. Supported solvers and modeling languages include:
-
-* `GurobiPyomoSolver`: Gurobi with Pyomo (default).
-* `CplexPyomoSolver`: [IBM ILOG CPLEX](https://www.ibm.com/products/ilog-cplex-optimization-studio) with Pyomo.
-* `XpressPyomoSolver`: [FICO XPRESS Solver](https://www.fico.com/en/products/fico-xpress-solver) with Pyomo.
-* `GurobiSolver`: Gurobi without any modeling language.
-
-To switch between solvers, provide the desired class using the `solver` argument:
-
-```python
-from miplearn import LearningSolver, CplexPyomoSolver
-solver = LearningSolver(solver=CplexPyomoSolver)
-```
-
-To configure a particular solver, use the `params` constructor argument, as shown below.
-
-```python
-from miplearn import LearningSolver, GurobiPyomoSolver
-solver = LearningSolver(
-    solver=lambda: GurobiPyomoSolver(
-        params={
-            "TimeLimit": 900,
-            "MIPGap": 1e-3,
-            "NodeLimit": 1000,
-        }
-    ),
-)
-```
-
-
-## Customizing solver components
-
-`LearningSolver` is composed by a number of individual machine-learning components, each targeting a different part of the solution process. Each component can be individually enabled, disabled or customized. The following components are enabled by default:
-
-* `LazyConstraintComponent`: Predicts which lazy constraint to initially enforce.
-* `ObjectiveValueComponent`: Predicts the optimal value of the optimization problem, given the optimal solution to the LP relaxation.
-* `PrimalSolutionComponent`: Predicts optimal values for binary decision variables. In heuristic mode, this component fixes the variables to their predicted values. In exact mode, the predicted values are provided to the solver as a (partial) MIP start.
-
-The following components are also available, but not enabled by default:
-
-* `BranchPriorityComponent`: Predicts good branch priorities for decision variables.
-
-### Selecting components
-
-To create a `LearningSolver` with a specific set of components, the `components` constructor argument may be used, as the next example shows:
-
-```python
-# Create a solver without any components
-solver1 = LearningSolver(components=[])
-
-# Create a solver with only two components
-solver2 = LearningSolver(components=[
-    LazyConstraintComponent(...),
-    PrimalSolutionComponent(...),
-])
-```
-
-### Adjusting component aggressiveness
-
-The aggressiveness of classification components, such as `PrimalSolutionComponent` and `LazyConstraintComponent`, can be adjusted through the `threshold` constructor argument. Internally, these components ask the machine learning models how confident are they on each prediction they make, then automatically discard all predictions that have low confidence. The `threshold` argument specifies how confident should the ML models be for a prediction to be considered trustworthy. Lowering a component's threshold increases its aggressiveness, while raising a component's threshold makes it more conservative.
-
-For example, if the ML model predicts that a certain binary variable will assume value `1.0` in the optimal solution with 75% confidence, and if the `PrimalSolutionComponent` is configured to discard all predictions with less than 90% confidence, then this variable will not be included in the predicted MIP start.
-
-MIPLearn currently provides two types of thresholds:
-
-* `MinProbabilityThreshold(p: List[float])` A threshold which indicates that a prediction is trustworthy if its probability of being correct, as computed by the machine learning model, is above a fixed value.
-* `MinPrecisionThreshold(p: List[float])` A dynamic threshold which automatically adjusts itself during training to ensure that the component achieves at least a given precision on the training data set. Note that increasing a component's precision may reduce its recall.
-
-The example below shows how to build a `PrimalSolutionComponent` which fixes variables to zero with at least 80% precision, and to one with at least 95% precision. Other components are configured similarly.
-
-```python
-from miplearn import PrimalSolutionComponent, MinPrecisionThreshold
-
-PrimalSolutionComponent(
-    mode="heuristic",
-    threshold=MinPrecisionThreshold([0.80, 0.95]),
-)
-```
-
-### Evaluating component performance
-
-MIPLearn allows solver components to be modified, trained and evaluated in isolation. In the following example, we build and
-fit `PrimalSolutionComponent` outside the solver, then evaluate its performance.
-
-```python
-from miplearn import PrimalSolutionComponent
-
-# User-provided set of previously-solved instances
-train_instances = [...]
-
-# Construct and fit component on a subset of training instances
-comp = PrimalSolutionComponent()
-comp.fit(train_instances[:100])
-
-# Evaluate performance on an additional set of training instances
-ev = comp.evaluate(train_instances[100:150])
-``` 
-
-The method `evaluate` returns a dictionary with performance evaluation statistics for each training instance provided,
-and for each type of prediction the component makes. To obtain a summary across all instances, pandas may be used, as below:
-
-```python
-import pandas as pd
-pd.DataFrame(ev["Fix one"]).mean(axis=1)
-```
-```text
-Predicted positive          3.120000
-Predicted negative        196.880000
-Condition positive         62.500000
-Condition negative        137.500000
-True positive               3.060000
-True negative             137.440000
-False positive              0.060000
-False negative             59.440000
-Accuracy                    0.702500
-F1 score                    0.093050
-Recall                      0.048921
-Precision                   0.981667
-Predicted positive (%)      1.560000
-Predicted negative (%)     98.440000
-Condition positive (%)     31.250000
-Condition negative (%)     68.750000
-True positive (%)           1.530000
-True negative (%)          68.720000
-False positive (%)          0.030000
-False negative (%)         29.720000
-dtype: float64
-```
-
-Regression components (such as `ObjectiveValueComponent`) can also be trained and evaluated similarly,
-as the next example shows:
-
-```python
-from miplearn import ObjectiveValueComponent
-comp = ObjectiveValueComponent()
-comp.fit(train_instances[:100])
-ev = comp.evaluate(train_instances[100:150])
-
-import pandas as pd
-pd.DataFrame(ev).mean(axis=1)
-```
-```text
-Mean squared error       7001.977827
-Explained variance          0.519790
-Max error                 242.375804
-Mean absolute error        65.843924
-R2                          0.517612
-Median absolute error      65.843924
-dtype: float64
-```
-
-### Using customized ML classifiers and regressors
-
-By default, given a training set of instantes, MIPLearn trains a fixed set of ML classifiers and regressors, then selects the best one based on cross-validation performance. Alternatively, the user may specify which ML model a component should use through the `classifier` or `regressor` contructor parameters. Scikit-learn classifiers and regressors are currently supported. A future version of the package will add compatibility with Keras models.
-
-The example below shows how to construct a `PrimalSolutionComponent` which internally uses scikit-learn's `KNeighborsClassifiers`. Any other scikit-learn classifier or pipeline can be used. It needs to be wrapped in `ScikitLearnClassifier` to ensure that all the proper data transformations are applied.
-
-```python
-from miplearn import PrimalSolutionComponent, ScikitLearnClassifier
-from sklearn.neighbors import KNeighborsClassifier
-
-comp = PrimalSolutionComponent(
-    classifier=ScikitLearnClassifier(
-        KNeighborsClassifier(n_neighbors=5),
-    ),
-)
-comp.fit(train_instances)
-``` 

docs/guide/collectors.ipynb

@@ -0,0 +1,277 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "505cea0b-5f5d-478a-9107-42bb5515937d",
+   "metadata": {},
+   "source": [
+    "# Training Data Collectors\n",
+    "The first step in solving mixed-integer optimization problems with the assistance of supervised machine learning methods is solving a large set of training instances and collecting the raw training data. In this section, we describe the various training data collectors included in MIPLearn. Additionally, the framework follows the convention of storing all training data in files with a specific data format (namely, HDF5). In this section, we briefly describe this format and the rationale for choosing it.\n",
+    "\n",
+    "## Overview\n",
+    "\n",
+    "In MIPLearn, a **collector** is a class that solves or analyzes the problem and collects raw data which may be later useful for machine learning methods. Collectors, by convention, take as input: (i) a list of problem data filenames, in gzipped pickle format, ending with `.pkl.gz`; (ii) a function that builds the optimization model, such as `build_tsp_model`. After processing is done, collectors store the training data in a HDF5 file located alongside with the problem data. For example, if the problem data is stored in file `problem.pkl.gz`, then the collector writes to `problem.h5`. Collectors are, in general, very time consuming, as they may need to solve the problem to optimality, potentially multiple times.\n",
+    "\n",
+    "## HDF5 Format\n",
+    "\n",
+    "MIPLearn stores all training data in [HDF5](HDF5) (Hierarchical Data Format, Version 5) files. The HDF format was originally developed by the [National Center for Supercomputing Applications][NCSA] (NCSA) for storing and organizing large amounts of data, and supports a variety of data types, including integers, floating-point numbers, strings, and arrays. Compared to other formats, such as CSV, JSON or SQLite, the HDF5 format provides several advantages for MIPLearn, including:\n",
+    "\n",
+    "- *Storage of multiple scalars, vectors and matrices in a single file* --- This allows MIPLearn to store all training data related to a given problem instance in a single file, which makes training data easier to store, organize and transfer.\n",
+    "- *High-performance partial I/O* --- Partial I/O allows MIPLearn to read a single element from the training data (e.g. value of the optimal solution) without loading the entire file to memory or reading it from beginning to end, which dramatically improves performance and reduces memory requirements. This is especially important when processing a large number of training data files.\n",
+    "- *On-the-fly compression* --- HDF5 files can be transparently compressed, using the gzip method, which reduces storage requirements and accelerates network transfers.\n",
+    "- *Stable, portable and well-supported data format* --- Training data files are typically expensive to generate. Having a stable and well supported data format ensures that these files remain usable in the future, potentially even by other non-Python MIP/ML frameworks.\n",
+    "\n",
+    "MIPLearn currently uses HDF5 as simple key-value storage for numerical data; more advanced features of the format, such as metadata, are not currently used. Although files generated by MIPLearn can be read with any HDF5 library, such as [h5py][h5py], some convenience functions are provided to make the access more simple and less error-prone. Specifically, the class [H5File][H5File], which is built on top of h5py, provides the methods [put_scalar][put_scalar], [put_array][put_array], [put_sparse][put_sparse], [put_bytes][put_bytes] to store, respectively, scalar values, dense multi-dimensional arrays, sparse multi-dimensional arrays and arbitrary binary data. The corresponding *get* methods are also provided. Compared to pure h5py methods, these methods automatically perform type-checking and gzip compression. The example below shows their usage.\n",
+    "\n",
+    "[HDF5]: https://en.wikipedia.org/wiki/Hierarchical_Data_Format\n",
+    "[NCSA]: https://en.wikipedia.org/wiki/National_Center_for_Supercomputing_Applications\n",
+    "[h5py]: https://www.h5py.org/\n",
+    "[H5File]: ../../api/helpers/#miplearn.h5.H5File\n",
+    "[put_scalar]: ../../api/helpers/#miplearn.h5.H5File.put_scalar\n",
+    "[put_array]: ../../api/helpers/#miplearn.h5.H5File.put_scalar\n",
+    "[put_sparse]: ../../api/helpers/#miplearn.h5.H5File.put_scalar\n",
+    "[put_bytes]: ../../api/helpers/#miplearn.h5.H5File.put_scalar\n",
+    "\n",
+    "\n",
+    "### Example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "f906fe9c",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "x1 = 1\n",
+      "x2 = hello world\n",
+      "x3 = [1 2 3]\n",
+      "x4 = [[0.37454012 0.9507143  0.7319939 ]\n",
+      " [0.5986585  0.15601864 0.15599452]\n",
+      " [0.05808361 0.8661761  0.601115  ]]\n",
+      "x5 =   (2, 3)\t0.68030757\n",
+      "  (3, 2)\t0.45049927\n",
+      "  (4, 0)\t0.013264962\n",
+      "  (0, 2)\t0.94220173\n",
+      "  (4, 2)\t0.5632882\n",
+      "  (2, 1)\t0.3854165\n",
+      "  (1, 1)\t0.015966251\n",
+      "  (3, 0)\t0.23089382\n",
+      "  (4, 4)\t0.24102546\n",
+      "  (1, 3)\t0.68326354\n",
+      "  (3, 1)\t0.6099967\n",
+      "  (0, 3)\t0.8331949\n"
+     ]
+    }
+   ],
+   "source": [
+    "import numpy as np\n",
+    "import scipy.sparse\n",
+    "\n",
+    "from miplearn.h5 import H5File\n",
+    "\n",
+    "# Set random seed to make example reproducible\n",
+    "np.random.seed(42)\n",
+    "\n",
+    "# Create a new empty HDF5 file\n",
+    "with H5File(\"test.h5\", \"w\") as h5:\n",
+    "    # Store a scalar\n",
+    "    h5.put_scalar(\"x1\", 1)\n",
+    "    h5.put_scalar(\"x2\", \"hello world\")\n",
+    "\n",
+    "    # Store a dense array and a dense matrix\n",
+    "    h5.put_array(\"x3\", np.array([1, 2, 3]))\n",
+    "    h5.put_array(\"x4\", np.random.rand(3, 3))\n",
+    "\n",
+    "    # Store a sparse matrix\n",
+    "    h5.put_sparse(\"x5\", scipy.sparse.random(5, 5, 0.5))\n",
+    "\n",
+    "# Re-open the file we just created and print\n",
+    "# previously-stored data\n",
+    "with H5File(\"test.h5\", \"r\") as h5:\n",
+    "    print(\"x1 =\", h5.get_scalar(\"x1\"))\n",
+    "    print(\"x2 =\", h5.get_scalar(\"x2\"))\n",
+    "    print(\"x3 =\", h5.get_array(\"x3\"))\n",
+    "    print(\"x4 =\", h5.get_array(\"x4\"))\n",
+    "    print(\"x5 =\", h5.get_sparse(\"x5\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "50441907",
+   "metadata": {},
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d0000c8d",
+   "metadata": {},
+   "source": [
+    "## Basic collector\n",
+    "\n",
+    "[BasicCollector][BasicCollector] is the most fundamental collector, and performs the following steps:\n",
+    "\n",
+    "1. Extracts all model data, such as objective function and constraint right-hand sides into numpy arrays, which can later be easily and efficiently accessed without rebuilding the model or invoking the solver;\n",
+    "2. Solves the linear relaxation of the problem and stores its optimal solution, basis status and sensitivity information, among other information;\n",
+    "3. Solves the original mixed-integer optimization problem to optimality and stores its optimal solution, along with solve statistics, such as number of explored nodes and wallclock time.\n",
+    "\n",
+    "Data extracted in Phases 1, 2 and 3 above are prefixed, respectively as `static_`, `lp_` and `mip_`. The entire set of fields is shown in the table below.\n",
+    "\n",
+    "[BasicCollector]: ../../api/collectors/#miplearn.collectors.basic.BasicCollector\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6529f667",
+   "metadata": {},
+   "source": [
+    "### Data fields\n",
+    "\n",
+    "| Field                             | Type                | Description                                                                                                                                 |\n",
+    "|-----------------------------------|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------|\n",
+    "| `static_constr_lhs`               | `(nconstrs, nvars)` | Constraint left-hand sides, in sparse matrix format                                                                                         |\n",
+    "| `static_constr_names`             | `(nconstrs,)`       | Constraint names                                                                                                                            |\n",
+    "| `static_constr_rhs`               | `(nconstrs,)`       | Constraint right-hand sides                                                                                                                 |\n",
+    "| `static_constr_sense`             | `(nconstrs,)`       | Constraint senses (`\"<\"`, `\">\"` or `\"=\"`)                                                                                                   |\n",
+    "| `static_obj_offset`               | `float`             | Constant value added to the objective function                                                                                              |\n",
+    "| `static_sense`                    | `str`               | `\"min\"` if minimization problem or `\"max\"` otherwise                                                                                        |\n",
+    "| `static_var_lower_bounds`         | `(nvars,)`          | Variable lower bounds                                                                                                                       |\n",
+    "| `static_var_names`                | `(nvars,)`          | Variable names                                                                                                                              |\n",
+    "| `static_var_obj_coeffs`           | `(nvars,)`          | Objective coefficients                                                                                                                      |\n",
+    "| `static_var_types`                | `(nvars,)`          | Types of the decision variables (`\"C\"`, `\"B\"` and `\"I\"` for continuous, binary and integer, respectively)                                   |\n",
+    "| `static_var_upper_bounds`         | `(nvars,)`          | Variable upper bounds                                                                                                                       |\n",
+    "| `lp_constr_basis_status`          | `(nconstr,)`        | Constraint basis status (`0` for basic, `-1` for non-basic)                                                                                 |\n",
+    "| `lp_constr_dual_values`           | `(nconstr,)`        | Constraint dual value (or shadow price)                                                                                                     |\n",
+    "| `lp_constr_sa_rhs_{up,down}`      | `(nconstr,)`        | Sensitivity information for the constraint RHS                                                                                              |\n",
+    "| `lp_constr_slacks`                | `(nconstr,)`        | Constraint slack in the solution to the LP relaxation                                                                                       |\n",
+    "| `lp_obj_value`                    | `float`             | Optimal value of the LP relaxation                                                                                                          |\n",
+    "| `lp_var_basis_status`             | `(nvars,)`          | Variable basis status (`0`, `-1`, `-2` or `-3` for basic, non-basic at lower bound, non-basic at upper bound, and superbasic, respectively) |\n",
+    "| `lp_var_reduced_costs`            | `(nvars,)`          | Variable reduced costs                                                                                                                      |\n",
+    "| `lp_var_sa_{obj,ub,lb}_{up,down}` | `(nvars,)`          | Sensitivity information for the variable objective coefficient, lower and upper bound.                                                      |\n",
+    "| `lp_var_values`                   | `(nvars,)`          | Optimal solution to the LP relaxation                                                                                                       |\n",
+    "| `lp_wallclock_time`               | `float`             | Time taken to solve the LP relaxation (in seconds)                                                                                          |\n",
+    "| `mip_constr_slacks`               | `(nconstrs,)`       | Constraint slacks in the best MIP solution                                                                                                  |\n",
+    "| `mip_gap`                         | `float`             | Relative MIP optimality gap                                                                                                                 |\n",
+    "| `mip_node_count`                  | `float`             | Number of explored branch-and-bound nodes                                                                                                   |\n",
+    "| `mip_obj_bound`                   | `float`             | Dual bound                                                                                                                                  |\n",
+    "| `mip_obj_value`                   | `float`             | Value of the best MIP solution                                                                                                              |\n",
+    "| `mip_var_values`                  | `(nvars,)`          | Best MIP solution                                                                                                                           |\n",
+    "| `mip_wallclock_time`              | `float`             | Time taken to solve the MIP (in seconds)                                                                                                    |"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f2894594",
+   "metadata": {},
+   "source": [
+    "### Example\n",
+    "\n",
+    "The example below shows how to generate a few random instances of the traveling salesman problem, store its problem data, run the collector and print some of the training data to screen."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "ac6f8c6f",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "lp_obj_value =  2909.0\n",
+      "mip_obj_value =  2921.0\n"
+     ]
+    }
+   ],
+   "source": [
+    "import random\n",
+    "import numpy as np\n",
+    "from scipy.stats import uniform, randint\n",
+    "from glob import glob\n",
+    "\n",
+    "from miplearn.problems.tsp import (\n",
+    "    TravelingSalesmanGenerator,\n",
+    "    build_tsp_model,\n",
+    ")\n",
+    "from miplearn.io import write_pkl_gz\n",
+    "from miplearn.h5 import H5File\n",
+    "from miplearn.collectors.basic import BasicCollector\n",
+    "\n",
+    "# Set random seed to make example reproducible.\n",
+    "random.seed(42)\n",
+    "np.random.seed(42)\n",
+    "\n",
+    "# Generate a few instances of the traveling salesman problem.\n",
+    "data = TravelingSalesmanGenerator(\n",
+    "    n=randint(low=10, high=11),\n",
+    "    x=uniform(loc=0.0, scale=1000.0),\n",
+    "    y=uniform(loc=0.0, scale=1000.0),\n",
+    "    gamma=uniform(loc=0.90, scale=0.20),\n",
+    "    fix_cities=True,\n",
+    "    round=True,\n",
+    ").generate(10)\n",
+    "\n",
+    "# Save instance data to data/tsp/00000.pkl.gz, data/tsp/00001.pkl.gz, ...\n",
+    "write_pkl_gz(data, \"data/tsp\")\n",
+    "\n",
+    "# Solve all instances and collect basic solution information.\n",
+    "# Process at most four instances in parallel.\n",
+    "bc = BasicCollector()\n",
+    "bc.collect(glob(\"data/tsp/*.pkl.gz\"), build_tsp_model, n_jobs=4)\n",
+    "\n",
+    "# Read and print some training data for the first instance.\n",
+    "with H5File(\"data/tsp/00000.h5\", \"r\") as h5:\n",
+    "    print(\"lp_obj_value = \", h5.get_scalar(\"lp_obj_value\"))\n",
+    "    print(\"mip_obj_value = \", h5.get_scalar(\"mip_obj_value\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "78f0b07a",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/guide/features.ipynb

@@ -0,0 +1,334 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "cdc6ebe9-d1d4-4de1-9b5a-4fc8ef57b11b",
+   "metadata": {},
+   "source": [
+    "# Feature Extractors\n",
+    "\n",
+    "In the previous page, we introduced *training data collectors*, which solve the optimization problem and collect raw training data, such as the optimal solution. In this page, we introduce **feature extractors**, which take the raw training data, stored in HDF5 files, and extract relevant information in order to train a machine learning model."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b4026de5",
+   "metadata": {},
+   "source": [
+    "\n",
+    "## Overview\n",
+    "\n",
+    "Feature extraction is an important step of the process of building a machine learning model because it helps to reduce the complexity of the data and convert it into a format that is more easily processed. Previous research has proposed converting absolute variable coefficients, for example, into relative values which are invariant  to various transformations, such as problem scaling, making them more amenable to learning. Various other transformations have also been described.\n",
+    "\n",
+    "In the framework, we treat data collection and feature extraction as two separate steps to accelerate the model development cycle. Specifically, collectors are typically time-consuming, as they often need to solve the problem to optimality, and therefore focus on collecting and storing all data that may or may not be relevant, in its raw format. Feature extractors, on the other hand, focus entirely on filtering the data and improving its representation, and are therefore much faster to run. Experimenting with new data representations, therefore, can be done without resolving the instances.\n",
+    "\n",
+    "In MIPLearn, extractors implement the abstract class [FeatureExtractor][FeatureExtractor], which has methods that take as input an [H5File][H5File] and produce either: (i) instance features, which describe the entire instances; (ii) variable features, which describe a particular decision variables; or (iii) constraint features, which describe a particular constraint. The extractor is free to implement only a subset of these methods, if it is known that it will not be used with a machine learning component that requires the other types of features.\n",
+    "\n",
+    "[FeatureExtractor]: ../../api/collectors/#miplearn.features.fields.FeaturesExtractor\n",
+    "[H5File]: ../../api/helpers/#miplearn.h5.H5File"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b2d9736c",
+   "metadata": {},
+   "source": [
+    "\n",
+    "## H5FieldsExtractor\n",
+    "\n",
+    "[H5FieldsExtractor][H5FieldsExtractor], the most simple extractor in MIPLearn, simple extracts data that is already available in the HDF5 file, assembles it into a matrix and returns it as-is. The fields used to build instance, variable and constraint features are user-specified. The class also performs checks to ensure that the shapes of the returned matrices make sense."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e8184dff",
+   "metadata": {},
+   "source": [
+    "### Example\n",
+    "\n",
+    "The example below demonstrates the usage of H5FieldsExtractor in a randomly generated instance of the multi-dimensional knapsack problem."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "ed9a18c8",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "instance features (11,) \n",
+      " [-1531.24308771  -350.          -692.          -454.\n",
+      "  -709.          -605.          -543.          -321.\n",
+      "  -674.          -571.          -341.        ]\n",
+      "variable features (10, 4) \n",
+      " [[-1.53124309e+03 -3.50000000e+02  0.00000000e+00  9.43468018e+01]\n",
+      " [-1.53124309e+03 -6.92000000e+02  2.51703322e-01  0.00000000e+00]\n",
+      " [-1.53124309e+03 -4.54000000e+02  0.00000000e+00  8.25504150e+01]\n",
+      " [-1.53124309e+03 -7.09000000e+02  1.11373022e-01  0.00000000e+00]\n",
+      " [-1.53124309e+03 -6.05000000e+02  1.00000000e+00 -1.26055283e+02]\n",
+      " [-1.53124309e+03 -5.43000000e+02  0.00000000e+00  1.68693771e+02]\n",
+      " [-1.53124309e+03 -3.21000000e+02  1.07488781e-01  0.00000000e+00]\n",
+      " [-1.53124309e+03 -6.74000000e+02  8.82293701e-01  0.00000000e+00]\n",
+      " [-1.53124309e+03 -5.71000000e+02  0.00000000e+00  1.41129074e+02]\n",
+      " [-1.53124309e+03 -3.41000000e+02  1.28830120e-01  0.00000000e+00]]\n",
+      "constraint features (5, 3) \n",
+      " [[ 1.3100000e+03 -1.5978307e-01  0.0000000e+00]\n",
+      " [ 9.8800000e+02 -3.2881632e-01  0.0000000e+00]\n",
+      " [ 1.0040000e+03 -4.0601316e-01  0.0000000e+00]\n",
+      " [ 1.2690000e+03 -1.3659772e-01  0.0000000e+00]\n",
+      " [ 1.0070000e+03 -2.8800571e-01  0.0000000e+00]]\n"
+     ]
+    }
+   ],
+   "source": [
+    "from glob import glob\n",
+    "from shutil import rmtree\n",
+    "\n",
+    "import numpy as np\n",
+    "from scipy.stats import uniform, randint\n",
+    "\n",
+    "from miplearn.collectors.basic import BasicCollector\n",
+    "from miplearn.extractors.fields import H5FieldsExtractor\n",
+    "from miplearn.h5 import H5File\n",
+    "from miplearn.io import write_pkl_gz\n",
+    "from miplearn.problems.multiknapsack import (\n",
+    "    MultiKnapsackGenerator,\n",
+    "    build_multiknapsack_model,\n",
+    ")\n",
+    "\n",
+    "# Set random seed to make example reproducible\n",
+    "np.random.seed(42)\n",
+    "\n",
+    "# Generate some random multiknapsack instances\n",
+    "rmtree(\"data/multiknapsack/\", ignore_errors=True)\n",
+    "write_pkl_gz(\n",
+    "    MultiKnapsackGenerator(\n",
+    "        n=randint(low=10, high=11),\n",
+    "        m=randint(low=5, high=6),\n",
+    "        w=uniform(loc=0, scale=1000),\n",
+    "        K=uniform(loc=100, scale=0),\n",
+    "        u=uniform(loc=1, scale=0),\n",
+    "        alpha=uniform(loc=0.25, scale=0),\n",
+    "        w_jitter=uniform(loc=0.95, scale=0.1),\n",
+    "        p_jitter=uniform(loc=0.75, scale=0.5),\n",
+    "        fix_w=True,\n",
+    "    ).generate(10),\n",
+    "    \"data/multiknapsack\",\n",
+    ")\n",
+    "\n",
+    "# Run the basic collector\n",
+    "BasicCollector().collect(\n",
+    "    glob(\"data/multiknapsack/*\"),\n",
+    "    build_multiknapsack_model,\n",
+    "    n_jobs=4,\n",
+    ")\n",
+    "\n",
+    "ext = H5FieldsExtractor(\n",
+    "    # Use as instance features the value of the LP relaxation and the\n",
+    "    # vector of objective coefficients.\n",
+    "    instance_fields=[\n",
+    "        \"lp_obj_value\",\n",
+    "        \"static_var_obj_coeffs\",\n",
+    "    ],\n",
+    "    # For each variable, use as features the optimal value of the LP\n",
+    "    # relaxation, the variable objective coefficient, the variable's\n",
+    "    # value its reduced cost.\n",
+    "    var_fields=[\n",
+    "        \"lp_obj_value\",\n",
+    "        \"static_var_obj_coeffs\",\n",
+    "        \"lp_var_values\",\n",
+    "        \"lp_var_reduced_costs\",\n",
+    "    ],\n",
+    "    # For each constraint, use as features the RHS, dual value and slack.\n",
+    "    constr_fields=[\n",
+    "        \"static_constr_rhs\",\n",
+    "        \"lp_constr_dual_values\",\n",
+    "        \"lp_constr_slacks\",\n",
+    "    ],\n",
+    ")\n",
+    "\n",
+    "with H5File(\"data/multiknapsack/00000.h5\") as h5:\n",
+    "    # Extract and print instance features\n",
+    "    x1 = ext.get_instance_features(h5)\n",
+    "    print(\"instance features\", x1.shape, \"\\n\", x1)\n",
+    "\n",
+    "    # Extract and print variable features\n",
+    "    x2 = ext.get_var_features(h5)\n",
+    "    print(\"variable features\", x2.shape, \"\\n\", x2)\n",
+    "\n",
+    "    # Extract and print constraint features\n",
+    "    x3 = ext.get_constr_features(h5)\n",
+    "    print(\"constraint features\", x3.shape, \"\\n\", x3)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2da2e74e",
+   "metadata": {},
+   "source": [
+    "\n",
+    "[H5FieldsExtractor]: ../../api/collectors/#miplearn.features.fields.H5FieldsExtractor"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d879c0d3",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-warning\">\n",
+    "Warning\n",
+    "\n",
+    "You should ensure that the number of features remains the same for all relevant HDF5 files. In the previous example, to illustrate this issue, we used variable objective coefficients as instance features. While this is allowed, note that this requires all problem instances to have the same number of variables; otherwise the number of features would vary from instance to instance and MIPLearn would be unable to concatenate the matrices.\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cd0ba071",
+   "metadata": {},
+   "source": [
+    "## AlvLouWeh2017Extractor\n",
+    "\n",
+    "Alvarez, Louveaux and Wehenkel (2017) proposed a set features to describe a particular decision variable in a given node of the branch-and-bound tree, and applied it to the problem of mimicking strong branching decisions. The class [AlvLouWeh2017Extractor][] implements a subset of these features (40 out of 64), which are available outside of the branch-and-bound tree. Some features are derived from the static defintion of the problem (i.e. from objective function and constraint data), while some features are derived from the solution to the LP relaxation. The features have been designed to be: (i) independent of the size of the problem; (ii) invariant with respect to irrelevant problem transformations, such as row and column permutation; and (iii) independent of the scale of the problem. We refer to the paper for a more complete description.\n",
+    "\n",
+    "### Example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "a1bc38fe",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "x1 (10, 40) \n",
+      " [[-1.00e+00  1.00e+20  1.00e-01  1.00e+00  0.00e+00  1.00e+00  6.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  6.00e-01  1.00e+00  1.75e+01  1.00e+00  2.00e-01\n",
+      "   1.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00 -1.00e+00  0.00e+00  1.00e+20]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  1.00e-01  1.00e+00  1.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  7.00e-01  1.00e+00  5.10e+00  1.00e+00  2.00e-01\n",
+      "   1.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   3.00e-01 -1.00e+00 -1.00e+00  0.00e+00  0.00e+00]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  0.00e+00  1.00e+00  9.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  5.00e-01  1.00e+00  1.30e+01  1.00e+00  2.00e-01\n",
+      "   1.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00 -1.00e+00  0.00e+00  1.00e+20]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  2.00e-01  1.00e+00  9.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  8.00e-01  1.00e+00  3.40e+00  1.00e+00  2.00e-01\n",
+      "   1.00e+00  1.00e-01  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   1.00e-01 -1.00e+00 -1.00e+00  0.00e+00  0.00e+00]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  1.00e-01  1.00e+00  7.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  6.00e-01  1.00e+00  3.80e+00  1.00e+00  2.00e-01\n",
+      "   1.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00 -1.00e+00 -1.00e+00  0.00e+00  0.00e+00]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  1.00e-01  1.00e+00  8.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  7.00e-01  1.00e+00  3.30e+00  1.00e+00  2.00e-01\n",
+      "   1.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00 -1.00e+00  0.00e+00  1.00e+20]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  0.00e+00  1.00e+00  3.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  1.00e+00  1.00e+00  5.70e+00  1.00e+00  1.00e-01\n",
+      "   1.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   1.00e-01 -1.00e+00 -1.00e+00  0.00e+00  0.00e+00]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  1.00e-01  1.00e+00  6.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  8.00e-01  1.00e+00  6.80e+00  1.00e+00  2.00e-01\n",
+      "   1.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   1.00e-01 -1.00e+00 -1.00e+00  0.00e+00  0.00e+00]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  4.00e-01  1.00e+00  6.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  8.00e-01  1.00e+00  1.40e+00  1.00e+00  1.00e-01\n",
+      "   1.00e+00  1.00e-01  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00 -1.00e+00  0.00e+00  1.00e+20]\n",
+      " [-1.00e+00  1.00e+20  1.00e-01  1.00e+00  0.00e+00  1.00e+00  5.00e-01\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  1.00e+00  5.00e-01  1.00e+00  7.60e+00  1.00e+00  1.00e-01\n",
+      "   1.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00  0.00e+00\n",
+      "   1.00e-01 -1.00e+00 -1.00e+00  0.00e+00  0.00e+00]]\n"
+     ]
+    }
+   ],
+   "source": [
+    "from miplearn.extractors.AlvLouWeh2017 import AlvLouWeh2017Extractor\n",
+    "from miplearn.h5 import H5File\n",
+    "\n",
+    "# Build the extractor\n",
+    "ext = AlvLouWeh2017Extractor()\n",
+    "\n",
+    "# Open previously-created multiknapsack training data\n",
+    "with H5File(\"data/multiknapsack/00000.h5\") as h5:\n",
+    "    # Extract and print variable features\n",
+    "    x1 = ext.get_var_features(h5)\n",
+    "    print(\"x1\", x1.shape, \"\\n\", x1.round(1))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "286c9927",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "References\n",
+    "\n",
+    "* **Alvarez, Alejandro Marcos.** *Computational and theoretical synergies between linear optimization and supervised machine learning.* (2016). University of Liège.\n",
+    "* **Alvarez, Alejandro Marcos, Quentin Louveaux, and Louis Wehenkel.** *A machine learning-based approximation of strong branching.* INFORMS Journal on Computing 29.1 (2017): 185-195.\n",
+    "\n",
+    "</div>"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/guide/primal.ipynb

@@ -0,0 +1,291 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "880cf4c7-d3c4-4b92-85c7-04a32264cdae",
+   "metadata": {},
+   "source": [
+    "# Primal Components\n",
+    "\n",
+    "In MIPLearn, a **primal component** is class that uses machine learning to predict a (potentially partial) assignment of values to the decision variables of the problem. Predicting high-quality primal solutions may be beneficial, as they allow the MIP solver to prune potentially large portions of the search space. Alternatively, if proof of optimality is not required, the MIP solver can be used to complete the partial solution generated by the machine learning model and and double-check its feasibility. MIPLearn allows both of these usage patterns.\n",
+    "\n",
+    "In this page, we describe the four primal components currently included in MIPLearn, which employ machine learning in different ways. Each component is highly configurable, and accepts an user-provided machine learning model, which it uses for all predictions. Each component can also be configured to provide the solution to the solver in multiple ways, depending on whether proof of optimality is required.\n",
+    "\n",
+    "## Primal component actions\n",
+    "\n",
+    "Before presenting the primal components themselves, we briefly discuss the three ways a solution may be provided to the solver. Each approach has benefits and limitations, which we also discuss in this section. All primal components can be configured to use any of the following approaches.\n",
+    "\n",
+    "The first approach is to provide the solution to the solver as a **warm start**. This is implemented by the class [SetWarmStart](SetWarmStart). The main advantage is that this method maintains all optimality and feasibility guarantees of the MIP solver, while still providing significant performance benefits for various classes of problems. If the machine learning model is able to predict multiple solutions, it is also possible to set multiple warm starts. In this case, the solver evaluates each warm start, discards the infeasible ones, then proceeds with the one that has the best objective value. The main disadvantage of this approach, compared to the next two, is that it provides relatively modest speedups for most problem classes, and no speedup at all for many others, even when the machine learning predictions are 100% accurate.\n",
+    "\n",
+    "[SetWarmStart]: ../../api/components/#miplearn.components.primal.actions.SetWarmStart\n",
+    "\n",
+    "The second approach is to **fix the decision variables** to their predicted values, then solve a restricted optimization problem on the remaining variables. This approach is implemented by the class `FixVariables`. The main advantage is its potential speedup: if machine learning can accurately predict values for a significant portion of the decision variables, then the MIP solver can typically complete the solution in a small fraction of the time it would take to find the same solution from scratch. The main disadvantage of this approach is that it loses optimality guarantees; that is, the complete solution found by the MIP solver may no longer be globally optimal. Also, if the machine learning predictions are not sufficiently accurate, there might not even be a feasible assignment for the variables that were left free.\n",
+    "\n",
+    "Finally, the third approach, which tries to strike a balance between the two previous ones, is to **enforce proximity** to a given solution. This strategy is implemented by the class `EnforceProximity`. More precisely, given values $\\bar{x}_1,\\ldots,\\bar{x}_n$ for a subset of binary decision variables $x_1,\\ldots,x_n$, this approach adds the constraint\n",
+    "\n",
+    "$$\n",
+    "\\sum_{i : \\bar{x}_i=0} x_i + \\sum_{i : \\bar{x}_i=1} \\left(1 - x_i\\right) \\leq k,\n",
+    "$$\n",
+    "to the problem, where $k$ is a user-defined parameter, which indicates how many of the predicted variables are allowed to deviate from the machine learning suggestion. The main advantage of this approach, compared to fixing variables, is its tolerance to lower-quality machine learning predictions. Its main disadvantage is that it typically leads to smaller speedups, especially for larger values of $k$. This approach also loses optimality guarantees.\n",
+    "\n",
+    "## Memorizing primal component\n",
+    "\n",
+    "A simple machine learning strategy for the prediction of primal solutions is to memorize all distinct solutions seen during training, then try to predict, during inference time, which of those memorized solutions are most likely to be feasible and to provide a good objective value for the current instance. The most promising solutions may alternatively be combined into a single partial solution, which is then provided to the MIP solver. Both variations of this strategy are implemented by the `MemorizingPrimalComponent` class. Note that it is only applicable if the problem size, and in fact if the meaning of the decision variables, remains the same across problem instances.\n",
+    "\n",
+    "More precisely, let $I_1,\\ldots,I_n$ be the training instances, and let $\\bar{x}^1,\\ldots,\\bar{x}^n$ be their respective optimal solutions. Given a new instance $I_{n+1}$, `MemorizingPrimalComponent` expects a user-provided binary classifier that assigns (through the `predict_proba` method, following scikit-learn's conventions) a score $\\delta_i$ to each solution $\\bar{x}^i$, such that solutions with higher score are more likely to be good solutions for $I_{n+1}$. The features provided to the classifier are the instance features computed by an user-provided extractor. Given these scores, the component then performs one of the following to actions, as decided by the user:\n",
+    "\n",
+    "1. Selects the top $k$ solutions with the highest scores and provides them to the solver; this is implemented by `SelectTopSolutions`, and it is typically used with the `SetWarmStart` action.\n",
+    "\n",
+    "2. Merges the top $k$ solutions into a single partial solution, then provides it to the solver. This is implemented by `MergeTopSolutions`. More precisely, suppose that the machine learning regressor ordered the solutions in the sequence $\\bar{x}^{i_1},\\ldots,\\bar{x}^{i_n}$, with the most promising solutions appearing first, and with ties being broken arbitrarily. The component starts by keeping only the $k$ most promising solutions $\\bar{x}^{i_1},\\ldots,\\bar{x}^{i_k}$. Then it computes, for each binary decision variable $x_l$, its average assigned value $\\tilde{x}_l$:\n",
+    "$$\n",
+    "    \\tilde{x}_l = \\frac{1}{k} \\sum_{j=1}^k \\bar{x}^{i_j}_l.\n",
+    "$$\n",
+    "  Finally, the component constructs a merged solution $y$, defined as:\n",
+    "$$\n",
+    "    y_j = \\begin{cases}\n",
+    "        0 & \\text{ if } \\tilde{x}_l \\le \\theta_0 \\\\\n",
+    "        1 & \\text{ if } \\tilde{x}_l \\ge \\theta_1 \\\\\n",
+    "        \\square & \\text{otherwise,}\n",
+    "    \\end{cases}\n",
+    "$$\n",
+    "  where $\\theta_0$ and $\\theta_1$ are user-specified parameters, and where $\\square$ indicates that the variable is left undefined. The solution $y$ is then provided by the solver using any of the three approaches defined in the previous section.\n",
+    "\n",
+    "The above specification of `MemorizingPrimalComponent` is meant to be as general as possible. Simpler strategies can be implemented by configuring this component in specific ways. For example, a simpler approach employed in the literature is to collect all optimal solutions, then provide the entire list of solutions to the solver as warm starts, without any filtering or post-processing. This strategy can be implemented with `MemorizingPrimalComponent` by using a model that returns a constant value for all solutions (e.g. [scikit-learn's DummyClassifier][DummyClassifier]), then selecting the top $n$ (instead of $k$) solutions. See example below. Another simple approach is taking the solution to the most similar instance, and using it, by itself, as a warm start. This can be implemented by using a model that computes distances between the current instance and the training ones (e.g. [scikit-learn's KNeighborsClassifier][KNeighborsClassifier]), then select the solution to the nearest one. See also example below. More complex strategies, of course, can also be configured.\n",
+    "\n",
+    "[DummyClassifier]: https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyClassifier.html\n",
+    "[KNeighborsClassifier]: https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html\n",
+    "\n",
+    "### Examples"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "253adbf4",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from sklearn.dummy import DummyClassifier\n",
+    "from sklearn.neighbors import KNeighborsClassifier\n",
+    "\n",
+    "from miplearn.components.primal.actions import (\n",
+    "    SetWarmStart,\n",
+    "    FixVariables,\n",
+    "    EnforceProximity,\n",
+    ")\n",
+    "from miplearn.components.primal.mem import (\n",
+    "    MemorizingPrimalComponent,\n",
+    "    SelectTopSolutions,\n",
+    "    MergeTopSolutions,\n",
+    ")\n",
+    "from miplearn.extractors.dummy import DummyExtractor\n",
+    "from miplearn.extractors.fields import H5FieldsExtractor\n",
+    "\n",
+    "# Configures a memorizing primal component that collects\n",
+    "# all distinct solutions seen during training and provides\n",
+    "# them to the solver without any filtering or post-processing.\n",
+    "comp1 = MemorizingPrimalComponent(\n",
+    "    clf=DummyClassifier(),\n",
+    "    extractor=DummyExtractor(),\n",
+    "    constructor=SelectTopSolutions(1_000_000),\n",
+    "    action=SetWarmStart(),\n",
+    ")\n",
+    "\n",
+    "# Configures a memorizing primal component that finds the\n",
+    "# training instance with the closest objective function, then\n",
+    "# fixes the decision variables to the values they assumed\n",
+    "# at the optimal solution for that instance.\n",
+    "comp2 = MemorizingPrimalComponent(\n",
+    "    clf=KNeighborsClassifier(n_neighbors=1),\n",
+    "    extractor=H5FieldsExtractor(\n",
+    "        instance_fields=[\"static_var_obj_coeffs\"],\n",
+    "    ),\n",
+    "    constructor=SelectTopSolutions(1),\n",
+    "    action=FixVariables(),\n",
+    ")\n",
+    "\n",
+    "# Configures a memorizing primal component that finds the distinct\n",
+    "# solutions to the 10 most similar training problem instances,\n",
+    "# selects the 3 solutions that were most often optimal to these\n",
+    "# training instances, combines them into a single partial solution,\n",
+    "# then enforces proximity, allowing at most 3 variables to deviate\n",
+    "# from the machine learning suggestion.\n",
+    "comp3 = MemorizingPrimalComponent(\n",
+    "    clf=KNeighborsClassifier(n_neighbors=10),\n",
+    "    extractor=H5FieldsExtractor(instance_fields=[\"static_var_obj_coeffs\"]),\n",
+    "    constructor=MergeTopSolutions(k=3, thresholds=[0.25, 0.75]),\n",
+    "    action=EnforceProximity(3),\n",
+    ")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f194a793",
+   "metadata": {},
+   "source": [
+    "## Independent vars primal component\n",
+    "\n",
+    "Instead of memorizing previously-seen primal solutions, it is also natural to use machine learning models to directly predict the values of the decision variables, constructing a solution from scratch. This approach has the benefit of potentially constructing novel high-quality solutions, never observed in the training data. Two variations of this strategy are supported by MIPLearn: (i) predicting the values of the decision variables independently, using multiple ML models; or (ii) predicting the values jointly, with a single model. We describe the first variation in this section, and the second variation in the next section.\n",
+    "\n",
+    "Let $I_1,\\ldots,I_n$ be the training instances, and let $\\bar{x}^1,\\ldots,\\bar{x}^n$ be their respective optimal solutions. For each binary decision variable $x_j$, the component `IndependentVarsPrimalComponent` creates a copy of a user-provided binary classifier and trains it to predict the optimal value of $x_j$, given $\\bar{x}^1_j,\\ldots,\\bar{x}^n_j$ as training labels. The features provided to the model are the variable features computed by an user-provided extractor. During inference time, the component uses these $n$ binary classifiers to construct a solution and provides it to the solver using one of the available actions.\n",
+    "\n",
+    "Three issues often arise in practice when using this approach:\n",
+    "\n",
+    " 1. For certain binary variables $x_j$, it is frequently the case that its optimal value is either always zero or always one in the training dataset, which poses problems to some standard scikit-learn classifiers, since they do not expect a single class. The wrapper `SingleClassFix` can be used to fix this issue (see example below).\n",
+    "2. It is also frequently the case that machine learning classifier can only reliably predict the values of some variables with high accuracy, not all of them. In this situation, instead of computing a complete primal solution, it may be more beneficial to construct a partial solution containing values only for the variables for which the ML made a high-confidence prediction. The meta-classifier `MinProbabilityClassifier` can be used for this purpose. It asks the base classifier for the probability of the value being zero or one (using the `predict_proba` method) and erases from the primal solution all values whose probabilities are below a given threshold.\n",
+    "3. To make multiple copies of the provided ML classifier, MIPLearn uses the standard `sklearn.base.clone` method, which may not be suitable for classifiers from other frameworks. To handle this, it is possible to override the clone function using the `clone_fn` constructor argument.\n",
+    "\n",
+    "### Examples"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "3fc0b5d1",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from sklearn.linear_model import LogisticRegression\n",
+    "from miplearn.classifiers.minprob import MinProbabilityClassifier\n",
+    "from miplearn.classifiers.singleclass import SingleClassFix\n",
+    "from miplearn.components.primal.indep import IndependentVarsPrimalComponent\n",
+    "from miplearn.extractors.AlvLouWeh2017 import AlvLouWeh2017Extractor\n",
+    "from miplearn.components.primal.actions import SetWarmStart\n",
+    "\n",
+    "# Configures a primal component that independently predicts the value of each\n",
+    "# binary variable using logistic regression and provides it to the solver as\n",
+    "# warm start. Erases predictions with probability less than 99%; applies\n",
+    "# single-class fix; and uses AlvLouWeh2017 features.\n",
+    "comp = IndependentVarsPrimalComponent(\n",
+    "    base_clf=SingleClassFix(\n",
+    "        MinProbabilityClassifier(\n",
+    "            base_clf=LogisticRegression(),\n",
+    "            thresholds=[0.99, 0.99],\n",
+    "        ),\n",
+    "    ),\n",
+    "    extractor=AlvLouWeh2017Extractor(),\n",
+    "    action=SetWarmStart(),\n",
+    ")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "45107a0c",
+   "metadata": {},
+   "source": [
+    "## Joint vars primal component\n",
+    "In the previous subsection, we used multiple machine learning models to independently predict the values of the binary decision variables. When these values are correlated, an alternative approach is to jointly predict the values of all binary variables using a single machine learning model. This strategy is implemented by `JointVarsPrimalComponent`. Compared to the previous ones, this component is much more straightforwad. It simply extracts instance features, using the user-provided feature extractor, then directly trains the user-provided binary classifier (using the `fit` method), without making any copies. The trained classifier is then used to predict entire solutions (using the `predict` method), which are given to the solver using one of the previously discussed methods. In the example below, we illustrate the usage of this component with a simple feed-forward neural network.\n",
+    "\n",
+    "`JointVarsPrimalComponent` can also be used to implement strategies that use multiple machine learning models, but not indepedently. For example, a common strategy in multioutput prediction is building a *classifier chain*. In this approach, the first decision variable is predicted using the instance features alone; but the $n$-th decision variable is predicted using the instance features plus the predicted values of the $n-1$ previous variables. This can be easily implemented using scikit-learn's `ClassifierChain` estimator, as shown in the example below.\n",
+    "\n",
+    "### Examples"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "cf9b52dd",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from sklearn.multioutput import ClassifierChain\n",
+    "from sklearn.neural_network import MLPClassifier\n",
+    "from miplearn.components.primal.joint import JointVarsPrimalComponent\n",
+    "from miplearn.extractors.fields import H5FieldsExtractor\n",
+    "from miplearn.components.primal.actions import SetWarmStart\n",
+    "\n",
+    "# Configures a primal component that uses a feedforward neural network\n",
+    "# to jointly predict the values of the binary variables, based on the\n",
+    "# objective cost function, and provides the solution to the solver as\n",
+    "# a warm start.\n",
+    "comp = JointVarsPrimalComponent(\n",
+    "    clf=MLPClassifier(),\n",
+    "    extractor=H5FieldsExtractor(\n",
+    "        instance_fields=[\"static_var_obj_coeffs\"],\n",
+    "    ),\n",
+    "    action=SetWarmStart(),\n",
+    ")\n",
+    "\n",
+    "# Configures a primal component that uses a chain of logistic regression\n",
+    "# models to jointly predict the values of the binary variables, based on\n",
+    "# the objective function.\n",
+    "comp = JointVarsPrimalComponent(\n",
+    "    clf=ClassifierChain(SingleClassFix(LogisticRegression())),\n",
+    "    extractor=H5FieldsExtractor(\n",
+    "        instance_fields=[\"static_var_obj_coeffs\"],\n",
+    "    ),\n",
+    "    action=SetWarmStart(),\n",
+    ")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dddf7be4",
+   "metadata": {},
+   "source": [
+    "## Expert primal component\n",
+    "\n",
+    "Before spending time and effort choosing a machine learning strategy and tweaking its parameters, it is usually a good idea to evaluate what would be the performance impact of the model if its predictions were 100% accurate. This is especially important for the prediction of warm starts, since they are not always very beneficial. To simplify this task, MIPLearn provides `ExpertPrimalComponent`, a component which simply loads the optimal solution from the HDF5 file, assuming that it has already been computed, then directly provides it to the solver using one of the available methods. This component is useful in benchmarks, to evaluate how close to the best theoretical performance the machine learning components are.\n",
+    "\n",
+    "### Example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "9e2e81b9",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from miplearn.components.primal.expert import ExpertPrimalComponent\n",
+    "from miplearn.components.primal.actions import SetWarmStart\n",
+    "\n",
+    "# Configures an expert primal component, which reads a pre-computed\n",
+    "# optimal solution from the HDF5 file and provides it to the solver\n",
+    "# as warm start.\n",
+    "comp = ExpertPrimalComponent(action=SetWarmStart())\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/guide/solvers.ipynb

@@ -0,0 +1,247 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "9ec1907b-db93-4840-9439-c9005902b968",
+   "metadata": {},
+   "source": [
+    "# Learning Solver\n",
+    "\n",
+    "On previous pages, we discussed various components of the MIPLearn framework, including training data collectors, feature extractors, and individual machine learning components. In this page, we introduce **LearningSolver**, the main class of the framework which integrates all the aforementioned components into a cohesive whole. Using **LearningSolver** involves three steps: (i) configuring the solver; (ii) training the ML components; and (iii) solving new MIP instances. In the following, we describe each of these steps, then conclude with a complete runnable example.\n",
+    "\n",
+    "### Configuring the solver\n",
+    "\n",
+    "**LearningSolver** is composed by multiple individual machine learning components, each targeting a different part of the solution process, or implementing a different machine learning strategy. This architecture allows strategies to be easily enabled, disabled or customized, making the framework flexible. By default, no components are provided and **LearningSolver** is equivalent to a traditional MIP solver. To specify additional components, the `components` constructor argument may be used:\n",
+    "\n",
+    "```python\n",
+    "solver = LearningSolver(\n",
+    "    components=[\n",
+    "        comp1,\n",
+    "        comp2,\n",
+    "        comp3,\n",
+    "    ]\n",
+    ")\n",
+    "```\n",
+    "\n",
+    "In this example, three components `comp1`, `comp2` and `comp3` are provided. The strategies implemented by these components are applied sequentially when solving the problem. For example, `comp1` and `comp2` could fix a subset of decision variables, while `comp3` constructs a warm start for the remaining problem.\n",
+    "\n",
+    "### Training and solving new instances\n",
+    "\n",
+    "Once a solver is configured, its ML components need to be trained. This can be achieved by the `solver.fit` method, as illustrated below. The method accepts a list of HDF5 files and trains each individual component sequentially. Once the solver is trained, new instances can be solved using `solver.optimize`. The method returns a dictionary of statistics collected by each component, such as the number of variables fixed.\n",
+    "\n",
+    "```python\n",
+    "# Build instances\n",
+    "train_data = ...\n",
+    "test_data = ...\n",
+    "\n",
+    "# Collect training data\n",
+    "bc = BasicCollector()\n",
+    "bc.collect(train_data, build_model)\n",
+    "\n",
+    "# Build solver\n",
+    "solver = LearningSolver(...)\n",
+    "\n",
+    "# Train components\n",
+    "solver.fit(train_data)\n",
+    "\n",
+    "# Solve a new test instance\n",
+    "stats = solver.optimize(test_data[0], build_model)\n",
+    "\n",
+    "```\n",
+    "\n",
+    "### Complete example\n",
+    "\n",
+    "In the example below, we illustrate the usage of **LearningSolver** by building instances of the Traveling Salesman Problem, collecting training data, training the ML components, then solving a new instance."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "92b09b98",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]\n",
+      "Thread count: 16 physical cores, 32 logical processors, using up to 32 threads\n",
+      "\n",
+      "Optimize a model with 10 rows, 45 columns and 90 nonzeros\n",
+      "Model fingerprint: 0x6ddcd141\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 1e+00]\n",
+      "  Objective range  [4e+01, 1e+03]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [2e+00, 2e+00]\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 10 rows, 45 columns, 90 nonzeros\n",
+      "\n",
+      "Iteration    Objective       Primal Inf.    Dual Inf.      Time\n",
+      "       0    6.3600000e+02   1.700000e+01   0.000000e+00      0s\n",
+      "      15    2.7610000e+03   0.000000e+00   0.000000e+00      0s\n",
+      "\n",
+      "Solved in 15 iterations and 0.00 seconds (0.00 work units)\n",
+      "Optimal objective  2.761000000e+03\n",
+      "Set parameter LazyConstraints to value 1\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]\n",
+      "Thread count: 16 physical cores, 32 logical processors, using up to 32 threads\n",
+      "\n",
+      "Optimize a model with 10 rows, 45 columns and 90 nonzeros\n",
+      "Model fingerprint: 0x74ca3d0a\n",
+      "Variable types: 0 continuous, 45 integer (45 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 1e+00]\n",
+      "  Objective range  [4e+01, 1e+03]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [2e+00, 2e+00]\n",
+      "\n",
+      "User MIP start produced solution with objective 2796 (0.00s)\n",
+      "Loaded user MIP start with objective 2796\n",
+      "\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 10 rows, 45 columns, 90 nonzeros\n",
+      "Variable types: 0 continuous, 45 integer (45 binary)\n",
+      "\n",
+      "Root relaxation: objective 2.761000e+03, 14 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 2761.00000    0    - 2796.00000 2761.00000  1.25%     -    0s\n",
+      "     0     0     cutoff    0      2796.00000 2796.00000  0.00%     -    0s\n",
+      "\n",
+      "Cutting planes:\n",
+      "  Lazy constraints: 3\n",
+      "\n",
+      "Explored 1 nodes (16 simplex iterations) in 0.01 seconds (0.00 work units)\n",
+      "Thread count was 32 (of 32 available processors)\n",
+      "\n",
+      "Solution count 1: 2796 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 2.796000000000e+03, best bound 2.796000000000e+03, gap 0.0000%\n",
+      "\n",
+      "User-callback calls 110, time in user-callback 0.00 sec\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "{'WS: Count': 1, 'WS: Number of variables set': 41.0}"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import random\n",
+    "\n",
+    "import numpy as np\n",
+    "from scipy.stats import uniform, randint\n",
+    "from sklearn.linear_model import LogisticRegression\n",
+    "\n",
+    "from miplearn.classifiers.minprob import MinProbabilityClassifier\n",
+    "from miplearn.classifiers.singleclass import SingleClassFix\n",
+    "from miplearn.collectors.basic import BasicCollector\n",
+    "from miplearn.components.primal.actions import SetWarmStart\n",
+    "from miplearn.components.primal.indep import IndependentVarsPrimalComponent\n",
+    "from miplearn.extractors.AlvLouWeh2017 import AlvLouWeh2017Extractor\n",
+    "from miplearn.io import write_pkl_gz\n",
+    "from miplearn.problems.tsp import (\n",
+    "    TravelingSalesmanGenerator,\n",
+    "    build_tsp_model,\n",
+    ")\n",
+    "from miplearn.solvers.learning import LearningSolver\n",
+    "\n",
+    "# Set random seed to make example reproducible.\n",
+    "random.seed(42)\n",
+    "np.random.seed(42)\n",
+    "\n",
+    "# Generate a few instances of the traveling salesman problem.\n",
+    "data = TravelingSalesmanGenerator(\n",
+    "    n=randint(low=10, high=11),\n",
+    "    x=uniform(loc=0.0, scale=1000.0),\n",
+    "    y=uniform(loc=0.0, scale=1000.0),\n",
+    "    gamma=uniform(loc=0.90, scale=0.20),\n",
+    "    fix_cities=True,\n",
+    "    round=True,\n",
+    ").generate(50)\n",
+    "\n",
+    "# Save instance data to data/tsp/00000.pkl.gz, data/tsp/00001.pkl.gz, ...\n",
+    "all_data = write_pkl_gz(data, \"data/tsp\")\n",
+    "\n",
+    "# Split train/test data\n",
+    "train_data = all_data[:40]\n",
+    "test_data = all_data[40:]\n",
+    "\n",
+    "# Collect training data\n",
+    "bc = BasicCollector()\n",
+    "bc.collect(train_data, build_tsp_model, n_jobs=4)\n",
+    "\n",
+    "# Build learning solver\n",
+    "solver = LearningSolver(\n",
+    "    components=[\n",
+    "        IndependentVarsPrimalComponent(\n",
+    "            base_clf=SingleClassFix(\n",
+    "                MinProbabilityClassifier(\n",
+    "                    base_clf=LogisticRegression(),\n",
+    "                    thresholds=[0.95, 0.95],\n",
+    "                ),\n",
+    "            ),\n",
+    "            extractor=AlvLouWeh2017Extractor(),\n",
+    "            action=SetWarmStart(),\n",
+    "        )\n",
+    "    ]\n",
+    ")\n",
+    "\n",
+    "# Train ML models\n",
+    "solver.fit(train_data)\n",
+    "\n",
+    "# Solve a test instance\n",
+    "solver.optimize(test_data[0], build_tsp_model)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e27d2cbd-5341-461d-bbc1-8131aee8d949",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/index.md

@@ -1,35 +0,0 @@
-# MIPLearn
-
-**MIPLearn** is an extensible framework for solving discrete optimization problems using a combination of Mixed-Integer Linear Programming (MIP) and Machine Learning (ML). The framework uses ML methods to automatically identify patterns in previously solved instances of the problem, then uses these patterns to accelerate the performance of conventional state-of-the-art MIP solvers (such as CPLEX, Gurobi or XPRESS).
-
-Unlike pure ML methods, MIPLearn is not only able to find high-quality solutions to discrete optimization problems, but it can also prove the optimality and feasibility of these solutions.
-Unlike conventional MIP solvers, MIPLearn can take full advantage of very specific observations that happen to be true in a particular family of instances (such as the observation that a particular constraint is typically redundant, or that a particular variable typically assumes a certain value). 
-
-For certain classes of problems, this approach has been shown to provide significant performance benefits (see [benchmarks](benchmark.md) and [references](about.md)).
-
-## Features
-
-* **MIPLearn proposes a flexible problem specification format,** which allows users to describe their particular optimization problems to a Learning-Enhanced MIP solver, both from the MIP perspective and from the ML perspective, without making any assumptions on the problem being modeled, the mathematical formulation of the problem, or ML encoding. While the format is very flexible, some constraints are enforced to ensure that it is usable by an actual solver.
-
-* **MIPLearn provides a reference implementation of a *Learning-Enhanced Solver*,** which can use the above problem specification format to automatically predict, based on previously solved instances, a number of hints to accelerate MIP performance. Currently, the reference solver is able to predict: (i) partial solutions which are likely to work well as MIP starts; (ii) an initial set of lazy constraints to enforce; (iii) variable branching priorities to accelerate the exploration of the branch-and-bound tree; (iv) the optimal objective value based on the solution to the LP relaxation. The usage of the solver is very straightforward. The most suitable ML models are automatically selected, trained, cross-validated and applied to the problem with no user intervention.
-
-* **MIPLearn provides a set of benchmark problems and random instance generators,** covering applications from different domains, which can be used to quickly evaluate new learning-enhanced MIP techniques in a measurable and reproducible way.
-
-* **MIPLearn is customizable and extensible**. For MIP and ML researchers exploring new techniques to accelerate MIP performance based on historical data, each component of the reference solver can be individually replaced, extended or customized.
-
-## Site contents
-
-```{toctree}
----
-maxdepth: 2
----
-usage.md
-benchmark.md
-customization.md
-about.md
-```
-
-## Source Code
-
-* [https://github.com/ANL-CEEESA/MIPLearn](https://github.com/ANL-CEEESA/MIPLearn)
-

docs/index.rst

@@ -0,0 +1,67 @@
+MIPLearn
+========
+**MIPLearn** is an extensible framework for solving discrete optimization problems using a combination of Mixed-Integer Linear Programming (MIP) and Machine Learning (ML). MIPLearn uses ML methods to automatically identify patterns in previously solved instances of the problem, then uses these patterns to accelerate the performance of conventional state-of-the-art MIP solvers such as CPLEX, Gurobi or XPRESS.
+
+Unlike pure ML methods, MIPLearn is not only able to find high-quality solutions to discrete optimization problems, but it can also prove the optimality and feasibility of these solutions. Unlike conventional MIP solvers, MIPLearn can take full advantage of very specific observations that happen to be true in a particular family of instances (such as the observation that a particular constraint is typically redundant, or that a particular variable typically assumes a certain value). For certain classes of problems, this approach may provide significant performance benefits.
+
+
+Contents
+--------
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Tutorials
+   :numbered: 2
+
+   tutorials/getting-started-pyomo
+   tutorials/getting-started-gurobipy
+   tutorials/getting-started-jump
+
+.. toctree::
+   :maxdepth: 2
+   :caption: User Guide
+   :numbered: 2
+
+   guide/problems
+   guide/collectors
+   guide/features
+   guide/primal
+   guide/solvers
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Python API Reference
+   :numbered: 2
+
+   api/problems
+   api/collectors
+   api/components
+   api/solvers
+   api/helpers
+
+
+Authors
+-------
+
+- **Alinson S. Xavier** (Argonne National Laboratory)
+- **Feng Qiu** (Argonne National Laboratory)
+- **Xiaoyi Gu** (Georgia Institute of Technology)
+- **Berkay Becu** (Georgia Institute of Technology)
+- **Santanu S. Dey**  (Georgia Institute of Technology)
+
+
+Acknowledgments
+---------------
+* Based upon work supported by **Laboratory Directed Research and Development** (LDRD) funding from Argonne National Laboratory, provided by the Director, Office of Science, of the U.S. Department of Energy.
+* Based upon work supported by the **U.S. Department of Energy Advanced Grid Modeling Program**.
+
+Citing MIPLearn
+---------------
+
+If you use MIPLearn in your research (either the solver or the included problem generators), we kindly request that you cite the package as follows:
+
+* **Alinson S. Xavier, Feng Qiu, Xiaoyi Gu, Berkay Becu, Santanu S. Dey.** *MIPLearn: An Extensible Framework for Learning-Enhanced Optimization (Version 0.3)*. Zenodo (2023). DOI: https://doi.org/10.5281/zenodo.4287567
+
+If you use MIPLearn in the field of power systems optimization, we kindly request that you cite the reference below, in which the main techniques implemented in MIPLearn were first developed:
+
+* **Alinson S. Xavier, Feng Qiu, Shabbir Ahmed.** *Learning to Solve Large-Scale Unit Commitment Problems.* INFORMS Journal on Computing (2020). DOI: https://doi.org/10.1287/ijoc.2020.0976

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=.
+set BUILDDIR=_build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/tutorials/Manifest.toml

@@ -0,0 +1,637 @@
+# This file is machine-generated - editing it directly is not advised
+
+julia_version = "1.9.0"
+manifest_format = "2.0"
+project_hash = "acf9261f767ae18f2b4613fd5590ea6a33f31e10"
+
+[[deps.ArgTools]]
+uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
+version = "1.1.1"
+
+[[deps.Artifacts]]
+uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"
+
+[[deps.Base64]]
+uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
+
+[[deps.BenchmarkTools]]
+deps = ["JSON", "Logging", "Printf", "Profile", "Statistics", "UUIDs"]
+git-tree-sha1 = "d9a9701b899b30332bbcb3e1679c41cce81fb0e8"
+uuid = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
+version = "1.3.2"
+
+[[deps.Bzip2_jll]]
+deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
+git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
+uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
+version = "1.0.8+0"
+
+[[deps.Calculus]]
+deps = ["LinearAlgebra"]
+git-tree-sha1 = "f641eb0a4f00c343bbc32346e1217b86f3ce9dad"
+uuid = "49dc2e85-a5d0-5ad3-a950-438e2897f1b9"
+version = "0.5.1"
+
+[[deps.CodecBzip2]]
+deps = ["Bzip2_jll", "Libdl", "TranscodingStreams"]
+git-tree-sha1 = "2e62a725210ce3c3c2e1a3080190e7ca491f18d7"
+uuid = "523fee87-0ab8-5b00-afb7-3ecf72e48cfd"
+version = "0.7.2"
+
+[[deps.CodecZlib]]
+deps = ["TranscodingStreams", "Zlib_jll"]
+git-tree-sha1 = "9c209fb7536406834aa938fb149964b985de6c83"
+uuid = "944b1d66-785c-5afd-91f1-9de20f533193"
+version = "0.7.1"
+
+[[deps.CommonSubexpressions]]
+deps = ["MacroTools", "Test"]
+git-tree-sha1 = "7b8a93dba8af7e3b42fecabf646260105ac373f7"
+uuid = "bbf7d656-a473-5ed7-a52c-81e309532950"
+version = "0.3.0"
+
+[[deps.Compat]]
+deps = ["UUIDs"]
+git-tree-sha1 = "7a60c856b9fa189eb34f5f8a6f6b5529b7942957"
+uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
+version = "4.6.1"
+weakdeps = ["Dates", "LinearAlgebra"]
+
+    [deps.Compat.extensions]
+    CompatLinearAlgebraExt = "LinearAlgebra"
+
+[[deps.CompilerSupportLibraries_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"
+version = "1.0.2+0"
+
+[[deps.Conda]]
+deps = ["Downloads", "JSON", "VersionParsing"]
+git-tree-sha1 = "e32a90da027ca45d84678b826fffd3110bb3fc90"
+uuid = "8f4d0f93-b110-5947-807f-2305c1781a2d"
+version = "1.8.0"
+
+[[deps.DataAPI]]
+git-tree-sha1 = "8da84edb865b0b5b0100c0666a9bc9a0b71c553c"
+uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
+version = "1.15.0"
+
+[[deps.DataStructures]]
+deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
+git-tree-sha1 = "d1fff3a548102f48987a52a2e0d114fa97d730f0"
+uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
+version = "0.18.13"
+
+[[deps.Dates]]
+deps = ["Printf"]
+uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"
+
+[[deps.DiffResults]]
+deps = ["StaticArraysCore"]
+git-tree-sha1 = "782dd5f4561f5d267313f23853baaaa4c52ea621"
+uuid = "163ba53b-c6d8-5494-b064-1a9d43ac40c5"
+version = "1.1.0"
+
+[[deps.DiffRules]]
+deps = ["IrrationalConstants", "LogExpFunctions", "NaNMath", "Random", "SpecialFunctions"]
+git-tree-sha1 = "23163d55f885173722d1e4cf0f6110cdbaf7e272"
+uuid = "b552c78f-8df3-52c6-915a-8e097449b14b"
+version = "1.15.1"
+
+[[deps.Distributions]]
+deps = ["FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SparseArrays", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns", "Test"]
+git-tree-sha1 = "c72970914c8a21b36bbc244e9df0ed1834a0360b"
+uuid = "31c24e10-a181-5473-b8eb-7969acd0382f"
+version = "0.25.95"
+
+    [deps.Distributions.extensions]
+    DistributionsChainRulesCoreExt = "ChainRulesCore"
+    DistributionsDensityInterfaceExt = "DensityInterface"
+
+    [deps.Distributions.weakdeps]
+    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
+    DensityInterface = "b429d917-457f-4dbc-8f4c-0cc954292b1d"
+
+[[deps.DocStringExtensions]]
+deps = ["LibGit2"]
+git-tree-sha1 = "2fb1e02f2b635d0845df5d7c167fec4dd739b00d"
+uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
+version = "0.9.3"
+
+[[deps.Downloads]]
+deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"]
+uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
+version = "1.6.0"
+
+[[deps.DualNumbers]]
+deps = ["Calculus", "NaNMath", "SpecialFunctions"]
+git-tree-sha1 = "5837a837389fccf076445fce071c8ddaea35a566"
+uuid = "fa6b7ba4-c1ee-5f82-b5fc-ecf0adba8f74"
+version = "0.6.8"
+
+[[deps.ExprTools]]
+git-tree-sha1 = "c1d06d129da9f55715c6c212866f5b1bddc5fa00"
+uuid = "e2ba6199-217a-4e67-a87a-7c52f15ade04"
+version = "0.1.9"
+
+[[deps.FileIO]]
+deps = ["Pkg", "Requires", "UUIDs"]
+git-tree-sha1 = "299dc33549f68299137e51e6d49a13b5b1da9673"
+uuid = "5789e2e9-d7fb-5bc7-8068-2c6fae9b9549"
+version = "1.16.1"
+
+[[deps.FileWatching]]
+uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee"
+
+[[deps.FillArrays]]
+deps = ["LinearAlgebra", "Random", "SparseArrays", "Statistics"]
+git-tree-sha1 = "589d3d3bff204bdd80ecc53293896b4f39175723"
+uuid = "1a297f60-69ca-5386-bcde-b61e274b549b"
+version = "1.1.1"
+
+[[deps.ForwardDiff]]
+deps = ["CommonSubexpressions", "DiffResults", "DiffRules", "LinearAlgebra", "LogExpFunctions", "NaNMath", "Preferences", "Printf", "Random", "SpecialFunctions"]
+git-tree-sha1 = "00e252f4d706b3d55a8863432e742bf5717b498d"
+uuid = "f6369f11-7733-5829-9624-2563aa707210"
+version = "0.10.35"
+
+    [deps.ForwardDiff.extensions]
+    ForwardDiffStaticArraysExt = "StaticArrays"
+
+    [deps.ForwardDiff.weakdeps]
+    StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
+
+[[deps.Gurobi]]
+deps = ["LazyArtifacts", "Libdl", "MathOptInterface"]
+git-tree-sha1 = "22439b1c2bacb7d50ed0df7dbd10211e0b4cd379"
+uuid = "2e9cd046-0924-5485-92f1-d5272153d98b"
+version = "1.0.1"
+
+[[deps.HDF5]]
+deps = ["Compat", "HDF5_jll", "Libdl", "Mmap", "Random", "Requires", "UUIDs"]
+git-tree-sha1 = "c73fdc3d9da7700691848b78c61841274076932a"
+uuid = "f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f"
+version = "0.16.15"
+
+[[deps.HDF5_jll]]
+deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "LLVMOpenMP_jll", "LazyArtifacts", "LibCURL_jll", "Libdl", "MPICH_jll", "MPIPreferences", "MPItrampoline_jll", "MicrosoftMPI_jll", "OpenMPI_jll", "OpenSSL_jll", "TOML", "Zlib_jll", "libaec_jll"]
+git-tree-sha1 = "3b20c3ce9c14aedd0adca2bc8c882927844bd53d"
+uuid = "0234f1f7-429e-5d53-9886-15a909be8d59"
+version = "1.14.0+0"
+
+[[deps.HiGHS]]
+deps = ["HiGHS_jll", "MathOptInterface", "PrecompileTools", "SparseArrays"]
+git-tree-sha1 = "bbd4ab443dfac4c9d5c5b40dd45f598dfad2e26a"
+uuid = "87dc4568-4c63-4d18-b0c0-bb2238e4078b"
+version = "1.5.2"
+
+[[deps.HiGHS_jll]]
+deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl"]
+git-tree-sha1 = "216e7198aeb256e7c7921ef2937d7e1e589ba6fd"
+uuid = "8fd58aa0-07eb-5a78-9b36-339c94fd15ea"
+version = "1.5.3+0"
+
+[[deps.HypergeometricFunctions]]
+deps = ["DualNumbers", "LinearAlgebra", "OpenLibm_jll", "SpecialFunctions"]
+git-tree-sha1 = "84204eae2dd237500835990bcade263e27674a93"
+uuid = "34004b35-14d8-5ef3-9330-4cdb6864b03a"
+version = "0.3.16"
+
+[[deps.InteractiveUtils]]
+deps = ["Markdown"]
+uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
+
+[[deps.IrrationalConstants]]
+git-tree-sha1 = "630b497eafcc20001bba38a4651b327dcfc491d2"
+uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
+version = "0.2.2"
+
+[[deps.JLD2]]
+deps = ["FileIO", "MacroTools", "Mmap", "OrderedCollections", "Pkg", "Printf", "Reexport", "Requires", "TranscodingStreams", "UUIDs"]
+git-tree-sha1 = "42c17b18ced77ff0be65957a591d34f4ed57c631"
+uuid = "033835bb-8acc-5ee8-8aae-3f567f8a3819"
+version = "0.4.31"
+
+[[deps.JLLWrappers]]
+deps = ["Preferences"]
+git-tree-sha1 = "abc9885a7ca2052a736a600f7fa66209f96506e1"
+uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
+version = "1.4.1"
+
+[[deps.JSON]]
+deps = ["Dates", "Mmap", "Parsers", "Unicode"]
+git-tree-sha1 = "31e996f0a15c7b280ba9f76636b3ff9e2ae58c9a"
+uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
+version = "0.21.4"
+
+[[deps.JuMP]]
+deps = ["LinearAlgebra", "MathOptInterface", "MutableArithmetics", "OrderedCollections", "Printf", "SnoopPrecompile", "SparseArrays"]
+git-tree-sha1 = "3e4a73edf2ca1bfe97f1fc86eb4364f95ef0fccd"
+uuid = "4076af6c-e467-56ae-b986-b466b2749572"
+version = "1.11.1"
+
+[[deps.KLU]]
+deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse_jll"]
+git-tree-sha1 = "764164ed65c30738750965d55652db9c94c59bfe"
+uuid = "ef3ab10e-7fda-4108-b977-705223b18434"
+version = "0.4.0"
+
+[[deps.LLVMOpenMP_jll]]
+deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
+git-tree-sha1 = "f689897ccbe049adb19a065c495e75f372ecd42b"
+uuid = "1d63c593-3942-5779-bab2-d838dc0a180e"
+version = "15.0.4+0"
+
+[[deps.LazyArtifacts]]
+deps = ["Artifacts", "Pkg"]
+uuid = "4af54fe1-eca0-43a8-85a7-787d91b784e3"
+
+[[deps.LibCURL]]
+deps = ["LibCURL_jll", "MozillaCACerts_jll"]
+uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
+version = "0.6.3"
+
+[[deps.LibCURL_jll]]
+deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
+uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
+version = "7.84.0+0"
+
+[[deps.LibGit2]]
+deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
+uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"
+
+[[deps.LibSSH2_jll]]
+deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
+uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
+version = "1.10.2+0"
+
+[[deps.Libdl]]
+uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
+
+[[deps.LinearAlgebra]]
+deps = ["Libdl", "OpenBLAS_jll", "libblastrampoline_jll"]
+uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
+
+[[deps.LogExpFunctions]]
+deps = ["DocStringExtensions", "IrrationalConstants", "LinearAlgebra"]
+git-tree-sha1 = "c3ce8e7420b3a6e071e0fe4745f5d4300e37b13f"
+uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
+version = "0.3.24"
+
+    [deps.LogExpFunctions.extensions]
+    LogExpFunctionsChainRulesCoreExt = "ChainRulesCore"
+    LogExpFunctionsChangesOfVariablesExt = "ChangesOfVariables"
+    LogExpFunctionsInverseFunctionsExt = "InverseFunctions"
+
+    [deps.LogExpFunctions.weakdeps]
+    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
+    ChangesOfVariables = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
+    InverseFunctions = "3587e190-3f89-42d0-90ee-14403ec27112"
+
+[[deps.Logging]]
+uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"
+
+[[deps.MIPLearn]]
+deps = ["Conda", "DataStructures", "HDF5", "HiGHS", "JLD2", "JuMP", "KLU", "LinearAlgebra", "MathOptInterface", "OrderedCollections", "Printf", "PyCall", "Random", "Requires", "SparseArrays", "Statistics", "TimerOutputs"]
+path = "/home/axavier/Packages/MIPLearn.jl/dev/"
+uuid = "2b1277c3-b477-4c49-a15e-7ba350325c68"
+version = "0.3.0"
+
+[[deps.MPICH_jll]]
+deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "MPIPreferences", "TOML"]
+git-tree-sha1 = "d790fbd913f85e8865c55bf4725aff197c5155c8"
+uuid = "7cb0a576-ebde-5e09-9194-50597f1243b4"
+version = "4.1.1+1"
+
+[[deps.MPIPreferences]]
+deps = ["Libdl", "Preferences"]
+git-tree-sha1 = "d86a788b336e8ae96429c0c42740ccd60ac0dfcc"
+uuid = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267"
+version = "0.1.8"
+
+[[deps.MPItrampoline_jll]]
+deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "MPIPreferences", "TOML"]
+git-tree-sha1 = "b3dcf8e1c610a10458df3c62038c8cc3a4d6291d"
+uuid = "f1f71cc9-e9ae-5b93-9b94-4fe0e1ad3748"
+version = "5.3.0+0"
+
+[[deps.MacroTools]]
+deps = ["Markdown", "Random"]
+git-tree-sha1 = "42324d08725e200c23d4dfb549e0d5d89dede2d2"
+uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
+version = "0.5.10"
+
+[[deps.Markdown]]
+deps = ["Base64"]
+uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"
+
+[[deps.MathOptInterface]]
+deps = ["BenchmarkTools", "CodecBzip2", "CodecZlib", "DataStructures", "ForwardDiff", "JSON", "LinearAlgebra", "MutableArithmetics", "NaNMath", "OrderedCollections", "PrecompileTools", "Printf", "SparseArrays", "SpecialFunctions", "Test", "Unicode"]
+git-tree-sha1 = "19a3636968e802918f8891d729c74bd64dff6d00"
+uuid = "b8f27783-ece8-5eb3-8dc8-9495eed66fee"
+version = "1.17.1"
+
+[[deps.MbedTLS_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"
+version = "2.28.2+0"
+
+[[deps.MicrosoftMPI_jll]]
+deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
+git-tree-sha1 = "a8027af3d1743b3bfae34e54872359fdebb31422"
+uuid = "9237b28f-5490-5468-be7b-bb81f5f5e6cf"
+version = "10.1.3+4"
+
+[[deps.Missings]]
+deps = ["DataAPI"]
+git-tree-sha1 = "f66bdc5de519e8f8ae43bdc598782d35a25b1272"
+uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
+version = "1.1.0"
+
+[[deps.Mmap]]
+uuid = "a63ad114-7e13-5084-954f-fe012c677804"
+
+[[deps.MozillaCACerts_jll]]
+uuid = "14a3606d-f60d-562e-9121-12d972cd8159"
+version = "2022.10.11"
+
+[[deps.MutableArithmetics]]
+deps = ["LinearAlgebra", "SparseArrays", "Test"]
+git-tree-sha1 = "964cb1a7069723727025ae295408747a0b36a854"
+uuid = "d8a4904e-b15c-11e9-3269-09a3773c0cb0"
+version = "1.3.0"
+
+[[deps.NaNMath]]
+deps = ["OpenLibm_jll"]
+git-tree-sha1 = "0877504529a3e5c3343c6f8b4c0381e57e4387e4"
+uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
+version = "1.0.2"
+
+[[deps.NetworkOptions]]
+uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
+version = "1.2.0"
+
+[[deps.OpenBLAS_jll]]
+deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
+uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"
+version = "0.3.21+4"
+
+[[deps.OpenLibm_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "05823500-19ac-5b8b-9628-191a04bc5112"
+version = "0.8.1+0"
+
+[[deps.OpenMPI_jll]]
+deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "MPIPreferences", "TOML"]
+git-tree-sha1 = "f3080f4212a8ba2ceb10a34b938601b862094314"
+uuid = "fe0851c0-eecd-5654-98d4-656369965a5c"
+version = "4.1.5+0"
+
+[[deps.OpenSSL_jll]]
+deps = ["Artifacts", "JLLWrappers", "Libdl"]
+git-tree-sha1 = "cae3153c7f6cf3f069a853883fd1919a6e5bab5b"
+uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
+version = "3.0.9+0"
+
+[[deps.OpenSpecFun_jll]]
+deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
+git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1"
+uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
+version = "0.5.5+0"
+
+[[deps.OrderedCollections]]
+git-tree-sha1 = "d321bf2de576bf25ec4d3e4360faca399afca282"
+uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
+version = "1.6.0"
+
+[[deps.PDMats]]
+deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"]
+git-tree-sha1 = "67eae2738d63117a196f497d7db789821bce61d1"
+uuid = "90014a1f-27ba-587c-ab20-58faa44d9150"
+version = "0.11.17"
+
+[[deps.Parsers]]
+deps = ["Dates", "PrecompileTools", "UUIDs"]
+git-tree-sha1 = "a5aef8d4a6e8d81f171b2bd4be5265b01384c74c"
+uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
+version = "2.5.10"
+
+[[deps.Pkg]]
+deps = ["Artifacts", "Dates", "Downloads", "FileWatching", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
+uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
+version = "1.9.0"
+
+[[deps.PrecompileTools]]
+deps = ["Preferences"]
+git-tree-sha1 = "9673d39decc5feece56ef3940e5dafba15ba0f81"
+uuid = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
+version = "1.1.2"
+
+[[deps.Preferences]]
+deps = ["TOML"]
+git-tree-sha1 = "7eb1686b4f04b82f96ed7a4ea5890a4f0c7a09f1"
+uuid = "21216c6a-2e73-6563-6e65-726566657250"
+version = "1.4.0"
+
+[[deps.Printf]]
+deps = ["Unicode"]
+uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"
+
+[[deps.Profile]]
+deps = ["Printf"]
+uuid = "9abbd945-dff8-562f-b5e8-e1ebf5ef1b79"
+
+[[deps.PyCall]]
+deps = ["Conda", "Dates", "Libdl", "LinearAlgebra", "MacroTools", "Serialization", "VersionParsing"]
+git-tree-sha1 = "62f417f6ad727987c755549e9cd88c46578da562"
+uuid = "438e738f-606a-5dbb-bf0a-cddfbfd45ab0"
+version = "1.95.1"
+
+[[deps.QuadGK]]
+deps = ["DataStructures", "LinearAlgebra"]
+git-tree-sha1 = "6ec7ac8412e83d57e313393220879ede1740f9ee"
+uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc"
+version = "2.8.2"
+
+[[deps.REPL]]
+deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
+uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
+
+[[deps.Random]]
+deps = ["SHA", "Serialization"]
+uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
+
+[[deps.Reexport]]
+git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
+uuid = "189a3867-3050-52da-a836-e630ba90ab69"
+version = "1.2.2"
+
+[[deps.Requires]]
+deps = ["UUIDs"]
+git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
+uuid = "ae029012-a4dd-5104-9daa-d747884805df"
+version = "1.3.0"
+
+[[deps.Rmath]]
+deps = ["Random", "Rmath_jll"]
+git-tree-sha1 = "f65dcb5fa46aee0cf9ed6274ccbd597adc49aa7b"
+uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa"
+version = "0.7.1"
+
+[[deps.Rmath_jll]]
+deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
+git-tree-sha1 = "6ed52fdd3382cf21947b15e8870ac0ddbff736da"
+uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f"
+version = "0.4.0+0"
+
+[[deps.SHA]]
+uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
+version = "0.7.0"
+
+[[deps.Serialization]]
+uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
+
+[[deps.SnoopPrecompile]]
+deps = ["Preferences"]
+git-tree-sha1 = "e760a70afdcd461cf01a575947738d359234665c"
+uuid = "66db9d55-30c0-4569-8b51-7e840670fc0c"
+version = "1.0.3"
+
+[[deps.Sockets]]
+uuid = "6462fe0b-24de-5631-8697-dd941f90decc"
+
+[[deps.SortingAlgorithms]]
+deps = ["DataStructures"]
+git-tree-sha1 = "a4ada03f999bd01b3a25dcaa30b2d929fe537e00"
+uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
+version = "1.1.0"
+
+[[deps.SparseArrays]]
+deps = ["Libdl", "LinearAlgebra", "Random", "Serialization", "SuiteSparse_jll"]
+uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
+
+[[deps.SpecialFunctions]]
+deps = ["IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
+git-tree-sha1 = "ef28127915f4229c971eb43f3fc075dd3fe91880"
+uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
+version = "2.2.0"
+
+    [deps.SpecialFunctions.extensions]
+    SpecialFunctionsChainRulesCoreExt = "ChainRulesCore"
+
+    [deps.SpecialFunctions.weakdeps]
+    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
+
+[[deps.StaticArraysCore]]
+git-tree-sha1 = "6b7ba252635a5eff6a0b0664a41ee140a1c9e72a"
+uuid = "1e83bf80-4336-4d27-bf5d-d5a4f845583c"
+version = "1.4.0"
+
+[[deps.Statistics]]
+deps = ["LinearAlgebra", "SparseArrays"]
+uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
+version = "1.9.0"
+
+[[deps.StatsAPI]]
+deps = ["LinearAlgebra"]
+git-tree-sha1 = "45a7769a04a3cf80da1c1c7c60caf932e6f4c9f7"
+uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
+version = "1.6.0"
+
+[[deps.StatsBase]]
+deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
+git-tree-sha1 = "75ebe04c5bed70b91614d684259b661c9e6274a4"
+uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
+version = "0.34.0"
+
+[[deps.StatsFuns]]
+deps = ["HypergeometricFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"]
+git-tree-sha1 = "f625d686d5a88bcd2b15cd81f18f98186fdc0c9a"
+uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c"
+version = "1.3.0"
+
+    [deps.StatsFuns.extensions]
+    StatsFunsChainRulesCoreExt = "ChainRulesCore"
+    StatsFunsInverseFunctionsExt = "InverseFunctions"
+
+    [deps.StatsFuns.weakdeps]
+    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
+    InverseFunctions = "3587e190-3f89-42d0-90ee-14403ec27112"
+
+[[deps.SuiteSparse]]
+deps = ["Libdl", "LinearAlgebra", "Serialization", "SparseArrays"]
+uuid = "4607b0f0-06f3-5cda-b6b1-a6196a1729e9"
+
+[[deps.SuiteSparse_jll]]
+deps = ["Artifacts", "Libdl", "Pkg", "libblastrampoline_jll"]
+uuid = "bea87d4a-7f5b-5778-9afe-8cc45184846c"
+version = "5.10.1+6"
+
+[[deps.Suppressor]]
+git-tree-sha1 = "c6ed566db2fe3931292865b966d6d140b7ef32a9"
+uuid = "fd094767-a336-5f1f-9728-57cf17d0bbfb"
+version = "0.2.1"
+
+[[deps.TOML]]
+deps = ["Dates"]
+uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
+version = "1.0.3"
+
+[[deps.Tar]]
+deps = ["ArgTools", "SHA"]
+uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
+version = "1.10.0"
+
+[[deps.Test]]
+deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
+uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
+
+[[deps.TimerOutputs]]
+deps = ["ExprTools", "Printf"]
+git-tree-sha1 = "f548a9e9c490030e545f72074a41edfd0e5bcdd7"
+uuid = "a759f4b9-e2f1-59dc-863e-4aeb61b1ea8f"
+version = "0.5.23"
+
+[[deps.TranscodingStreams]]
+deps = ["Random", "Test"]
+git-tree-sha1 = "9a6ae7ed916312b41236fcef7e0af564ef934769"
+uuid = "3bb67fe8-82b1-5028-8e26-92a6c54297fa"
+version = "0.9.13"
+
+[[deps.UUIDs]]
+deps = ["Random", "SHA"]
+uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
+
+[[deps.Unicode]]
+uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
+
+[[deps.VersionParsing]]
+git-tree-sha1 = "58d6e80b4ee071f5efd07fda82cb9fbe17200868"
+uuid = "81def892-9a0e-5fdd-b105-ffc91e053289"
+version = "1.3.0"
+
+[[deps.Zlib_jll]]
+deps = ["Libdl"]
+uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
+version = "1.2.13+0"
+
+[[deps.libaec_jll]]
+deps = ["Artifacts", "JLLWrappers", "Libdl"]
+git-tree-sha1 = "eddd19a8dea6b139ea97bdc8a0e2667d4b661720"
+uuid = "477f73a3-ac25-53e9-8cc3-50b2fa2566f0"
+version = "1.0.6+1"
+
+[[deps.libblastrampoline_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
+version = "5.7.0+0"
+
+[[deps.nghttp2_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
+version = "1.48.0+0"
+
+[[deps.p7zip_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
+version = "17.4.0+0"

docs/tutorials/Project.toml

@@ -0,0 +1,7 @@
+[deps]
+Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
+Gurobi = "2e9cd046-0924-5485-92f1-d5272153d98b"
+JuMP = "4076af6c-e467-56ae-b986-b466b2749572"
+MIPLearn = "2b1277c3-b477-4c49-a15e-7ba350325c68"
+PyCall = "438e738f-606a-5dbb-bf0a-cddfbfd45ab0"
+Suppressor = "fd094767-a336-5f1f-9728-57cf17d0bbfb"

docs/tutorials/getting-started-gurobipy.ipynb

@@ -0,0 +1,849 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "6b8983b1",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "# Getting started (Gurobipy)\n",
+    "\n",
+    "## Introduction\n",
+    "\n",
+    "**MIPLearn** is an open source framework that uses machine learning (ML) to accelerate the performance of mixed-integer programming solvers (e.g. Gurobi, CPLEX, XPRESS). In this tutorial, we will:\n",
+    "\n",
+    "1. Install the Python/Gurobipy version of MIPLearn\n",
+    "2. Model a simple optimization problem using Gurobipy\n",
+    "3. Generate training data and train the ML models\n",
+    "4. Use the ML models together Gurobi to solve new instances\n",
+    "\n",
+    "<div class=\"alert alert-info\">\n",
+    "Note\n",
+    "    \n",
+    "The Python/Gurobipy version of MIPLearn is only compatible with the Gurobi Optimizer. For broader solver compatibility, see the Python/Pyomo and Julia/JuMP versions of the package.\n",
+    "</div>\n",
+    "\n",
+    "<div class=\"alert alert-warning\">\n",
+    "Warning\n",
+    "    \n",
+    "MIPLearn is still in early development stage. If run into any bugs or issues, please submit a bug report in our GitHub repository. Comments, suggestions and pull requests are also very welcome!\n",
+    "    \n",
+    "</div>\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "02f0a927",
+   "metadata": {},
+   "source": [
+    "## Installation\n",
+    "\n",
+    "MIPLearn is available in two versions:\n",
+    "\n",
+    "- Python version, compatible with the Pyomo and Gurobipy modeling languages,\n",
+    "- Julia version, compatible with the JuMP modeling language.\n",
+    "\n",
+    "In this tutorial, we will demonstrate how to use and install the Python/Gurobipy version of the package. The first step is to install Python 3.8+ in your computer. See the [official Python website for more instructions](https://www.python.org/downloads/). After Python is installed, we proceed to install MIPLearn using `pip`:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "cd8a69c1",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:18:02.381829278Z",
+     "start_time": "2023-06-06T20:18:02.381532300Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "# !pip install MIPLearn==0.3.0"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e8274543",
+   "metadata": {},
+   "source": [
+    "In addition to MIPLearn itself, we will also install Gurobi 10.0, a state-of-the-art commercial MILP solver. This step also install a demo license for Gurobi, which should able to solve the small optimization problems in this tutorial. A license is required for solving larger-scale problems."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "dcc8756c",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:18:15.537811992Z",
+     "start_time": "2023-06-06T20:18:13.449177860Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: gurobipy<10.1,>=10 in /home/axavier/Software/anaconda3/envs/miplearn/lib/python3.8/site-packages (10.0.1)\n"
+     ]
+    }
+   ],
+   "source": [
+    "!pip install 'gurobipy>=10,<10.1'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a14e4550",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "    \n",
+    "Note\n",
+    "    \n",
+    "In the code above, we install specific version of all packages to ensure that this tutorial keeps running in the future, even when newer (and possibly incompatible) versions of the packages are released. This is usually a recommended practice for all Python projects.\n",
+    "    \n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "16b86823",
+   "metadata": {},
+   "source": [
+    "## Modeling a simple optimization problem\n",
+    "\n",
+    "To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the **unit commitment problem,** a practical optimization problem solved daily by electric grid operators around the world. \n",
+    "\n",
+    "Suppose that a utility company needs to decide which electrical generators should be online at each hour of the day, as well as how much power should each generator produce. More specifically, assume that the company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs the company $c^\\text{fix}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing and costs nothing. The total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts).\n",
+    "\n",
+    "This simple problem can be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem is then given by:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f12c3702",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align}\n",
+    "\\text{minimize } \\quad & \\sum_{i=1}^n \\left( c^\\text{fix}_i x_i + c^\\text{var}_i y_i \\right) \\\\\n",
+    "\\text{subject to } \\quad & y_i \\leq p^\\text{max}_i x_i & i=1,\\ldots,n \\\\\n",
+    "& y_i \\geq p^\\text{min}_i x_i & i=1,\\ldots,n \\\\\n",
+    "& \\sum_{i=1}^n y_i = d \\\\\n",
+    "& x_i \\in \\{0,1\\} & i=1,\\ldots,n \\\\\n",
+    "& y_i \\geq 0 & i=1,\\ldots,n\n",
+    "\\end{align}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "be3989ed",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "\n",
+    "Note\n",
+    "\n",
+    "We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem.\n",
+    "\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a5fd33f6",
+   "metadata": {},
+   "source": [
+    "Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Python and Pyomo. We start by defining a data class `UnitCommitmentData`, which holds all the input data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "22a67170-10b4-43d3-8708-014d91141e73",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:18:25.442346786Z",
+     "start_time": "2023-06-06T20:18:25.329017476Z"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from dataclasses import dataclass\n",
+    "from typing import List\n",
+    "\n",
+    "import numpy as np\n",
+    "\n",
+    "\n",
+    "@dataclass\n",
+    "class UnitCommitmentData:\n",
+    "    demand: float\n",
+    "    pmin: List[float]\n",
+    "    pmax: List[float]\n",
+    "    cfix: List[float]\n",
+    "    cvar: List[float]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "29f55efa-0751-465a-9b0a-a821d46a3d40",
+   "metadata": {},
+   "source": [
+    "Next, we write a `build_uc_model` function, which converts the input data into a concrete Pyomo model. The function accepts `UnitCommitmentData`, the data structure we previously defined, or the path to a compressed pickle file containing this data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "2f67032f-0d74-4317-b45c-19da0ec859e9",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:48:05.953902842Z",
+     "start_time": "2023-06-06T20:48:05.909747925Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import gurobipy as gp\n",
+    "from gurobipy import GRB, quicksum\n",
+    "from typing import Union\n",
+    "from miplearn.io import read_pkl_gz\n",
+    "from miplearn.solvers.gurobi import GurobiModel\n",
+    "\n",
+    "def build_uc_model(data: Union[str, UnitCommitmentData]) -> GurobiModel:\n",
+    "    if isinstance(data, str):\n",
+    "        data = read_pkl_gz(data)\n",
+    "\n",
+    "    model = gp.Model()\n",
+    "    n = len(data.pmin)\n",
+    "    x = model._x = model.addVars(n, vtype=GRB.BINARY, name=\"x\")\n",
+    "    y = model._y = model.addVars(n, name=\"y\")\n",
+    "    model.setObjective(\n",
+    "        quicksum(\n",
+    "            data.cfix[i] * x[i] + data.cvar[i] * y[i] for i in range(n)\n",
+    "        )\n",
+    "    )\n",
+    "    model.addConstrs(y[i] <= data.pmax[i] * x[i] for i in range(n))\n",
+    "    model.addConstrs(y[i] >= data.pmin[i] * x[i] for i in range(n))\n",
+    "    model.addConstr(quicksum(y[i] for i in range(n)) == data.demand)\n",
+    "    return GurobiModel(model)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c22714a3",
+   "metadata": {},
+   "source": [
+    "At this point, we can already use Pyomo and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "2a896f47",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:49:14.266758244Z",
+     "start_time": "2023-06-06T20:49:14.223514806Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Restricted license - for non-production use only - expires 2024-10-28\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 7 rows, 6 columns and 15 nonzeros\n",
+      "Model fingerprint: 0x58dfdd53\n",
+      "Variable types: 3 continuous, 3 integer (3 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 7e+01]\n",
+      "  Objective range  [2e+00, 7e+02]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [1e+02, 1e+02]\n",
+      "Presolve removed 2 rows and 1 columns\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 5 rows, 5 columns, 13 nonzeros\n",
+      "Variable types: 0 continuous, 5 integer (3 binary)\n",
+      "Found heuristic solution: objective 1400.0000000\n",
+      "\n",
+      "Root relaxation: objective 1.035000e+03, 3 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 1035.00000    0    1 1400.00000 1035.00000  26.1%     -    0s\n",
+      "     0     0 1105.71429    0    1 1400.00000 1105.71429  21.0%     -    0s\n",
+      "*    0     0               0    1320.0000000 1320.00000  0.00%     -    0s\n",
+      "\n",
+      "Explored 1 nodes (5 simplex iterations) in 0.01 seconds (0.00 work units)\n",
+      "Thread count was 12 (of 12 available processors)\n",
+      "\n",
+      "Solution count 2: 1320 1400 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 1.320000000000e+03, best bound 1.320000000000e+03, gap 0.0000%\n",
+      "obj = 1320.0\n",
+      "x = [-0.0, 1.0, 1.0]\n",
+      "y = [0.0, 60.0, 40.0]\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = build_uc_model(\n",
+    "    UnitCommitmentData(\n",
+    "        demand=100.0,\n",
+    "        pmin=[10, 20, 30],\n",
+    "        pmax=[50, 60, 70],\n",
+    "        cfix=[700, 600, 500],\n",
+    "        cvar=[1.5, 2.0, 2.5],\n",
+    "    )\n",
+    ")\n",
+    "\n",
+    "model.optimize()\n",
+    "print(\"obj =\", model.inner.objVal)\n",
+    "print(\"x =\", [model.inner._x[i].x for i in range(3)])\n",
+    "print(\"y =\", [model.inner._y[i].x for i in range(3)])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "41b03bbc",
+   "metadata": {},
+   "source": [
+    "Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "01f576e1-1790-425e-9e5c-9fa07b6f4c26",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "    \n",
+    "Note\n",
+    "\n",
+    "- In the example above, `GurobiModel` is just a thin wrapper around a standard Gurobi model. This wrapper allows MIPLearn to be solver- and modeling-language-agnostic. The wrapper provides only a few basic methods, such as `optimize`. For more control, and to query the solution, the original Gurobi model can be accessed through `model.inner`, as illustrated above.\n",
+    "- To ensure training data consistency, MIPLearn requires all decision variables to have names.\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cf60c1dd",
+   "metadata": {},
+   "source": [
+    "## Generating training data\n",
+    "\n",
+    "Although Gurobi could solve the small example above in a fraction of a second, it gets slower for larger and more complex versions of the problem. If this is a problem that needs to be solved frequently, as it is often the case in practice, it could make sense to spend some time upfront generating a **trained** solver, which can optimize new instances (similar to the ones it was trained on) faster.\n",
+    "\n",
+    "In the following, we will use MIPLearn to train machine learning models that is able to predict the optimal solution for instances that follow a given probability distribution, then it will provide this predicted solution to Gurobi as a warm start. Before we can train the model, we need to collect training data by solving a large number of instances. In real-world situations, we may construct these training instances based on historical data. In this tutorial, we will construct them using a random instance generator:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "5eb09fab",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:49:22.758192368Z",
+     "start_time": "2023-06-06T20:49:22.724784572Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from scipy.stats import uniform\n",
+    "from typing import List\n",
+    "import random\n",
+    "\n",
+    "\n",
+    "def random_uc_data(samples: int, n: int, seed: int = 42) -> List[UnitCommitmentData]:\n",
+    "    random.seed(seed)\n",
+    "    np.random.seed(seed)\n",
+    "    pmin = uniform(loc=100_000.0, scale=400_000.0).rvs(n)\n",
+    "    pmax = pmin * uniform(loc=2.0, scale=2.5).rvs(n)\n",
+    "    cfix = pmin * uniform(loc=100.0, scale=25.0).rvs(n)\n",
+    "    cvar = uniform(loc=1.25, scale=0.25).rvs(n)\n",
+    "    return [\n",
+    "        UnitCommitmentData(\n",
+    "            demand=pmax.sum() * uniform(loc=0.5, scale=0.25).rvs(),\n",
+    "            pmin=pmin,\n",
+    "            pmax=pmax,\n",
+    "            cfix=cfix,\n",
+    "            cvar=cvar,\n",
+    "        )\n",
+    "        for _ in range(samples)\n",
+    "    ]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a03a7ac",
+   "metadata": {},
+   "source": [
+    "In this example, for simplicity, only the demands change from one instance to the next. We could also have randomized the costs, production limits or even the number of units. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
+    "\n",
+    "Now we generate 500 instances of this problem, each one with 50 generators, and we use 450 of these instances for training. After generating the instances, we write them to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold in memory the entire training data, as well as the concrete Pyomo models. Files also make it much easier to solve multiple instances simultaneously, potentially on multiple machines. The code below generates the files `uc/train/00000.pkl.gz`, `uc/train/00001.pkl.gz`, etc., which contain the input data in compressed (gzipped) pickle format."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "6156752c",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:49:24.811192929Z",
+     "start_time": "2023-06-06T20:49:24.575639142Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from miplearn.io import write_pkl_gz\n",
+    "\n",
+    "data = random_uc_data(samples=500, n=500)\n",
+    "train_data = write_pkl_gz(data[0:450], \"uc/train\")\n",
+    "test_data = write_pkl_gz(data[450:500], \"uc/test\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b17af877",
+   "metadata": {},
+   "source": [
+    "Finally, we use `BasicCollector` to collect the optimal solutions and other useful training data for all training instances. The data is stored in HDF5 files `uc/train/00000.h5`, `uc/train/00001.h5`, etc. The optimization models are also exported to compressed MPS files `uc/train/00000.mps.gz`, `uc/train/00001.mps.gz`, etc."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "7623f002",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:49:34.936729253Z",
+     "start_time": "2023-06-06T20:49:25.936126612Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from miplearn.collectors.basic import BasicCollector\n",
+    "\n",
+    "bc = BasicCollector()\n",
+    "bc.collect(train_data, build_uc_model, n_jobs=4)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c42b1be1-9723-4827-82d8-974afa51ef9f",
+   "metadata": {},
+   "source": [
+    "## Training and solving test instances"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a33c6aa4-f0b8-4ccb-9935-01f7d7de2a1c",
+   "metadata": {},
+   "source": [
+    "With training data in hand, we can now design and train a machine learning model to accelerate solver performance. In this tutorial, for illustration purposes, we will use ML to generate a good warm start using $k$-nearest neighbors. More specifically, the strategy is to:\n",
+    "\n",
+    "1. Memorize the optimal solutions of all training instances;\n",
+    "2. Given a test instance, find the 25 most similar training instances, based on constraint right-hand sides;\n",
+    "3. Merge their optimal solutions into a single partial solution; specifically, only assign values to the binary variables that agree unanimously.\n",
+    "4. Provide this partial solution to the solver as a warm start.\n",
+    "\n",
+    "This simple strategy can be implemented as shown below, using `MemorizingPrimalComponent`. For more advanced strategies, and for the usage of more advanced classifiers, see the user guide."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "435f7bf8-4b09-4889-b1ec-b7b56e7d8ed2",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:49:38.997939600Z",
+     "start_time": "2023-06-06T20:49:38.968261432Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from sklearn.neighbors import KNeighborsClassifier\n",
+    "from miplearn.components.primal.actions import SetWarmStart\n",
+    "from miplearn.components.primal.mem import (\n",
+    "    MemorizingPrimalComponent,\n",
+    "    MergeTopSolutions,\n",
+    ")\n",
+    "from miplearn.extractors.fields import H5FieldsExtractor\n",
+    "\n",
+    "comp = MemorizingPrimalComponent(\n",
+    "    clf=KNeighborsClassifier(n_neighbors=25),\n",
+    "    extractor=H5FieldsExtractor(\n",
+    "        instance_fields=[\"static_constr_rhs\"],\n",
+    "    ),\n",
+    "    constructor=MergeTopSolutions(25, [0.0, 1.0]),\n",
+    "    action=SetWarmStart(),\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9536e7e4-0b0d-49b0-bebd-4a848f839e94",
+   "metadata": {},
+   "source": [
+    "Having defined the ML strategy, we next construct `LearningSolver`, train the ML component and optimize one of the test instances."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "9d13dd50-3dcf-4673-a757-6f44dcc0dedf",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:49:42.072345411Z",
+     "start_time": "2023-06-06T20:49:41.294040974Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0xa8b70287\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "Presolve removed 1000 rows and 500 columns\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1 rows, 500 columns, 500 nonzeros\n",
+      "\n",
+      "Iteration    Objective       Primal Inf.    Dual Inf.      Time\n",
+      "       0    6.6166537e+09   5.648803e+04   0.000000e+00      0s\n",
+      "       1    8.2906219e+09   0.000000e+00   0.000000e+00      0s\n",
+      "\n",
+      "Solved in 1 iterations and 0.01 seconds (0.00 work units)\n",
+      "Optimal objective  8.290621916e+09\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x4ccd7ae3\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "\n",
+      "User MIP start produced solution with objective 8.30129e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.29184e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.29146e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.29146e+09 (0.01s)\n",
+      "Loaded user MIP start with objective 8.29146e+09\n",
+      "\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "\n",
+      "Root relaxation: objective 8.290622e+09, 512 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 8.2906e+09    0    1 8.2915e+09 8.2906e+09  0.01%     -    0s\n",
+      "\n",
+      "Cutting planes:\n",
+      "  Cover: 1\n",
+      "  Flow cover: 2\n",
+      "\n",
+      "Explored 1 nodes (512 simplex iterations) in 0.07 seconds (0.01 work units)\n",
+      "Thread count was 12 (of 12 available processors)\n",
+      "\n",
+      "Solution count 3: 8.29146e+09 8.29184e+09 8.30129e+09 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 8.291459497797e+09, best bound 8.290645029670e+09, gap 0.0098%\n"
+     ]
+    }
+   ],
+   "source": [
+    "from miplearn.solvers.learning import LearningSolver\n",
+    "\n",
+    "solver_ml = LearningSolver(components=[comp])\n",
+    "solver_ml.fit(train_data)\n",
+    "solver_ml.optimize(test_data[0], build_uc_model);"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "61da6dad-7f56-4edb-aa26-c00eb5f946c0",
+   "metadata": {},
+   "source": [
+    "By examining the solve log above, specifically the line `Loaded user MIP start with objective...`, we can see that MIPLearn was able to construct an initial solution which turned out to be very close to the optimal solution to the problem. Now let us repeat the code above, but a solver which does not apply any ML strategies. Note that our previously-defined component is not provided."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "2ff391ed-e855-4228-aa09-a7641d8c2893",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:49:44.012782276Z",
+     "start_time": "2023-06-06T20:49:43.813974362Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0xa8b70287\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "Presolve removed 1000 rows and 500 columns\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1 rows, 500 columns, 500 nonzeros\n",
+      "\n",
+      "Iteration    Objective       Primal Inf.    Dual Inf.      Time\n",
+      "       0    6.6166537e+09   5.648803e+04   0.000000e+00      0s\n",
+      "       1    8.2906219e+09   0.000000e+00   0.000000e+00      0s\n",
+      "\n",
+      "Solved in 1 iterations and 0.01 seconds (0.00 work units)\n",
+      "Optimal objective  8.290621916e+09\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x4cbbf7c7\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Found heuristic solution: objective 9.757128e+09\n",
+      "\n",
+      "Root relaxation: objective 8.290622e+09, 512 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 8.2906e+09    0    1 9.7571e+09 8.2906e+09  15.0%     -    0s\n",
+      "H    0     0                    8.298273e+09 8.2906e+09  0.09%     -    0s\n",
+      "     0     0 8.2907e+09    0    4 8.2983e+09 8.2907e+09  0.09%     -    0s\n",
+      "     0     0 8.2907e+09    0    1 8.2983e+09 8.2907e+09  0.09%     -    0s\n",
+      "     0     0 8.2907e+09    0    4 8.2983e+09 8.2907e+09  0.09%     -    0s\n",
+      "H    0     0                    8.293980e+09 8.2907e+09  0.04%     -    0s\n",
+      "     0     0 8.2907e+09    0    5 8.2940e+09 8.2907e+09  0.04%     -    0s\n",
+      "     0     0 8.2907e+09    0    1 8.2940e+09 8.2907e+09  0.04%     -    0s\n",
+      "     0     0 8.2907e+09    0    2 8.2940e+09 8.2907e+09  0.04%     -    0s\n",
+      "     0     0 8.2908e+09    0    1 8.2940e+09 8.2908e+09  0.04%     -    0s\n",
+      "     0     0 8.2908e+09    0    4 8.2940e+09 8.2908e+09  0.04%     -    0s\n",
+      "     0     0 8.2908e+09    0    4 8.2940e+09 8.2908e+09  0.04%     -    0s\n",
+      "H    0     0                    8.291465e+09 8.2908e+09  0.01%     -    0s\n",
+      "\n",
+      "Cutting planes:\n",
+      "  Gomory: 2\n",
+      "  MIR: 1\n",
+      "\n",
+      "Explored 1 nodes (1031 simplex iterations) in 0.07 seconds (0.03 work units)\n",
+      "Thread count was 12 (of 12 available processors)\n",
+      "\n",
+      "Solution count 4: 8.29147e+09 8.29398e+09 8.29827e+09 9.75713e+09 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 8.291465302389e+09, best bound 8.290781665333e+09, gap 0.0082%\n"
+     ]
+    }
+   ],
+   "source": [
+    "solver_baseline = LearningSolver(components=[])\n",
+    "solver_baseline.fit(train_data)\n",
+    "solver_baseline.optimize(test_data[0], build_uc_model);"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b6d37b88-9fcc-43ee-ac1e-2a7b1e51a266",
+   "metadata": {},
+   "source": [
+    "In the log above, the `MIP start` line is missing, and Gurobi had to start with a significantly inferior initial solution. The solver was still able to find the optimal solution at the end, but it required using its own internal heuristic procedures. In this example, because we solve very small optimization problems, there was almost no difference in terms of running time, but the difference can be significant for larger problems."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eec97f06",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Accessing the solution\n",
+    "\n",
+    "In the example above, we used `LearningSolver.solve` together with data files to solve both the training and the test instances. The optimal solutions were saved to HDF5 files in the train/test folders, and could be retrieved by reading theses files, but that is not very convenient. In the following example, we show how to build and solve a Pyomo model entirely in-memory, using our trained solver."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "67a6cd18",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:50:12.869892930Z",
+     "start_time": "2023-06-06T20:50:12.509410473Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x19042f12\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "Presolve removed 1000 rows and 500 columns\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1 rows, 500 columns, 500 nonzeros\n",
+      "\n",
+      "Iteration    Objective       Primal Inf.    Dual Inf.      Time\n",
+      "       0    6.5917580e+09   5.627453e+04   0.000000e+00      0s\n",
+      "       1    8.2535968e+09   0.000000e+00   0.000000e+00      0s\n",
+      "\n",
+      "Solved in 1 iterations and 0.01 seconds (0.00 work units)\n",
+      "Optimal objective  8.253596777e+09\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x8ee64638\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "\n",
+      "User MIP start produced solution with objective 8.25814e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.25512e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.25459e+09 (0.04s)\n",
+      "User MIP start produced solution with objective 8.25459e+09 (0.04s)\n",
+      "Loaded user MIP start with objective 8.25459e+09\n",
+      "\n",
+      "Presolve time: 0.01s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "\n",
+      "Root relaxation: objective 8.253597e+09, 512 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 8.2536e+09    0    1 8.2546e+09 8.2536e+09  0.01%     -    0s\n",
+      "     0     0 8.2537e+09    0    3 8.2546e+09 8.2537e+09  0.01%     -    0s\n",
+      "     0     0 8.2537e+09    0    1 8.2546e+09 8.2537e+09  0.01%     -    0s\n",
+      "     0     0 8.2537e+09    0    4 8.2546e+09 8.2537e+09  0.01%     -    0s\n",
+      "     0     0 8.2537e+09    0    4 8.2546e+09 8.2537e+09  0.01%     -    0s\n",
+      "     0     0 8.2538e+09    0    4 8.2546e+09 8.2538e+09  0.01%     -    0s\n",
+      "     0     0 8.2538e+09    0    5 8.2546e+09 8.2538e+09  0.01%     -    0s\n",
+      "     0     0 8.2538e+09    0    6 8.2546e+09 8.2538e+09  0.01%     -    0s\n",
+      "\n",
+      "Cutting planes:\n",
+      "  Cover: 1\n",
+      "  MIR: 2\n",
+      "  StrongCG: 1\n",
+      "  Flow cover: 1\n",
+      "\n",
+      "Explored 1 nodes (575 simplex iterations) in 0.12 seconds (0.01 work units)\n",
+      "Thread count was 12 (of 12 available processors)\n",
+      "\n",
+      "Solution count 3: 8.25459e+09 8.25512e+09 8.25814e+09 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 8.254590409970e+09, best bound 8.253768093811e+09, gap 0.0100%\n",
+      "obj = 8254590409.969726\n",
+      "x = [1.0, 1.0, 0.0]\n",
+      "y = [935662.0949263407, 1604270.0218116897, 0.0]\n"
+     ]
+    }
+   ],
+   "source": [
+    "data = random_uc_data(samples=1, n=500)[0]\n",
+    "model = build_uc_model(data)\n",
+    "solver_ml.optimize(model)\n",
+    "print(\"obj =\", model.inner.objVal)\n",
+    "print(\"x =\", [model.inner._x[i].x for i in range(3)])\n",
+    "print(\"y =\", [model.inner._y[i].x for i in range(3)])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5593d23a-83bd-4e16-8253-6300f5e3f63b",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/tutorials/getting-started-jump.ipynb

@@ -0,0 +1,680 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "6b8983b1",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "# Getting started (JuMP)\n",
+    "\n",
+    "## Introduction\n",
+    "\n",
+    "**MIPLearn** is an open source framework that uses machine learning (ML) to accelerate the performance of mixed-integer programming solvers (e.g. Gurobi, CPLEX, XPRESS). In this tutorial, we will:\n",
+    "\n",
+    "1. Install the Julia/JuMP version of MIPLearn\n",
+    "2. Model a simple optimization problem using JuMP\n",
+    "3. Generate training data and train the ML models\n",
+    "4. Use the ML models together Gurobi to solve new instances\n",
+    "\n",
+    "<div class=\"alert alert-warning\">\n",
+    "Warning\n",
+    "    \n",
+    "MIPLearn is still in early development stage. If run into any bugs or issues, please submit a bug report in our GitHub repository. Comments, suggestions and pull requests are also very welcome!\n",
+    "    \n",
+    "</div>\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "02f0a927",
+   "metadata": {},
+   "source": [
+    "## Installation\n",
+    "\n",
+    "MIPLearn is available in two versions:\n",
+    "\n",
+    "- Python version, compatible with the Pyomo and Gurobipy modeling languages,\n",
+    "- Julia version, compatible with the JuMP modeling language.\n",
+    "\n",
+    "In this tutorial, we will demonstrate how to use and install the Python/Pyomo version of the package. The first step is to install Julia in your machine. See the [official Julia website for more instructions](https://julialang.org/downloads/). After Julia is installed, launch the Julia REPL, type `]` to enter package mode, then install MIPLearn:\n",
+    "\n",
+    "```\n",
+    "pkg> add MIPLearn@0.3\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e8274543",
+   "metadata": {},
+   "source": [
+    "In addition to MIPLearn itself, we will also install:\n",
+    "\n",
+    "- the JuMP modeling language\n",
+    "- Gurobi, a state-of-the-art commercial MILP solver\n",
+    "- Distributions, to generate random data\n",
+    "- PyCall, to access ML model from Scikit-Learn\n",
+    "- Suppressor, to make the output cleaner\n",
+    "\n",
+    "```\n",
+    "pkg> add JuMP@1, Gurobi@1, Distributions@0.25, PyCall@1, Suppressor@0.2\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a14e4550",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "    \n",
+    "Note\n",
+    "\n",
+    "- If you do not have a Gurobi license available, you can also follow the tutorial by installing an open-source solver, such as `HiGHS`, and replacing `Gurobi.Optimizer` by `HiGHS.Optimizer` in all the code examples.\n",
+    "- In the code above, we install specific version of all packages to ensure that this tutorial keeps running in the future, even when newer (and possibly incompatible) versions of the packages are released. This is usually a recommended practice for all Julia projects.\n",
+    "   \n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "16b86823",
+   "metadata": {},
+   "source": [
+    "## Modeling a simple optimization problem\n",
+    "\n",
+    "To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the **unit commitment problem,** a practical optimization problem solved daily by electric grid operators around the world. \n",
+    "\n",
+    "Suppose that a utility company needs to decide which electrical generators should be online at each hour of the day, as well as how much power should each generator produce. More specifically, assume that the company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs the company $c^\\text{fix}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing and costs nothing. The total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts).\n",
+    "\n",
+    "This simple problem can be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem is then given by:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f12c3702",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align}\n",
+    "\\text{minimize } \\quad & \\sum_{i=1}^n \\left( c^\\text{fix}_i x_i + c^\\text{var}_i y_i \\right) \\\\\n",
+    "\\text{subject to } \\quad & y_i \\leq p^\\text{max}_i x_i & i=1,\\ldots,n \\\\\n",
+    "& y_i \\geq p^\\text{min}_i x_i & i=1,\\ldots,n \\\\\n",
+    "& \\sum_{i=1}^n y_i = d \\\\\n",
+    "& x_i \\in \\{0,1\\} & i=1,\\ldots,n \\\\\n",
+    "& y_i \\geq 0 & i=1,\\ldots,n\n",
+    "\\end{align}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "be3989ed",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "\n",
+    "Note\n",
+    "\n",
+    "We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem.\n",
+    "\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a5fd33f6",
+   "metadata": {},
+   "source": [
+    "Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Julia and JuMP. We start by defining a data class `UnitCommitmentData`, which holds all the input data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "c62ebff1-db40-45a1-9997-d121837f067b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "struct UnitCommitmentData\n",
+    "    demand::Float64\n",
+    "    pmin::Vector{Float64}\n",
+    "    pmax::Vector{Float64}\n",
+    "    cfix::Vector{Float64}\n",
+    "    cvar::Vector{Float64}\n",
+    "end;"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "29f55efa-0751-465a-9b0a-a821d46a3d40",
+   "metadata": {},
+   "source": [
+    "Next, we write a `build_uc_model` function, which converts the input data into a concrete JuMP model. The function accepts `UnitCommitmentData`, the data structure we previously defined, or the path to a JLD2 file containing this data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "79ef7775-18ca-4dfa-b438-49860f762ad0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "using MIPLearn\n",
+    "using JuMP\n",
+    "using Gurobi\n",
+    "\n",
+    "function build_uc_model(data)\n",
+    "    if data isa String\n",
+    "        data = read_jld2(data)\n",
+    "    end\n",
+    "    model = Model(Gurobi.Optimizer)\n",
+    "    G = 1:length(data.pmin)\n",
+    "    @variable(model, x[G], Bin)\n",
+    "    @variable(model, y[G] >= 0)\n",
+    "    @objective(model, Min, sum(data.cfix[g] * x[g] + data.cvar[g] * y[g] for g in G))\n",
+    "    @constraint(model, eq_max_power[g in G], y[g] <= data.pmax[g] * x[g])\n",
+    "    @constraint(model, eq_min_power[g in G], y[g] >= data.pmin[g] * x[g])\n",
+    "    @constraint(model, eq_demand, sum(y[g] for g in G) == data.demand)\n",
+    "    return JumpModel(model)\n",
+    "end;"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c22714a3",
+   "metadata": {},
+   "source": [
+    "At this point, we can already use Gurobi to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "dd828d68-fd43-4d2a-a058-3e2628d99d9e",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:01:10.993801745Z",
+     "start_time": "2023-06-06T20:01:10.887580927Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]\n",
+      "Thread count: 16 physical cores, 32 logical processors, using up to 32 threads\n",
+      "\n",
+      "Optimize a model with 7 rows, 6 columns and 15 nonzeros\n",
+      "Model fingerprint: 0x55e33a07\n",
+      "Variable types: 3 continuous, 3 integer (3 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 7e+01]\n",
+      "  Objective range  [2e+00, 7e+02]\n",
+      "  Bounds range     [0e+00, 0e+00]\n",
+      "  RHS range        [1e+02, 1e+02]\n",
+      "Presolve removed 2 rows and 1 columns\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 5 rows, 5 columns, 13 nonzeros\n",
+      "Variable types: 0 continuous, 5 integer (3 binary)\n",
+      "Found heuristic solution: objective 1400.0000000\n",
+      "\n",
+      "Root relaxation: objective 1.035000e+03, 3 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 1035.00000    0    1 1400.00000 1035.00000  26.1%     -    0s\n",
+      "     0     0 1105.71429    0    1 1400.00000 1105.71429  21.0%     -    0s\n",
+      "*    0     0               0    1320.0000000 1320.00000  0.00%     -    0s\n",
+      "\n",
+      "Explored 1 nodes (5 simplex iterations) in 0.00 seconds (0.00 work units)\n",
+      "Thread count was 32 (of 32 available processors)\n",
+      "\n",
+      "Solution count 2: 1320 1400 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 1.320000000000e+03, best bound 1.320000000000e+03, gap 0.0000%\n",
+      "\n",
+      "User-callback calls 371, time in user-callback 0.00 sec\n",
+      "objective_value(model.inner) = 1320.0\n",
+      "Vector(value.(model.inner[:x])) = [-0.0, 1.0, 1.0]\n",
+      "Vector(value.(model.inner[:y])) = [0.0, 60.0, 40.0]\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = build_uc_model(\n",
+    "    UnitCommitmentData(\n",
+    "        100.0,  # demand\n",
+    "        [10, 20, 30],  # pmin\n",
+    "        [50, 60, 70],  # pmax\n",
+    "        [700, 600, 500],  # cfix\n",
+    "        [1.5, 2.0, 2.5],  # cvar\n",
+    "    )\n",
+    ")\n",
+    "model.optimize()\n",
+    "@show objective_value(model.inner)\n",
+    "@show Vector(value.(model.inner[:x]))\n",
+    "@show Vector(value.(model.inner[:y]));"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "41b03bbc",
+   "metadata": {},
+   "source": [
+    "Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "01f576e1-1790-425e-9e5c-9fa07b6f4c26",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "    \n",
+    "Notes\n",
+    "    \n",
+    "- In the example above, `JumpModel` is just a thin wrapper around a standard JuMP model. This wrapper allows MIPLearn to be solver- and modeling-language-agnostic. The wrapper provides only a few basic methods, such as `optimize`. For more control, and to query the solution, the original JuMP model can be accessed through `model.inner`, as illustrated above.\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cf60c1dd",
+   "metadata": {},
+   "source": [
+    "## Generating training data\n",
+    "\n",
+    "Although Gurobi could solve the small example above in a fraction of a second, it gets slower for larger and more complex versions of the problem. If this is a problem that needs to be solved frequently, as it is often the case in practice, it could make sense to spend some time upfront generating a **trained** solver, which can optimize new instances (similar to the ones it was trained on) faster.\n",
+    "\n",
+    "In the following, we will use MIPLearn to train machine learning models that is able to predict the optimal solution for instances that follow a given probability distribution, then it will provide this predicted solution to Gurobi as a warm start. Before we can train the model, we need to collect training data by solving a large number of instances. In real-world situations, we may construct these training instances based on historical data. In this tutorial, we will construct them using a random instance generator:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "1326efd7-3869-4137-ab6b-df9cb609a7e0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "using Distributions\n",
+    "using Random\n",
+    "\n",
+    "function random_uc_data(; samples::Int, n::Int, seed::Int=42)::Vector\n",
+    "    Random.seed!(seed)\n",
+    "    pmin = rand(Uniform(100_000, 500_000), n)\n",
+    "    pmax = pmin .* rand(Uniform(2, 2.5), n)\n",
+    "    cfix = pmin .* rand(Uniform(100, 125), n)\n",
+    "    cvar = rand(Uniform(1.25, 1.50), n)\n",
+    "    return [\n",
+    "        UnitCommitmentData(\n",
+    "            sum(pmax) * rand(Uniform(0.5, 0.75)),\n",
+    "            pmin,\n",
+    "            pmax,\n",
+    "            cfix,\n",
+    "            cvar,\n",
+    "        )\n",
+    "        for _ in 1:samples\n",
+    "    ]\n",
+    "end;"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a03a7ac",
+   "metadata": {},
+   "source": [
+    "In this example, for simplicity, only the demands change from one instance to the next. We could also have randomized the costs, production limits or even the number of units. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
+    "\n",
+    "Now we generate 500 instances of this problem, each one with 50 generators, and we use 450 of these instances for training. After generating the instances, we write them to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold in memory the entire training data, as well as the concrete Pyomo models. Files also make it much easier to solve multiple instances simultaneously, potentially on multiple machines. The code below generates the files `uc/train/00001.jld2`, `uc/train/00002.jld2`, etc., which contain the input data in JLD2 format."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "6156752c",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:03:04.782830561Z",
+     "start_time": "2023-06-06T20:03:04.530421396Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "data = random_uc_data(samples=500, n=500)\n",
+    "train_data = write_jld2(data[1:450], \"uc/train\")\n",
+    "test_data = write_jld2(data[451:500], \"uc/test\");"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b17af877",
+   "metadata": {},
+   "source": [
+    "Finally, we use `BasicCollector` to collect the optimal solutions and other useful training data for all training instances. The data is stored in HDF5 files `uc/train/00001.h5`, `uc/train/00002.h5`, etc. The optimization models are also exported to compressed MPS files `uc/train/00001.mps.gz`, `uc/train/00002.mps.gz`, etc."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "7623f002",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:03:35.571497019Z",
+     "start_time": "2023-06-06T20:03:25.804104036Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "using Suppressor\n",
+    "@suppress_out begin\n",
+    "    bc = BasicCollector()\n",
+    "    bc.collect(train_data, build_uc_model)\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c42b1be1-9723-4827-82d8-974afa51ef9f",
+   "metadata": {},
+   "source": [
+    "## Training and solving test instances"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a33c6aa4-f0b8-4ccb-9935-01f7d7de2a1c",
+   "metadata": {},
+   "source": [
+    "With training data in hand, we can now design and train a machine learning model to accelerate solver performance. In this tutorial, for illustration purposes, we will use ML to generate a good warm start using $k$-nearest neighbors. More specifically, the strategy is to:\n",
+    "\n",
+    "1. Memorize the optimal solutions of all training instances;\n",
+    "2. Given a test instance, find the 25 most similar training instances, based on constraint right-hand sides;\n",
+    "3. Merge their optimal solutions into a single partial solution; specifically, only assign values to the binary variables that agree unanimously.\n",
+    "4. Provide this partial solution to the solver as a warm start.\n",
+    "\n",
+    "This simple strategy can be implemented as shown below, using `MemorizingPrimalComponent`. For more advanced strategies, and for the usage of more advanced classifiers, see the user guide."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "435f7bf8-4b09-4889-b1ec-b7b56e7d8ed2",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:05:20.497772794Z",
+     "start_time": "2023-06-06T20:05:20.484821405Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "# Load kNN classifier from Scikit-Learn\n",
+    "using PyCall\n",
+    "KNeighborsClassifier = pyimport(\"sklearn.neighbors\").KNeighborsClassifier\n",
+    "\n",
+    "# Build the MIPLearn component\n",
+    "comp = MemorizingPrimalComponent(\n",
+    "    clf=KNeighborsClassifier(n_neighbors=25),\n",
+    "    extractor=H5FieldsExtractor(\n",
+    "        instance_fields=[\"static_constr_rhs\"],\n",
+    "    ),\n",
+    "    constructor=MergeTopSolutions(25, [0.0, 1.0]),\n",
+    "    action=SetWarmStart(),\n",
+    ");"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9536e7e4-0b0d-49b0-bebd-4a848f839e94",
+   "metadata": {},
+   "source": [
+    "Having defined the ML strategy, we next construct `LearningSolver`, train the ML component and optimize one of the test instances."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "9d13dd50-3dcf-4673-a757-6f44dcc0dedf",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:05:22.672002339Z",
+     "start_time": "2023-06-06T20:05:21.447466634Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]\n",
+      "Thread count: 16 physical cores, 32 logical processors, using up to 32 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0xd2378195\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 1e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [0e+00, 0e+00]\n",
+      "  RHS range        [2e+08, 2e+08]\n",
+      "\n",
+      "User MIP start produced solution with objective 1.02165e+10 (0.00s)\n",
+      "Loaded user MIP start with objective 1.02165e+10\n",
+      "\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "\n",
+      "Root relaxation: objective 1.021568e+10, 510 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 1.0216e+10    0    1 1.0217e+10 1.0216e+10  0.01%     -    0s\n",
+      "\n",
+      "Explored 1 nodes (510 simplex iterations) in 0.01 seconds (0.00 work units)\n",
+      "Thread count was 32 (of 32 available processors)\n",
+      "\n",
+      "Solution count 1: 1.02165e+10 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 1.021651058978e+10, best bound 1.021567971257e+10, gap 0.0081%\n",
+      "\n",
+      "User-callback calls 169, time in user-callback 0.00 sec\n"
+     ]
+    }
+   ],
+   "source": [
+    "solver_ml = LearningSolver(components=[comp])\n",
+    "solver_ml.fit(train_data)\n",
+    "solver_ml.optimize(test_data[1], build_uc_model);"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "61da6dad-7f56-4edb-aa26-c00eb5f946c0",
+   "metadata": {},
+   "source": [
+    "By examining the solve log above, specifically the line `Loaded user MIP start with objective...`, we can see that MIPLearn was able to construct an initial solution which turned out to be very close to the optimal solution to the problem. Now let us repeat the code above, but a solver which does not apply any ML strategies. Note that our previously-defined component is not provided."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "2ff391ed-e855-4228-aa09-a7641d8c2893",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:05:46.969575966Z",
+     "start_time": "2023-06-06T20:05:46.420803286Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]\n",
+      "Thread count: 16 physical cores, 32 logical processors, using up to 32 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0xb45c0594\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 1e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [0e+00, 0e+00]\n",
+      "  RHS range        [2e+08, 2e+08]\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Found heuristic solution: objective 1.071463e+10\n",
+      "\n",
+      "Root relaxation: objective 1.021568e+10, 510 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 1.0216e+10    0    1 1.0715e+10 1.0216e+10  4.66%     -    0s\n",
+      "H    0     0                    1.025162e+10 1.0216e+10  0.35%     -    0s\n",
+      "     0     0 1.0216e+10    0    1 1.0252e+10 1.0216e+10  0.35%     -    0s\n",
+      "H    0     0                    1.023090e+10 1.0216e+10  0.15%     -    0s\n",
+      "H    0     0                    1.022335e+10 1.0216e+10  0.07%     -    0s\n",
+      "H    0     0                    1.022281e+10 1.0216e+10  0.07%     -    0s\n",
+      "H    0     0                    1.021753e+10 1.0216e+10  0.02%     -    0s\n",
+      "H    0     0                    1.021752e+10 1.0216e+10  0.02%     -    0s\n",
+      "     0     0 1.0216e+10    0    3 1.0218e+10 1.0216e+10  0.02%     -    0s\n",
+      "     0     0 1.0216e+10    0    1 1.0218e+10 1.0216e+10  0.02%     -    0s\n",
+      "H    0     0                    1.021651e+10 1.0216e+10  0.01%     -    0s\n",
+      "\n",
+      "Explored 1 nodes (764 simplex iterations) in 0.03 seconds (0.02 work units)\n",
+      "Thread count was 32 (of 32 available processors)\n",
+      "\n",
+      "Solution count 7: 1.02165e+10 1.02175e+10 1.02228e+10 ... 1.07146e+10\n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 1.021651058978e+10, best bound 1.021573363741e+10, gap 0.0076%\n",
+      "\n",
+      "User-callback calls 204, time in user-callback 0.00 sec\n"
+     ]
+    }
+   ],
+   "source": [
+    "solver_baseline = LearningSolver(components=[])\n",
+    "solver_baseline.fit(train_data)\n",
+    "solver_baseline.optimize(test_data[1], build_uc_model);"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b6d37b88-9fcc-43ee-ac1e-2a7b1e51a266",
+   "metadata": {},
+   "source": [
+    "In the log above, the `MIP start` line is missing, and Gurobi had to start with a significantly inferior initial solution. The solver was still able to find the optimal solution at the end, but it required using its own internal heuristic procedures. In this example, because we solve very small optimization problems, there was almost no difference in terms of running time, but the difference can be significant for larger problems."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eec97f06",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Accessing the solution\n",
+    "\n",
+    "In the example above, we used `LearningSolver.solve` together with data files to solve both the training and the test instances. The optimal solutions were saved to HDF5 files in the train/test folders, and could be retrieved by reading theses files, but that is not very convenient. In the following example, we show how to build and solve a JuMP model entirely in-memory, using our trained solver."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "67a6cd18",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:06:26.913448568Z",
+     "start_time": "2023-06-06T20:06:26.169047914Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]\n",
+      "Thread count: 16 physical cores, 32 logical processors, using up to 32 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x974a7fba\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 1e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [0e+00, 0e+00]\n",
+      "  RHS range        [2e+08, 2e+08]\n",
+      "\n",
+      "User MIP start produced solution with objective 9.86729e+09 (0.00s)\n",
+      "User MIP start produced solution with objective 9.86675e+09 (0.00s)\n",
+      "User MIP start produced solution with objective 9.86654e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 9.8661e+09 (0.01s)\n",
+      "Loaded user MIP start with objective 9.8661e+09\n",
+      "\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "\n",
+      "Root relaxation: objective 9.865344e+09, 510 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 9.8653e+09    0    1 9.8661e+09 9.8653e+09  0.01%     -    0s\n",
+      "\n",
+      "Explored 1 nodes (510 simplex iterations) in 0.02 seconds (0.01 work units)\n",
+      "Thread count was 32 (of 32 available processors)\n",
+      "\n",
+      "Solution count 4: 9.8661e+09 9.86654e+09 9.86675e+09 9.86729e+09 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 9.866096485614e+09, best bound 9.865343669936e+09, gap 0.0076%\n",
+      "\n",
+      "User-callback calls 182, time in user-callback 0.00 sec\n",
+      "objective_value(model.inner) = 9.866096485613789e9\n"
+     ]
+    }
+   ],
+   "source": [
+    "data = random_uc_data(samples=1, n=500)[1]\n",
+    "model = build_uc_model(data)\n",
+    "solver_ml.optimize(model)\n",
+    "@show objective_value(model.inner);"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Julia 1.9.0",
+   "language": "julia",
+   "name": "julia-1.9"
+  },
+  "language_info": {
+   "file_extension": ".jl",
+   "mimetype": "application/julia",
+   "name": "julia",
+   "version": "1.9.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/tutorials/getting-started-pyomo.ipynb

@@ -0,0 +1,869 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "6b8983b1",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "# Getting started (Pyomo)\n",
+    "\n",
+    "## Introduction\n",
+    "\n",
+    "**MIPLearn** is an open source framework that uses machine learning (ML) to accelerate the performance of mixed-integer programming solvers (e.g. Gurobi, CPLEX, XPRESS). In this tutorial, we will:\n",
+    "\n",
+    "1. Install the Python/Pyomo version of MIPLearn\n",
+    "2. Model a simple optimization problem using Pyomo\n",
+    "3. Generate training data and train the ML models\n",
+    "4. Use the ML models together Gurobi to solve new instances\n",
+    "\n",
+    "<div class=\"alert alert-info\">\n",
+    "Note\n",
+    "    \n",
+    "The Python/Pyomo version of MIPLearn is currently only compatible with Pyomo persistent solvers (Gurobi, CPLEX and XPRESS). For broader solver compatibility, see the Julia/JuMP version of the package.\n",
+    "</div>\n",
+    "\n",
+    "<div class=\"alert alert-warning\">\n",
+    "Warning\n",
+    "    \n",
+    "MIPLearn is still in early development stage. If run into any bugs or issues, please submit a bug report in our GitHub repository. Comments, suggestions and pull requests are also very welcome!\n",
+    "    \n",
+    "</div>\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "02f0a927",
+   "metadata": {},
+   "source": [
+    "## Installation\n",
+    "\n",
+    "MIPLearn is available in two versions:\n",
+    "\n",
+    "- Python version, compatible with the Pyomo and Gurobipy modeling languages,\n",
+    "- Julia version, compatible with the JuMP modeling language.\n",
+    "\n",
+    "In this tutorial, we will demonstrate how to use and install the Python/Pyomo version of the package. The first step is to install Python 3.8+ in your computer. See the [official Python website for more instructions](https://www.python.org/downloads/). After Python is installed, we proceed to install MIPLearn using `pip`:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "cd8a69c1",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T19:57:33.202580815Z",
+     "start_time": "2023-06-06T19:57:33.198341886Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "# !pip install MIPLearn==0.3.0"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e8274543",
+   "metadata": {},
+   "source": [
+    "In addition to MIPLearn itself, we will also install Gurobi 10.0, a state-of-the-art commercial MILP solver. This step also install a demo license for Gurobi, which should able to solve the small optimization problems in this tutorial. A license is required for solving larger-scale problems."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "dcc8756c",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T19:57:35.756831801Z",
+     "start_time": "2023-06-06T19:57:33.201767088Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: gurobipy<10.1,>=10 in /home/axavier/Software/anaconda3/envs/miplearn/lib/python3.8/site-packages (10.0.1)\n"
+     ]
+    }
+   ],
+   "source": [
+    "!pip install 'gurobipy>=10,<10.1'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a14e4550",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "    \n",
+    "Note\n",
+    "    \n",
+    "In the code above, we install specific version of all packages to ensure that this tutorial keeps running in the future, even when newer (and possibly incompatible) versions of the packages are released. This is usually a recommended practice for all Python projects.\n",
+    "    \n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "16b86823",
+   "metadata": {},
+   "source": [
+    "## Modeling a simple optimization problem\n",
+    "\n",
+    "To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the **unit commitment problem,** a practical optimization problem solved daily by electric grid operators around the world. \n",
+    "\n",
+    "Suppose that a utility company needs to decide which electrical generators should be online at each hour of the day, as well as how much power should each generator produce. More specifically, assume that the company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs the company $c^\\text{fix}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing and costs nothing. The total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts).\n",
+    "\n",
+    "This simple problem can be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem is then given by:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f12c3702",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align}\n",
+    "\\text{minimize } \\quad & \\sum_{i=1}^n \\left( c^\\text{fix}_i x_i + c^\\text{var}_i y_i \\right) \\\\\n",
+    "\\text{subject to } \\quad & y_i \\leq p^\\text{max}_i x_i & i=1,\\ldots,n \\\\\n",
+    "& y_i \\geq p^\\text{min}_i x_i & i=1,\\ldots,n \\\\\n",
+    "& \\sum_{i=1}^n y_i = d \\\\\n",
+    "& x_i \\in \\{0,1\\} & i=1,\\ldots,n \\\\\n",
+    "& y_i \\geq 0 & i=1,\\ldots,n\n",
+    "\\end{align}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "be3989ed",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "\n",
+    "Note\n",
+    "\n",
+    "We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem.\n",
+    "\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a5fd33f6",
+   "metadata": {},
+   "source": [
+    "Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Python and Pyomo. We start by defining a data class `UnitCommitmentData`, which holds all the input data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "22a67170-10b4-43d3-8708-014d91141e73",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:00:03.278853343Z",
+     "start_time": "2023-06-06T20:00:03.123324067Z"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from dataclasses import dataclass\n",
+    "from typing import List\n",
+    "\n",
+    "import numpy as np\n",
+    "\n",
+    "\n",
+    "@dataclass\n",
+    "class UnitCommitmentData:\n",
+    "    demand: float\n",
+    "    pmin: List[float]\n",
+    "    pmax: List[float]\n",
+    "    cfix: List[float]\n",
+    "    cvar: List[float]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "29f55efa-0751-465a-9b0a-a821d46a3d40",
+   "metadata": {},
+   "source": [
+    "Next, we write a `build_uc_model` function, which converts the input data into a concrete Pyomo model. The function accepts `UnitCommitmentData`, the data structure we previously defined, or the path to a compressed pickle file containing this data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "2f67032f-0d74-4317-b45c-19da0ec859e9",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:00:45.890126754Z",
+     "start_time": "2023-06-06T20:00:45.637044282Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import pyomo.environ as pe\n",
+    "from typing import Union\n",
+    "from miplearn.io import read_pkl_gz\n",
+    "from miplearn.solvers.pyomo import PyomoModel\n",
+    "\n",
+    "\n",
+    "def build_uc_model(data: Union[str, UnitCommitmentData]) -> PyomoModel:\n",
+    "    if isinstance(data, str):\n",
+    "        data = read_pkl_gz(data)\n",
+    "\n",
+    "    model = pe.ConcreteModel()\n",
+    "    n = len(data.pmin)\n",
+    "    model.x = pe.Var(range(n), domain=pe.Binary)\n",
+    "    model.y = pe.Var(range(n), domain=pe.NonNegativeReals)\n",
+    "    model.obj = pe.Objective(\n",
+    "        expr=sum(\n",
+    "            data.cfix[i] * model.x[i] + data.cvar[i] * model.y[i] for i in range(n)\n",
+    "        )\n",
+    "    )\n",
+    "    model.eq_max_power = pe.ConstraintList()\n",
+    "    model.eq_min_power = pe.ConstraintList()\n",
+    "    for i in range(n):\n",
+    "        model.eq_max_power.add(model.y[i] <= data.pmax[i] * model.x[i])\n",
+    "        model.eq_min_power.add(model.y[i] >= data.pmin[i] * model.x[i])\n",
+    "    model.eq_demand = pe.Constraint(\n",
+    "        expr=sum(model.y[i] for i in range(n)) == data.demand,\n",
+    "    )\n",
+    "    return PyomoModel(model, \"gurobi_persistent\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c22714a3",
+   "metadata": {},
+   "source": [
+    "At this point, we can already use Pyomo and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "2a896f47",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:01:10.993801745Z",
+     "start_time": "2023-06-06T20:01:10.887580927Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Restricted license - for non-production use only - expires 2024-10-28\n",
+      "Set parameter QCPDual to value 1\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 7 rows, 6 columns and 15 nonzeros\n",
+      "Model fingerprint: 0x15c7a953\n",
+      "Variable types: 3 continuous, 3 integer (3 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 7e+01]\n",
+      "  Objective range  [2e+00, 7e+02]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [1e+02, 1e+02]\n",
+      "Presolve removed 2 rows and 1 columns\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 5 rows, 5 columns, 13 nonzeros\n",
+      "Variable types: 0 continuous, 5 integer (3 binary)\n",
+      "Found heuristic solution: objective 1400.0000000\n",
+      "\n",
+      "Root relaxation: objective 1.035000e+03, 3 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 1035.00000    0    1 1400.00000 1035.00000  26.1%     -    0s\n",
+      "     0     0 1105.71429    0    1 1400.00000 1105.71429  21.0%     -    0s\n",
+      "*    0     0               0    1320.0000000 1320.00000  0.00%     -    0s\n",
+      "\n",
+      "Explored 1 nodes (5 simplex iterations) in 0.01 seconds (0.00 work units)\n",
+      "Thread count was 12 (of 12 available processors)\n",
+      "\n",
+      "Solution count 2: 1320 1400 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 1.320000000000e+03, best bound 1.320000000000e+03, gap 0.0000%\n",
+      "WARNING: Cannot get reduced costs for MIP.\n",
+      "WARNING: Cannot get duals for MIP.\n",
+      "obj = 1320.0\n",
+      "x = [-0.0, 1.0, 1.0]\n",
+      "y = [0.0, 60.0, 40.0]\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = build_uc_model(\n",
+    "    UnitCommitmentData(\n",
+    "        demand=100.0,\n",
+    "        pmin=[10, 20, 30],\n",
+    "        pmax=[50, 60, 70],\n",
+    "        cfix=[700, 600, 500],\n",
+    "        cvar=[1.5, 2.0, 2.5],\n",
+    "    )\n",
+    ")\n",
+    "\n",
+    "model.optimize()\n",
+    "print(\"obj =\", model.inner.obj())\n",
+    "print(\"x =\", [model.inner.x[i].value for i in range(3)])\n",
+    "print(\"y =\", [model.inner.y[i].value for i in range(3)])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "41b03bbc",
+   "metadata": {},
+   "source": [
+    "Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "01f576e1-1790-425e-9e5c-9fa07b6f4c26",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-info\">\n",
+    "    \n",
+    "Notes\n",
+    "    \n",
+    "- In the example above, `PyomoModel` is just a thin wrapper around a standard Pyomo model. This wrapper allows MIPLearn to be solver- and modeling-language-agnostic. The wrapper provides only a few basic methods, such as `optimize`. For more control, and to query the solution, the original Pyomo model can be accessed through `model.inner`, as illustrated above.    \n",
+    "- To use CPLEX or XPRESS, instead of Gurobi, replace `gurobi_persistent` by `cplex_persistent` or `xpress_persistent` in the `build_uc_model`. Note that only persistent Pyomo solvers are currently supported. Pull requests adding support for other types of solver are very welcome.\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cf60c1dd",
+   "metadata": {},
+   "source": [
+    "## Generating training data\n",
+    "\n",
+    "Although Gurobi could solve the small example above in a fraction of a second, it gets slower for larger and more complex versions of the problem. If this is a problem that needs to be solved frequently, as it is often the case in practice, it could make sense to spend some time upfront generating a **trained** solver, which can optimize new instances (similar to the ones it was trained on) faster.\n",
+    "\n",
+    "In the following, we will use MIPLearn to train machine learning models that is able to predict the optimal solution for instances that follow a given probability distribution, then it will provide this predicted solution to Gurobi as a warm start. Before we can train the model, we need to collect training data by solving a large number of instances. In real-world situations, we may construct these training instances based on historical data. In this tutorial, we will construct them using a random instance generator:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "5eb09fab",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:02:27.324208900Z",
+     "start_time": "2023-06-06T20:02:26.990044230Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from scipy.stats import uniform\n",
+    "from typing import List\n",
+    "import random\n",
+    "\n",
+    "\n",
+    "def random_uc_data(samples: int, n: int, seed: int = 42) -> List[UnitCommitmentData]:\n",
+    "    random.seed(seed)\n",
+    "    np.random.seed(seed)\n",
+    "    pmin = uniform(loc=100_000.0, scale=400_000.0).rvs(n)\n",
+    "    pmax = pmin * uniform(loc=2.0, scale=2.5).rvs(n)\n",
+    "    cfix = pmin * uniform(loc=100.0, scale=25.0).rvs(n)\n",
+    "    cvar = uniform(loc=1.25, scale=0.25).rvs(n)\n",
+    "    return [\n",
+    "        UnitCommitmentData(\n",
+    "            demand=pmax.sum() * uniform(loc=0.5, scale=0.25).rvs(),\n",
+    "            pmin=pmin,\n",
+    "            pmax=pmax,\n",
+    "            cfix=cfix,\n",
+    "            cvar=cvar,\n",
+    "        )\n",
+    "        for _ in range(samples)\n",
+    "    ]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a03a7ac",
+   "metadata": {},
+   "source": [
+    "In this example, for simplicity, only the demands change from one instance to the next. We could also have randomized the costs, production limits or even the number of units. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
+    "\n",
+    "Now we generate 500 instances of this problem, each one with 50 generators, and we use 450 of these instances for training. After generating the instances, we write them to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold in memory the entire training data, as well as the concrete Pyomo models. Files also make it much easier to solve multiple instances simultaneously, potentially on multiple machines. The code below generates the files `uc/train/00000.pkl.gz`, `uc/train/00001.pkl.gz`, etc., which contain the input data in compressed (gzipped) pickle format."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "6156752c",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:03:04.782830561Z",
+     "start_time": "2023-06-06T20:03:04.530421396Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from miplearn.io import write_pkl_gz\n",
+    "\n",
+    "data = random_uc_data(samples=500, n=500)\n",
+    "train_data = write_pkl_gz(data[0:450], \"uc/train\")\n",
+    "test_data = write_pkl_gz(data[450:500], \"uc/test\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b17af877",
+   "metadata": {},
+   "source": [
+    "Finally, we use `BasicCollector` to collect the optimal solutions and other useful training data for all training instances. The data is stored in HDF5 files `uc/train/00000.h5`, `uc/train/00001.h5`, etc. The optimization models are also exported to compressed MPS files `uc/train/00000.mps.gz`, `uc/train/00001.mps.gz`, etc."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "7623f002",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:03:35.571497019Z",
+     "start_time": "2023-06-06T20:03:25.804104036Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from miplearn.collectors.basic import BasicCollector\n",
+    "\n",
+    "bc = BasicCollector()\n",
+    "bc.collect(train_data, build_uc_model, n_jobs=4)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c42b1be1-9723-4827-82d8-974afa51ef9f",
+   "metadata": {},
+   "source": [
+    "## Training and solving test instances"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a33c6aa4-f0b8-4ccb-9935-01f7d7de2a1c",
+   "metadata": {},
+   "source": [
+    "With training data in hand, we can now design and train a machine learning model to accelerate solver performance. In this tutorial, for illustration purposes, we will use ML to generate a good warm start using $k$-nearest neighbors. More specifically, the strategy is to:\n",
+    "\n",
+    "1. Memorize the optimal solutions of all training instances;\n",
+    "2. Given a test instance, find the 25 most similar training instances, based on constraint right-hand sides;\n",
+    "3. Merge their optimal solutions into a single partial solution; specifically, only assign values to the binary variables that agree unanimously.\n",
+    "4. Provide this partial solution to the solver as a warm start.\n",
+    "\n",
+    "This simple strategy can be implemented as shown below, using `MemorizingPrimalComponent`. For more advanced strategies, and for the usage of more advanced classifiers, see the user guide."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "435f7bf8-4b09-4889-b1ec-b7b56e7d8ed2",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:05:20.497772794Z",
+     "start_time": "2023-06-06T20:05:20.484821405Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from sklearn.neighbors import KNeighborsClassifier\n",
+    "from miplearn.components.primal.actions import SetWarmStart\n",
+    "from miplearn.components.primal.mem import (\n",
+    "    MemorizingPrimalComponent,\n",
+    "    MergeTopSolutions,\n",
+    ")\n",
+    "from miplearn.extractors.fields import H5FieldsExtractor\n",
+    "\n",
+    "comp = MemorizingPrimalComponent(\n",
+    "    clf=KNeighborsClassifier(n_neighbors=25),\n",
+    "    extractor=H5FieldsExtractor(\n",
+    "        instance_fields=[\"static_constr_rhs\"],\n",
+    "    ),\n",
+    "    constructor=MergeTopSolutions(25, [0.0, 1.0]),\n",
+    "    action=SetWarmStart(),\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9536e7e4-0b0d-49b0-bebd-4a848f839e94",
+   "metadata": {},
+   "source": [
+    "Having defined the ML strategy, we next construct `LearningSolver`, train the ML component and optimize one of the test instances."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "9d13dd50-3dcf-4673-a757-6f44dcc0dedf",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:05:22.672002339Z",
+     "start_time": "2023-06-06T20:05:21.447466634Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Set parameter QCPDual to value 1\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x5e67c6ee\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "Presolve removed 1000 rows and 500 columns\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1 rows, 500 columns, 500 nonzeros\n",
+      "\n",
+      "Iteration    Objective       Primal Inf.    Dual Inf.      Time\n",
+      "       0    6.6166537e+09   5.648803e+04   0.000000e+00      0s\n",
+      "       1    8.2906219e+09   0.000000e+00   0.000000e+00      0s\n",
+      "\n",
+      "Solved in 1 iterations and 0.01 seconds (0.00 work units)\n",
+      "Optimal objective  8.290621916e+09\n",
+      "Set parameter QCPDual to value 1\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0xa4a7961e\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "\n",
+      "User MIP start produced solution with objective 8.30129e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.29184e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.29146e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.29146e+09 (0.02s)\n",
+      "Loaded user MIP start with objective 8.29146e+09\n",
+      "\n",
+      "Presolve time: 0.01s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "\n",
+      "Root relaxation: objective 8.290622e+09, 512 iterations, 0.01 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 8.2906e+09    0    1 8.2915e+09 8.2906e+09  0.01%     -    0s\n",
+      "\n",
+      "Cutting planes:\n",
+      "  Cover: 1\n",
+      "  Flow cover: 2\n",
+      "\n",
+      "Explored 1 nodes (512 simplex iterations) in 0.09 seconds (0.01 work units)\n",
+      "Thread count was 12 (of 12 available processors)\n",
+      "\n",
+      "Solution count 3: 8.29146e+09 8.29184e+09 8.30129e+09 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 8.291459497797e+09, best bound 8.290645029670e+09, gap 0.0098%\n",
+      "WARNING: Cannot get reduced costs for MIP.\n",
+      "WARNING: Cannot get duals for MIP.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from miplearn.solvers.learning import LearningSolver\n",
+    "\n",
+    "solver_ml = LearningSolver(components=[comp])\n",
+    "solver_ml.fit(train_data)\n",
+    "solver_ml.optimize(test_data[0], build_uc_model);"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "61da6dad-7f56-4edb-aa26-c00eb5f946c0",
+   "metadata": {},
+   "source": [
+    "By examining the solve log above, specifically the line `Loaded user MIP start with objective...`, we can see that MIPLearn was able to construct an initial solution which turned out to be very close to the optimal solution to the problem. Now let us repeat the code above, but a solver which does not apply any ML strategies. Note that our previously-defined component is not provided."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "2ff391ed-e855-4228-aa09-a7641d8c2893",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:05:46.969575966Z",
+     "start_time": "2023-06-06T20:05:46.420803286Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Set parameter QCPDual to value 1\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x5e67c6ee\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "Presolve removed 1000 rows and 500 columns\n",
+      "Presolve time: 0.01s\n",
+      "Presolved: 1 rows, 500 columns, 500 nonzeros\n",
+      "\n",
+      "Iteration    Objective       Primal Inf.    Dual Inf.      Time\n",
+      "       0    6.6166537e+09   5.648803e+04   0.000000e+00      0s\n",
+      "       1    8.2906219e+09   0.000000e+00   0.000000e+00      0s\n",
+      "\n",
+      "Solved in 1 iterations and 0.01 seconds (0.00 work units)\n",
+      "Optimal objective  8.290621916e+09\n",
+      "Set parameter QCPDual to value 1\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x8a0f9587\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "Presolve time: 0.00s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Found heuristic solution: objective 9.757128e+09\n",
+      "\n",
+      "Root relaxation: objective 8.290622e+09, 512 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 8.2906e+09    0    1 9.7571e+09 8.2906e+09  15.0%     -    0s\n",
+      "H    0     0                    8.298273e+09 8.2906e+09  0.09%     -    0s\n",
+      "     0     0 8.2907e+09    0    4 8.2983e+09 8.2907e+09  0.09%     -    0s\n",
+      "     0     0 8.2907e+09    0    1 8.2983e+09 8.2907e+09  0.09%     -    0s\n",
+      "     0     0 8.2907e+09    0    4 8.2983e+09 8.2907e+09  0.09%     -    0s\n",
+      "H    0     0                    8.293980e+09 8.2907e+09  0.04%     -    0s\n",
+      "     0     0 8.2907e+09    0    5 8.2940e+09 8.2907e+09  0.04%     -    0s\n",
+      "     0     0 8.2907e+09    0    1 8.2940e+09 8.2907e+09  0.04%     -    0s\n",
+      "     0     0 8.2907e+09    0    2 8.2940e+09 8.2907e+09  0.04%     -    0s\n",
+      "     0     0 8.2908e+09    0    1 8.2940e+09 8.2908e+09  0.04%     -    0s\n",
+      "     0     0 8.2908e+09    0    4 8.2940e+09 8.2908e+09  0.04%     -    0s\n",
+      "     0     0 8.2908e+09    0    4 8.2940e+09 8.2908e+09  0.04%     -    0s\n",
+      "H    0     0                    8.291465e+09 8.2908e+09  0.01%     -    0s\n",
+      "\n",
+      "Cutting planes:\n",
+      "  Gomory: 2\n",
+      "  MIR: 1\n",
+      "\n",
+      "Explored 1 nodes (1025 simplex iterations) in 0.08 seconds (0.03 work units)\n",
+      "Thread count was 12 (of 12 available processors)\n",
+      "\n",
+      "Solution count 4: 8.29147e+09 8.29398e+09 8.29827e+09 9.75713e+09 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 8.291465302389e+09, best bound 8.290781665333e+09, gap 0.0082%\n",
+      "WARNING: Cannot get reduced costs for MIP.\n",
+      "WARNING: Cannot get duals for MIP.\n"
+     ]
+    }
+   ],
+   "source": [
+    "solver_baseline = LearningSolver(components=[])\n",
+    "solver_baseline.fit(train_data)\n",
+    "solver_baseline.optimize(test_data[0], build_uc_model);"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b6d37b88-9fcc-43ee-ac1e-2a7b1e51a266",
+   "metadata": {},
+   "source": [
+    "In the log above, the `MIP start` line is missing, and Gurobi had to start with a significantly inferior initial solution. The solver was still able to find the optimal solution at the end, but it required using its own internal heuristic procedures. In this example, because we solve very small optimization problems, there was almost no difference in terms of running time, but the difference can be significant for larger problems."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eec97f06",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Accessing the solution\n",
+    "\n",
+    "In the example above, we used `LearningSolver.solve` together with data files to solve both the training and the test instances. The optimal solutions were saved to HDF5 files in the train/test folders, and could be retrieved by reading theses files, but that is not very convenient. In the following example, we show how to build and solve a Pyomo model entirely in-memory, using our trained solver."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "67a6cd18",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-06-06T20:06:26.913448568Z",
+     "start_time": "2023-06-06T20:06:26.169047914Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Set parameter QCPDual to value 1\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x2dfe4e1c\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "Presolve removed 1000 rows and 500 columns\n",
+      "Presolve time: 0.01s\n",
+      "Presolved: 1 rows, 500 columns, 500 nonzeros\n",
+      "\n",
+      "Iteration    Objective       Primal Inf.    Dual Inf.      Time\n",
+      "       0    6.5917580e+09   5.627453e+04   0.000000e+00      0s\n",
+      "       1    8.2535968e+09   0.000000e+00   0.000000e+00      0s\n",
+      "\n",
+      "Solved in 1 iterations and 0.01 seconds (0.00 work units)\n",
+      "Optimal objective  8.253596777e+09\n",
+      "Set parameter QCPDual to value 1\n",
+      "Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)\n",
+      "\n",
+      "CPU model: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, instruction set [SSE2|AVX|AVX2]\n",
+      "Thread count: 6 physical cores, 12 logical processors, using up to 12 threads\n",
+      "\n",
+      "Optimize a model with 1001 rows, 1000 columns and 2500 nonzeros\n",
+      "Model fingerprint: 0x20637200\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "Coefficient statistics:\n",
+      "  Matrix range     [1e+00, 2e+06]\n",
+      "  Objective range  [1e+00, 6e+07]\n",
+      "  Bounds range     [1e+00, 1e+00]\n",
+      "  RHS range        [3e+08, 3e+08]\n",
+      "\n",
+      "User MIP start produced solution with objective 8.25814e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.25512e+09 (0.01s)\n",
+      "User MIP start produced solution with objective 8.25459e+09 (0.04s)\n",
+      "User MIP start produced solution with objective 8.25459e+09 (0.04s)\n",
+      "Loaded user MIP start with objective 8.25459e+09\n",
+      "\n",
+      "Presolve time: 0.01s\n",
+      "Presolved: 1001 rows, 1000 columns, 2500 nonzeros\n",
+      "Variable types: 500 continuous, 500 integer (500 binary)\n",
+      "\n",
+      "Root relaxation: objective 8.253597e+09, 512 iterations, 0.00 seconds (0.00 work units)\n",
+      "\n",
+      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
+      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
+      "\n",
+      "     0     0 8.2536e+09    0    1 8.2546e+09 8.2536e+09  0.01%     -    0s\n",
+      "     0     0 8.2537e+09    0    3 8.2546e+09 8.2537e+09  0.01%     -    0s\n",
+      "     0     0 8.2537e+09    0    1 8.2546e+09 8.2537e+09  0.01%     -    0s\n",
+      "     0     0 8.2537e+09    0    4 8.2546e+09 8.2537e+09  0.01%     -    0s\n",
+      "     0     0 8.2537e+09    0    4 8.2546e+09 8.2537e+09  0.01%     -    0s\n",
+      "     0     0 8.2538e+09    0    4 8.2546e+09 8.2538e+09  0.01%     -    0s\n",
+      "     0     0 8.2538e+09    0    5 8.2546e+09 8.2538e+09  0.01%     -    0s\n",
+      "     0     0 8.2538e+09    0    6 8.2546e+09 8.2538e+09  0.01%     -    0s\n",
+      "\n",
+      "Cutting planes:\n",
+      "  Cover: 1\n",
+      "  MIR: 2\n",
+      "  StrongCG: 1\n",
+      "  Flow cover: 1\n",
+      "\n",
+      "Explored 1 nodes (575 simplex iterations) in 0.11 seconds (0.01 work units)\n",
+      "Thread count was 12 (of 12 available processors)\n",
+      "\n",
+      "Solution count 3: 8.25459e+09 8.25512e+09 8.25814e+09 \n",
+      "\n",
+      "Optimal solution found (tolerance 1.00e-04)\n",
+      "Best objective 8.254590409970e+09, best bound 8.253768093811e+09, gap 0.0100%\n",
+      "WARNING: Cannot get reduced costs for MIP.\n",
+      "WARNING: Cannot get duals for MIP.\n",
+      "obj = 8254590409.96973\n",
+      " x = [1.0, 1.0, 0.0, 1.0, 1.0]\n",
+      " y = [935662.0949263407, 1604270.0218116897, 0.0, 1369560.835229226, 602828.5321028307]\n"
+     ]
+    }
+   ],
+   "source": [
+    "data = random_uc_data(samples=1, n=500)[0]\n",
+    "model = build_uc_model(data)\n",
+    "solver_ml.optimize(model)\n",
+    "print(\"obj =\", model.inner.obj())\n",
+    "print(\" x =\", [model.inner.x[i].value for i in range(5)])\n",
+    "print(\" y =\", [model.inner.y[i].value for i in range(5)])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5593d23a-83bd-4e16-8253-6300f5e3f63b",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/usage.md

@@ -1,246 +0,0 @@
-```{sectnum}
----
-start: 1
-depth: 2
-suffix: .
----
-```
-
-# Using MIPLearn
-
-## Installation
-
-In these docs, we describe the Python/Pyomo version of the package, although a [Julia/JuMP version](https://github.com/ANL-CEEESA/MIPLearn.jl) is also available. A mixed-integer solver is also required and its Python bindings must be properly installed. Supported solvers are currently CPLEX, Gurobi and XPRESS.
-
-To install MIPLearn, run: 
-
-```bash
-pip3 install --upgrade miplearn==0.2.*
-```
-
-After installation, the package `miplearn` should become available to Python. It can be imported
-as follows:
-
-```python
-import miplearn
-```
-
-## Using `LearningSolver`
-
-The main class provided by this package is `LearningSolver`, a learning-enhanced MIP solver which uses information from previously solved instances to accelerate the solution of new instances. The following example shows its basic usage:
-
-```python
-from miplearn import LearningSolver
-
-# List of user-provided instances
-training_instances = [...] 
-test_instances = [...]
-
-# Create solver
-solver = LearningSolver()
-
-# Solve all training instances
-for instance in training_instances:
-    solver.solve(instance)
-
-# Learn from training instances
-solver.fit(training_instances)
-
-# Solve all test instances
-for instance in test_instances:
-    solver.solve(instance)
-```
-
-In this example, we have two lists of user-provided instances: `training_instances` and `test_instances`. We start by solving all training instances. Since there is no historical information available at this point, the instances will be processed from scratch, with no ML acceleration. After solving each instance, the solver stores within each `instance` object the optimal solution, the optimal objective value, and other information that can be used to accelerate future solves. After all training instances are solved, we call `solver.fit(training_instances)`. This instructs the solver to train all its internal machine-learning models based on the solutions of the (solved) trained instances. Subsequent calls to `solver.solve(instance)` will automatically use the trained Machine Learning models to accelerate the solution process.
-
-
-## Describing problem instances
-
-Instances to be solved by `LearningSolver` must derive from the abstract class `miplearn.Instance`. The following three abstract methods must be implemented:
-
-* `instance.to_model()`, which returns a concrete Pyomo model corresponding to the instance;
-* `instance.get_instance_features()`, which returns a 1-dimensional Numpy array of (numerical) features describing the entire instance;
-* `instance.get_variable_features(var_name, index)`, which returns a 1-dimensional array of (numerical) features describing a particular decision variable.
-
-The first method is used by `LearningSolver` to construct a concrete Pyomo model, which will be provided to the internal MIP solver. The second and third methods provide an encoding of the instance, which can be used by the ML models to make predictions. In the knapsack problem, for example, an implementation may decide to provide as instance features the average weights, average prices, number of items and the size of the knapsack. The weight and the price of each individual item could be provided as variable features. See `src/python/miplearn/problems/knapsack.py` for a concrete example.
-
-An optional method which can be implemented is `instance.get_variable_category(var_name, index)`, which returns a category (a string, an integer or any hashable type) for each decision variable. If two variables have the same category, `LearningSolver` will use the same internal ML model to predict the values of both variables. By default, all variables belong to the `"default"` category, and therefore only one ML model is used for all variables. If the returned category is `None`, ML predictors will ignore the variable.
-
-It is not necessary to have a one-to-one correspondence between features and problem instances. One important (and deliberate) limitation of MIPLearn, however, is that `get_instance_features()` must always return arrays of same length for all relevant instances of the problem. Similarly, `get_variable_features(var_name, index)` must also always return arrays of same length for all variables in each category. It is up to the user to decide how to encode variable-length characteristics of the problem into fixed-length vectors. In graph problems, for example, graph embeddings can be used to reduce the (variable-length) lists of nodes and edges into a fixed-length structure that still preserves some properties of the graph. Different instance encodings may have significant impact on performance.
-
-
-## Describing lazy constraints
-
-For many MIP formulations, it is not desirable to add all constraints up-front, either because the total number of constraints is very large, or because some of the constraints, even in relatively small numbers, can still cause significant performance impact when added to the formulation. In these situations, it may be desirable to generate and add constraints incrementaly, during the solution process itself. Conventional MIP solvers typically start by solving the problem without any lazy constraints. Whenever a candidate solution is found, the solver finds all violated lazy constraints and adds them to the formulation. MIPLearn significantly accelerates this process by using ML to predict which lazy constraints should be enforced from the very beginning of the optimization process, even before a candidate solution is available.
-
-MIPLearn supports two types of lazy constraints: through constraint annotations and through callbacks.
-
-###  Adding lazy constraints through annotations
-
-The easiest way to create lazy constraints in MIPLearn is to add them to the model (just like any regular constraints), then annotate them as lazy, as described below. Just before the optimization starts, MIPLearn removes all lazy constraints from the model and places them in a lazy constraint pool. If any trained ML models are available, MIPLearn queries these models to decide which of these constraints should be moved back into the formulation. After this step, the optimization starts, and lazy constraints from the pool are added to the model in the conventional fashion.
-
-To tag a constraint as lazy, the following methods must be implemented:
-
-* `instance.has_static_lazy_constraints()`, which returns `True` if the model has any annotated lazy constraints. By default, this method returns `False`.
-* `instance.is_constraint_lazy(cid)`, which returns `True` if the constraint with name `cid` should be treated as a lazy constraint, and `False` otherwise.
-* `instance.get_constraint_features(cid)`, which returns a 1-dimensional Numpy array of (numerical) features describing the constraint.
-
-For instances such that `has_lazy_constraints` returns `True`, MIPLearn calls `is_constraint_lazy` for each constraint in the formulation, providing the name of the constraint. For constraints such that `is_constraint_lazy` returns `True`, MIPLearn additionally calls `get_constraint_features` to gather a ML representation of each constraint. These features are used to predict which lazy constraints should be initially enforced.
-
-An additional method that can be implemented is `get_lazy_constraint_category(cid)`, which returns a category (a string or any other hashable type) for each lazy constraint. Similarly to decision variable categories, if two lazy constraints have the same category, then MIPLearn will use the same internal ML model to decide whether to initially enforce them. By default, all lazy constraints belong to the `"default"` category, and therefore a single ML model is used.
-
-!!! warning
-    If two lazy constraints belong to the same category, their feature vectors should have the same length.
-
-### Adding lazy constraints through callbacks
-
-Although convenient, the method described in the previous subsection still requires the generation of all lazy constraints ahead of time, which can be prohibitively expensive. An alternative method is through a lazy constraint callbacks, described below. During the solution process, MIPLearn will repeatedly call a user-provided function to identify any violated lazy constraints. If violated constraints are identified, MIPLearn will additionally call another user-provided function to generate the constraint and add it to the formulation.
-
-To describe lazy constraints through user callbacks, the following methods need to be implemented:
-
-* `instance.has_dynamic_lazy_constraints()`, which returns `True` if the model has any lazy constraints generated by user callbacks. By default, this method returns `False`.
-* `instance.find_violated_lazy_constraints(model)`, which returns a list of identifiers corresponding to the lazy constraints found to be violated by the current solution. These identifiers should be strings, tuples or any other hashable type.
-* `instance.build_violated_lazy_constraints(model, cid)`, which returns either a list of Pyomo constraints, or a single Pyomo constraint, corresponding to the given lazy constraint identifier.
-* `instance.get_constraint_features(cid)`, which returns a 1-dimensional Numpy array of (numerical) features describing the constraint. If this constraint is not valid, returns `None`.
-* `instance.get_lazy_constraint_category(cid)`, which returns a category (a string or any other hashable type) for each lazy constraint, indicating which ML model to use. By default, returns `"default"`.
-
-
-Assuming that trained ML models are available, immediately after calling `solver.solve`, MIPLearn will call `get_constraint_features` for each lazy constraint identifier found in the training set. For constraints such that `get_constraint_features` returns a vector (instead of `None`), MIPLearn will call `get_constraint_category` to decide which trained ML model to use. It will then query the ML model to decide whether the constraint should be initially enforced. Assuming that the ML predicts this constraint will be necessary, MIPLearn calls `build_violated_constraints` then adds the returned list of Pyomo constraints to the model. The optimization then starts. When no trained ML models are available, this entire initial process is skipped, and MIPLearn behaves like a conventional solver.
-
-After the optimization process starts, MIPLearn will periodically call `find_violated_lazy_constraints` to verify if the current solution violates any lazy constraints. If any violated lazy constraints are found, MIPLearn will call the method `build_violated_lazy_constraints` and add the returned constraints to the formulation.
-
-```{tip}
-When implementing `find_violated_lazy_constraints(self, model)`, the current solution may be accessed through `self.solution[var_name][index]`.
-```
-
-## Obtaining heuristic solutions
-
-By default, `LearningSolver` uses Machine Learning to accelerate the MIP solution process, while maintaining all optimality guarantees provided by the MIP solver. In the default mode of operation, for example, predicted optimal solutions are used only as MIP starts.
-
-For more significant performance benefits, `LearningSolver` can also be configured to place additional trust in the Machine Learning predictors, by using the `mode="heuristic"` constructor argument. When operating in this mode, if a ML model is statistically shown (through *stratified k-fold cross validation*) to have exceptionally high accuracy, the solver may decide to restrict the search space based on its predictions. The parts of the solution which the ML models cannot predict accurately will still be explored using traditional (branch-and-bound) methods.  For particular applications, this mode has been shown to quickly produce optimal or near-optimal solutions (see [references](about.md#references) and [benchmark results](benchmark.md)).
-
-
-```{danger}
-The `heuristic` mode provides no optimality guarantees, and therefore should only be used if the solver is first trained on a large and representative set of training instances. Training on a small or non-representative set of instances may produce low-quality solutions, or make the solver incorrectly classify new instances as infeasible.
-```
-
-## Scaling Up
-
-### Saving and loading solver state
-
-After solving a large number of training instances, it may be desirable to save the current state of `LearningSolver` to disk, so that the solver can still use the acquired knowledge after the application restarts. This can be accomplished by using the the utility functions `write_pickle_gz` and `read_pickle_gz`, as the following example illustrates:
-
-```python
-from miplearn import LearningSolver, write_pickle_gz, read_pickle_gz
-
-# Solve training instances
-training_instances = [...]
-solver = LearningSolver()
-for instance in training_instances:
-    solver.solve(instance)
-
-# Train machine-learning models
-solver.fit(training_instances)
-
-# Save trained solver to disk
-write_pickle_gz(solver, "solver.pkl.gz")
-
-# Application restarts...
-
-# Load trained solver from disk
-solver = read_pickle_gz("solver.pkl.gz")
-
-# Solve additional instances
-test_instances = [...]
-for instance in test_instances:
-    solver.solve(instance)
-```
-
-
-### Solving instances in parallel
-
-In many situations, instances can be solved in parallel to accelerate the training process. `LearningSolver` provides the method `parallel_solve(instances)` to easily achieve this:
-
-```python
-from miplearn import LearningSolver
-
-training_instances = [...]
-solver = LearningSolver()
-solver.parallel_solve(training_instances, n_jobs=4)
-solver.fit(training_instances)
-
-# Test phase...
-test_instances = [...]
-solver.parallel_solve(test_instances)
-```
-
-
-### Solving instances from the disk
-
-In all examples above, we have assumed that instances are available as Python objects, stored in memory. When problem instances are very large, or when there is a large number of problem instances, this approach may require an excessive amount of memory. To reduce memory requirements, MIPLearn can also operate on instances that are stored on disk, through the `PickleGzInstance` class, as the next example illustrates.
-
-```python
-import pickle
-from miplearn import (
-    LearningSolver,
-    PickleGzInstance,
-    write_pickle_gz,
-)
-
-# Construct and pickle 600 problem instances
-for i in range(600):
-    instance = MyProblemInstance([...])
-    write_pickle_gz(instance, "instance_%03d.pkl" % i)
-        
-# Split instances into training and test
-test_instances  = [PickleGzInstance("instance_%03d.pkl" % i) for i in range(500)]
-train_instances = [PickleGzInstance("instance_%03d.pkl" % i) for i in range(500, 600)]
-
-# Create solver
-solver = LearningSolver([...])
-
-# Solve training instances 
-solver.parallel_solve(train_instances, n_jobs=4)
-
-# Train ML models
-solver.fit(train_instances)
-
-# Solve test instances 
-solver.parallel_solve(test_instances, n_jobs=4)
-```
-
-
-By default, `solve` and `parallel_solve` modify files in place. That is, after the instances are loaded from disk and solved, MIPLearn writes them back to the disk, overwriting the original files. To discard the modifications instead, use `LearningSolver(..., discard_outputs=True)`. This can be useful, for example, during benchmarks.
-
-## Running benchmarks
-
-MIPLearn provides the utility class `BenchmarkRunner`, which simplifies the task of comparing the performance of different solvers. The snippet below shows its basic usage:
-
-```python
-from miplearn import BenchmarkRunner, LearningSolver
-
-# Create train and test instances
-train_instances = [...]
-test_instances  = [...]
-
-# Training phase...
-training_solver = LearningSolver(...)
-training_solver.parallel_solve(train_instances, n_jobs=10)
-
-# Test phase...
-benchmark = BenchmarkRunner({
-    "Baseline": LearningSolver(...),
-    "Strategy A": LearningSolver(...),
-    "Strategy B": LearningSolver(...),
-    "Strategy C": LearningSolver(...),
-})
-benchmark.fit(train_instances)
-benchmark.parallel_solve(test_instances, n_jobs=5)
-benchmark.write_csv("results.csv")
-```
-
-The method `fit` trains the ML models for each individual solver. The method `parallel_solve` solves the test instances in parallel, and collects solver statistics such as running time and optimal value. Finally, `write_csv` produces a table of results. The columns in the CSV file depend on the components added to the solver.
-
-## Current Limitations
-
-* Only binary and continuous decision variables are currently supported. General integer variables are not currently supported by some solver components.

miplearn/__init__.py

@@ -1,29 +1,3 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
-
-from .benchmark import BenchmarkRunner
-from .classifiers import Classifier, Regressor
-from .classifiers.adaptive import AdaptiveClassifier
-from .classifiers.sklearn import ScikitLearnRegressor, ScikitLearnClassifier
-from .classifiers.threshold import MinPrecisionThreshold
-from .components.component import Component
-from .components.dynamic_lazy import DynamicLazyConstraintsComponent
-from .components.dynamic_user_cuts import UserCutsComponent
-from .components.objective import ObjectiveValueComponent
-from .components.primal import PrimalSolutionComponent
-from .components.static_lazy import StaticLazyConstraintsComponent
-from .instance.base import Instance
-from .instance.picklegz import (
-    PickleGzInstance,
-    write_pickle_gz,
-    read_pickle_gz,
-    write_pickle_gz_multiple,
-)
-from .log import setup_logger
-from .solvers.gurobi import GurobiSolver
-from .solvers.internal import InternalSolver
-from .solvers.learning import LearningSolver
-from .solvers.pyomo.base import BasePyomoSolver
-from .solvers.pyomo.cplex import CplexPyomoSolver
-from .solvers.pyomo.gurobi import GurobiPyomoSolver

miplearn/benchmark.py

@@ -1,129 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-import os
-from typing import Dict, List
-
-import pandas as pd
-
-from miplearn.components.component import Component
-from miplearn.instance.base import Instance
-from miplearn.solvers.learning import LearningSolver
-
-logger = logging.getLogger(__name__)
-
-
-class BenchmarkRunner:
-    """
-    Utility class that simplifies the task of comparing the performance of different
-    solvers.
-
-    Example
-    -------
-    ```python
-    benchmark = BenchmarkRunner({
-        "Baseline": LearningSolver(...),
-        "Strategy A": LearningSolver(...),
-        "Strategy B": LearningSolver(...),
-        "Strategy C": LearningSolver(...),
-    })
-    benchmark.fit(train_instances)
-    benchmark.parallel_solve(test_instances, n_jobs=5)
-    benchmark.save_results("result.csv")
-    ```
-
-    Parameters
-    ----------
-    solvers: Dict[str, LearningSolver]
-        Dictionary containing the solvers to compare. Solvers may have different
-        arguments and components. The key should be the name of the solver. It
-        appears in the exported tables of results.
-    """
-
-    def __init__(self, solvers: Dict[str, LearningSolver]) -> None:
-        self.solvers: Dict[str, LearningSolver] = solvers
-        self.results = pd.DataFrame(
-            columns=[
-                "Solver",
-                "Instance",
-            ]
-        )
-
-    def parallel_solve(
-        self,
-        instances: List[Instance],
-        n_jobs: int = 1,
-        n_trials: int = 3,
-    ) -> None:
-        """
-        Solves the given instances in parallel and collect benchmark statistics.
-
-        Parameters
-        ----------
-        instances: List[Instance]
-            List of instances to solve. This can either be a list of instances
-            already loaded in memory, or a list of filenames pointing to pickled (and
-            optionally gzipped) files.
-        n_jobs: int
-            List of instances to solve in parallel at a time.
-        n_trials: int
-            How many times each instance should be solved.
-        """
-        self._silence_miplearn_logger()
-        trials = instances * n_trials
-        for (solver_name, solver) in self.solvers.items():
-            results = solver.parallel_solve(
-                trials,
-                n_jobs=n_jobs,
-                label="Solve (%s)" % solver_name,
-                discard_outputs=True,
-            )
-            for i in range(len(trials)):
-                idx = i % len(instances)
-                results[i]["Solver"] = solver_name
-                results[i]["Instance"] = idx
-                self.results = self.results.append(pd.DataFrame([results[i]]))
-        self._restore_miplearn_logger()
-
-    def write_csv(self, filename: str) -> None:
-        """
-        Writes the collected results to a CSV file.
-
-        Parameters
-        ----------
-        filename: str
-            The name of the file.
-        """
-        os.makedirs(os.path.dirname(filename), exist_ok=True)
-        self.results.to_csv(filename)
-
-    def fit(self, instances: List[Instance], n_jobs: int = 1) -> None:
-        """
-        Trains all solvers with the provided training instances.
-
-        Parameters
-        ----------
-        instances:  List[Instance]
-            List of training instances.
-        n_jobs: int
-            Number of parallel processes to use.
-        """
-        components: List[Component] = []
-        for solver in self.solvers.values():
-            components += solver.components.values()
-        Component.fit_multiple(
-            components,
-            instances,
-            n_jobs=n_jobs,
-        )
-
-    def _silence_miplearn_logger(self) -> None:
-        miplearn_logger = logging.getLogger("miplearn")
-        self.prev_log_level = miplearn_logger.getEffectiveLevel()
-        miplearn_logger.setLevel(logging.WARNING)
-
-    def _restore_miplearn_logger(self) -> None:
-        miplearn_logger = logging.getLogger("miplearn")
-        miplearn_logger.setLevel(self.prev_log_level)

miplearn/classifiers/__init__.py

@@ -1,163 +1,3 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
-
-from abc import ABC, abstractmethod
-from typing import Optional
-
-import numpy as np
-
-
-class Classifier(ABC):
-    """
-    A Classifier decides which class each sample belongs to, based on historical
-    data.
-    """
-
-    def __init__(self) -> None:
-        self.n_features: Optional[int] = None
-        self.n_classes: Optional[int] = None
-
-    @abstractmethod
-    def fit(self, x_train: np.ndarray, y_train: np.ndarray) -> None:
-        """
-        Trains the classifier.
-
-        Parameters
-        ----------
-        x_train: np.ndarray
-            An array of features with shape (`n_samples`, `n_features`). Each entry
-            must be a float.
-        y_train: np.ndarray
-            An array of labels with shape (`n_samples`, `n_classes`). Each entry must be
-            a bool, and there must be exactly one True element in each row.
-        """
-        assert isinstance(x_train, np.ndarray)
-        assert isinstance(y_train, np.ndarray)
-        assert x_train.dtype in [
-            np.float16,
-            np.float32,
-            np.float64,
-        ], f"x_train.dtype shoule be float. Found {x_train.dtype} instead."
-        assert y_train.dtype == np.bool8
-        assert len(x_train.shape) == 2
-        assert len(y_train.shape) == 2
-        (n_samples_x, n_features) = x_train.shape
-        (n_samples_y, n_classes) = y_train.shape
-        assert n_samples_y == n_samples_x
-        self.n_features = n_features
-        self.n_classes = n_classes
-
-    @abstractmethod
-    def predict_proba(self, x_test: np.ndarray) -> np.ndarray:
-        """
-        Predicts the probability of each sample belonging to each class. Must be called
-        after fit.
-
-        Parameters
-        ----------
-        x_test: np.ndarray
-            An array of features with shape (`n_samples`, `n_features`). The number of
-            features in `x_test` must match the number of features in `x_train` provided
-            to `fit`.
-
-        Returns
-        -------
-        np.ndarray
-            An array of predicted probabilities with shape (`n_samples`, `n_classes`),
-            where `n_classes` is the number of columns in `y_train` provided to `fit`.
-        """
-        assert self.n_features is not None
-        assert isinstance(x_test, np.ndarray)
-        assert len(x_test.shape) == 2
-        (n_samples, n_features_x) = x_test.shape
-        assert n_features_x == self.n_features, (
-            f"Test and training data have different number of "
-            f"features: {n_features_x} != {self.n_features}"
-        )
-        return np.ndarray([])
-
-    @abstractmethod
-    def clone(self) -> "Classifier":
-        """
-        Returns an unfitted copy of this classifier with the same hyperparameters.
-        """
-        pass
-
-
-class Regressor(ABC):
-    """
-    A Regressor tries to predict the values of some continous variables, given the
-    values of other variables.
-    """
-
-    def __init__(self) -> None:
-        self.n_inputs: Optional[int] = None
-
-    @abstractmethod
-    def fit(self, x_train: np.ndarray, y_train: np.ndarray) -> None:
-        """
-        Trains the regressor.
-
-        Parameters
-        ----------
-        x_train: np.ndarray
-            An array of inputs with shape (`n_samples`, `n_inputs`). Each entry must be
-            a float.
-        y_train: np.ndarray
-            An array of outputs with shape (`n_samples`, `n_outputs`). Each entry must
-            be a float.
-        """
-        assert isinstance(x_train, np.ndarray)
-        assert isinstance(y_train, np.ndarray)
-        assert x_train.dtype in [np.float16, np.float32, np.float64]
-        assert y_train.dtype in [np.float16, np.float32, np.float64]
-        assert len(x_train.shape) == 2, (
-            f"Parameter x_train should be a square matrix. "
-            f"Found {x_train.shape} ndarray instead."
-        )
-        assert len(y_train.shape) == 2, (
-            f"Parameter y_train should be a square matrix. "
-            f"Found {y_train.shape} ndarray instead."
-        )
-        (n_samples_x, n_inputs) = x_train.shape
-        (n_samples_y, n_outputs) = y_train.shape
-        assert n_samples_y == n_samples_x
-        self.n_inputs = n_inputs
-
-    @abstractmethod
-    def predict(self, x_test: np.ndarray) -> np.ndarray:
-        """
-        Predicts the values of the output variables. Must be called after fit.
-
-        Parameters
-        ----------
-        x_test: np.ndarray
-            An array of inputs with shape (`n_samples`, `n_inputs`), where `n_inputs`
-            must match the number of columns in `x_train` provided to `fit`.
-
-        Returns
-        -------
-        np.ndarray
-            An array of outputs  with shape (`n_samples`, `n_outputs`), where
-            `n_outputs` is the number of columns in `y_train` provided to `fit`.
-        """
-        assert self.n_inputs is not None
-        assert isinstance(x_test, np.ndarray), (
-            f"Parameter x_train must be np.ndarray. "
-            f"Found {x_test.__class__.__name__} instead."
-        )
-        assert len(x_test.shape) == 2
-        (n_samples, n_inputs_x) = x_test.shape
-        assert n_inputs_x == self.n_inputs, (
-            f"Test and training data have different number of "
-            f"inputs: {n_inputs_x} != {self.n_inputs}"
-        )
-        return np.ndarray([])
-
-    @abstractmethod
-    def clone(self) -> "Regressor":
-        """
-        Returns an unfitted copy of this regressor with the same hyperparameters.
-        """
-        pass

miplearn/classifiers/adaptive.py

@@ -1,120 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from typing import Dict, Optional
-
-import numpy as np
-from sklearn.linear_model import LogisticRegression
-from sklearn.metrics import roc_auc_score
-from sklearn.neighbors import KNeighborsClassifier
-from sklearn.pipeline import make_pipeline
-from sklearn.preprocessing import StandardScaler
-
-from miplearn.classifiers import Classifier
-from miplearn.classifiers.counting import CountingClassifier
-from miplearn.classifiers.sklearn import ScikitLearnClassifier
-
-logger = logging.getLogger(__name__)
-
-
-class CandidateClassifierSpecs:
-    """
-    Specifications describing how to construct a certain classifier, and under
-    which circumstances it can be used.
-
-    Parameters
-    ----------
-    min_samples: int
-        Minimum number of samples for this classifier to be considered.
-    classifier: Callable[[], Classifier]
-        Callable that constructs the classifier.
-    """
-
-    def __init__(
-        self,
-        classifier: Classifier,
-        min_samples: int = 0,
-    ) -> None:
-        self.min_samples = min_samples
-        self.classifier = classifier
-
-
-class AdaptiveClassifier(Classifier):
-    """
-    A meta-classifier which dynamically selects what actual classifier to use
-    based on its cross-validation score on a particular training data set.
-
-    Parameters
-    ----------
-    candidates: Dict[str, CandidateClassifierSpecs]
-        A dictionary of candidate classifiers to consider, mapping the name of the
-        candidate to its specs, which describes how to construct it and under what
-        scenarios. If no candidates are provided, uses a fixed set of defaults,
-        which includes `CountingClassifier`, `KNeighborsClassifier` and
-        `LogisticRegression`.
-    """
-
-    def __init__(
-        self,
-        candidates: Optional[Dict[str, CandidateClassifierSpecs]] = None,
-    ) -> None:
-        super().__init__()
-        if candidates is None:
-            candidates = {
-                "knn(100)": CandidateClassifierSpecs(
-                    classifier=ScikitLearnClassifier(
-                        KNeighborsClassifier(n_neighbors=100)
-                    ),
-                    min_samples=100,
-                ),
-                "logistic": CandidateClassifierSpecs(
-                    classifier=ScikitLearnClassifier(
-                        make_pipeline(
-                            StandardScaler(),
-                            LogisticRegression(),
-                        )
-                    ),
-                    min_samples=30,
-                ),
-                "counting": CandidateClassifierSpecs(
-                    classifier=CountingClassifier(),
-                ),
-            }
-        self.candidates = candidates
-        self.classifier: Optional[Classifier] = None
-
-    def fit(self, x_train: np.ndarray, y_train: np.ndarray) -> None:
-        super().fit(x_train, y_train)
-        n_samples = x_train.shape[0]
-        assert y_train.shape == (n_samples, 2)
-
-        # If almost all samples belong to the same class, return a fixed prediction and
-        # skip all the other steps.
-        if y_train[:, 0].mean() > 0.999 or y_train[:, 1].mean() > 0.999:
-            self.classifier = CountingClassifier()
-            self.classifier.fit(x_train, y_train)
-            return
-
-        best_name, best_clf, best_score = None, None, -float("inf")
-        for (name, specs) in self.candidates.items():
-            if n_samples < specs.min_samples:
-                continue
-            clf = specs.classifier.clone()
-            clf.fit(x_train, y_train)
-            proba = clf.predict_proba(x_train)
-            # FIXME: Switch to k-fold cross validation
-            score = roc_auc_score(y_train[:, 1], proba[:, 1])
-            if score > best_score:
-                best_name, best_clf, best_score = name, clf, score
-        logger.debug("Best classifier: %s (score=%.3f)" % (best_name, best_score))
-        self.classifier = best_clf
-
-    def predict_proba(self, x_test: np.ndarray) -> np.ndarray:
-        super().predict_proba(x_test)
-        assert self.classifier is not None
-        return self.classifier.predict_proba(x_test)
-
-    def clone(self) -> "AdaptiveClassifier":
-        return AdaptiveClassifier(self.candidates)

miplearn/classifiers/counting.py

@@ -1,45 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-from typing import Optional, cast
-
-import numpy as np
-
-from miplearn.classifiers import Classifier
-
-
-class CountingClassifier(Classifier):
-    """
-
-    A classifier that generates constant predictions, based only on the frequency of
-    the training labels. For example, suppose `y_train` is given by:
-    ```python
-    y_train = np.array([
-        [True, False],
-        [False, True],
-        [False, True],
-    ])
-    ```
-    Then `predict_proba` always returns `[0.33 0.66]` for every sample, regardless of
-    `x_train`. It essentially counts how many times each label appeared, hence the name.
-
-    """
-
-    def __init__(self) -> None:
-        super().__init__()
-        self.mean: Optional[np.ndarray] = None
-
-    def fit(self, x_train: np.ndarray, y_train: np.ndarray) -> None:
-        super().fit(x_train, y_train)
-        self.mean = cast(np.ndarray, np.mean(y_train, axis=0))
-
-    def predict_proba(self, x_test: np.ndarray) -> np.ndarray:
-        super().predict_proba(x_test)
-        n_samples = x_test.shape[0]
-        return np.array([self.mean for _ in range(n_samples)])
-
-    def __repr__(self) -> str:
-        return "CountingClassifier(mean=%s)" % self.mean
-
-    def clone(self) -> "CountingClassifier":
-        return CountingClassifier()

miplearn/classifiers/cv.py

@@ -1,132 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from typing import Optional, List
-
-import numpy as np
-from sklearn.dummy import DummyClassifier
-from sklearn.linear_model import LogisticRegression
-from sklearn.model_selection import cross_val_score
-
-from miplearn.classifiers import Classifier
-from miplearn.classifiers.sklearn import ScikitLearnClassifier
-
-logger = logging.getLogger(__name__)
-
-
-class CrossValidatedClassifier(Classifier):
-    """
-    A meta-classifier that, upon training, evaluates the performance of another
-    candidate classifier on the training data set, using k-fold cross validation,
-    then either adopts it, if its cv-score is high enough, or returns constant
-    predictions for every x_test, otherwise.
-
-    Parameters
-    ----------
-    classifier: Callable[[], ScikitLearnClassifier]
-        A callable that constructs the candidate classifier.
-    threshold: float
-        Number from zero to one indicating how well must the candidate classifier
-        perform to be adopted. The threshold is specified in comparison to a dummy
-        classifier trained on the same dataset. For example, a threshold of 0.0
-        indicates that any classifier as good as the dummy predictor is acceptable. A
-        threshold of 1.0 indicates that only classifiers with perfect
-        cross-validation scores are acceptable. Other numbers are a linear
-        interpolation of these two extremes.
-    constant: Optional[List[bool]]
-        If the candidate classifier fails to meet the threshold, use a dummy classifier
-        which always returns this prediction instead. The list should have exactly as
-        many elements as the number of columns of `x_train` provided to `fit`.
-    cv: int
-        Number of folds.
-    scoring: str
-        Scoring function.
-    """
-
-    def __init__(
-        self,
-        classifier: ScikitLearnClassifier = ScikitLearnClassifier(LogisticRegression()),
-        threshold: float = 0.75,
-        constant: Optional[List[bool]] = None,
-        cv: int = 5,
-        scoring: str = "accuracy",
-    ):
-        super().__init__()
-        if constant is None:
-            constant = [True, False]
-        self.n_classes = len(constant)
-        self.classifier: Optional[ScikitLearnClassifier] = None
-        self.classifier_prototype = classifier
-        self.constant: List[bool] = constant
-        self.threshold = threshold
-        self.cv = cv
-        self.scoring = scoring
-
-    def fit(self, x_train: np.ndarray, y_train: np.ndarray) -> None:
-        super().fit(x_train, y_train)
-        (n_samples, n_classes) = x_train.shape
-        assert n_classes == self.n_classes
-
-        # Calculate dummy score and absolute score threshold
-        y_train_avg = np.average(y_train)
-        dummy_score = max(y_train_avg, 1 - y_train_avg)
-        absolute_threshold = 1.0 * self.threshold + dummy_score * (1 - self.threshold)
-
-        # Calculate cross validation score and decide which classifier to use
-        clf = self.classifier_prototype.clone()
-        assert clf is not None
-        assert isinstance(clf, ScikitLearnClassifier), (
-            f"The provided classifier callable must return a ScikitLearnClassifier. "
-            f"Found {clf.__class__.__name__} instead. If this is a scikit-learn "
-            f"classifier, you must wrap it with ScikitLearnClassifier."
-        )
-
-        cv_score = float(
-            np.mean(
-                cross_val_score(
-                    clf.inner_clf,
-                    x_train,
-                    y_train[:, 1],
-                    cv=self.cv,
-                    scoring=self.scoring,
-                )
-            )
-        )
-        if cv_score >= absolute_threshold:
-            logger.debug(
-                "cv_score is above threshold (%.2f >= %.2f); keeping"
-                % (cv_score, absolute_threshold)
-            )
-            self.classifier = clf
-        else:
-            logger.debug(
-                "cv_score is below threshold (%.2f < %.2f); discarding"
-                % (cv_score, absolute_threshold)
-            )
-            self.classifier = ScikitLearnClassifier(
-                DummyClassifier(
-                    strategy="constant",
-                    constant=self.constant[1],
-                )
-            )
-
-        # Train chosen classifier
-        assert self.classifier is not None
-        assert isinstance(self.classifier, ScikitLearnClassifier)
-        self.classifier.fit(x_train, y_train)
-
-    def predict_proba(self, x_test: np.ndarray) -> np.ndarray:
-        super().predict_proba(x_test)
-        assert self.classifier is not None
-        return self.classifier.predict_proba(x_test)
-
-    def clone(self) -> "CrossValidatedClassifier":
-        return CrossValidatedClassifier(
-            classifier=self.classifier_prototype,
-            threshold=self.threshold,
-            constant=self.constant,
-            cv=self.cv,
-            scoring=self.scoring,
-        )

miplearn/classifiers/minprob.py

@@ -0,0 +1,61 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+from typing import List, Any, Callable, Optional
+
+import numpy as np
+import sklearn
+from sklearn.base import BaseEstimator
+from sklearn.utils.multiclass import unique_labels
+
+
+class MinProbabilityClassifier(BaseEstimator):
+    """
+    Meta-classifier that returns NaN for predictions made by a base classifier that
+    have probability below a given threshold. More specifically, this meta-classifier
+    calls base_clf.predict_proba and compares the result against the provided
+    thresholds. If the probability for one of the classes is above its threshold,
+    the meta-classifier returns that prediction. Otherwise, it returns NaN.
+    """
+
+    def __init__(
+        self,
+        base_clf: Any,
+        thresholds: List[float],
+        clone_fn: Callable[[Any], Any] = sklearn.base.clone,
+    ) -> None:
+        assert len(thresholds) == 2
+        self.base_clf = base_clf
+        self.thresholds = thresholds
+        self.clone_fn = clone_fn
+        self.clf_: Optional[Any] = None
+        self.classes_: Optional[List[Any]] = None
+
+    def fit(self, x: np.ndarray, y: np.ndarray) -> None:
+        assert len(y.shape) == 1
+        assert len(x.shape) == 2
+        classes = unique_labels(y)
+        assert len(classes) == len(self.thresholds)
+
+        self.clf_ = self.clone_fn(self.base_clf)
+        self.clf_.fit(x, y)
+        self.classes_ = self.clf_.classes_
+
+    def predict(self, x: np.ndarray) -> np.ndarray:
+        assert self.clf_ is not None
+        assert self.classes_ is not None
+
+        y_proba = self.clf_.predict_proba(x)
+        assert len(y_proba.shape) == 2
+        assert y_proba.shape[0] == x.shape[0]
+        assert y_proba.shape[1] == 2
+        n_samples = x.shape[0]
+
+        y_pred = []
+        for sample_idx in range(n_samples):
+            yi = float("nan")
+            for (class_idx, class_val) in enumerate(self.classes_):
+                if y_proba[sample_idx, class_idx] >= self.thresholds[class_idx]:
+                    yi = class_val
+            y_pred.append(yi)
+        return np.array(y_pred)

miplearn/classifiers/singleclass.py

@@ -0,0 +1,51 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+from typing import Callable, Optional
+
+import numpy as np
+import sklearn.base
+from sklearn.base import BaseEstimator
+from sklearn.utils.multiclass import unique_labels
+
+
+class SingleClassFix(BaseEstimator):
+    """
+    Some sklearn classifiers, such as logistic regression, have issues with datasets
+    that contain a single class. This meta-classifier fixes the issue. If the
+    training data contains a single class, this meta-classifier always returns that
+    class as a prediction. Otherwise, it fits the provided base classifier,
+    and returns its predictions instead.
+    """
+
+    def __init__(
+        self,
+        base_clf: BaseEstimator,
+        clone_fn: Callable = sklearn.base.clone,
+    ):
+        self.base_clf = base_clf
+        self.clf_: Optional[BaseEstimator] = None
+        self.constant_ = None
+        self.classes_ = None
+        self.clone_fn = clone_fn
+
+    def fit(self, x: np.ndarray, y: np.ndarray) -> None:
+        classes = unique_labels(y)
+        if len(classes) == 1:
+            assert classes[0] is not None
+            self.clf_ = None
+            self.constant_ = classes[0]
+            self.classes_ = classes
+        else:
+            self.clf_ = self.clone_fn(self.base_clf)
+            assert self.clf_ is not None
+            self.clf_.fit(x, y)
+            self.constant_ = None
+            self.classes_ = self.clf_.classes_
+
+    def predict(self, x: np.ndarray) -> np.ndarray:
+        if self.constant_ is not None:
+            return np.full(x.shape[0], self.constant_)
+        else:
+            assert self.clf_ is not None
+            return self.clf_.predict(x)

miplearn/classifiers/sklearn.py

@@ -1,93 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-from typing import Optional, Any, cast
-
-import numpy as np
-import sklearn
-
-from miplearn.classifiers import Classifier, Regressor
-
-
-class ScikitLearnClassifier(Classifier):
-    """
-    Wrapper for ScikitLearn classifiers, which makes sure inputs and outputs have the
-    correct dimensions and types.
-    """
-
-    def __init__(self, clf: Any) -> None:
-        super().__init__()
-        self.inner_clf = clf
-        self.constant: Optional[np.ndarray] = None
-
-    def fit(self, x_train: np.ndarray, y_train: np.ndarray) -> None:
-        super().fit(x_train, y_train)
-        (n_samples, n_classes) = y_train.shape
-        assert n_classes == 2, (
-            f"Scikit-learn classifiers must have exactly two classes. "
-            f"{n_classes} classes were provided instead."
-        )
-
-        # When all samples belong to the same class, sklearn's predict_proba returns
-        # an array with a single column. The following check avoid this strange
-        # behavior.
-        mean = cast(np.ndarray, y_train.astype(float).mean(axis=0))
-        if mean.max() == 1.0:
-            self.constant = mean
-            return
-
-        self.inner_clf.fit(x_train, y_train[:, 1])
-
-    def predict_proba(self, x_test: np.ndarray) -> np.ndarray:
-        super().predict_proba(x_test)
-        n_samples = x_test.shape[0]
-        if self.constant is not None:
-            return np.array([self.constant for n in range(n_samples)])
-        sklearn_proba = self.inner_clf.predict_proba(x_test)
-        if isinstance(sklearn_proba, list):
-            assert len(sklearn_proba) == self.n_classes
-            for pb in sklearn_proba:
-                assert isinstance(pb, np.ndarray)
-                assert pb.dtype in [np.float16, np.float32, np.float64]
-                assert pb.shape == (n_samples, 2)
-            proba = np.hstack([pb[:, [1]] for pb in sklearn_proba])
-            assert proba.shape == (n_samples, self.n_classes)
-            return proba
-        else:
-            assert isinstance(sklearn_proba, np.ndarray)
-            assert sklearn_proba.shape == (n_samples, 2)
-            return sklearn_proba
-
-    def clone(self) -> "ScikitLearnClassifier":
-        return ScikitLearnClassifier(
-            clf=sklearn.base.clone(self.inner_clf),
-        )
-
-
-class ScikitLearnRegressor(Regressor):
-    """
-    Wrapper for ScikitLearn regressors, which makes sure inputs and outputs have the
-    correct dimensions and types.
-    """
-
-    def __init__(self, reg: Any) -> None:
-        super().__init__()
-        self.inner_reg = reg
-
-    def fit(self, x_train: np.ndarray, y_train: np.ndarray) -> None:
-        super().fit(x_train, y_train)
-        self.inner_reg.fit(x_train, y_train)
-
-    def predict(self, x_test: np.ndarray) -> np.ndarray:
-        super().predict(x_test)
-        n_samples = x_test.shape[0]
-        sklearn_pred = self.inner_reg.predict(x_test)
-        assert isinstance(sklearn_pred, np.ndarray)
-        assert sklearn_pred.shape[0] == n_samples
-        return sklearn_pred
-
-    def clone(self) -> "ScikitLearnRegressor":
-        return ScikitLearnRegressor(
-            reg=sklearn.base.clone(self.inner_reg),
-        )

miplearn/classifiers/threshold.py

@@ -1,128 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-from abc import abstractmethod, ABC
-from typing import Optional, List
-
-import numpy as np
-from sklearn.metrics._ranking import _binary_clf_curve
-
-from miplearn.classifiers import Classifier
-
-
-class Threshold(ABC):
-    """
-    Solver components ask the machine learning models how confident are they on each
-    prediction they make, then automatically discard all predictions that have low
-    confidence. A Threshold specifies how confident should the ML models be for a
-    prediction to be considered trustworthy.
-
-    To model dynamic thresholds, which automatically adjust themselves during
-    training to reach some desired target (such as minimum precision, or minimum
-    recall), thresholds behave somewhat similar to ML models themselves, with `fit`
-    and `predict` methods.
-    """
-
-    @abstractmethod
-    def fit(
-        self,
-        clf: Classifier,
-        x_train: np.ndarray,
-        y_train: np.ndarray,
-    ) -> None:
-        """
-        Given a trained binary classifier `clf`, calibrates itself based on the
-        classifier's performance on the given training data set.
-        """
-        assert isinstance(clf, Classifier)
-        assert isinstance(x_train, np.ndarray)
-        assert isinstance(y_train, np.ndarray)
-        n_samples = x_train.shape[0]
-        assert y_train.shape[0] == n_samples
-
-    @abstractmethod
-    def predict(self, x_test: np.ndarray) -> List[float]:
-        """
-        Returns the minimum probability for a machine learning prediction to be
-        considered trustworthy. There is one value for each label.
-        """
-        pass
-
-    @abstractmethod
-    def clone(self) -> "Threshold":
-        """
-        Returns an unfitted copy of this threshold with the same hyperparameters.
-        """
-        pass
-
-
-class MinProbabilityThreshold(Threshold):
-    """
-    A threshold which considers predictions trustworthy if their probability of being
-    correct, as computed by the machine learning models, are above a fixed value.
-    """
-
-    def __init__(self, min_probability: List[float]):
-        self.min_probability = min_probability
-
-    def fit(self, clf: Classifier, x_train: np.ndarray, y_train: np.ndarray) -> None:
-        pass
-
-    def predict(self, x_test: np.ndarray) -> List[float]:
-        return self.min_probability
-
-    def clone(self) -> "MinProbabilityThreshold":
-        return MinProbabilityThreshold(self.min_probability)
-
-
-class MinPrecisionThreshold(Threshold):
-    """
-    A dynamic threshold which automatically adjusts itself during training to ensure
-    that the component achieves at least a given precision `p` on the training data
-    set. Note that increasing a component's minimum precision may reduce its recall.
-    """
-
-    def __init__(self, min_precision: List[float]) -> None:
-        self.min_precision = min_precision
-        self._computed_threshold: Optional[List[float]] = None
-
-    def fit(
-        self,
-        clf: Classifier,
-        x_train: np.ndarray,
-        y_train: np.ndarray,
-    ) -> None:
-        super().fit(clf, x_train, y_train)
-        (n_samples, n_classes) = y_train.shape
-        proba = clf.predict_proba(x_train)
-        self._computed_threshold = [
-            self._compute(
-                y_train[:, i],
-                proba[:, i],
-                self.min_precision[i],
-            )
-            for i in range(n_classes)
-        ]
-
-    def predict(self, x_test: np.ndarray) -> List[float]:
-        assert self._computed_threshold is not None
-        return self._computed_threshold
-
-    @staticmethod
-    def _compute(
-        y_actual: np.ndarray,
-        y_prob: np.ndarray,
-        min_precision: float,
-    ) -> float:
-        fps, tps, thresholds = _binary_clf_curve(y_actual, y_prob)
-        precision = tps / (tps + fps)
-        for k in reversed(range(len(precision))):
-            if precision[k] >= min_precision:
-                return thresholds[k]
-        return float("inf")
-
-    def clone(self) -> "MinPrecisionThreshold":
-        return MinPrecisionThreshold(
-            min_precision=self.min_precision,
-        )

miplearn/collectors/basic.py

@@ -0,0 +1,86 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+import json
+import os
+from io import StringIO
+from os.path import exists
+from typing import Callable, List
+
+from ..h5 import H5File
+from ..io import _RedirectOutput, gzip, _to_h5_filename
+from ..parallel import p_umap
+
+
+class BasicCollector:
+    def collect(
+        self,
+        filenames: List[str],
+        build_model: Callable,
+        n_jobs: int = 1,
+        progress: bool = False,
+    ) -> None:
+        def _collect(data_filename):
+            h5_filename = _to_h5_filename(data_filename)
+            mps_filename = h5_filename.replace(".h5", ".mps")
+
+            if exists(h5_filename):
+                # Try to read optimal solution
+                mip_var_values = None
+                try:
+                    with H5File(h5_filename, "r") as h5:
+                        mip_var_values = h5.get_array("mip_var_values")
+                except:
+                    pass
+
+                if mip_var_values is None:
+                    print(f"Removing empty/corrupted h5 file: {h5_filename}")
+                    os.remove(h5_filename)
+                else:
+                    return
+
+            with H5File(h5_filename, "w") as h5:
+                streams = [StringIO()]
+                with _RedirectOutput(streams):
+                    # Load and extract static features
+                    model = build_model(data_filename)
+                    model.extract_after_load(h5)
+
+                    # Solve LP relaxation
+                    relaxed = model.relax()
+                    relaxed.optimize()
+                    relaxed.extract_after_lp(h5)
+
+                    # Solve MIP
+                    model.optimize()
+                    model.extract_after_mip(h5)
+
+                    # Add lazy constraints to model
+                    if (
+                        hasattr(model, "fix_violations")
+                        and model.fix_violations is not None
+                    ):
+                        model.fix_violations(model, model.violations_, "aot")
+                        h5.put_scalar(
+                            "mip_constr_violations", json.dumps(model.violations_)
+                        )
+
+                    # Save MPS file
+                    model.write(mps_filename)
+                    gzip(mps_filename)
+
+                h5.put_scalar("mip_log", streams[0].getvalue())
+
+        if n_jobs > 1:
+            p_umap(
+                _collect,
+                filenames,
+                num_cpus=n_jobs,
+                desc="collect",
+                smoothing=0,
+                disable=not progress,
+            )
+        else:
+            for filename in filenames:
+                _collect(filename)

miplearn/collectors/lazy.py

@@ -0,0 +1,117 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from io import StringIO
+from typing import Callable
+
+import gurobipy as gp
+import numpy as np
+from gurobipy import GRB, LinExpr
+
+from ..h5 import H5File
+from ..io import _RedirectOutput
+
+
+class LazyCollector:
+    def __init__(
+        self,
+        min_constrs: int = 100_000,
+        time_limit: float = 900,
+    ) -> None:
+        self.min_constrs = min_constrs
+        self.time_limit = time_limit
+
+    def collect(
+        self, data_filename: str, build_model: Callable, tol: float = 1e-6
+    ) -> None:
+        h5_filename = f"{data_filename}.h5"
+        with H5File(h5_filename, "r+") as h5:
+            streams = [StringIO()]
+            lazy = None
+            with _RedirectOutput(streams):
+                slacks = h5.get_array("mip_constr_slacks")
+                assert slacks is not None
+
+                # Check minimum problem size
+                if len(slacks) < self.min_constrs:
+                    print("Problem is too small. Skipping.")
+                    h5.put_array("mip_constr_lazy", np.zeros(len(slacks)))
+                    return
+
+                # Load model
+                print("Loading model...")
+                model = build_model(data_filename)
+                model.params.LazyConstraints = True
+                model.params.timeLimit = self.time_limit
+                gp_constrs = np.array(model.getConstrs())
+                gp_vars = np.array(model.getVars())
+
+                # Load constraints
+                lhs = h5.get_sparse("static_constr_lhs")
+                rhs = h5.get_array("static_constr_rhs")
+                sense = h5.get_array("static_constr_sense")
+                assert lhs is not None
+                assert rhs is not None
+                assert sense is not None
+                lhs_csr = lhs.tocsr()
+                lhs_csc = lhs.tocsc()
+                constr_idx = np.array(range(len(rhs)))
+                lazy = np.zeros(len(rhs))
+
+                # Drop loose constraints
+                selected = (slacks > 0) & ((sense == b"<") | (sense == b">"))
+                loose_constrs = gp_constrs[selected]
+                print(
+                    f"Removing {len(loose_constrs):,d} constraints (out of {len(rhs):,d})..."
+                )
+                model.remove(list(loose_constrs))
+
+                # Filter to constraints that were dropped
+                lhs_csr = lhs_csr[selected, :]
+                lhs_csc = lhs_csc[selected, :]
+                rhs = rhs[selected]
+                sense = sense[selected]
+                constr_idx = constr_idx[selected]
+                lazy[selected] = 1
+
+                # Load warm start
+                var_names = h5.get_array("static_var_names")
+                var_values = h5.get_array("mip_var_values")
+                assert var_values is not None
+                assert var_names is not None
+                for (var_idx, var_name) in enumerate(var_names):
+                    var = model.getVarByName(var_name.decode())
+                    var.start = var_values[var_idx]
+
+                print("Solving MIP with lazy constraints callback...")
+
+                def callback(model: gp.Model, where: int) -> None:
+                    assert rhs is not None
+                    assert lazy is not None
+                    assert sense is not None
+
+                    if where == GRB.Callback.MIPSOL:
+                        x_val = np.array(model.cbGetSolution(model.getVars()))
+                        slack = lhs_csc * x_val - rhs
+                        slack[sense == b">"] *= -1
+                        is_violated = slack > tol
+
+                        for (j, rhs_j) in enumerate(rhs):
+                            if is_violated[j]:
+                                lazy[constr_idx[j]] = 0
+                                expr = LinExpr(
+                                    lhs_csr[j, :].data, gp_vars[lhs_csr[j, :].indices]
+                                )
+                                if sense[j] == b"<":
+                                    model.cbLazy(expr <= rhs_j)
+                                elif sense[j] == b">":
+                                    model.cbLazy(expr >= rhs_j)
+                                else:
+                                    raise RuntimeError(f"Unknown sense: {sense[j]}")
+
+                model.optimize(callback)
+                print(f"Marking {lazy.sum():,.0f} constraints as lazy...")
+
+            h5.put_array("mip_constr_lazy", lazy)
+            h5.put_scalar("mip_constr_lazy_log", streams[0].getvalue())

miplearn/collectors/priority.py

@@ -0,0 +1,49 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+import os
+import subprocess
+from typing import Callable
+
+from ..h5 import H5File
+
+
+class BranchPriorityCollector:
+    def __init__(
+        self,
+        time_limit: float = 900.0,
+        print_interval: int = 1,
+        node_limit: int = 500,
+    ) -> None:
+        self.time_limit = time_limit
+        self.print_interval = print_interval
+        self.node_limit = node_limit
+
+    def collect(self, data_filename: str, _: Callable) -> None:
+        basename = data_filename.replace(".pkl.gz", "")
+        env = os.environ.copy()
+        env["JULIA_NUM_THREADS"] = "1"
+        ret = subprocess.run(
+            [
+                "julia",
+                "--project=.",
+                "-e",
+                (
+                    f"using CPLEX, JuMP, MIPLearn.BB; "
+                    f"BB.solve!("
+                    f'    optimizer_with_attributes(CPLEX.Optimizer, "CPXPARAM_Threads" => 1),'
+                    f'    "{basename}",'
+                    f"    print_interval={self.print_interval},"
+                    f"    time_limit={self.time_limit:.2f},"
+                    f"    node_limit={self.node_limit},"
+                    f")"
+                ),
+            ],
+            check=True,
+            capture_output=True,
+            env=env,
+        )
+        h5_filename = f"{basename}.h5"
+        with H5File(h5_filename, "r+") as h5:
+            h5.put_scalar("bb_log", ret.stdout)

miplearn/components/__init__.py

@@ -1,47 +1,3 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
-from typing import Dict
-
-
-def classifier_evaluation_dict(
-    tp: int,
-    tn: int,
-    fp: int,
-    fn: int,
-) -> Dict[str, float]:
-    p = tp + fn
-    n = fp + tn
-    d: Dict = {
-        "Predicted positive": fp + tp,
-        "Predicted negative": fn + tn,
-        "Condition positive": p,
-        "Condition negative": n,
-        "True positive": tp,
-        "True negative": tn,
-        "False positive": fp,
-        "False negative": fn,
-        "Accuracy": (tp + tn) / (p + n),
-        "F1 score": (2 * tp) / (2 * tp + fp + fn),
-    }
-
-    if p > 0:
-        d["Recall"] = tp / p
-    else:
-        d["Recall"] = 1.0
-
-    if tp + fp > 0:
-        d["Precision"] = tp / (tp + fp)
-    else:
-        d["Precision"] = 1.0
-
-    t = (p + n) / 100.0
-    d["Predicted positive (%)"] = d["Predicted positive"] / t
-    d["Predicted negative (%)"] = d["Predicted negative"] / t
-    d["Condition positive (%)"] = d["Condition positive"] / t
-    d["Condition negative (%)"] = d["Condition negative"] / t
-    d["True positive (%)"] = d["True positive"] / t
-    d["True negative (%)"] = d["True negative"] / t
-    d["False positive (%)"] = d["False positive"] / t
-    d["False negative (%)"] = d["False negative"] / t
-    return d

miplearn/components/component.py

@@ -1,254 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-from typing import Any, List, TYPE_CHECKING, Tuple, Dict, Optional
-
-import numpy as np
-from p_tqdm import p_umap
-
-from miplearn.features.sample import Sample
-from miplearn.instance.base import Instance
-from miplearn.types import LearningSolveStats, Category
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import LearningSolver
-
-
-# noinspection PyMethodMayBeStatic
-class Component:
-    """
-    A Component is an object which adds functionality to a LearningSolver.
-
-    For better code maintainability, LearningSolver simply delegates most of its
-    functionality to Components. Each Component is responsible for exactly one ML
-    strategy.
-    """
-
-    def after_solve_lp(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        """
-        Method called by LearningSolver after the root LP relaxation is solved.
-        See before_solve_lp for a description of the parameters.
-        """
-        return
-
-    def after_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        """
-        Method called by LearningSolver after the MIP is solved.
-        See before_solve_lp for a description of the parameters.
-        """
-        return
-
-    def before_solve_lp(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        """
-        Method called by LearningSolver before the root LP relaxation is solved.
-
-        Parameters
-        ----------
-        solver: LearningSolver
-            The solver calling this method.
-        instance: Instance
-            The instance being solved.
-        model
-            The concrete optimization model being solved.
-        stats: LearningSolveStats
-            A dictionary containing statistics about the solution process, such as
-            number of nodes explored and running time. Components are free to add
-            their own statistics here. For example, PrimalSolutionComponent adds
-            statistics regarding the number of predicted variables. All statistics in
-            this dictionary are exported to the benchmark CSV file.
-        sample: miplearn.features.Sample
-            An object containing data that may be useful for training machine
-            learning models and accelerating the solution process. Components are
-            free to add their own training data here.
-        """
-        return
-
-    def before_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        """
-        Method called by LearningSolver before the MIP is solved.
-        See before_solve_lp for a description of the parameters.
-        """
-        return
-
-    def fit_xy(
-        self,
-        x: Dict[Category, np.ndarray],
-        y: Dict[Category, np.ndarray],
-    ) -> None:
-        """
-        Given two dictionaries x and y, mapping the name of the category to matrices
-        of features and targets, this function does two things. First, for each
-        category, it creates a clone of the prototype regressor/classifier. Second,
-        it passes (x[category], y[category]) to the clone's fit method.
-        """
-        return
-
-    def iteration_cb(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-    ) -> bool:
-        """
-        Method called by LearningSolver at the end of each iteration.
-
-        After solving the MIP, LearningSolver calls `iteration_cb` of each component,
-        giving them a chance to modify the problem and resolve it before the solution
-        process ends. For example, the lazy constraint component uses `iteration_cb`
-        to check that all lazy constraints are satisfied.
-
-        If `iteration_cb` returns False for all components, the solution process
-        ends. If it retunrs True for any component, the MIP is solved again.
-
-        Parameters
-        ----------
-        solver: LearningSolver
-            The solver calling this method.
-        instance: Instance
-            The instance being solved.
-        model: Any
-            The concrete optimization model being solved.
-        """
-        return False
-
-    def lazy_cb(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-    ) -> None:
-        return
-
-    def sample_evaluate(
-        self,
-        instance: Optional[Instance],
-        sample: Sample,
-    ) -> Dict[str, Dict[str, float]]:
-        return {}
-
-    def sample_xy(
-        self,
-        instance: Optional[Instance],
-        sample: Sample,
-    ) -> Tuple[Dict, Dict]:
-        """
-        Returns a pair of x and y dictionaries containing, respectively, the matrices
-        of ML features and the labels for the sample. If the training sample does not
-        include label information, returns (x, {}).
-        """
-        pass
-
-    def pre_fit(self, pre: List[Any]) -> None:
-        pass
-
-    def user_cut_cb(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-    ) -> None:
-        return
-
-    def pre_sample_xy(self, instance: Instance, sample: Sample) -> Any:
-        pass
-
-    @staticmethod
-    def fit_multiple(
-        components: List["Component"],
-        instances: List[Instance],
-        n_jobs: int = 1,
-    ) -> None:
-
-        # Part I: Pre-fit
-        def _pre_sample_xy(instance: Instance) -> Dict:
-            pre_instance: Dict = {}
-            for (cidx, comp) in enumerate(components):
-                pre_instance[cidx] = []
-            instance.load()
-            for sample in instance.get_samples():
-                for (cidx, comp) in enumerate(components):
-                    pre_instance[cidx].append(comp.pre_sample_xy(instance, sample))
-            instance.free()
-            return pre_instance
-
-        if n_jobs == 1:
-            pre = [_pre_sample_xy(instance) for instance in instances]
-        else:
-            pre = p_umap(_pre_sample_xy, instances, num_cpus=n_jobs)
-        pre_combined: Dict = {}
-        for (cidx, comp) in enumerate(components):
-            pre_combined[cidx] = []
-            for p in pre:
-                pre_combined[cidx].extend(p[cidx])
-        for (cidx, comp) in enumerate(components):
-            comp.pre_fit(pre_combined[cidx])
-
-        # Part II: Fit
-        def _sample_xy(instance: Instance) -> Tuple[Dict, Dict]:
-            x_instance: Dict = {}
-            y_instance: Dict = {}
-            for (cidx, comp) in enumerate(components):
-                x_instance[cidx] = {}
-                y_instance[cidx] = {}
-            instance.load()
-            for sample in instance.get_samples():
-                for (cidx, comp) in enumerate(components):
-                    x = x_instance[cidx]
-                    y = y_instance[cidx]
-                    x_sample, y_sample = comp.sample_xy(instance, sample)
-                    for cat in x_sample.keys():
-                        if cat not in x:
-                            x[cat] = []
-                            y[cat] = []
-                        x[cat] += x_sample[cat]
-                        y[cat] += y_sample[cat]
-            instance.free()
-            return x_instance, y_instance
-
-        if n_jobs == 1:
-            xy = [_sample_xy(instance) for instance in instances]
-        else:
-            xy = p_umap(_sample_xy, instances)
-        for (cidx, comp) in enumerate(components):
-            x_comp: Dict = {}
-            y_comp: Dict = {}
-            for (x, y) in xy:
-                for cat in x[cidx].keys():
-                    if cat not in x_comp:
-                        x_comp[cat] = []
-                        y_comp[cat] = []
-                    x_comp[cat].extend(x[cidx][cat])
-                    y_comp[cat].extend(y[cidx][cat])
-            for cat in x_comp.keys():
-                x_comp[cat] = np.array(x_comp[cat], dtype=np.float32)
-                y_comp[cat] = np.array(y_comp[cat])
-            comp.fit_xy(x_comp, y_comp)

miplearn/components/dynamic_common.py

@@ -1,171 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from typing import Dict, List, Tuple, Optional, Any, Set
-
-import numpy as np
-from overrides import overrides
-
-from miplearn.features.extractor import FeaturesExtractor
-from miplearn.classifiers import Classifier
-from miplearn.classifiers.threshold import Threshold
-from miplearn.components import classifier_evaluation_dict
-from miplearn.components.component import Component
-from miplearn.features.sample import Sample
-from miplearn.instance.base import Instance
-from miplearn.types import ConstraintCategory, ConstraintName
-
-logger = logging.getLogger(__name__)
-
-
-class DynamicConstraintsComponent(Component):
-    """
-    Base component used by both DynamicLazyConstraintsComponent and UserCutsComponent.
-    """
-
-    def __init__(
-        self,
-        attr: str,
-        classifier: Classifier,
-        threshold: Threshold,
-    ):
-        assert isinstance(classifier, Classifier)
-        self.threshold_prototype: Threshold = threshold
-        self.classifier_prototype: Classifier = classifier
-        self.classifiers: Dict[ConstraintCategory, Classifier] = {}
-        self.thresholds: Dict[ConstraintCategory, Threshold] = {}
-        self.known_cids: List[ConstraintName] = []
-        self.attr = attr
-
-    def sample_xy_with_cids(
-        self,
-        instance: Optional[Instance],
-        sample: Sample,
-    ) -> Tuple[
-        Dict[ConstraintCategory, List[List[float]]],
-        Dict[ConstraintCategory, List[List[bool]]],
-        Dict[ConstraintCategory, List[ConstraintName]],
-    ]:
-        if len(self.known_cids) == 0:
-            return {}, {}, {}
-        assert instance is not None
-        x: Dict[ConstraintCategory, List[List[float]]] = {}
-        y: Dict[ConstraintCategory, List[List[bool]]] = {}
-        cids: Dict[ConstraintCategory, List[ConstraintName]] = {}
-        known_cids = np.array(self.known_cids, dtype="S")
-
-        enforced_cids = None
-        enforced_cids_np = sample.get_array(self.attr)
-        if enforced_cids_np is not None:
-            enforced_cids = list(enforced_cids_np)
-
-        # Get user-provided constraint features
-        (
-            constr_features,
-            constr_categories,
-            constr_lazy,
-        ) = FeaturesExtractor._extract_user_features_constrs(instance, known_cids)
-
-        # Augment with instance features
-        instance_features = sample.get_array("static_instance_features")
-        assert instance_features is not None
-        constr_features = np.hstack(
-            [
-                instance_features.reshape(1, -1).repeat(len(known_cids), axis=0),
-                constr_features,
-            ]
-        )
-
-        categories = np.unique(constr_categories)
-        for c in categories:
-            x[c] = constr_features[constr_categories == c].tolist()
-            cids[c] = known_cids[constr_categories == c].tolist()
-            if enforced_cids is not None:
-                tmp = np.isin(cids[c], enforced_cids).reshape(-1, 1)
-                y[c] = np.hstack([~tmp, tmp]).tolist()  # type: ignore
-
-        return x, y, cids
-
-    @overrides
-    def sample_xy(
-        self,
-        instance: Optional[Instance],
-        sample: Sample,
-    ) -> Tuple[Dict, Dict]:
-        x, y, _ = self.sample_xy_with_cids(instance, sample)
-        return x, y
-
-    @overrides
-    def pre_fit(self, pre: List[Any]) -> None:
-        assert pre is not None
-        known_cids: Set = set()
-        for cids in pre:
-            known_cids |= set(list(cids))
-        self.known_cids.clear()
-        self.known_cids.extend(sorted(known_cids))
-
-    def sample_predict(
-        self,
-        instance: Instance,
-        sample: Sample,
-    ) -> List[ConstraintName]:
-        pred: List[ConstraintName] = []
-        if len(self.known_cids) == 0:
-            logger.info("Classifiers not fitted. Skipping.")
-            return pred
-        x, _, cids = self.sample_xy_with_cids(instance, sample)
-        for category in x.keys():
-            assert category in self.classifiers
-            assert category in self.thresholds
-            clf = self.classifiers[category]
-            thr = self.thresholds[category]
-            nx = np.array(x[category])
-            proba = clf.predict_proba(nx)
-            t = thr.predict(nx)
-            for i in range(proba.shape[0]):
-                if proba[i][1] > t[1]:
-                    pred += [cids[category][i]]
-        return pred
-
-    @overrides
-    def pre_sample_xy(self, instance: Instance, sample: Sample) -> Any:
-        return sample.get_array(self.attr)
-
-    @overrides
-    def fit_xy(
-        self,
-        x: Dict[ConstraintCategory, np.ndarray],
-        y: Dict[ConstraintCategory, np.ndarray],
-    ) -> None:
-        for category in x.keys():
-            self.classifiers[category] = self.classifier_prototype.clone()
-            self.thresholds[category] = self.threshold_prototype.clone()
-            npx = np.array(x[category])
-            npy = np.array(y[category])
-            self.classifiers[category].fit(npx, npy)
-            self.thresholds[category].fit(self.classifiers[category], npx, npy)
-
-    @overrides
-    def sample_evaluate(
-        self,
-        instance: Instance,
-        sample: Sample,
-    ) -> Dict[str, float]:
-        actual = sample.get_array(self.attr)
-        assert actual is not None
-        pred = set(self.sample_predict(instance, sample))
-        tp, tn, fp, fn = 0, 0, 0, 0
-        for cid in self.known_cids:
-            if cid in pred:
-                if cid in actual:
-                    tp += 1
-                else:
-                    fp += 1
-            else:
-                if cid in actual:
-                    fn += 1
-                else:
-                    tn += 1
-        return classifier_evaluation_dict(tp=tp, tn=tn, fp=fp, fn=fn)

miplearn/components/dynamic_lazy.py

@@ -1,145 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-import pdb
-from typing import Dict, List, TYPE_CHECKING, Tuple, Any, Optional, Set
-
-import numpy as np
-from overrides import overrides
-
-from miplearn.classifiers import Classifier
-from miplearn.classifiers.counting import CountingClassifier
-from miplearn.classifiers.threshold import MinProbabilityThreshold, Threshold
-from miplearn.components.component import Component
-from miplearn.components.dynamic_common import DynamicConstraintsComponent
-from miplearn.features.sample import Sample
-from miplearn.instance.base import Instance
-from miplearn.types import LearningSolveStats, ConstraintName, ConstraintCategory
-
-logger = logging.getLogger(__name__)
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import LearningSolver
-
-
-class DynamicLazyConstraintsComponent(Component):
-    """
-    A component that predicts which lazy constraints to enforce.
-    """
-
-    def __init__(
-        self,
-        classifier: Classifier = CountingClassifier(),
-        threshold: Threshold = MinProbabilityThreshold([0, 0.05]),
-    ):
-        self.dynamic: DynamicConstraintsComponent = DynamicConstraintsComponent(
-            classifier=classifier,
-            threshold=threshold,
-            attr="mip_constr_lazy_enforced",
-        )
-        self.classifiers = self.dynamic.classifiers
-        self.thresholds = self.dynamic.thresholds
-        self.known_cids = self.dynamic.known_cids
-        self.lazy_enforced: Set[ConstraintName] = set()
-
-    @staticmethod
-    def enforce(
-        cids: List[ConstraintName],
-        instance: Instance,
-        model: Any,
-        solver: "LearningSolver",
-    ) -> None:
-        assert solver.internal_solver is not None
-        for cid in cids:
-            instance.enforce_lazy_constraint(solver.internal_solver, model, cid)
-
-    @overrides
-    def before_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        self.lazy_enforced.clear()
-        logger.info("Predicting violated (dynamic) lazy constraints...")
-        cids = self.dynamic.sample_predict(instance, sample)
-        logger.info("Enforcing %d lazy constraints..." % len(cids))
-        self.enforce(cids, instance, model, solver)
-
-    @overrides
-    def after_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        sample.put_array(
-            "mip_constr_lazy_enforced",
-            np.array(list(self.lazy_enforced), dtype="S"),
-        )
-
-    @overrides
-    def iteration_cb(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-    ) -> bool:
-        assert solver.internal_solver is not None
-        logger.debug("Finding violated lazy constraints...")
-        cids = instance.find_violated_lazy_constraints(solver.internal_solver, model)
-        if len(cids) == 0:
-            logger.debug("No violations found")
-            return False
-        else:
-            self.lazy_enforced |= set(cids)
-            logger.debug("    %d violations found" % len(cids))
-            self.enforce(cids, instance, model, solver)
-            return True
-
-    # Delegate ML methods to self.dynamic
-    # -------------------------------------------------------------------
-    @overrides
-    def sample_xy(
-        self,
-        instance: Optional[Instance],
-        sample: Sample,
-    ) -> Tuple[Dict, Dict]:
-        return self.dynamic.sample_xy(instance, sample)
-
-    @overrides
-    def pre_fit(self, pre: List[Any]) -> None:
-        self.dynamic.pre_fit(pre)
-
-    def sample_predict(
-        self,
-        instance: Instance,
-        sample: Sample,
-    ) -> List[ConstraintName]:
-        return self.dynamic.sample_predict(instance, sample)
-
-    @overrides
-    def pre_sample_xy(self, instance: Instance, sample: Sample) -> Any:
-        return self.dynamic.pre_sample_xy(instance, sample)
-
-    @overrides
-    def fit_xy(
-        self,
-        x: Dict[ConstraintCategory, np.ndarray],
-        y: Dict[ConstraintCategory, np.ndarray],
-    ) -> None:
-        self.dynamic.fit_xy(x, y)
-
-    @overrides
-    def sample_evaluate(
-        self,
-        instance: Instance,
-        sample: Sample,
-    ) -> Dict[ConstraintCategory, Dict[str, float]]:
-        return self.dynamic.sample_evaluate(instance, sample)

miplearn/components/dynamic_user_cuts.py

@@ -1,137 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from typing import Any, TYPE_CHECKING, Set, Tuple, Dict, List, Optional
-
-import numpy as np
-from overrides import overrides
-
-from miplearn.classifiers import Classifier
-from miplearn.classifiers.counting import CountingClassifier
-from miplearn.classifiers.threshold import Threshold, MinProbabilityThreshold
-from miplearn.components.component import Component
-from miplearn.components.dynamic_common import DynamicConstraintsComponent
-from miplearn.features.sample import Sample
-from miplearn.instance.base import Instance
-from miplearn.types import LearningSolveStats, ConstraintName, ConstraintCategory
-
-logger = logging.getLogger(__name__)
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import LearningSolver
-
-
-class UserCutsComponent(Component):
-    def __init__(
-        self,
-        classifier: Classifier = CountingClassifier(),
-        threshold: Threshold = MinProbabilityThreshold([0.50, 0.50]),
-    ) -> None:
-        self.dynamic = DynamicConstraintsComponent(
-            classifier=classifier,
-            threshold=threshold,
-            attr="mip_user_cuts_enforced",
-        )
-        self.enforced: Set[ConstraintName] = set()
-        self.n_added_in_callback = 0
-
-    @overrides
-    def before_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: "Instance",
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        assert solver.internal_solver is not None
-        self.enforced.clear()
-        self.n_added_in_callback = 0
-        logger.info("Predicting violated user cuts...")
-        cids = self.dynamic.sample_predict(instance, sample)
-        logger.info("Enforcing %d user cuts ahead-of-time..." % len(cids))
-        for cid in cids:
-            instance.enforce_user_cut(solver.internal_solver, model, cid)
-        stats["UserCuts: Added ahead-of-time"] = len(cids)
-
-    @overrides
-    def user_cut_cb(
-        self,
-        solver: "LearningSolver",
-        instance: "Instance",
-        model: Any,
-    ) -> None:
-        assert solver.internal_solver is not None
-        logger.debug("Finding violated user cuts...")
-        cids = instance.find_violated_user_cuts(model)
-        logger.debug(f"Found {len(cids)} violated user cuts")
-        logger.debug("Building violated user cuts...")
-        for cid in cids:
-            if cid in self.enforced:
-                continue
-            assert isinstance(cid, ConstraintName)
-            instance.enforce_user_cut(solver.internal_solver, model, cid)
-            self.enforced.add(cid)
-            self.n_added_in_callback += 1
-        if len(cids) > 0:
-            logger.debug(f"Added {len(cids)} violated user cuts")
-
-    @overrides
-    def after_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: "Instance",
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        sample.put_array(
-            "mip_user_cuts_enforced",
-            np.array(list(self.enforced), dtype="S"),
-        )
-        stats["UserCuts: Added in callback"] = self.n_added_in_callback
-        if self.n_added_in_callback > 0:
-            logger.info(f"{self.n_added_in_callback} user cuts added in callback")
-
-    # Delegate ML methods to self.dynamic
-    # -------------------------------------------------------------------
-    @overrides
-    def sample_xy(
-        self,
-        instance: "Instance",
-        sample: Sample,
-    ) -> Tuple[Dict, Dict]:
-        return self.dynamic.sample_xy(instance, sample)
-
-    @overrides
-    def pre_fit(self, pre: List[Any]) -> None:
-        self.dynamic.pre_fit(pre)
-
-    def sample_predict(
-        self,
-        instance: "Instance",
-        sample: Sample,
-    ) -> List[ConstraintName]:
-        return self.dynamic.sample_predict(instance, sample)
-
-    @overrides
-    def pre_sample_xy(self, instance: Instance, sample: Sample) -> Any:
-        return self.dynamic.pre_sample_xy(instance, sample)
-
-    @overrides
-    def fit_xy(
-        self,
-        x: Dict[ConstraintCategory, np.ndarray],
-        y: Dict[ConstraintCategory, np.ndarray],
-    ) -> None:
-        self.dynamic.fit_xy(x, y)
-
-    @overrides
-    def sample_evaluate(
-        self,
-        instance: "Instance",
-        sample: Sample,
-    ) -> Dict[ConstraintCategory, Dict[str, float]]:
-        return self.dynamic.sample_evaluate(instance, sample)

miplearn/components/lazy.py

@@ -0,0 +1,43 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+import json
+from typing import Any, Dict, List
+
+import gurobipy as gp
+
+from ..h5 import H5File
+
+
+class ExpertLazyComponent:
+    def __init__(self) -> None:
+        pass
+
+    def fit(self, train_h5: List[str]) -> None:
+        pass
+
+    def before_mip(self, test_h5: str, model: gp.Model, stats: Dict[str, Any]) -> None:
+        with H5File(test_h5, "r") as h5:
+            constr_names = h5.get_array("static_constr_names")
+            constr_lazy = h5.get_array("mip_constr_lazy")
+            constr_violations = h5.get_scalar("mip_constr_violations")
+
+            assert constr_names is not None
+            assert constr_violations is not None
+
+            # Static lazy constraints
+            n_static_lazy = 0
+            if constr_lazy is not None:
+                for (constr_idx, constr_name) in enumerate(constr_names):
+                    if constr_lazy[constr_idx]:
+                        constr = model.getConstrByName(constr_name.decode())
+                        constr.lazy = 3
+                        n_static_lazy += 1
+            stats.update({"Static lazy constraints": n_static_lazy})
+
+            # Dynamic lazy constraints
+            if hasattr(model, "_fix_violations"):
+                violations = json.loads(constr_violations)
+                model._fix_violations(model, violations, "aot")
+                stats.update({"Dynamic lazy constraints": len(violations)})

miplearn/components/objective.py

@@ -1,126 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from typing import List, Dict, Any, TYPE_CHECKING, Tuple, Optional, cast
-
-import numpy as np
-from overrides import overrides
-from sklearn.linear_model import LinearRegression
-
-from miplearn.classifiers import Regressor
-from miplearn.classifiers.sklearn import ScikitLearnRegressor
-from miplearn.components.component import Component
-from miplearn.features.sample import Sample
-from miplearn.instance.base import Instance
-from miplearn.types import LearningSolveStats
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import LearningSolver
-
-logger = logging.getLogger(__name__)
-
-
-class ObjectiveValueComponent(Component):
-    """
-    A Component which predicts the optimal objective value of the problem.
-    """
-
-    def __init__(
-        self,
-        regressor: Regressor = ScikitLearnRegressor(LinearRegression()),
-    ) -> None:
-        assert isinstance(regressor, Regressor)
-        self.regressors: Dict[str, Regressor] = {}
-        self.regressor_prototype = regressor
-
-    @overrides
-    def before_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        logger.info("Predicting optimal value...")
-        pred = self.sample_predict(sample)
-        for (c, v) in pred.items():
-            logger.info(f"Predicted {c.lower()}: %.6e" % v)
-            stats[f"Objective: Predicted {c.lower()}"] = v  # type: ignore
-
-    @overrides
-    def fit_xy(
-        self,
-        x: Dict[str, np.ndarray],
-        y: Dict[str, np.ndarray],
-    ) -> None:
-        for c in ["Upper bound", "Lower bound"]:
-            if c in y:
-                self.regressors[c] = self.regressor_prototype.clone()
-                self.regressors[c].fit(x[c], y[c])
-
-    def sample_predict(self, sample: Sample) -> Dict[str, float]:
-        pred: Dict[str, float] = {}
-        x, _ = self.sample_xy(None, sample)
-        for c in ["Upper bound", "Lower bound"]:
-            if c in self.regressors is not None:
-                pred[c] = self.regressors[c].predict(np.array(x[c]))[0, 0]
-            else:
-                logger.info(f"{c} regressor not fitted. Skipping.")
-        return pred
-
-    @overrides
-    def sample_xy(
-        self,
-        _: Optional[Instance],
-        sample: Sample,
-    ) -> Tuple[Dict[str, List[List[float]]], Dict[str, List[List[float]]]]:
-        lp_instance_features_np = sample.get_array("lp_instance_features")
-        if lp_instance_features_np is None:
-            lp_instance_features_np = sample.get_array("static_instance_features")
-        assert lp_instance_features_np is not None
-        lp_instance_features = cast(List[float], lp_instance_features_np.tolist())
-
-        # Features
-        x: Dict[str, List[List[float]]] = {
-            "Upper bound": [lp_instance_features],
-            "Lower bound": [lp_instance_features],
-        }
-
-        # Labels
-        y: Dict[str, List[List[float]]] = {}
-        mip_lower_bound = sample.get_scalar("mip_lower_bound")
-        mip_upper_bound = sample.get_scalar("mip_upper_bound")
-        if mip_lower_bound is not None:
-            y["Lower bound"] = [[mip_lower_bound]]
-        if mip_upper_bound is not None:
-            y["Upper bound"] = [[mip_upper_bound]]
-
-        return x, y
-
-    @overrides
-    def sample_evaluate(
-        self,
-        instance: Instance,
-        sample: Sample,
-    ) -> Dict[str, Dict[str, float]]:
-        def compare(y_pred: float, y_actual: float) -> Dict[str, float]:
-            err = np.round(abs(y_pred - y_actual), 8)
-            return {
-                "Actual value": y_actual,
-                "Predicted value": y_pred,
-                "Absolute error": err,
-                "Relative error": err / y_actual,
-            }
-
-        result: Dict[str, Dict[str, float]] = {}
-        pred = self.sample_predict(sample)
-        actual_ub = sample.get_scalar("mip_upper_bound")
-        actual_lb = sample.get_scalar("mip_lower_bound")
-        if actual_ub is not None:
-            result["Upper bound"] = compare(pred["Upper bound"], actual_ub)
-        if actual_lb is not None:
-            result["Lower bound"] = compare(pred["Lower bound"], actual_lb)
-        return result

miplearn/components/primal.py

@@ -1,242 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from typing import Dict, List, Any, TYPE_CHECKING, Tuple, Optional
-
-import numpy as np
-from overrides import overrides
-
-from miplearn.classifiers import Classifier
-from miplearn.classifiers.adaptive import AdaptiveClassifier
-from miplearn.classifiers.threshold import MinPrecisionThreshold, Threshold
-from miplearn.components import classifier_evaluation_dict
-from miplearn.components.component import Component
-from miplearn.features.sample import Sample
-from miplearn.instance.base import Instance
-from miplearn.types import (
-    LearningSolveStats,
-    Category,
-    Solution,
-)
-
-logger = logging.getLogger(__name__)
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import LearningSolver
-
-
-class PrimalSolutionComponent(Component):
-    """
-    A component that predicts the optimal primal values for the binary decision
-    variables.
-
-    In exact mode, predicted primal solutions are provided to the solver as MIP
-    starts. In heuristic mode, this component fixes the decision variables to their
-    predicted values.
-    """
-
-    def __init__(
-        self,
-        classifier: Classifier = AdaptiveClassifier(),
-        mode: str = "exact",
-        threshold: Threshold = MinPrecisionThreshold([0.98, 0.98]),
-    ) -> None:
-        assert isinstance(classifier, Classifier)
-        assert isinstance(threshold, Threshold)
-        assert mode in ["exact", "heuristic"]
-        self.mode = mode
-        self.classifiers: Dict[Category, Classifier] = {}
-        self.thresholds: Dict[Category, Threshold] = {}
-        self.threshold_prototype = threshold
-        self.classifier_prototype = classifier
-
-    @overrides
-    def before_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: Instance,
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        logger.info("Predicting primal solution...")
-
-        # Do nothing if models are not trained
-        if len(self.classifiers) == 0:
-            logger.info("Classifiers not fitted. Skipping.")
-            return
-
-        # Predict solution and provide it to the solver
-        solution = self.sample_predict(sample)
-        assert solver.internal_solver is not None
-        if self.mode == "heuristic":
-            solver.internal_solver.fix(solution)
-        else:
-            solver.internal_solver.set_warm_start(solution)
-
-        # Update statistics
-        stats["Primal: Free"] = 0
-        stats["Primal: Zero"] = 0
-        stats["Primal: One"] = 0
-        for (var_name, value) in solution.items():
-            if value is None:
-                stats["Primal: Free"] += 1
-            else:
-                if value < 0.5:
-                    stats["Primal: Zero"] += 1
-                else:
-                    stats["Primal: One"] += 1
-        logger.info(
-            f"Predicted: free: {stats['Primal: Free']}, "
-            f"zero: {stats['Primal: Zero']}, "
-            f"one: {stats['Primal: One']}"
-        )
-
-    def sample_predict(self, sample: Sample) -> Solution:
-        var_names = sample.get_array("static_var_names")
-        var_categories = sample.get_array("static_var_categories")
-        assert var_names is not None
-        assert var_categories is not None
-
-        # Compute y_pred
-        x, _ = self.sample_xy(None, sample)
-        y_pred = {}
-        for category in x.keys():
-            assert category in self.classifiers, (
-                f"Classifier for category {category} has not been trained. "
-                f"Please call component.fit before component.predict."
-            )
-            xc = np.array(x[category])
-            proba = self.classifiers[category].predict_proba(xc)
-            thr = self.thresholds[category].predict(xc)
-            y_pred[category] = np.vstack(
-                [
-                    proba[:, 0] >= thr[0],
-                    proba[:, 1] >= thr[1],
-                ]
-            ).T
-
-        # Convert y_pred into solution
-        solution: Solution = {v: None for v in var_names}
-        category_offset: Dict[Category, int] = {cat: 0 for cat in x.keys()}
-        for (i, var_name) in enumerate(var_names):
-            category = var_categories[i]
-            if category not in category_offset:
-                continue
-            offset = category_offset[category]
-            category_offset[category] += 1
-            if y_pred[category][offset, 0]:
-                solution[var_name] = 0.0
-            if y_pred[category][offset, 1]:
-                solution[var_name] = 1.0
-
-        return solution
-
-    @overrides
-    def sample_xy(
-        self,
-        _: Optional[Instance],
-        sample: Sample,
-    ) -> Tuple[Dict[Category, List[List[float]]], Dict[Category, List[List[float]]]]:
-        x: Dict = {}
-        y: Dict = {}
-        instance_features = sample.get_array("static_instance_features")
-        mip_var_values = sample.get_array("mip_var_values")
-        var_features = sample.get_array("lp_var_features")
-        var_names = sample.get_array("static_var_names")
-        var_types = sample.get_array("static_var_types")
-        var_categories = sample.get_array("static_var_categories")
-        if var_features is None:
-            var_features = sample.get_array("static_var_features")
-        assert instance_features is not None
-        assert var_features is not None
-        assert var_names is not None
-        assert var_types is not None
-        assert var_categories is not None
-
-        for (i, var_name) in enumerate(var_names):
-            # Skip non-binary variables
-            if var_types[i] != b"B":
-                continue
-
-            # Initialize categories
-            category = var_categories[i]
-            if len(category) == 0:
-                continue
-            if category not in x.keys():
-                x[category] = []
-                y[category] = []
-
-            # Features
-            features = list(instance_features)
-            features.extend(var_features[i])
-            x[category].append(features)
-
-            # Labels
-            if mip_var_values is not None:
-                opt_value = mip_var_values[i]
-                assert opt_value is not None
-                y[category].append([opt_value < 0.5, opt_value >= 0.5])
-        return x, y
-
-    @overrides
-    def sample_evaluate(
-        self,
-        _: Optional[Instance],
-        sample: Sample,
-    ) -> Dict[str, Dict[str, float]]:
-        mip_var_values = sample.get_array("mip_var_values")
-        var_names = sample.get_array("static_var_names")
-        assert mip_var_values is not None
-        assert var_names is not None
-
-        solution_actual = {
-            var_name: mip_var_values[i] for (i, var_name) in enumerate(var_names)
-        }
-        solution_pred = self.sample_predict(sample)
-        vars_all, vars_one, vars_zero = set(), set(), set()
-        pred_one_positive, pred_zero_positive = set(), set()
-        for (var_name, value_actual) in solution_actual.items():
-            vars_all.add(var_name)
-            if value_actual > 0.5:
-                vars_one.add(var_name)
-            else:
-                vars_zero.add(var_name)
-            value_pred = solution_pred[var_name]
-            if value_pred is not None:
-                if value_pred > 0.5:
-                    pred_one_positive.add(var_name)
-                else:
-                    pred_zero_positive.add(var_name)
-        pred_one_negative = vars_all - pred_one_positive
-        pred_zero_negative = vars_all - pred_zero_positive
-        return {
-            "0": classifier_evaluation_dict(
-                tp=len(pred_zero_positive & vars_zero),
-                tn=len(pred_zero_negative & vars_one),
-                fp=len(pred_zero_positive & vars_one),
-                fn=len(pred_zero_negative & vars_zero),
-            ),
-            "1": classifier_evaluation_dict(
-                tp=len(pred_one_positive & vars_one),
-                tn=len(pred_one_negative & vars_zero),
-                fp=len(pred_one_positive & vars_zero),
-                fn=len(pred_one_negative & vars_one),
-            ),
-        }
-
-    @overrides
-    def fit_xy(
-        self,
-        x: Dict[Category, np.ndarray],
-        y: Dict[Category, np.ndarray],
-    ) -> None:
-        for category in x.keys():
-            clf = self.classifier_prototype.clone()
-            thr = self.threshold_prototype.clone()
-            clf.fit(x[category], y[category])
-            thr.fit(clf, x[category], y[category])
-            self.classifiers[category] = clf
-            self.thresholds[category] = thr

miplearn/components/primal/__init__.py

@@ -0,0 +1,29 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+from typing import Tuple
+
+import numpy as np
+
+from miplearn.h5 import H5File
+
+
+def _extract_bin_var_names_values(
+    h5: H5File,
+) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    bin_var_names, bin_var_indices = _extract_bin_var_names(h5)
+    var_values = h5.get_array("mip_var_values")
+    assert var_values is not None
+    bin_var_values = var_values[bin_var_indices].astype(int)
+    return bin_var_names, bin_var_values, bin_var_indices
+
+
+def _extract_bin_var_names(h5: H5File) -> Tuple[np.ndarray, np.ndarray]:
+    var_types = h5.get_array("static_var_types")
+    var_names = h5.get_array("static_var_names")
+    assert var_types is not None
+    assert var_names is not None
+    bin_var_indices = np.where(var_types == b"B")[0]
+    bin_var_names = var_names[bin_var_indices]
+    assert len(bin_var_names.shape) == 1
+    return bin_var_names, bin_var_indices

miplearn/components/primal/actions.py

@@ -0,0 +1,93 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+import logging
+from abc import ABC, abstractmethod
+from typing import Optional, Dict
+
+import numpy as np
+
+from miplearn.solvers.abstract import AbstractModel
+
+logger = logging.getLogger()
+
+
+class PrimalComponentAction(ABC):
+    @abstractmethod
+    def perform(
+        self,
+        model: AbstractModel,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict],
+    ) -> None:
+        pass
+
+
+class SetWarmStart(PrimalComponentAction):
+    def perform(
+        self,
+        model: AbstractModel,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict],
+    ) -> None:
+        logger.info("Setting warm starts...")
+        model.set_warm_starts(var_names, var_values, stats)
+
+
+class FixVariables(PrimalComponentAction):
+    def perform(
+        self,
+        model: AbstractModel,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict],
+    ) -> None:
+        logger.info("Fixing variables...")
+        assert len(var_values.shape) == 2
+        assert var_values.shape[0] == 1
+        var_values = var_values.reshape(-1)
+        model.fix_variables(var_names, var_values, stats)
+        if stats is not None:
+            stats["Heuristic"] = True
+
+
+class EnforceProximity(PrimalComponentAction):
+    def __init__(self, tol: float) -> None:
+        self.tol = tol
+
+    def perform(
+        self,
+        model: AbstractModel,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict],
+    ) -> None:
+        assert len(var_values.shape) == 2
+        assert var_values.shape[0] == 1
+        var_values = var_values.reshape(-1)
+
+        constr_lhs = []
+        constr_vars = []
+        constr_rhs = 0.0
+        for (i, var_name) in enumerate(var_names):
+            if np.isnan(var_values[i]):
+                continue
+            constr_lhs.append(1.0 if var_values[i] < 0.5 else -1.0)
+            constr_rhs -= var_values[i]
+            constr_vars.append(var_name)
+
+        constr_rhs += len(constr_vars) * self.tol
+        logger.info(
+            f"Adding proximity constraint (tol={self.tol}, nz={len(constr_vars)})..."
+        )
+
+        model.add_constrs(
+            np.array(constr_vars),
+            np.array([constr_lhs]),
+            np.array(["<"], dtype="S"),
+            np.array([constr_rhs]),
+        )
+        if stats is not None:
+            stats["Heuristic"] = True

miplearn/components/primal/expert.py

@@ -0,0 +1,32 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+import logging
+from typing import Any, Dict, List
+
+from . import _extract_bin_var_names_values
+from .actions import PrimalComponentAction
+from ...solvers.abstract import AbstractModel
+from ...h5 import H5File
+
+logger = logging.getLogger(__name__)
+
+
+class ExpertPrimalComponent:
+    def __init__(self, action: PrimalComponentAction):
+        self.action = action
+
+    """
+    Component that predicts warm starts by peeking at the optimal solution.
+    """
+
+    def fit(self, train_h5: List[str]) -> None:
+        pass
+
+    def before_mip(
+        self, test_h5: str, model: AbstractModel, stats: Dict[str, Any]
+    ) -> None:
+        with H5File(test_h5, "r") as h5:
+            names, values, _ = _extract_bin_var_names_values(h5)
+            self.action.perform(model, names, values.reshape(1, -1), stats)

miplearn/components/primal/indep.py

@@ -0,0 +1,129 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+import logging
+from typing import Any, Dict, List, Callable, Optional
+
+import numpy as np
+import sklearn
+
+from miplearn.components.primal import (
+    _extract_bin_var_names_values,
+    _extract_bin_var_names,
+)
+from miplearn.components.primal.actions import PrimalComponentAction
+from miplearn.extractors.abstract import FeaturesExtractor
+from miplearn.solvers.abstract import AbstractModel
+from miplearn.h5 import H5File
+
+logger = logging.getLogger(__name__)
+
+
+class IndependentVarsPrimalComponent:
+    def __init__(
+        self,
+        base_clf: Any,
+        extractor: FeaturesExtractor,
+        action: PrimalComponentAction,
+        clone_fn: Callable[[Any], Any] = sklearn.clone,
+    ):
+        self.base_clf = base_clf
+        self.extractor = extractor
+        self.clf_: Dict[bytes, Any] = {}
+        self.bin_var_names_: Optional[np.ndarray] = None
+        self.n_features_: Optional[int] = None
+        self.clone_fn = clone_fn
+        self.action = action
+
+    def fit(self, train_h5: List[str]) -> None:
+        logger.info("Reading training data...")
+        self.bin_var_names_ = None
+        n_bin_vars: Optional[int] = None
+        n_vars: Optional[int] = None
+        x, y = [], []
+        for h5_filename in train_h5:
+            with H5File(h5_filename, "r") as h5:
+                # Get number of variables
+                var_types = h5.get_array("static_var_types")
+                assert var_types is not None
+                n_vars = len(var_types)
+
+                # Extract features
+                (
+                    bin_var_names,
+                    bin_var_values,
+                    bin_var_indices,
+                ) = _extract_bin_var_names_values(h5)
+
+                # Store/check variable names
+                if self.bin_var_names_ is None:
+                    self.bin_var_names_ = bin_var_names
+                    n_bin_vars = len(self.bin_var_names_)
+                else:
+                    assert np.all(bin_var_names == self.bin_var_names_)
+
+                # Build x and y vectors
+                x_sample = self.extractor.get_var_features(h5)
+                assert len(x_sample.shape) == 2
+                assert x_sample.shape[0] == n_vars
+                x_sample = x_sample[bin_var_indices]
+                if self.n_features_ is None:
+                    self.n_features_ = x_sample.shape[1]
+                else:
+                    assert x_sample.shape[1] == self.n_features_
+                x.append(x_sample)
+                y.append(bin_var_values)
+
+        assert n_bin_vars is not None
+        assert self.bin_var_names_ is not None
+
+        logger.info("Constructing matrices...")
+        x_np = np.vstack(x)
+        y_np = np.hstack(y)
+        n_samples = len(train_h5) * n_bin_vars
+        assert x_np.shape == (n_samples, self.n_features_)
+        assert y_np.shape == (n_samples,)
+        logger.info(
+            f"Dataset has {n_bin_vars} binary variables, "
+            f"{len(train_h5):,d} samples per variable, "
+            f"{self.n_features_:,d} features, 1 target and 2 classes"
+        )
+
+        logger.info(f"Training {n_bin_vars} classifiers...")
+        self.clf_ = {}
+        for (var_idx, var_name) in enumerate(self.bin_var_names_):
+            self.clf_[var_name] = self.clone_fn(self.base_clf)
+            self.clf_[var_name].fit(
+                x_np[var_idx::n_bin_vars, :], y_np[var_idx::n_bin_vars]
+            )
+
+        logger.info("Done fitting.")
+
+    def before_mip(
+        self, test_h5: str, model: AbstractModel, stats: Dict[str, Any]
+    ) -> None:
+        assert self.bin_var_names_ is not None
+        assert self.n_features_ is not None
+
+        # Read features
+        with H5File(test_h5, "r") as h5:
+            x_sample = self.extractor.get_var_features(h5)
+            bin_var_names, bin_var_indices = _extract_bin_var_names(h5)
+            assert np.all(bin_var_names == self.bin_var_names_)
+            x_sample = x_sample[bin_var_indices]
+
+        assert x_sample.shape == (len(self.bin_var_names_), self.n_features_)
+
+        # Predict optimal solution
+        logger.info("Predicting warm starts...")
+        y_pred = []
+        for (var_idx, var_name) in enumerate(self.bin_var_names_):
+            x_var = x_sample[var_idx, :].reshape(1, -1)
+            y_var = self.clf_[var_name].predict(x_var)
+            assert y_var.shape == (1,)
+            y_pred.append(y_var[0])
+
+        # Construct warm starts, based on prediction
+        y_pred_np = np.array(y_pred).reshape(1, -1)
+        assert y_pred_np.shape == (1, len(self.bin_var_names_))
+        self.action.perform(model, self.bin_var_names_, y_pred_np, stats)

miplearn/components/primal/joint.py

@@ -0,0 +1,88 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+import logging
+from typing import List, Dict, Any, Optional
+
+import numpy as np
+
+from miplearn.components.primal import _extract_bin_var_names_values
+from miplearn.components.primal.actions import PrimalComponentAction
+from miplearn.extractors.abstract import FeaturesExtractor
+from miplearn.solvers.abstract import AbstractModel
+from miplearn.h5 import H5File
+
+logger = logging.getLogger(__name__)
+
+
+class JointVarsPrimalComponent:
+    def __init__(
+        self, clf: Any, extractor: FeaturesExtractor, action: PrimalComponentAction
+    ):
+        self.clf = clf
+        self.extractor = extractor
+        self.bin_var_names_: Optional[np.ndarray] = None
+        self.action = action
+
+    def fit(self, train_h5: List[str]) -> None:
+        logger.info("Reading training data...")
+        self.bin_var_names_ = None
+        x, y, n_samples, n_features = [], [], len(train_h5), None
+        for h5_filename in train_h5:
+            with H5File(h5_filename, "r") as h5:
+                bin_var_names, bin_var_values, _ = _extract_bin_var_names_values(h5)
+
+                # Store/check variable names
+                if self.bin_var_names_ is None:
+                    self.bin_var_names_ = bin_var_names
+                else:
+                    assert np.all(bin_var_names == self.bin_var_names_)
+
+                # Build x and y vectors
+                x_sample = self.extractor.get_instance_features(h5)
+                assert len(x_sample.shape) == 1
+                if n_features is None:
+                    n_features = len(x_sample)
+                else:
+                    assert len(x_sample) == n_features
+                x.append(x_sample)
+                y.append(bin_var_values)
+        assert self.bin_var_names_ is not None
+
+        logger.info("Constructing matrices...")
+        x_np = np.vstack(x)
+        y_np = np.array(y)
+        assert len(x_np.shape) == 2
+        assert x_np.shape[0] == n_samples
+        assert x_np.shape[1] == n_features
+        assert y_np.shape == (n_samples, len(self.bin_var_names_))
+        logger.info(
+            f"Dataset has {n_samples:,d} samples, "
+            f"{n_features:,d} features and {y_np.shape[1]:,d} targets"
+        )
+
+        logger.info("Training classifier...")
+        self.clf.fit(x_np, y_np)
+
+        logger.info("Done fitting.")
+
+    def before_mip(
+        self, test_h5: str, model: AbstractModel, stats: Dict[str, Any]
+    ) -> None:
+        assert self.bin_var_names_ is not None
+
+        # Read features
+        with H5File(test_h5, "r") as h5:
+            x_sample = self.extractor.get_instance_features(h5)
+        assert len(x_sample.shape) == 1
+        x_sample = x_sample.reshape(1, -1)
+
+        # Predict optimal solution
+        logger.info("Predicting warm starts...")
+        y_pred = self.clf.predict(x_sample)
+        assert len(y_pred.shape) == 2
+        assert y_pred.shape[0] == 1
+        assert y_pred.shape[1] == len(self.bin_var_names_)
+
+        # Construct warm starts, based on prediction
+        self.action.perform(model, self.bin_var_names_, y_pred, stats)

miplearn/components/primal/mem.py

@@ -0,0 +1,167 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+import logging
+from abc import ABC, abstractmethod
+from typing import List, Dict, Any, Optional, Tuple
+
+import numpy as np
+
+from . import _extract_bin_var_names_values
+from .actions import PrimalComponentAction
+from ...extractors.abstract import FeaturesExtractor
+from ...solvers.abstract import AbstractModel
+from ...h5 import H5File
+
+logger = logging.getLogger()
+
+
+class SolutionConstructor(ABC):
+    @abstractmethod
+    def construct(self, y_proba: np.ndarray, solutions: np.ndarray) -> np.ndarray:
+        pass
+
+
+class MemorizingPrimalComponent:
+    """
+    Component that memorizes all solutions seen during training, then fits a
+    single classifier to predict which of the memorized solutions should be
+    provided to the solver. Optionally combines multiple memorized solutions
+    into a single, partial one.
+    """
+
+    def __init__(
+        self,
+        clf: Any,
+        extractor: FeaturesExtractor,
+        constructor: SolutionConstructor,
+        action: PrimalComponentAction,
+    ) -> None:
+        assert clf is not None
+        self.clf = clf
+        self.extractor = extractor
+        self.constructor = constructor
+        self.solutions_: Optional[np.ndarray] = None
+        self.bin_var_names_: Optional[np.ndarray] = None
+        self.action = action
+
+    def fit(self, train_h5: List[str]) -> None:
+        logger.info("Reading training data...")
+        n_samples = len(train_h5)
+        solutions_ = []
+        self.bin_var_names_ = None
+        x, y, n_features = [], [], None
+        solution_to_idx: Dict[Tuple, int] = {}
+        for h5_filename in train_h5:
+            with H5File(h5_filename, "r") as h5:
+                bin_var_names, bin_var_values, _ = _extract_bin_var_names_values(h5)
+
+                # Store/check variable names
+                if self.bin_var_names_ is None:
+                    self.bin_var_names_ = bin_var_names
+                else:
+                    assert np.all(bin_var_names == self.bin_var_names_)
+
+                # Store solution
+                sol = tuple(np.where(bin_var_values)[0])
+                if sol not in solution_to_idx:
+                    solutions_.append(bin_var_values)
+                    solution_to_idx[sol] = len(solution_to_idx)
+                y.append(solution_to_idx[sol])
+
+                # Extract features
+                x_sample = self.extractor.get_instance_features(h5)
+                assert len(x_sample.shape) == 1
+                if n_features is None:
+                    n_features = len(x_sample)
+                else:
+                    assert len(x_sample) == n_features
+                x.append(x_sample)
+
+        logger.info("Constructing matrices...")
+        x_np = np.vstack(x)
+        y_np = np.array(y)
+        assert len(x_np.shape) == 2
+        assert x_np.shape[0] == n_samples
+        assert x_np.shape[1] == n_features
+        assert y_np.shape == (n_samples,)
+        self.solutions_ = np.array(solutions_)
+        n_classes = len(solution_to_idx)
+        logger.info(
+            f"Dataset has {n_samples:,d} samples, "
+            f"{n_features:,d} features and {n_classes:,d} classes"
+        )
+
+        logger.info("Training classifier...")
+        self.clf.fit(x_np, y_np)
+
+        logger.info("Done fitting.")
+
+    def before_mip(
+        self, test_h5: str, model: AbstractModel, stats: Dict[str, Any]
+    ) -> None:
+        assert self.solutions_ is not None
+        assert self.bin_var_names_ is not None
+
+        # Read features
+        with H5File(test_h5, "r") as h5:
+            x_sample = self.extractor.get_instance_features(h5)
+        assert len(x_sample.shape) == 1
+        x_sample = x_sample.reshape(1, -1)
+
+        # Predict optimal solution
+        logger.info("Predicting primal solution...")
+        y_proba = self.clf.predict_proba(x_sample)
+        assert len(y_proba.shape) == 2
+        assert y_proba.shape[0] == 1
+        assert y_proba.shape[1] == len(self.solutions_)
+
+        # Construct warm starts, based on prediction
+        starts = self.constructor.construct(y_proba[0, :], self.solutions_)
+        self.action.perform(model, self.bin_var_names_, starts, stats)
+
+
+class SelectTopSolutions(SolutionConstructor):
+    """
+    Warm start construction strategy that selects and returns the top k solutions.
+    """
+
+    def __init__(self, k: int) -> None:
+        self.k = k
+
+    def construct(self, y_proba: np.ndarray, solutions: np.ndarray) -> np.ndarray:
+        # Check arguments
+        assert len(y_proba.shape) == 1
+        assert len(solutions.shape) == 2
+        assert len(y_proba) == solutions.shape[0]
+
+        # Select top k solutions
+        ind = np.argsort(-y_proba, kind="stable")
+        selected = ind[: min(self.k, len(ind))]
+        return solutions[selected, :]
+
+
+class MergeTopSolutions(SolutionConstructor):
+    """
+    Warm start construction strategy that first selects the top k solutions,
+    then merges them into a single solution.
+
+    To merge the solutions, the strategy first computes the mean optimal value of each
+    decision variable, then: (i) sets the variable to zero if the mean is below
+    thresholds[0]; (ii) sets the variable to one if the mean is above thresholds[1];
+    (iii) leaves the variable free otherwise.
+    """
+
+    def __init__(self, k: int, thresholds: List[float]):
+        assert len(thresholds) == 2
+        self.k = k
+        self.thresholds = thresholds
+
+    def construct(self, y_proba: np.ndarray, solutions: np.ndarray) -> np.ndarray:
+        filtered = SelectTopSolutions(self.k).construct(y_proba, solutions)
+        mean = filtered.mean(axis=0)
+        start = np.full((1, solutions.shape[1]), float("nan"))
+        start[0, mean <= self.thresholds[0]] = 0
+        start[0, mean >= self.thresholds[1]] = 1
+        return start

miplearn/components/priority.py

@@ -0,0 +1,31 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from math import log
+from typing import List, Dict, Any
+import numpy as np
+
+import gurobipy as gp
+
+from ..h5 import H5File
+
+
+class ExpertBranchPriorityComponent:
+    def __init__(self) -> None:
+        pass
+
+    def fit(self, train_h5: List[str]) -> None:
+        pass
+
+    def before_mip(self, test_h5: str, model: gp.Model, _: Dict[str, Any]) -> None:
+        with H5File(test_h5, "r") as h5:
+            var_names = h5.get_array("static_var_names")
+            var_priority = h5.get_array("bb_var_priority")
+            assert var_priority is not None
+            assert var_names is not None
+
+            for (var_idx, var_name) in enumerate(var_names):
+                if np.isfinite(var_priority[var_idx]):
+                    var = model.getVarByName(var_name.decode())
+                    var.branchPriority = int(log(1 + var_priority[var_idx]))

miplearn/components/static_lazy.py

@@ -1,252 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from typing import Dict, Tuple, List, Any, TYPE_CHECKING, Set, Optional
-
-import numpy as np
-from overrides import overrides
-
-from miplearn.classifiers import Classifier
-from miplearn.classifiers.counting import CountingClassifier
-from miplearn.classifiers.threshold import MinProbabilityThreshold, Threshold
-from miplearn.components.component import Component
-from miplearn.features.sample import Sample
-from miplearn.solvers.internal import Constraints
-from miplearn.instance.base import Instance
-from miplearn.types import LearningSolveStats, ConstraintName, ConstraintCategory
-
-logger = logging.getLogger(__name__)
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import LearningSolver
-
-
-class LazyConstraint:
-    def __init__(self, cid: ConstraintName, obj: Any) -> None:
-        self.cid = cid
-        self.obj = obj
-
-
-class StaticLazyConstraintsComponent(Component):
-    """
-    Component that decides which of the constraints tagged as lazy should
-    be kept in the formulation, and which should be removed.
-    """
-
-    def __init__(
-        self,
-        classifier: Classifier = CountingClassifier(),
-        threshold: Threshold = MinProbabilityThreshold([0.50, 0.50]),
-        violation_tolerance: float = -0.5,
-    ) -> None:
-        assert isinstance(classifier, Classifier)
-        self.classifier_prototype: Classifier = classifier
-        self.threshold_prototype: Threshold = threshold
-        self.classifiers: Dict[ConstraintCategory, Classifier] = {}
-        self.thresholds: Dict[ConstraintCategory, Threshold] = {}
-        self.pool: Constraints = Constraints()
-        self.violation_tolerance: float = violation_tolerance
-        self.enforced_cids: Set[ConstraintName] = set()
-        self.n_restored: int = 0
-        self.n_iterations: int = 0
-
-    @overrides
-    def after_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: "Instance",
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        sample.put_array(
-            "mip_constr_lazy_enforced",
-            np.array(list(self.enforced_cids), dtype="S"),
-        )
-        stats["LazyStatic: Restored"] = self.n_restored
-        stats["LazyStatic: Iterations"] = self.n_iterations
-
-    @overrides
-    def before_solve_mip(
-        self,
-        solver: "LearningSolver",
-        instance: "Instance",
-        model: Any,
-        stats: LearningSolveStats,
-        sample: Sample,
-    ) -> None:
-        assert solver.internal_solver is not None
-        static_lazy_count = sample.get_scalar("static_constr_lazy_count")
-        assert static_lazy_count is not None
-
-        logger.info("Predicting violated (static) lazy constraints...")
-        if static_lazy_count == 0:
-            logger.info("Instance does not have static lazy constraints. Skipping.")
-        self.enforced_cids = set(self.sample_predict(sample))
-        logger.info("Moving lazy constraints to the pool...")
-        constraints = Constraints.from_sample(sample)
-        assert constraints.lazy is not None
-        assert constraints.names is not None
-        selected = [
-            (constraints.lazy[i] and constraints.names[i] not in self.enforced_cids)
-            for i in range(len(constraints.lazy))
-        ]
-        n_removed = sum(selected)
-        n_kept = sum(constraints.lazy) - n_removed
-        self.pool = constraints[selected]
-        assert self.pool.names is not None
-        solver.internal_solver.remove_constraints(self.pool.names)
-        logger.info(f"{n_kept} lazy constraints kept; {n_removed} moved to the pool")
-        stats["LazyStatic: Removed"] = n_removed
-        stats["LazyStatic: Kept"] = n_kept
-        stats["LazyStatic: Restored"] = 0
-        self.n_restored = 0
-        self.n_iterations = 0
-
-    @overrides
-    def fit_xy(
-        self,
-        x: Dict[ConstraintCategory, np.ndarray],
-        y: Dict[ConstraintCategory, np.ndarray],
-    ) -> None:
-        for c in y.keys():
-            assert c in x
-            self.classifiers[c] = self.classifier_prototype.clone()
-            self.thresholds[c] = self.threshold_prototype.clone()
-            self.classifiers[c].fit(x[c], y[c])
-            self.thresholds[c].fit(self.classifiers[c], x[c], y[c])
-
-    @overrides
-    def iteration_cb(
-        self,
-        solver: "LearningSolver",
-        instance: "Instance",
-        model: Any,
-    ) -> bool:
-        if solver.use_lazy_cb:
-            return False
-        else:
-            return self._check_and_add(solver)
-
-    @overrides
-    def lazy_cb(
-        self,
-        solver: "LearningSolver",
-        instance: "Instance",
-        model: Any,
-    ) -> None:
-        self._check_and_add(solver)
-
-    def sample_predict(self, sample: Sample) -> List[ConstraintName]:
-        x, y, cids = self._sample_xy_with_cids(sample)
-        enforced_cids: List[ConstraintName] = []
-        for category in x.keys():
-            if category not in self.classifiers:
-                continue
-            npx = np.array(x[category])
-            proba = self.classifiers[category].predict_proba(npx)
-            thr = self.thresholds[category].predict(npx)
-            pred = list(proba[:, 1] > thr[1])
-            for (i, is_selected) in enumerate(pred):
-                if is_selected:
-                    enforced_cids += [cids[category][i]]
-        return enforced_cids
-
-    @overrides
-    def sample_xy(
-        self,
-        _: Optional[Instance],
-        sample: Sample,
-    ) -> Tuple[
-        Dict[ConstraintCategory, List[List[float]]],
-        Dict[ConstraintCategory, List[List[float]]],
-    ]:
-        x, y, __ = self._sample_xy_with_cids(sample)
-        return x, y
-
-    def _check_and_add(self, solver: "LearningSolver") -> bool:
-        assert solver.internal_solver is not None
-        assert self.pool.names is not None
-        if len(self.pool.names) == 0:
-            logger.info("Lazy constraint pool is empty. Skipping violation check.")
-            return False
-        self.n_iterations += 1
-        logger.info("Finding violated lazy constraints...")
-        is_satisfied = solver.internal_solver.are_constraints_satisfied(
-            self.pool,
-            tol=self.violation_tolerance,
-        )
-        is_violated = [not i for i in is_satisfied]
-        violated_constraints = self.pool[is_violated]
-        satisfied_constraints = self.pool[is_satisfied]
-        self.pool = satisfied_constraints
-        assert violated_constraints.names is not None
-        assert satisfied_constraints.names is not None
-        n_violated = len(violated_constraints.names)
-        n_satisfied = len(satisfied_constraints.names)
-        logger.info(f"Found {n_violated} violated lazy constraints found")
-        if n_violated > 0:
-            logger.info(
-                f"Enforcing {n_violated} lazy constraints; "
-                f"{n_satisfied} left in the pool..."
-            )
-            solver.internal_solver.add_constraints(violated_constraints)
-            for (i, name) in enumerate(violated_constraints.names):
-                self.enforced_cids.add(name)
-                self.n_restored += 1
-            return True
-        else:
-            return False
-
-    def _sample_xy_with_cids(
-        self, sample: Sample
-    ) -> Tuple[
-        Dict[ConstraintCategory, List[List[float]]],
-        Dict[ConstraintCategory, List[List[float]]],
-        Dict[ConstraintCategory, List[ConstraintName]],
-    ]:
-        x: Dict[ConstraintCategory, List[List[float]]] = {}
-        y: Dict[ConstraintCategory, List[List[float]]] = {}
-        cids: Dict[ConstraintCategory, List[ConstraintName]] = {}
-        instance_features = sample.get_array("static_instance_features")
-        constr_features = sample.get_array("lp_constr_features")
-        constr_names = sample.get_array("static_constr_names")
-        constr_categories = sample.get_array("static_constr_categories")
-        constr_lazy = sample.get_array("static_constr_lazy")
-        lazy_enforced = sample.get_array("mip_constr_lazy_enforced")
-        if constr_features is None:
-            constr_features = sample.get_array("static_constr_features")
-
-        assert instance_features is not None
-        assert constr_features is not None
-        assert constr_names is not None
-        assert constr_categories is not None
-        assert constr_lazy is not None
-
-        for (cidx, cname) in enumerate(constr_names):
-            # Initialize categories
-            if not constr_lazy[cidx]:
-                continue
-            category = constr_categories[cidx]
-            if len(category) == 0:
-                continue
-            if category not in x:
-                x[category] = []
-                y[category] = []
-                cids[category] = []
-
-            # Features
-            features = list(instance_features)
-            features.extend(constr_features[cidx])
-            x[category].append(features)
-            cids[category].append(cname)
-
-            # Labels
-            if lazy_enforced is not None:
-                if cname in lazy_enforced:
-                    y[category] += [[False, True]]
-                else:
-                    y[category] += [[True, False]]
-        return x, y, cids

miplearn/extractors/AlvLouWeh2017.py

@@ -0,0 +1,210 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from typing import Tuple, Optional
+
+import numpy as np
+
+from miplearn.extractors.abstract import FeaturesExtractor
+from miplearn.h5 import H5File
+
+
+class AlvLouWeh2017Extractor(FeaturesExtractor):
+    def __init__(
+        self,
+        with_m1: bool = True,
+        with_m2: bool = True,
+        with_m3: bool = True,
+    ):
+        self.with_m1 = with_m1
+        self.with_m2 = with_m2
+        self.with_m3 = with_m3
+
+    def get_instance_features(self, h5: H5File) -> np.ndarray:
+        raise NotImplemented()
+
+    def get_var_features(self, h5: H5File) -> np.ndarray:
+        """
+        Computes static variable features described in:
+            Alvarez, A. M., Louveaux, Q., & Wehenkel, L. (2017). A machine learning-based
+            approximation of strong branching. INFORMS Journal on Computing, 29(1),
+            185-195.
+        """
+        A = h5.get_sparse("static_constr_lhs")
+        b = h5.get_array("static_constr_rhs")
+        c = h5.get_array("static_var_obj_coeffs")
+        c_sa_up = h5.get_array("lp_var_sa_obj_up")
+        c_sa_down = h5.get_array("lp_var_sa_obj_down")
+        values = h5.get_array("lp_var_values")
+
+        assert A is not None
+        assert b is not None
+        assert c is not None
+
+        nvars = len(c)
+        curr = 0
+        max_n_features = 40
+        features = np.zeros((nvars, max_n_features))
+
+        def push(v: np.ndarray) -> None:
+            nonlocal curr
+            assert v.shape == (nvars,), f"{v.shape} != ({nvars},)"
+            features[:, curr] = v
+            curr += 1
+
+        def push_sign_abs(v: np.ndarray) -> None:
+            assert v.shape == (nvars,), f"{v.shape} != ({nvars},)"
+            push(np.sign(v))
+            push(np.abs(v))
+
+        def maxmin(M: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
+            M_max = np.ravel(M.max(axis=0).todense())
+            M_min = np.ravel(M.min(axis=0).todense())
+            return M_max, M_min
+
+        with np.errstate(divide="ignore", invalid="ignore"):
+            # Feature 1
+            push(np.sign(c))
+
+            # Feature 2
+            c_pos_sum = c[c > 0].sum()
+            push(np.abs(c) / c_pos_sum)
+
+            # Feature 3
+            c_neg_sum = -c[c < 0].sum()
+            push(np.abs(c) / c_neg_sum)
+
+            if A is not None and self.with_m1:
+                # Compute A_ji / |b_j|
+                M1 = A.T.multiply(1.0 / np.abs(b)).T.tocsr()
+
+                # Select rows with positive b_j and compute max/min
+                M1_pos = M1[b > 0, :]
+                if M1_pos.shape[0] > 0:
+                    M1_pos_max = np.asarray(M1_pos.max(axis=0).todense()).flatten()
+                    M1_pos_min = np.asarray(M1_pos.min(axis=0).todense()).flatten()
+                else:
+                    M1_pos_max = np.zeros(nvars)
+                    M1_pos_min = np.zeros(nvars)
+
+                # Select rows with negative b_j and compute max/min
+                M1_neg = M1[b < 0, :]
+                if M1_neg.shape[0] > 0:
+                    M1_neg_max = np.asarray(M1_neg.max(axis=0).todense()).flatten()
+                    M1_neg_min = np.asarray(M1_neg.min(axis=0).todense()).flatten()
+                else:
+                    M1_neg_max = np.zeros(nvars)
+                    M1_neg_min = np.zeros(nvars)
+
+                # Features 4-11
+                push_sign_abs(M1_pos_min)
+                push_sign_abs(M1_pos_max)
+                push_sign_abs(M1_neg_min)
+                push_sign_abs(M1_neg_max)
+
+            if A is not None and self.with_m2:
+                # Compute |c_i| / A_ij
+                M2 = A.power(-1).multiply(np.abs(c)).tocsc()
+
+                # Compute max/min
+                M2_max, M2_min = maxmin(M2)
+
+                # Make copies of M2 and erase elements based on sign(c)
+                M2_pos_max = M2_max.copy()
+                M2_neg_max = M2_max.copy()
+                M2_pos_min = M2_min.copy()
+                M2_neg_min = M2_min.copy()
+                M2_pos_max[c <= 0] = 0
+                M2_pos_min[c <= 0] = 0
+                M2_neg_max[c >= 0] = 0
+                M2_neg_min[c >= 0] = 0
+
+                # Features 12-19
+                push_sign_abs(M2_pos_min)
+                push_sign_abs(M2_pos_max)
+                push_sign_abs(M2_neg_min)
+                push_sign_abs(M2_neg_max)
+
+            if A is not None and self.with_m3:
+                # Compute row sums
+                S_pos = A.maximum(0).sum(axis=1)
+                S_neg = np.abs(A.minimum(0).sum(axis=1))
+
+                # Divide A by positive and negative row sums
+                M3_pos = A.multiply(1 / S_pos).tocsr()
+                M3_neg = A.multiply(1 / S_neg).tocsr()
+
+                # Remove +inf and -inf generated by division by zero
+                M3_pos.data[~np.isfinite(M3_pos.data)] = 0.0
+                M3_neg.data[~np.isfinite(M3_neg.data)] = 0.0
+                M3_pos.eliminate_zeros()
+                M3_neg.eliminate_zeros()
+
+                # Split each matrix into positive and negative parts
+                M3_pos_pos = M3_pos.maximum(0)
+                M3_pos_neg = -(M3_pos.minimum(0))
+                M3_neg_pos = M3_neg.maximum(0)
+                M3_neg_neg = -(M3_neg.minimum(0))
+
+                # Calculate max/min
+                M3_pos_pos_max, M3_pos_pos_min = maxmin(M3_pos_pos)
+                M3_pos_neg_max, M3_pos_neg_min = maxmin(M3_pos_neg)
+                M3_neg_pos_max, M3_neg_pos_min = maxmin(M3_neg_pos)
+                M3_neg_neg_max, M3_neg_neg_min = maxmin(M3_neg_neg)
+
+                # Features 20-35
+                push_sign_abs(M3_pos_pos_max)
+                push_sign_abs(M3_pos_pos_min)
+                push_sign_abs(M3_pos_neg_max)
+                push_sign_abs(M3_pos_neg_min)
+                push_sign_abs(M3_neg_pos_max)
+                push_sign_abs(M3_neg_pos_min)
+                push_sign_abs(M3_neg_neg_max)
+                push_sign_abs(M3_neg_neg_min)
+
+            # Feature 36: only available during B&B
+
+            # Feature 37
+            if values is not None:
+                push(
+                    np.minimum(
+                        values - np.floor(values),
+                        np.ceil(values) - values,
+                    )
+                )
+
+            # Features 38-43: only available during B&B
+
+            # Feature 44
+            if c_sa_up is not None:
+                assert c_sa_down is not None
+
+                # Features 44 and 46
+                push(np.sign(c_sa_up))
+                push(np.sign(c_sa_down))
+
+                # Feature 45 is duplicated
+
+                # Feature 47-48
+                push(np.log(c - c_sa_down / np.sign(c)))
+                push(np.log(c - c_sa_up / np.sign(c)))
+
+                # Features 49-64: only available during B&B
+
+        features = features[:, 0:curr]
+        _fix_infinity(features)
+        return features
+
+    def get_constr_features(self, h5: H5File) -> np.ndarray:
+        raise NotImplemented()
+
+
+def _fix_infinity(m: Optional[np.ndarray]) -> None:
+    if m is None:
+        return
+    masked = np.ma.masked_invalid(m)  # type: ignore
+    max_values = np.max(masked, axis=0)
+    min_values = np.min(masked, axis=0)
+    m[:] = np.maximum(np.minimum(m, max_values), min_values)
+    m[~np.isfinite(m)] = 0.0

miplearn/extractors/abstract.py

@@ -0,0 +1,19 @@
+from abc import ABC, abstractmethod
+
+import numpy as np
+
+from miplearn.h5 import H5File
+
+
+class FeaturesExtractor(ABC):
+    @abstractmethod
+    def get_instance_features(self, h5: H5File) -> np.ndarray:
+        pass
+
+    @abstractmethod
+    def get_var_features(self, h5: H5File) -> np.ndarray:
+        pass
+
+    @abstractmethod
+    def get_constr_features(self, h5: H5File) -> np.ndarray:
+        pass

miplearn/extractors/dummy.py

@@ -0,0 +1,24 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+import numpy as np
+
+from miplearn.extractors.abstract import FeaturesExtractor
+from miplearn.h5 import H5File
+
+
+class DummyExtractor(FeaturesExtractor):
+    def get_instance_features(self, h5: H5File) -> np.ndarray:
+        return np.zeros(1)
+
+    def get_var_features(self, h5: H5File) -> np.ndarray:
+        var_types = h5.get_array("static_var_types")
+        assert var_types is not None
+        n_vars = len(var_types)
+        return np.zeros((n_vars, 1))
+
+    def get_constr_features(self, h5: H5File) -> np.ndarray:
+        constr_sense = h5.get_array("static_constr_sense")
+        assert constr_sense is not None
+        n_constr = len(constr_sense)
+        return np.zeros((n_constr, 1))

miplearn/extractors/fields.py

@@ -0,0 +1,69 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+from typing import Optional, List
+
+import numpy as np
+
+from miplearn.extractors.abstract import FeaturesExtractor
+from miplearn.h5 import H5File
+
+
+class H5FieldsExtractor(FeaturesExtractor):
+    def __init__(
+        self,
+        instance_fields: Optional[List[str]] = None,
+        var_fields: Optional[List[str]] = None,
+        constr_fields: Optional[List[str]] = None,
+    ):
+        self.instance_fields = instance_fields
+        self.var_fields = var_fields
+        self.constr_fields = constr_fields
+
+    def get_instance_features(self, h5: H5File) -> np.ndarray:
+        if self.instance_fields is None:
+            raise Exception("No instance fields provided")
+        x = []
+        for field in self.instance_fields:
+            try:
+                data = h5.get_array(field)
+            except ValueError:
+                data = h5.get_scalar(field)
+            assert data is not None
+            x.append(data)
+        x = np.hstack(x)
+        assert len(x.shape) == 1
+        return x
+
+    def get_var_features(self, h5: H5File) -> np.ndarray:
+        var_types = h5.get_array("static_var_types")
+        assert var_types is not None
+        n_vars = len(var_types)
+        if self.var_fields is None:
+            raise Exception("No var fields provided")
+        return self._extract(h5, self.var_fields, n_vars)
+
+    def get_constr_features(self, h5: H5File) -> np.ndarray:
+        constr_sense = h5.get_array("static_constr_sense")
+        assert constr_sense is not None
+        n_constr = len(constr_sense)
+        if self.constr_fields is None:
+            raise Exception("No constr fields provided")
+        return self._extract(h5, self.constr_fields, n_constr)
+
+    def _extract(self, h5, fields, n_expected):
+        x = []
+        for field in fields:
+            try:
+                data = h5.get_array(field)
+            except ValueError:
+                v = h5.get_scalar(field)
+                data = np.repeat(v, n_expected)
+            assert data is not None
+            assert len(data.shape) == 1
+            assert data.shape[0] == n_expected
+            x.append(data)
+        features = np.vstack(x).T
+        assert len(features.shape) == 2
+        assert features.shape[0] == n_expected
+        return features

miplearn/features/extractor.py

@@ -1,406 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-from math import log, isfinite
-from typing import TYPE_CHECKING, List, Tuple, Optional
-
-import numpy as np
-
-from miplearn.features.sample import Sample
-from miplearn.solvers.internal import LPSolveStats
-
-if TYPE_CHECKING:
-    from miplearn.solvers.internal import InternalSolver
-    from miplearn.instance.base import Instance
-
-
-class FeaturesExtractor:
-    def __init__(
-        self,
-        with_sa: bool = True,
-        with_lhs: bool = True,
-    ) -> None:
-        self.with_sa = with_sa
-        self.with_lhs = with_lhs
-
-    def extract_after_load_features(
-        self,
-        instance: "Instance",
-        solver: "InternalSolver",
-        sample: Sample,
-    ) -> None:
-        variables = solver.get_variables(with_static=True)
-        constraints = solver.get_constraints(with_static=True, with_lhs=self.with_lhs)
-        assert constraints.names is not None
-        sample.put_array("static_var_lower_bounds", variables.lower_bounds)
-        sample.put_array("static_var_names", variables.names)
-        sample.put_array("static_var_obj_coeffs", variables.obj_coeffs)
-        sample.put_array("static_var_types", variables.types)
-        sample.put_array("static_var_upper_bounds", variables.upper_bounds)
-        sample.put_array("static_constr_names", constraints.names)
-        # sample.put("static_constr_lhs", constraints.lhs)
-        sample.put_array("static_constr_rhs", constraints.rhs)
-        sample.put_array("static_constr_senses", constraints.senses)
-
-        # Instance features
-        self._extract_user_features_instance(instance, sample)
-
-        # Constraint features
-        (
-            constr_features,
-            constr_categories,
-            constr_lazy,
-        ) = FeaturesExtractor._extract_user_features_constrs(
-            instance,
-            constraints.names,
-        )
-        sample.put_array("static_constr_features", constr_features)
-        sample.put_array("static_constr_categories", constr_categories)
-        sample.put_array("static_constr_lazy", constr_lazy)
-        sample.put_scalar("static_constr_lazy_count", int(constr_lazy.sum()))
-
-        # Variable features
-        (
-            vars_features_user,
-            var_categories,
-        ) = self._extract_user_features_vars(instance, sample)
-        sample.put_array("static_var_categories", var_categories)
-        assert variables.lower_bounds is not None
-        assert variables.obj_coeffs is not None
-        assert variables.upper_bounds is not None
-        sample.put_array(
-            "static_var_features",
-            np.hstack(
-                [
-                    vars_features_user,
-                    self._extract_var_features_AlvLouWeh2017(
-                        obj_coeffs=variables.obj_coeffs,
-                    ),
-                    variables.lower_bounds.reshape(-1, 1),
-                    variables.obj_coeffs.reshape(-1, 1),
-                    variables.upper_bounds.reshape(-1, 1),
-                ]
-            ),
-        )
-
-    def extract_after_lp_features(
-        self,
-        solver: "InternalSolver",
-        sample: Sample,
-        lp_stats: LPSolveStats,
-    ) -> None:
-        for (k, v) in lp_stats.__dict__.items():
-            sample.put_scalar(k, v)
-        variables = solver.get_variables(with_static=False, with_sa=self.with_sa)
-        constraints = solver.get_constraints(with_static=False, with_sa=self.with_sa)
-        sample.put_array("lp_var_basis_status", variables.basis_status)
-        sample.put_array("lp_var_reduced_costs", variables.reduced_costs)
-        sample.put_array("lp_var_sa_lb_down", variables.sa_lb_down)
-        sample.put_array("lp_var_sa_lb_up", variables.sa_lb_up)
-        sample.put_array("lp_var_sa_obj_down", variables.sa_obj_down)
-        sample.put_array("lp_var_sa_obj_up", variables.sa_obj_up)
-        sample.put_array("lp_var_sa_ub_down", variables.sa_ub_down)
-        sample.put_array("lp_var_sa_ub_up", variables.sa_ub_up)
-        sample.put_array("lp_var_values", variables.values)
-        sample.put_array("lp_constr_basis_status", constraints.basis_status)
-        sample.put_array("lp_constr_dual_values", constraints.dual_values)
-        sample.put_array("lp_constr_sa_rhs_down", constraints.sa_rhs_down)
-        sample.put_array("lp_constr_sa_rhs_up", constraints.sa_rhs_up)
-        sample.put_array("lp_constr_slacks", constraints.slacks)
-
-        # Variable features
-        lp_var_features_list = []
-        for f in [
-            sample.get_array("static_var_features"),
-            self._extract_var_features_AlvLouWeh2017(
-                obj_coeffs=sample.get_array("static_var_obj_coeffs"),
-                obj_sa_up=variables.sa_obj_up,
-                obj_sa_down=variables.sa_obj_down,
-                values=variables.values,
-            ),
-        ]:
-            if f is not None:
-                lp_var_features_list.append(f)
-        for f in [
-            variables.reduced_costs,
-            variables.sa_lb_down,
-            variables.sa_lb_up,
-            variables.sa_obj_down,
-            variables.sa_obj_up,
-            variables.sa_ub_down,
-            variables.sa_ub_up,
-            variables.values,
-        ]:
-            if f is not None:
-                lp_var_features_list.append(f.reshape(-1, 1))
-        lp_var_features = np.hstack(lp_var_features_list)
-        _fix_infinity(lp_var_features)
-        sample.put_array("lp_var_features", lp_var_features)
-
-        # Constraint features
-        lp_constr_features_list = []
-        for f in [sample.get_array("static_constr_features")]:
-            if f is not None:
-                lp_constr_features_list.append(f)
-        for f in [
-            sample.get_array("lp_constr_dual_values"),
-            sample.get_array("lp_constr_sa_rhs_down"),
-            sample.get_array("lp_constr_sa_rhs_up"),
-            sample.get_array("lp_constr_slacks"),
-        ]:
-            if f is not None:
-                lp_constr_features_list.append(f.reshape(-1, 1))
-        lp_constr_features = np.hstack(lp_constr_features_list)
-        _fix_infinity(lp_constr_features)
-        sample.put_array("lp_constr_features", lp_constr_features)
-
-        # Build lp_instance_features
-        static_instance_features = sample.get_array("static_instance_features")
-        assert static_instance_features is not None
-        assert lp_stats.lp_value is not None
-        assert lp_stats.lp_wallclock_time is not None
-        sample.put_array(
-            "lp_instance_features",
-            np.hstack(
-                [
-                    static_instance_features,
-                    lp_stats.lp_value,
-                    lp_stats.lp_wallclock_time,
-                ]
-            ),
-        )
-
-    def extract_after_mip_features(
-        self,
-        solver: "InternalSolver",
-        sample: Sample,
-    ) -> None:
-        variables = solver.get_variables(with_static=False, with_sa=False)
-        constraints = solver.get_constraints(with_static=False, with_sa=False)
-        sample.put_array("mip_var_values", variables.values)
-        sample.put_array("mip_constr_slacks", constraints.slacks)
-
-    # noinspection DuplicatedCode
-    def _extract_user_features_vars(
-        self,
-        instance: "Instance",
-        sample: Sample,
-    ) -> Tuple[np.ndarray, np.ndarray]:
-        # Query variable names
-        var_names = sample.get_array("static_var_names")
-        assert var_names is not None
-
-        # Query variable features
-        var_features = instance.get_variable_features(var_names)
-        assert isinstance(var_features, np.ndarray), (
-            f"Variable features must be a numpy array. "
-            f"Found {var_features.__class__} instead."
-        )
-        assert len(var_features.shape) == 2, (
-            f"Variable features must be 2-dimensional array. "
-            f"Found array with shape {var_features.shape} instead."
-        )
-        assert var_features.shape[0] == len(var_names), (
-            f"Variable features must have exactly {len(var_names)} rows. "
-            f"Found {var_features.shape[0]} rows instead."
-        )
-        assert var_features.dtype.kind in ["f"], (
-            f"Variable features must be floating point numbers. "
-            f"Found {var_features.dtype} instead."
-        )
-
-        # Query variable categories
-        var_categories = instance.get_variable_categories(var_names)
-        assert isinstance(var_categories, np.ndarray), (
-            f"Variable categories must be a numpy array. "
-            f"Found {var_categories.__class__} instead."
-        )
-        assert len(var_categories.shape) == 1, (
-            f"Variable categories must be a vector. "
-            f"Found array with shape {var_categories.shape} instead."
-        )
-        assert len(var_categories) == len(var_names), (
-            f"Variable categories must have exactly {len(var_names)} elements. "
-            f"Found {var_categories.shape[0]} elements instead."
-        )
-        assert var_categories.dtype.kind == "S", (
-            f"Variable categories must be a numpy array with dtype='S'. "
-            f"Found {var_categories.dtype} instead."
-        )
-        return var_features, var_categories
-
-    # noinspection DuplicatedCode
-    @classmethod
-    def _extract_user_features_constrs(
-        cls,
-        instance: "Instance",
-        constr_names: np.ndarray,
-    ) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
-        # Query constraint features
-        constr_features = instance.get_constraint_features(constr_names)
-        assert isinstance(constr_features, np.ndarray), (
-            f"get_constraint_features must return a numpy array. "
-            f"Found {constr_features.__class__} instead."
-        )
-        assert len(constr_features.shape) == 2, (
-            f"get_constraint_features must return a 2-dimensional array. "
-            f"Found array with shape {constr_features.shape} instead."
-        )
-        assert constr_features.shape[0] == len(constr_names), (
-            f"get_constraint_features must return an array with {len(constr_names)} "
-            f"rows. Found {constr_features.shape[0]} rows instead."
-        )
-        assert constr_features.dtype.kind in ["f"], (
-            f"get_constraint_features must return floating point numbers. "
-            f"Found {constr_features.dtype} instead."
-        )
-
-        # Query constraint categories
-        constr_categories = instance.get_constraint_categories(constr_names)
-        assert isinstance(constr_categories, np.ndarray), (
-            f"get_constraint_categories must return a numpy array. "
-            f"Found {constr_categories.__class__} instead."
-        )
-        assert len(constr_categories.shape) == 1, (
-            f"get_constraint_categories must return a vector. "
-            f"Found array with shape {constr_categories.shape} instead."
-        )
-        assert len(constr_categories) == len(constr_names), (
-            f"get_constraint_categories must return a vector with {len(constr_names)} "
-            f"elements. Found {constr_categories.shape[0]} elements instead."
-        )
-        assert constr_categories.dtype.kind == "S", (
-            f"get_constraint_categories must return a numpy array with dtype='S'. "
-            f"Found {constr_categories.dtype} instead."
-        )
-
-        # Query constraint lazy attribute
-        constr_lazy = instance.are_constraints_lazy(constr_names)
-        assert isinstance(constr_lazy, np.ndarray), (
-            f"are_constraints_lazy must return a numpy array. "
-            f"Found {constr_lazy.__class__} instead."
-        )
-        assert len(constr_lazy.shape) == 1, (
-            f"are_constraints_lazy must return a vector. "
-            f"Found array with shape {constr_lazy.shape} instead."
-        )
-        assert constr_lazy.shape[0] == len(constr_names), (
-            f"are_constraints_lazy must return a vector with {len(constr_names)} "
-            f"elements. Found {constr_lazy.shape[0]} elements instead."
-        )
-        assert constr_lazy.dtype.kind == "b", (
-            f"are_constraints_lazy must return a boolean array. "
-            f"Found {constr_lazy.dtype} instead."
-        )
-
-        return constr_features, constr_categories, constr_lazy
-
-    def _extract_user_features_instance(
-        self,
-        instance: "Instance",
-        sample: Sample,
-    ) -> None:
-        features = instance.get_instance_features()
-        assert isinstance(features, np.ndarray), (
-            f"Instance features must be a numpy array. "
-            f"Found {features.__class__} instead."
-        )
-        assert len(features.shape) == 1, (
-            f"Instance features must be a vector. "
-            f"Found array with shape {features.shape} instead."
-        )
-        assert features.dtype.kind in [
-            "f"
-        ], f"Instance features have unsupported {features.dtype}"
-        sample.put_array("static_instance_features", features)
-
-    # Alvarez, A. M., Louveaux, Q., & Wehenkel, L. (2017). A machine learning-based
-    # approximation of strong branching. INFORMS Journal on Computing, 29(1), 185-195.
-    # noinspection PyPep8Naming
-    def _extract_var_features_AlvLouWeh2017(
-        self,
-        obj_coeffs: Optional[np.ndarray] = None,
-        obj_sa_down: Optional[np.ndarray] = None,
-        obj_sa_up: Optional[np.ndarray] = None,
-        values: Optional[np.ndarray] = None,
-    ) -> np.ndarray:
-        assert obj_coeffs is not None
-        obj_coeffs = obj_coeffs.astype(float)
-        _fix_infinity(obj_coeffs)
-        nvars = len(obj_coeffs)
-
-        if obj_sa_down is not None:
-            obj_sa_down = obj_sa_down.astype(float)
-            _fix_infinity(obj_sa_down)
-
-        if obj_sa_up is not None:
-            obj_sa_up = obj_sa_up.astype(float)
-            _fix_infinity(obj_sa_up)
-
-        if values is not None:
-            values = values.astype(float)
-            _fix_infinity(values)
-
-        pos_obj_coeffs_sum = obj_coeffs[obj_coeffs > 0].sum()
-        neg_obj_coeffs_sum = -obj_coeffs[obj_coeffs < 0].sum()
-
-        curr = 0
-        max_n_features = 8
-        features = np.zeros((nvars, max_n_features))
-        with np.errstate(divide="ignore", invalid="ignore"):
-            # Feature 1
-            features[:, curr] = np.sign(obj_coeffs)
-            curr += 1
-
-            # Feature 2
-            if abs(pos_obj_coeffs_sum) > 0:
-                features[:, curr] = np.abs(obj_coeffs) / pos_obj_coeffs_sum
-                curr += 1
-
-            # Feature 3
-            if abs(neg_obj_coeffs_sum) > 0:
-                features[:, curr] = np.abs(obj_coeffs) / neg_obj_coeffs_sum
-                curr += 1
-
-            # Feature 37
-            if values is not None:
-                features[:, curr] = np.minimum(
-                    values - np.floor(values),
-                    np.ceil(values) - values,
-                )
-                curr += 1
-
-            # Feature 44
-            if obj_sa_up is not None:
-                features[:, curr] = np.sign(obj_sa_up)
-                curr += 1
-
-            # Feature 46
-            if obj_sa_down is not None:
-                features[:, curr] = np.sign(obj_sa_down)
-                curr += 1
-
-            # Feature 47
-            if obj_sa_down is not None:
-                features[:, curr] = np.log(
-                    obj_coeffs - obj_sa_down / np.sign(obj_coeffs)
-                )
-                curr += 1
-
-            # Feature 48
-            if obj_sa_up is not None:
-                features[:, curr] = np.log(obj_coeffs - obj_sa_up / np.sign(obj_coeffs))
-                curr += 1
-
-        features = features[:, 0:curr]
-        _fix_infinity(features)
-        return features
-
-
-def _fix_infinity(m: np.ndarray) -> None:
-    masked = np.ma.masked_invalid(m)
-    max_values = np.max(masked, axis=0)
-    min_values = np.min(masked, axis=0)
-    m[:] = np.maximum(np.minimum(m, max_values), min_values)
-    m[np.isnan(m)] = 0.0

miplearn/h5.py

@@ -1,19 +1,18 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
-import warnings
+
-from abc import ABC, abstractmethod
+from types import TracebackType
-from copy import deepcopy
+from typing import Optional, Any, Union, List, Type, Literal
-from typing import Dict, Optional, Any, Union, List, Tuple, cast, Set
-from scipy.sparse import coo_matrix
 
 import h5py
 import numpy as np
-from h5py import Dataset
+from scipy.sparse import coo_matrix
-from overrides import overrides
 
 Bytes = Union[bytes, bytearray]
+
 Scalar = Union[None, bool, str, int, float]
+
 Vector = Union[
     None,
     List[bool],
@@ -23,6 +22,7 @@ Vector = Union[
     List[Optional[str]],
     np.ndarray,
 ]
+
 VectorList = Union[
     List[List[bool]],
     List[List[str]],
@@ -35,113 +35,7 @@ VectorList = Union[
 ]
 
 
-class Sample(ABC):
+class H5File:
-    """Abstract dictionary-like class that stores training data."""
-
-    @abstractmethod
-    def get_scalar(self, key: str) -> Optional[Any]:
-        pass
-
-    @abstractmethod
-    def put_scalar(self, key: str, value: Scalar) -> None:
-        pass
-
-    @abstractmethod
-    def put_array(self, key: str, value: Optional[np.ndarray]) -> None:
-        pass
-
-    @abstractmethod
-    def get_array(self, key: str) -> Optional[np.ndarray]:
-        pass
-
-    @abstractmethod
-    def put_sparse(self, key: str, value: coo_matrix) -> None:
-        pass
-
-    @abstractmethod
-    def get_sparse(self, key: str) -> Optional[coo_matrix]:
-        pass
-
-    def _assert_is_scalar(self, value: Any) -> None:
-        if value is None:
-            return
-        if isinstance(value, (str, bool, int, float, bytes, np.bytes_)):
-            return
-        assert False, f"scalar expected; found instead: {value} ({value.__class__})"
-
-    def _assert_is_array(self, value: np.ndarray) -> None:
-        assert isinstance(
-            value, np.ndarray
-        ), f"np.ndarray expected; found instead: {value.__class__}"
-        assert value.dtype.kind in "biufS", f"Unsupported dtype: {value.dtype}"
-
-    def _assert_is_sparse(self, value: Any) -> None:
-        assert isinstance(value, coo_matrix)
-        self._assert_is_array(value.data)
-
-
-class MemorySample(Sample):
-    """Dictionary-like class that stores training data in-memory."""
-
-    def __init__(
-        self,
-        data: Optional[Dict[str, Any]] = None,
-    ) -> None:
-        if data is None:
-            data = {}
-        self._data: Dict[str, Any] = data
-
-    @overrides
-    def get_scalar(self, key: str) -> Optional[Any]:
-        return self._get(key)
-
-    @overrides
-    def put_scalar(self, key: str, value: Scalar) -> None:
-        if value is None:
-            return
-        self._assert_is_scalar(value)
-        self._put(key, value)
-
-    def _get(self, key: str) -> Optional[Any]:
-        if key in self._data:
-            return self._data[key]
-        else:
-            return None
-
-    def _put(self, key: str, value: Any) -> None:
-        self._data[key] = value
-
-    @overrides
-    def put_array(self, key: str, value: Optional[np.ndarray]) -> None:
-        if value is None:
-            return
-        self._assert_is_array(value)
-        self._put(key, value)
-
-    @overrides
-    def get_array(self, key: str) -> Optional[np.ndarray]:
-        return cast(Optional[np.ndarray], self._get(key))
-
-    @overrides
-    def put_sparse(self, key: str, value: coo_matrix) -> None:
-        if value is None:
-            return
-        self._assert_is_sparse(value)
-        self._put(key, value)
-
-    @overrides
-    def get_sparse(self, key: str) -> Optional[coo_matrix]:
-        return cast(Optional[coo_matrix], self._get(key))
-
-
-class Hdf5Sample(Sample):
-    """
-    Dictionary-like class that stores training data in an HDF5 file.
-
-    Unlike MemorySample, this class only loads to memory the parts of the data set that
-    are actually accessed, and therefore it is more scalable.
-    """
-
     def __init__(
         self,
         filename: str,
@@ -149,7 +43,6 @@ class Hdf5Sample(Sample):
     ) -> None:
         self.file = h5py.File(filename, mode, libver="latest")
 
-    @overrides
     def get_scalar(self, key: str) -> Optional[Any]:
         if key not in self.file:
             return None
@@ -162,7 +55,6 @@ class Hdf5Sample(Sample):
         else:
             return ds[()].tolist()
 
-    @overrides
     def put_scalar(self, key: str, value: Any) -> None:
         if value is None:
             return
@@ -171,24 +63,21 @@ class Hdf5Sample(Sample):
             del self.file[key]
         self.file.create_dataset(key, data=value)
 
-    @overrides
     def put_array(self, key: str, value: Optional[np.ndarray]) -> None:
         if value is None:
             return
         self._assert_is_array(value)
-        if len(value.shape) > 1 and value.dtype.kind == "f":
+        if value.dtype.kind == "f":
-            value = value.astype("float16")
+            value = value.astype("float32")
         if key in self.file:
             del self.file[key]
         return self.file.create_dataset(key, data=value, compression="gzip")
 
-    @overrides
     def get_array(self, key: str) -> Optional[np.ndarray]:
         if key not in self.file:
             return None
         return self.file[key][:]
 
-    @overrides
     def put_sparse(self, key: str, value: coo_matrix) -> None:
         if value is None:
             return
@@ -197,7 +86,6 @@ class Hdf5Sample(Sample):
         self.put_array(f"{key}_col", value.col)
         self.put_array(f"{key}_data", value.data)
 
-    @overrides
     def get_sparse(self, key: str) -> Optional[coo_matrix]:
         row = self.get_array(f"{key}_row")
         if row is None:
@@ -222,3 +110,37 @@ class Hdf5Sample(Sample):
             value, (bytes, bytearray)
         ), f"bytes expected; found: {value.__class__}"  # type: ignore
         self.put_array(key, np.frombuffer(value, dtype="uint8"))
+
+    def close(self):
+        self.file.close()
+
+    def __enter__(self) -> "H5File":
+        return self
+
+    def __exit__(
+        self,
+        exc_type: Optional[Type[BaseException]],
+        exc_val: Optional[BaseException],
+        exc_tb: Optional[TracebackType],
+    ) -> Literal[False]:
+        self.file.close()
+        return False
+
+    def _assert_is_scalar(self, value: Any) -> None:
+        if value is None:
+            return
+        if isinstance(value, (str, bool, int, float, bytes, np.bytes_)):
+            return
+        assert False, f"scalar expected; found instead: {value} ({value.__class__})"
+
+    def _assert_is_array(self, value: np.ndarray) -> None:
+        assert isinstance(
+            value, np.ndarray
+        ), f"np.ndarray expected; found instead: {value.__class__}"
+        assert value.dtype.kind in "biufS", f"Unsupported dtype: {value.dtype}"
+
+    def _assert_is_sparse(self, value: Any) -> None:
+        assert isinstance(
+            value, coo_matrix
+        ), f"coo_matrix expected; found: {value.__class__}"
+        self._assert_is_array(value.data)

miplearn/instance/__init__.py

@@ -1,3 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.

miplearn/instance/base.py

@@ -1,198 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from abc import ABC, abstractmethod
-from typing import Any, List, TYPE_CHECKING, Dict
-
-import numpy as np
-
-from miplearn.features.sample import Sample, MemorySample
-from miplearn.types import ConstraintName, ConstraintCategory
-
-logger = logging.getLogger(__name__)
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import InternalSolver
-
-
-# noinspection PyMethodMayBeStatic
-class Instance(ABC):
-    """
-    Abstract class holding all the data necessary to generate a concrete model of the
-    proble.
-
-    In the knapsack problem, for example, this class could hold the number of items,
-    their weights and costs, as well as the size of the knapsack. Objects
-    implementing this class are able to convert themselves into a concrete
-    optimization model, which can be optimized by a solver, or into arrays of
-    features, which can be provided as inputs to machine learning models.
-    """
-
-    def __init__(self) -> None:
-        self._samples: List[Sample] = []
-
-    @abstractmethod
-    def to_model(self) -> Any:
-        """
-        Returns the optimization model corresponding to this instance.
-        """
-        pass
-
-    def get_instance_features(self) -> np.ndarray:
-        """
-        Returns a 1-dimensional array of (numerical) features describing the
-        entire instance.
-
-        The array is used by LearningSolver to determine how similar two instances
-        are. It may also be used to predict, in combination with variable-specific
-        features, the values of binary decision variables in the problem.
-
-        There is not necessarily a one-to-one correspondence between models and
-        instance features: the features may encode only part of the data necessary to
-        generate the complete model. Features may also be statistics computed from
-        the original data. For example, in the knapsack problem, an implementation
-        may decide to provide as instance features only the average weights, average
-        prices, number of items and the size of the knapsack.
-
-        The returned array MUST have the same length for all relevant instances of
-        the problem. If two instances map into arrays of different lengths,
-        they cannot be solved by the same LearningSolver object.
-
-        By default, returns [0.0].
-        """
-        return np.zeros(1)
-
-    def get_variable_features(self, names: np.ndarray) -> np.ndarray:
-        """
-        Returns dictionary mapping the name of each variable to a (1-dimensional) list
-        of numerical features describing a particular decision variable.
-
-        In combination with instance features, variable features are used by
-        LearningSolver to predict, among other things, the optimal value of each
-        decision variable before the optimization takes place. In the knapsack
-        problem, for example, an implementation could provide as variable features
-        the weight and the price of a specific item.
-
-        Like instance features, the arrays returned by this method MUST have the same
-        length for all variables within the same category, for all relevant instances
-        of the problem.
-
-        If features are not provided for a given variable, MIPLearn will use a
-        default set of features.
-
-        By default, returns [[0.0], ..., [0.0]].
-        """
-        return np.zeros((len(names), 1))
-
-    def get_variable_categories(self, names: np.ndarray) -> np.ndarray:
-        """
-        Returns a dictionary mapping the name of each variable to its category.
-
-        If two variables have the same category, LearningSolver will use the same
-        internal ML model to predict the values of both variables. If a variable is not
-        listed in the dictionary, ML models will ignore the variable.
-
-        By default, returns `names`.
-        """
-        return names
-
-    def get_constraint_features(self, names: np.ndarray) -> np.ndarray:
-        return np.zeros((len(names), 1))
-
-    def get_constraint_categories(self, names: np.ndarray) -> np.ndarray:
-        return names
-
-    def has_dynamic_lazy_constraints(self) -> bool:
-        return False
-
-    def are_constraints_lazy(self, names: np.ndarray) -> np.ndarray:
-        return np.zeros(len(names), dtype=bool)
-
-    def find_violated_lazy_constraints(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-    ) -> List[ConstraintName]:
-        """
-        Returns lazy constraint violations found for the current solution.
-
-        After solving a model, LearningSolver will ask the instance to identify which
-        lazy constraints are violated by the current solution. For each identified
-        violation, LearningSolver will then call the enforce_lazy_constraint and
-        resolve the problem. The process repeats until no further lazy constraint
-        violations are found.
-
-        Each "violation" is simply a string which allows the instance to identify
-        unambiguously which lazy constraint should be generated. In the Traveling
-        Salesman Problem, for example, a subtour violation could be a string
-        containing the cities in the subtour.
-
-        The current solution can be queried with `solver.get_solution()`. If the solver
-        is configured to use lazy callbacks, this solution may be non-integer.
-
-        For a concrete example, see TravelingSalesmanInstance.
-        """
-        return []
-
-    def enforce_lazy_constraint(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-        violation: ConstraintName,
-    ) -> None:
-        """
-        Adds constraints to the model to ensure that the given violation is fixed.
-
-        This method is typically called immediately after
-        find_violated_lazy_constraints. The violation object provided to this method
-        is exactly the same object returned earlier by
-        find_violated_lazy_constraints. After some training, LearningSolver may
-        decide to proactively build some lazy constraints at the beginning of the
-        optimization process, before a solution is even available. In this case,
-        enforce_lazy_constraints will be called without a corresponding call to
-        find_violated_lazy_constraints.
-
-        Note that this method can be called either before the optimization starts or
-        from within a callback. To ensure that constraints are added correctly in
-        either case, it is recommended to use `solver.add_constraint`, instead of
-        modifying the `model` object directly.
-
-        For a concrete example, see TravelingSalesmanInstance.
-        """
-        pass
-
-    def has_user_cuts(self) -> bool:
-        return False
-
-    def find_violated_user_cuts(self, model: Any) -> List[ConstraintName]:
-        return []
-
-    def enforce_user_cut(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-        violation: ConstraintName,
-    ) -> Any:
-        return None
-
-    def load(self) -> None:
-        pass
-
-    def free(self) -> None:
-        pass
-
-    def flush(self) -> None:
-        """
-        Save any pending changes made to the instance to the underlying data store.
-        """
-        pass
-
-    def get_samples(self) -> List[Sample]:
-        return self._samples
-
-    def create_sample(self) -> Sample:
-        sample = MemorySample()
-        self._samples.append(sample)
-        return sample

miplearn/instance/file.py

@@ -1,131 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-import gc
-import os
-from typing import Any, Optional, List, Dict, TYPE_CHECKING
-import pickle
-
-import numpy as np
-from overrides import overrides
-
-from miplearn.features.sample import Hdf5Sample, Sample
-from miplearn.instance.base import Instance
-from miplearn.types import ConstraintName, ConstraintCategory
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import InternalSolver
-
-
-class FileInstance(Instance):
-    def __init__(self, filename: str) -> None:
-        super().__init__()
-        assert os.path.exists(filename), f"File not found: {filename}"
-        self.h5 = Hdf5Sample(filename)
-        self.instance: Optional[Instance] = None
-
-    # Delegation
-    # -------------------------------------------------------------------------
-    @overrides
-    def to_model(self) -> Any:
-        assert self.instance is not None
-        return self.instance.to_model()
-
-    @overrides
-    def get_instance_features(self) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_instance_features()
-
-    @overrides
-    def get_variable_features(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_variable_features(names)
-
-    @overrides
-    def get_variable_categories(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_variable_categories(names)
-
-    @overrides
-    def get_constraint_features(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_constraint_features(names)
-
-    @overrides
-    def get_constraint_categories(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_constraint_categories(names)
-
-    @overrides
-    def has_dynamic_lazy_constraints(self) -> bool:
-        assert self.instance is not None
-        return self.instance.has_dynamic_lazy_constraints()
-
-    @overrides
-    def are_constraints_lazy(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.are_constraints_lazy(names)
-
-    @overrides
-    def find_violated_lazy_constraints(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-    ) -> List[ConstraintName]:
-        assert self.instance is not None
-        return self.instance.find_violated_lazy_constraints(solver, model)
-
-    @overrides
-    def enforce_lazy_constraint(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-        violation: ConstraintName,
-    ) -> None:
-        assert self.instance is not None
-        self.instance.enforce_lazy_constraint(solver, model, violation)
-
-    @overrides
-    def find_violated_user_cuts(self, model: Any) -> List[ConstraintName]:
-        assert self.instance is not None
-        return self.instance.find_violated_user_cuts(model)
-
-    @overrides
-    def enforce_user_cut(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-        violation: ConstraintName,
-    ) -> None:
-        assert self.instance is not None
-        self.instance.enforce_user_cut(solver, model, violation)
-
-    # Input & Output
-    # -------------------------------------------------------------------------
-    @overrides
-    def free(self) -> None:
-        self.instance = None
-        gc.collect()
-
-    @overrides
-    def load(self) -> None:
-        if self.instance is not None:
-            return
-        pkl = self.h5.get_bytes("pickled")
-        assert pkl is not None
-        self.instance = pickle.loads(pkl)
-        assert isinstance(self.instance, Instance)
-
-    @classmethod
-    def save(cls, instance: Instance, filename: str) -> None:
-        h5 = Hdf5Sample(filename, mode="w")
-        instance_pkl = pickle.dumps(instance)
-        h5.put_bytes("pickled", instance_pkl)
-
-    @overrides
-    def create_sample(self) -> Sample:
-        return self.h5
-
-    @overrides
-    def get_samples(self) -> List[Sample]:
-        return [self.h5]

miplearn/instance/picklegz.py

@@ -1,155 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import gc
-import gzip
-import os
-import pickle
-from typing import Optional, Any, List, cast, IO, TYPE_CHECKING, Dict
-
-import numpy as np
-from overrides import overrides
-
-from miplearn.features.sample import Sample
-from miplearn.instance.base import Instance
-from miplearn.types import ConstraintName, ConstraintCategory
-
-if TYPE_CHECKING:
-    from miplearn.solvers.learning import InternalSolver
-
-
-class PickleGzInstance(Instance):
-    """
-    An instance backed by a gzipped pickle file.
-
-    The instance is only loaded to memory after an operation is called (for example,
-    `to_model`).
-
-    Parameters
-    ----------
-    filename: str
-        Path of the gzipped pickle file that should be loaded.
-    """
-
-    # noinspection PyMissingConstructor
-    def __init__(self, filename: str) -> None:
-        assert os.path.exists(filename), f"File not found: {filename}"
-        self.instance: Optional[Instance] = None
-        self.filename: str = filename
-
-    @overrides
-    def to_model(self) -> Any:
-        assert self.instance is not None
-        return self.instance.to_model()
-
-    @overrides
-    def get_instance_features(self) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_instance_features()
-
-    @overrides
-    def get_variable_features(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_variable_features(names)
-
-    @overrides
-    def get_variable_categories(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_variable_categories(names)
-
-    @overrides
-    def get_constraint_features(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_constraint_features(names)
-
-    @overrides
-    def get_constraint_categories(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.get_constraint_categories(names)
-
-    @overrides
-    def has_dynamic_lazy_constraints(self) -> bool:
-        assert self.instance is not None
-        return self.instance.has_dynamic_lazy_constraints()
-
-    @overrides
-    def are_constraints_lazy(self, names: np.ndarray) -> np.ndarray:
-        assert self.instance is not None
-        return self.instance.are_constraints_lazy(names)
-
-    @overrides
-    def find_violated_lazy_constraints(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-    ) -> List[ConstraintName]:
-        assert self.instance is not None
-        return self.instance.find_violated_lazy_constraints(solver, model)
-
-    @overrides
-    def enforce_lazy_constraint(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-        violation: ConstraintName,
-    ) -> None:
-        assert self.instance is not None
-        self.instance.enforce_lazy_constraint(solver, model, violation)
-
-    @overrides
-    def find_violated_user_cuts(self, model: Any) -> List[ConstraintName]:
-        assert self.instance is not None
-        return self.instance.find_violated_user_cuts(model)
-
-    @overrides
-    def enforce_user_cut(
-        self,
-        solver: "InternalSolver",
-        model: Any,
-        violation: ConstraintName,
-    ) -> None:
-        assert self.instance is not None
-        self.instance.enforce_user_cut(solver, model, violation)
-
-    @overrides
-    def load(self) -> None:
-        if self.instance is None:
-            obj = read_pickle_gz(self.filename)
-            assert isinstance(obj, Instance)
-            self.instance = obj
-
-    @overrides
-    def free(self) -> None:
-        self.instance = None  # type: ignore
-        gc.collect()
-
-    @overrides
-    def flush(self) -> None:
-        write_pickle_gz(self.instance, self.filename)
-
-    @overrides
-    def get_samples(self) -> List[Sample]:
-        assert self.instance is not None
-        return self.instance.get_samples()
-
-    @overrides
-    def create_sample(self) -> Sample:
-        assert self.instance is not None
-        return self.instance.create_sample()
-
-
-def write_pickle_gz(obj: Any, filename: str) -> None:
-    os.makedirs(os.path.dirname(filename), exist_ok=True)
-    with gzip.GzipFile(filename, "wb") as file:
-        pickle.dump(obj, cast(IO[bytes], file))
-
-
-def read_pickle_gz(filename: str) -> Any:
-    with gzip.GzipFile(filename, "rb") as file:
-        return pickle.load(cast(IO[bytes], file))
-
-
-def write_pickle_gz_multiple(objs: List[Any], dirname: str) -> None:
-    for (i, obj) in enumerate(objs):
-        write_pickle_gz(obj, f"{dirname}/{i:05d}.pkl.gz")

miplearn/io.py

@@ -0,0 +1,92 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from gzip import GzipFile
+import os
+import pickle
+import sys
+from typing import IO, Any, Callable, List, cast, TextIO
+
+from .parallel import p_umap
+import shutil
+
+
+class _RedirectOutput:
+    def __init__(self, streams: List[Any]) -> None:
+        self.streams = streams
+
+    def write(self, data: Any) -> None:
+        for stream in self.streams:
+            stream.write(data)
+
+    def flush(self) -> None:
+        for stream in self.streams:
+            stream.flush()
+
+    def __enter__(self) -> Any:
+        self._original_stdout = sys.stdout
+        self._original_stderr = sys.stderr
+        sys.stdout = cast(TextIO, self)
+        sys.stderr = cast(TextIO, self)
+        return self
+
+    def __exit__(
+        self,
+        _type: Any,
+        _value: Any,
+        _traceback: Any,
+    ) -> None:
+        sys.stdout = self._original_stdout
+        sys.stderr = self._original_stderr
+
+
+def write_pkl_gz(
+    objs: List[Any],
+    dirname: str,
+    prefix: str = "",
+    n_jobs: int = 1,
+    progress: bool = False,
+) -> List[str]:
+    filenames = [f"{dirname}/{prefix}{i:05d}.pkl.gz" for i in range(len(objs))]
+
+    def _process(i: int) -> None:
+        filename = filenames[i]
+        obj = objs[i]
+        os.makedirs(os.path.dirname(filename), exist_ok=True)
+        with GzipFile(filename, "wb") as file:
+            pickle.dump(obj, cast(IO[bytes], file))
+
+    if n_jobs > 1:
+        p_umap(
+            _process,
+            range(len(objs)),
+            smoothing=0,
+            num_cpus=n_jobs,
+            maxtasksperchild=None,
+            disable=not progress,
+        )
+    else:
+        for i in range(len(objs)):
+            _process(i)
+    return filenames
+
+
+def gzip(filename: str) -> None:
+    with open(filename, "rb") as input_file:
+        with GzipFile(f"{filename}.gz", "wb") as output_file:
+            shutil.copyfileobj(input_file, output_file)
+    os.remove(filename)
+
+
+def read_pkl_gz(filename: str) -> Any:
+    with GzipFile(filename, "rb") as file:
+        return pickle.load(cast(IO[bytes], file))
+
+
+def _to_h5_filename(data_filename: str) -> str:
+    output = f"{data_filename}.h5"
+    output = output.replace(".pkl.gz.h5", ".h5")
+    output = output.replace(".pkl.h5", ".h5")
+    output = output.replace(".jld2.h5", ".h5")
+    return output

miplearn/log.py

@@ -1,72 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-import sys
-import time
-import traceback
-import warnings
-from typing import Dict, Any, Optional
-
-_formatwarning = warnings.formatwarning
-
-
-class TimeFormatter(logging.Formatter):
-    def __init__(
-        self,
-        start_time: float,
-        log_colors: Dict[str, str],
-    ) -> None:
-        super().__init__()
-        self.start_time = start_time
-        self.log_colors = log_colors
-
-    def format(self, record: logging.LogRecord) -> str:
-        if record.levelno >= logging.ERROR:
-            color = self.log_colors["red"]
-        elif record.levelno >= logging.WARNING:
-            color = self.log_colors["yellow"]
-        else:
-            color = self.log_colors["green"]
-        return "%s[%12.3f]%s %s" % (
-            color,
-            record.created - self.start_time,
-            self.log_colors["reset"],
-            record.getMessage(),
-        )
-
-
-def formatwarning_tb(*args: Any, **kwargs: Any) -> str:
-    s = _formatwarning(*args, **kwargs)
-    tb = traceback.format_stack()
-    s += "".join(tb[:-1])
-    return s
-
-
-def setup_logger(
-    start_time: Optional[float] = None,
-    force_color: bool = False,
-) -> None:
-    if start_time is None:
-        start_time = time.time()
-    if sys.stdout.isatty() or force_color:
-        log_colors = {
-            "green": "\033[92m",
-            "yellow": "\033[93m",
-            "red": "\033[91m",
-            "reset": "\033[0m",
-        }
-    else:
-        log_colors = {
-            "green": "",
-            "yellow": "",
-            "red": "",
-            "reset": "",
-        }
-    handler = logging.StreamHandler()
-    handler.setFormatter(TimeFormatter(start_time, log_colors))
-    logging.getLogger().addHandler(handler)
-    logging.getLogger("miplearn").setLevel(logging.INFO)
-    warnings.formatwarning = formatwarning_tb
-    logging.captureWarnings(True)

miplearn/parallel.py

@@ -0,0 +1,32 @@
+# Modified version of: https://github.com/swansonk14/p_tqdm
+# Copyright (c) 2022 Kyle Swanson
+# MIT License
+
+from collections.abc import Sized
+from typing import Any, Callable, Generator, Iterable, List
+
+from pathos.multiprocessing import _ProcessPool as Pool
+from tqdm.auto import tqdm
+
+
+def _parallel(function: Callable, *iterables: Iterable, **kwargs: Any) -> Generator:
+    # Determine length of tqdm (equal to length of the shortest iterable or total kwarg)
+    total = kwargs.pop("total", None)
+    lengths = [len(iterable) for iterable in iterables if isinstance(iterable, Sized)]
+    length = total or (min(lengths) if lengths else None)
+
+    # Create parallel generator
+    num_cpus = kwargs.pop("num_cpus", 1)
+    maxtasksperchild = kwargs.pop("maxtasksperchild", 1)
+    chunksize = kwargs.pop("chunksize", 1)
+    with Pool(num_cpus, maxtasksperchild=maxtasksperchild) as pool:
+        for item in tqdm(
+            pool.imap_unordered(function, *iterables, chunksize=chunksize),
+            total=length,
+            **kwargs
+        ):
+            yield item
+
+
+def p_umap(function: Callable, *iterables: Iterable, **kwargs: Any) -> List[Any]:
+    return list(_parallel(function, *iterables, **kwargs))

miplearn/problems/__init__.py

@@ -1,3 +1,3 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.

miplearn/problems/binpack.py

@@ -0,0 +1,146 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from dataclasses import dataclass
+from typing import List, Optional, Union
+
+import gurobipy as gp
+import numpy as np
+from gurobipy import GRB, quicksum
+from scipy.stats import uniform, randint
+from scipy.stats.distributions import rv_frozen
+
+from miplearn.io import read_pkl_gz
+from miplearn.solvers.gurobi import GurobiModel
+
+
+@dataclass
+class BinPackData:
+    """Data for the bin packing problem.
+
+    Parameters
+    ----------
+    sizes
+        Sizes of the items
+    capacity
+        Capacity of the bin
+    """
+
+    sizes: np.ndarray
+    capacity: int
+
+
+class BinPackGenerator:
+    """Random instance generator for the bin packing problem.
+
+    If `fix_items=False`, the class samples the user-provided probability distributions
+    n, sizes and capacity to decide, respectively, the number of items, the sizes of
+    the items and capacity of the bin. All values are sampled independently.
+
+    If `fix_items=True`, the class creates a reference instance, using the method
+    previously described, then generates additional instances by perturbing its item
+    sizes and bin capacity. More specifically, the sizes of the items are set to `s_i
+    * gamma_i` where `s_i` is the size of the i-th item in the reference instance and
+    `gamma_i` is sampled from `sizes_jitter`. Similarly, the bin capacity is set to `B *
+    beta`, where `B` is the reference bin capacity and `beta` is sampled from
+    `capacity_jitter`. The number of items remains the same across all generated
+    instances.
+
+    Args
+    ----
+    n
+        Probability distribution for the number of items.
+    sizes
+        Probability distribution for the item sizes.
+    capacity
+        Probability distribution for the bin capacity.
+    sizes_jitter
+        Probability distribution for the item size randomization.
+    capacity_jitter
+        Probability distribution for the bin capacity.
+    fix_items
+        If `True`, generates a reference instance, then applies some perturbation to it.
+        If `False`, generates completely different instances.
+    """
+
+    def __init__(
+        self,
+        n: rv_frozen,
+        sizes: rv_frozen,
+        capacity: rv_frozen,
+        sizes_jitter: rv_frozen,
+        capacity_jitter: rv_frozen,
+        fix_items: bool,
+    ) -> None:
+        self.n = n
+        self.sizes = sizes
+        self.capacity = capacity
+        self.sizes_jitter = sizes_jitter
+        self.capacity_jitter = capacity_jitter
+        self.fix_items = fix_items
+        self.ref_data: Optional[BinPackData] = None
+
+    def generate(self, n_samples: int) -> List[BinPackData]:
+        """Generates random instances.
+
+        Parameters
+        ----------
+        n_samples
+            Number of samples to generate.
+        """
+
+        def _sample() -> BinPackData:
+            if self.ref_data is None:
+                n = self.n.rvs()
+                sizes = self.sizes.rvs(n)
+                capacity = self.capacity.rvs()
+                if self.fix_items:
+                    self.ref_data = BinPackData(sizes, capacity)
+            else:
+                n = self.ref_data.sizes.shape[0]
+                sizes = self.ref_data.sizes
+                capacity = self.ref_data.capacity
+
+            sizes = sizes * self.sizes_jitter.rvs(n)
+            capacity = capacity * self.capacity_jitter.rvs()
+            return BinPackData(sizes.round(2), capacity.round(2))
+
+        return [_sample() for n in range(n_samples)]
+
+
+def build_binpack_model(data: Union[str, BinPackData]) -> GurobiModel:
+    """Converts bin packing problem data into a concrete Gurobipy model."""
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, BinPackData)
+
+    model = gp.Model()
+    n = data.sizes.shape[0]
+
+    # Var: Use bin
+    y = model.addVars(n, name="y", vtype=GRB.BINARY)
+
+    # Var: Assign item to bin
+    x = model.addVars(n, n, name="x", vtype=GRB.BINARY)
+
+    # Obj: Minimize number of bins
+    model.setObjective(quicksum(y[i] for i in range(n)))
+
+    # Eq: Enforce bin capacity
+    model.addConstrs(
+        (
+            quicksum(data.sizes[i] * x[i, j] for i in range(n)) <= data.capacity * y[j]
+            for j in range(n)
+        ),
+        name="eq_capacity",
+    )
+
+    # Eq: Must assign all items to bins
+    model.addConstrs(
+        (quicksum(x[i, j] for j in range(n)) == 1 for i in range(n)),
+        name="eq_assign",
+    )
+
+    model.update()
+    return GurobiModel(model)

miplearn/problems/knapsack.py

@@ -1,247 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-from typing import List, Dict, Optional
-
-import numpy as np
-import pyomo.environ as pe
-from overrides import overrides
-from scipy.stats import uniform, randint, rv_discrete
-from scipy.stats.distributions import rv_frozen
-
-from miplearn.instance.base import Instance
-
-
-class ChallengeA:
-    """
-    - 250 variables, 10 constraints, fixed weights
-    - w ~ U(0, 1000), jitter ~ U(0.95, 1.05)
-    - K = 500, u ~ U(0., 1.)
-    - alpha = 0.25
-    """
-
-    def __init__(
-        self,
-        seed: int = 42,
-        n_training_instances: int = 500,
-        n_test_instances: int = 50,
-    ) -> None:
-        np.random.seed(seed)
-        self.gen = MultiKnapsackGenerator(
-            n=randint(low=250, high=251),
-            m=randint(low=10, high=11),
-            w=uniform(loc=0.0, scale=1000.0),
-            K=uniform(loc=500.0, scale=0.0),
-            u=uniform(loc=0.0, scale=1.0),
-            alpha=uniform(loc=0.25, scale=0.0),
-            fix_w=True,
-            w_jitter=uniform(loc=0.95, scale=0.1),
-        )
-        np.random.seed(seed + 1)
-        self.training_instances = self.gen.generate(n_training_instances)
-
-        np.random.seed(seed + 2)
-        self.test_instances = self.gen.generate(n_test_instances)
-
-
-class MultiKnapsackInstance(Instance):
-    """Representation of the Multidimensional 0-1 Knapsack Problem.
-
-    Given a set of n items and m knapsacks, the problem is to find a subset of items
-    S maximizing sum(prices[i] for i in S). If selected, each item i occupies
-    weights[i,j] units of space in each knapsack j. Furthermore, each knapsack j has
-    limited storage space, given by capacities[j].
-
-    This implementation assigns a different category for each decision variable,
-    and therefore trains one ML model per variable. It is only suitable when training
-    and test instances have same size and items don't shuffle around.
-    """
-
-    def __init__(
-        self,
-        prices: np.ndarray,
-        capacities: np.ndarray,
-        weights: np.ndarray,
-    ) -> None:
-        super().__init__()
-        assert isinstance(prices, np.ndarray)
-        assert isinstance(capacities, np.ndarray)
-        assert isinstance(weights, np.ndarray)
-        assert len(weights.shape) == 2
-        self.m, self.n = weights.shape
-        assert prices.shape == (self.n,)
-        assert capacities.shape == (self.m,)
-        self.prices = prices
-        self.capacities = capacities
-        self.weights = weights
-
-    @overrides
-    def to_model(self) -> pe.ConcreteModel:
-        model = pe.ConcreteModel()
-        model.x = pe.Var(range(self.n), domain=pe.Binary)
-        model.OBJ = pe.Objective(
-            expr=sum(model.x[j] * self.prices[j] for j in range(self.n)),
-            sense=pe.maximize,
-        )
-        model.eq_capacity = pe.ConstraintList()
-        for i in range(self.m):
-            model.eq_capacity.add(
-                sum(model.x[j] * self.weights[i, j] for j in range(self.n))
-                <= self.capacities[i]
-            )
-
-        return model
-
-    @overrides
-    def get_instance_features(self) -> np.ndarray:
-        return np.array([float(np.mean(self.prices))] + list(self.capacities))
-
-    @overrides
-    def get_variable_features(self, names: np.ndarray) -> np.ndarray:
-        features = []
-        for i in range(len(self.weights)):
-            f = [self.prices[i]]
-            f.extend(self.weights[:, i])
-            features.append(f)
-        return np.array(features)
-
-
-# noinspection PyPep8Naming
-class MultiKnapsackGenerator:
-    def __init__(
-        self,
-        n: rv_frozen = randint(low=100, high=101),
-        m: rv_frozen = randint(low=30, high=31),
-        w: rv_frozen = randint(low=0, high=1000),
-        K: rv_frozen = randint(low=500, high=501),
-        u: rv_frozen = uniform(loc=0.0, scale=1.0),
-        alpha: rv_frozen = uniform(loc=0.25, scale=0.0),
-        fix_w: bool = False,
-        w_jitter: rv_frozen = uniform(loc=1.0, scale=0.0),
-        round: bool = True,
-    ):
-        """Initialize the problem generator.
-
-        Instances have a random number of items (or variables) and a random number of
-        knapsacks (or constraints), as specified by the provided probability
-        distributions `n` and `m`, respectively. The weight of each item `i` on
-        knapsack `j` is sampled independently from the provided distribution `w`. The
-        capacity of knapsack `j` is set to:
-
-            alpha_j * sum(w[i,j] for i in range(n)),
-
-        where `alpha_j`, the tightness ratio, is sampled from the provided
-        probability distribution `alpha`. To make the instances more challenging,
-        the costs of the items are linearly correlated to their average weights. More
-        specifically, the weight of each item `i` is set to:
-
-            sum(w[i,j]/m for j in range(m)) + K * u_i,
-
-        where `K`, the correlation coefficient, and `u_i`, the correlation
-        multiplier, are sampled from the provided probability distributions. Note
-        that `K` is only sample once for the entire instance.
-
-        If fix_w=True is provided, then w[i,j] are kept the same in all generated
-        instances. This also implies that n and m are kept fixed. Although the prices
-        and capacities are derived from w[i,j], as long as u and K are not constants,
-        the generated instances will still not be completely identical.
-
-        If a probability distribution w_jitter is provided, then item weights will be
-        set to w[i,j] * gamma[i,j] where gamma[i,j] is sampled from w_jitter. When
-        combined with fix_w=True, this argument may be used to generate instances
-        where the weight of each item is roughly the same, but not exactly identical,
-        across all instances. The prices of the items and the capacities of the
-        knapsacks will be calculated as above, but using these perturbed weights
-        instead.
-
-        By default, all generated prices, weights and capacities are rounded to the
-        nearest integer number. If `round=False` is provided, this rounding will be
-        disabled.
-
-        Parameters
-        ----------
-        n: rv_discrete
-            Probability distribution for the number of items (or variables)
-        m: rv_discrete
-            Probability distribution for the number of knapsacks (or constraints)
-        w: rv_continuous
-            Probability distribution for the item weights
-        K: rv_continuous
-            Probability distribution for the profit correlation coefficient
-        u: rv_continuous
-            Probability distribution for the profit multiplier
-        alpha: rv_continuous
-            Probability distribution for the tightness ratio
-        fix_w: boolean
-            If true, weights are kept the same (minus the noise from w_jitter) in all
-            instances
-        w_jitter: rv_continuous
-            Probability distribution for random noise added to the weights
-        round: boolean
-            If true, all prices, weights and capacities are rounded to the nearest
-            integer
-        """
-        assert isinstance(n, rv_frozen), "n should be a SciPy probability distribution"
-        assert isinstance(m, rv_frozen), "m should be a SciPy probability distribution"
-        assert isinstance(w, rv_frozen), "w should be a SciPy probability distribution"
-        assert isinstance(K, rv_frozen), "K should be a SciPy probability distribution"
-        assert isinstance(u, rv_frozen), "u should be a SciPy probability distribution"
-        assert isinstance(
-            alpha, rv_frozen
-        ), "alpha should be a SciPy probability distribution"
-        assert isinstance(fix_w, bool), "fix_w should be boolean"
-        assert isinstance(
-            w_jitter, rv_frozen
-        ), "w_jitter should be a SciPy probability distribution"
-
-        self.n = n
-        self.m = m
-        self.w = w
-        self.u = u
-        self.K = K
-        self.alpha = alpha
-        self.w_jitter = w_jitter
-        self.round = round
-        self.fix_n: Optional[int] = None
-        self.fix_m: Optional[int] = None
-        self.fix_w: Optional[np.ndarray] = None
-        self.fix_u: Optional[np.ndarray] = None
-        self.fix_K: Optional[float] = None
-
-        if fix_w:
-            self.fix_n = self.n.rvs()
-            self.fix_m = self.m.rvs()
-            self.fix_w = np.array([self.w.rvs(self.fix_n) for _ in range(self.fix_m)])
-            self.fix_u = self.u.rvs(self.fix_n)
-            self.fix_K = self.K.rvs()
-
-    def generate(self, n_samples: int) -> List[MultiKnapsackInstance]:
-        def _sample() -> MultiKnapsackInstance:
-            if self.fix_w is not None:
-                assert self.fix_m is not None
-                assert self.fix_n is not None
-                assert self.fix_u is not None
-                assert self.fix_K is not None
-                n = self.fix_n
-                m = self.fix_m
-                w = self.fix_w
-                u = self.fix_u
-                K = self.fix_K
-            else:
-                n = self.n.rvs()
-                m = self.m.rvs()
-                w = np.array([self.w.rvs(n) for _ in range(m)])
-                u = self.u.rvs(n)
-                K = self.K.rvs()
-            w = w * np.array([self.w_jitter.rvs(n) for _ in range(m)])
-            alpha = self.alpha.rvs(m)
-            p = np.array([w[:, j].sum() / m + K * u[j] for j in range(n)])
-            b = np.array([w[i, :].sum() * alpha[i] for i in range(m)])
-            if self.round:
-                p = p.round()
-                b = b.round()
-                w = w.round()
-            return MultiKnapsackInstance(p, b, w)
-
-        return [_sample() for _ in range(n_samples)]

miplearn/problems/multiknapsack.py

@@ -0,0 +1,189 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from dataclasses import dataclass
+from typing import List, Optional, Union
+
+import gurobipy as gp
+import numpy as np
+from gurobipy import GRB
+from scipy.stats import uniform, randint
+from scipy.stats.distributions import rv_frozen
+
+from miplearn.io import read_pkl_gz
+from miplearn.solvers.gurobi import GurobiModel
+
+
+@dataclass
+class MultiKnapsackData:
+    """Data for the multi-dimensional knapsack problem
+
+    Args
+    ----
+    prices
+        Item prices.
+    capacities
+        Knapsack capacities.
+    weights
+        Matrix of item weights.
+    """
+
+    prices: np.ndarray
+    capacities: np.ndarray
+    weights: np.ndarray
+
+
+# noinspection PyPep8Naming
+class MultiKnapsackGenerator:
+    """Random instance generator for the multi-dimensional knapsack problem.
+
+    Instances have a random number of items (or variables) and a random number of
+    knapsacks (or constraints), as specified by the provided probability
+    distributions `n` and `m`, respectively. The weight of each item `i` on knapsack
+    `j` is sampled independently from the provided distribution `w`. The capacity of
+    knapsack `j` is set to ``alpha_j * sum(w[i,j] for i in range(n))``,
+    where `alpha_j`, the tightness ratio, is sampled from the provided probability
+    distribution `alpha`.
+
+    To make the instances more challenging, the costs of the items are linearly
+    correlated to their average weights. More specifically, the weight of each item
+    `i` is set to ``sum(w[i,j]/m for j in range(m)) + K * u_i``, where `K`,
+    the correlation coefficient, and `u_i`, the correlation multiplier, are sampled
+    from the provided probability distributions. Note that `K` is only sample once
+    for the entire instance.
+
+    If `fix_w=True`, then `weights[i,j]` are kept the same in all generated
+    instances. This also implies that n and m are kept fixed. Although the prices and
+    capacities are derived from `weights[i,j]`, as long as `u` and `K` are not
+    constants, the generated instances will still not be completely identical.
+
+    If a probability distribution `w_jitter` is provided, then item weights will be
+    set to ``w[i,j] * gamma[i,j]`` where `gamma[i,j]` is sampled from `w_jitter`.
+    When combined with `fix_w=True`, this argument may be used to generate instances
+    where the weight of each item is roughly the same, but not exactly identical,
+    across all instances. The prices of the items and the capacities of the knapsacks
+    will be calculated as above, but using these perturbed weights instead.
+
+    By default, all generated prices, weights and capacities are rounded to the
+    nearest integer number. If `round=False` is provided, this rounding will be
+    disabled.
+
+    Parameters
+    ----------
+    n: rv_discrete
+        Probability distribution for the number of items (or variables).
+    m: rv_discrete
+        Probability distribution for the number of knapsacks (or constraints).
+    w: rv_continuous
+        Probability distribution for the item weights.
+    K: rv_continuous
+        Probability distribution for the profit correlation coefficient.
+    u: rv_continuous
+        Probability distribution for the profit multiplier.
+    alpha: rv_continuous
+        Probability distribution for the tightness ratio.
+    fix_w: boolean
+        If true, weights are kept the same (minus the noise from w_jitter) in all
+        instances.
+    w_jitter: rv_continuous
+        Probability distribution for random noise added to the weights.
+    round: boolean
+        If true, all prices, weights and capacities are rounded to the nearest
+        integer.
+    """
+
+    def __init__(
+        self,
+        n: rv_frozen = randint(low=100, high=101),
+        m: rv_frozen = randint(low=30, high=31),
+        w: rv_frozen = randint(low=0, high=1000),
+        K: rv_frozen = randint(low=500, high=501),
+        u: rv_frozen = uniform(loc=0.0, scale=1.0),
+        alpha: rv_frozen = uniform(loc=0.25, scale=0.0),
+        fix_w: bool = False,
+        w_jitter: rv_frozen = uniform(loc=1.0, scale=0.0),
+        p_jitter: rv_frozen = uniform(loc=1.0, scale=0.0),
+        round: bool = True,
+    ):
+        assert isinstance(n, rv_frozen), "n should be a SciPy probability distribution"
+        assert isinstance(m, rv_frozen), "m should be a SciPy probability distribution"
+        assert isinstance(w, rv_frozen), "w should be a SciPy probability distribution"
+        assert isinstance(K, rv_frozen), "K should be a SciPy probability distribution"
+        assert isinstance(u, rv_frozen), "u should be a SciPy probability distribution"
+        assert isinstance(
+            alpha, rv_frozen
+        ), "alpha should be a SciPy probability distribution"
+        assert isinstance(fix_w, bool), "fix_w should be boolean"
+        assert isinstance(
+            w_jitter, rv_frozen
+        ), "w_jitter should be a SciPy probability distribution"
+
+        self.n = n
+        self.m = m
+        self.w = w
+        self.u = u
+        self.K = K
+        self.alpha = alpha
+        self.w_jitter = w_jitter
+        self.p_jitter = p_jitter
+        self.round = round
+        self.fix_n: Optional[int] = None
+        self.fix_m: Optional[int] = None
+        self.fix_w: Optional[np.ndarray] = None
+        self.fix_u: Optional[np.ndarray] = None
+        self.fix_K: Optional[float] = None
+
+        if fix_w:
+            self.fix_n = self.n.rvs()
+            self.fix_m = self.m.rvs()
+            self.fix_w = np.array([self.w.rvs(self.fix_n) for _ in range(self.fix_m)])
+            self.fix_u = self.u.rvs(self.fix_n)
+            self.fix_K = self.K.rvs()
+
+    def generate(self, n_samples: int) -> List[MultiKnapsackData]:
+        def _sample() -> MultiKnapsackData:
+            if self.fix_w is not None:
+                assert self.fix_m is not None
+                assert self.fix_n is not None
+                assert self.fix_u is not None
+                assert self.fix_K is not None
+                n = self.fix_n
+                m = self.fix_m
+                w = self.fix_w
+                u = self.fix_u
+                K = self.fix_K
+            else:
+                n = self.n.rvs()
+                m = self.m.rvs()
+                w = np.array([self.w.rvs(n) for _ in range(m)])
+                u = self.u.rvs(n)
+                K = self.K.rvs()
+            w = w * np.array([self.w_jitter.rvs(n) for _ in range(m)])
+            alpha = self.alpha.rvs(m)
+            p = np.array(
+                [w[:, j].sum() / m + K * u[j] for j in range(n)]
+            ) * self.p_jitter.rvs(n)
+            b = np.array([w[i, :].sum() * alpha[i] for i in range(m)])
+            if self.round:
+                p = p.round()
+                b = b.round()
+                w = w.round()
+            return MultiKnapsackData(p, b, w)
+
+        return [_sample() for _ in range(n_samples)]
+
+
+def build_multiknapsack_model(data: Union[str, MultiKnapsackData]) -> GurobiModel:
+    """Converts multi-knapsack problem data into a concrete Gurobipy model."""
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, MultiKnapsackData)
+
+    model = gp.Model()
+    m, n = data.weights.shape
+    x = model.addMVar(n, vtype=GRB.BINARY, name="x")
+    model.addConstr(data.weights @ x <= data.capacities)
+    model.setObjective(-data.prices @ x)
+    model.update()
+    return GurobiModel(model)

miplearn/problems/pmedian.py

@@ -0,0 +1,185 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from dataclasses import dataclass
+from typing import List, Optional, Union
+
+import gurobipy as gp
+import numpy as np
+from gurobipy import quicksum, GRB
+from scipy.spatial.distance import pdist, squareform
+from scipy.stats import uniform, randint
+from scipy.stats.distributions import rv_frozen
+
+from miplearn.io import read_pkl_gz
+from miplearn.solvers.gurobi import GurobiModel
+
+
+@dataclass
+class PMedianData:
+    """Data for the capacitated p-median problem
+
+    Args
+    ----
+    distances
+        Matrix of distances between customer i and facility j.
+    demands
+        Customer demands.
+    p
+        Number of medians that need to be chosen.
+    capacities
+        Facility capacities.
+    """
+
+    distances: np.ndarray
+    demands: np.ndarray
+    p: int
+    capacities: np.ndarray
+
+
+class PMedianGenerator:
+    """Random generator for the capacitated p-median problem.
+
+    This class first decides the number of customers and the parameter `p` by
+    sampling the provided `n` and `p` distributions, respectively. Then, for each
+    customer `i`, the class builds its geographical location `(xi, yi)` by sampling
+    the provided `x` and `y` distributions. For each `i`, the demand for customer `i`
+    and the capacity of facility `i` are decided by sampling the distributions
+    `demands` and `capacities`, respectively. Finally, the costs `w[i,j]` are set to
+    the Euclidean distance between the locations of customers `i` and `j`.
+
+    If `fixed=True`, then the number of customers, their locations, the parameter
+    `p`, the demands and the capacities are only sampled from their respective
+    distributions exactly once, to build a reference instance which is then
+    perturbed. Specifically, for each perturbation, the distances, demands and
+    capacities are multiplied by factors sampled from the distributions
+    `distances_jitter`, `demands_jitter` and `capacities_jitter`, respectively. The
+    result is a list of instances that have the same set of customers, but slightly
+    different demands, capacities and distances.
+
+    Parameters
+    ----------
+    x
+        Probability distribution for the x-coordinate of the points.
+    y
+        Probability distribution for the y-coordinate of the points.
+    n
+        Probability distribution for the number of customer.
+    p
+        Probability distribution for the number of medians.
+    demands
+        Probability distribution for the customer demands.
+    capacities
+        Probability distribution for the facility capacities.
+    distances_jitter
+        Probability distribution for the random scaling factor applied to distances.
+    demands_jitter
+        Probability distribution for the random scaling factor applied to demands.
+    capacities_jitter
+        Probability distribution for the random scaling factor applied to capacities.
+    fixed
+        If `True`, then customer are kept the same across instances.
+    """
+
+    def __init__(
+        self,
+        x: rv_frozen = uniform(loc=0.0, scale=100.0),
+        y: rv_frozen = uniform(loc=0.0, scale=100.0),
+        n: rv_frozen = randint(low=100, high=101),
+        p: rv_frozen = randint(low=10, high=11),
+        demands: rv_frozen = uniform(loc=0, scale=20),
+        capacities: rv_frozen = uniform(loc=0, scale=100),
+        distances_jitter: rv_frozen = uniform(loc=1.0, scale=0.0),
+        demands_jitter: rv_frozen = uniform(loc=1.0, scale=0.0),
+        capacities_jitter: rv_frozen = uniform(loc=1.0, scale=0.0),
+        fixed: bool = True,
+    ):
+        self.x = x
+        self.y = y
+        self.n = n
+        self.p = p
+        self.demands = demands
+        self.capacities = capacities
+        self.distances_jitter = distances_jitter
+        self.demands_jitter = demands_jitter
+        self.capacities_jitter = capacities_jitter
+        self.fixed = fixed
+        self.ref_data: Optional[PMedianData] = None
+
+    def generate(self, n_samples: int) -> List[PMedianData]:
+        def _sample() -> PMedianData:
+            if self.ref_data is None:
+                n = self.n.rvs()
+                p = self.p.rvs()
+                loc = np.array([(self.x.rvs(), self.y.rvs()) for _ in range(n)])
+                distances = squareform(pdist(loc))
+                demands = self.demands.rvs(n)
+                capacities = self.capacities.rvs(n)
+            else:
+                n = self.ref_data.demands.shape[0]
+                distances = self.ref_data.distances * self.distances_jitter.rvs(
+                    size=(n, n)
+                )
+                distances = np.tril(distances) + np.triu(distances.T, 1)
+                demands = self.ref_data.demands * self.demands_jitter.rvs(n)
+                capacities = self.ref_data.capacities * self.capacities_jitter.rvs(n)
+                p = self.ref_data.p
+
+            data = PMedianData(
+                distances=distances.round(2),
+                demands=demands.round(2),
+                p=p,
+                capacities=capacities.round(2),
+            )
+
+            if self.fixed and self.ref_data is None:
+                self.ref_data = data
+
+            return data
+
+        return [_sample() for _ in range(n_samples)]
+
+
+def build_pmedian_model(data: Union[str, PMedianData]) -> GurobiModel:
+    """Converts capacitated p-median data into a concrete Gurobipy model."""
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, PMedianData)
+
+    model = gp.Model()
+    n = len(data.demands)
+
+    # Decision variables
+    x = model.addVars(n, n, vtype=GRB.BINARY, name="x")
+    y = model.addVars(n, vtype=GRB.BINARY, name="y")
+
+    # Objective function
+    model.setObjective(
+        quicksum(data.distances[i, j] * x[i, j] for i in range(n) for j in range(n))
+    )
+
+    # Eq: Must serve each customer
+    model.addConstrs(
+        (quicksum(x[i, j] for j in range(n)) == 1 for i in range(n)),
+        name="eq_demand",
+    )
+
+    # Eq: Must choose p medians
+    model.addConstr(
+        quicksum(y[j] for j in range(n)) == data.p,
+        name="eq_choose",
+    )
+
+    # Eq: Must not exceed capacity
+    model.addConstrs(
+        (
+            quicksum(data.demands[i] * x[i, j] for i in range(n))
+            <= data.capacities[j] * y[j]
+            for j in range(n)
+        ),
+        name="eq_capacity",
+    )
+
+    model.update()
+    return GurobiModel(model)

miplearn/problems/setcover.py

@@ -0,0 +1,120 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from dataclasses import dataclass
+from typing import List, Union
+
+import gurobipy as gp
+import numpy as np
+import pyomo.environ as pe
+from gurobipy.gurobipy import GRB
+from scipy.stats import uniform, randint
+from scipy.stats.distributions import rv_frozen
+
+from miplearn.io import read_pkl_gz
+from miplearn.solvers.gurobi import GurobiModel
+from miplearn.solvers.pyomo import PyomoModel
+
+
+@dataclass
+class SetCoverData:
+    costs: np.ndarray
+    incidence_matrix: np.ndarray
+
+
+class SetCoverGenerator:
+    def __init__(
+        self,
+        n_elements: rv_frozen = randint(low=50, high=51),
+        n_sets: rv_frozen = randint(low=100, high=101),
+        costs: rv_frozen = uniform(loc=0.0, scale=100.0),
+        costs_jitter: rv_frozen = uniform(loc=-5.0, scale=10.0),
+        K: rv_frozen = uniform(loc=25.0, scale=0.0),
+        density: rv_frozen = uniform(loc=0.02, scale=0.00),
+        fix_sets: bool = True,
+    ):
+        self.n_elements = n_elements
+        self.n_sets = n_sets
+        self.costs = costs
+        self.costs_jitter = costs_jitter
+        self.density = density
+        self.K = K
+        self.fix_sets = fix_sets
+        self.fixed_costs = None
+        self.fixed_matrix = None
+
+    def generate(self, n_samples: int) -> List[SetCoverData]:
+        def _sample() -> SetCoverData:
+            if self.fixed_matrix is None:
+                n_sets = self.n_sets.rvs()
+                n_elements = self.n_elements.rvs()
+                density = self.density.rvs()
+
+                incidence_matrix = np.random.rand(n_elements, n_sets) < density
+                incidence_matrix = incidence_matrix.astype(int)
+
+                # Ensure each element belongs to at least one set
+                for j in range(n_elements):
+                    if incidence_matrix[j, :].sum() == 0:
+                        incidence_matrix[j, randint(low=0, high=n_sets).rvs()] = 1
+
+                # Ensure each set contains at least one element
+                for i in range(n_sets):
+                    if incidence_matrix[:, i].sum() == 0:
+                        incidence_matrix[randint(low=0, high=n_elements).rvs(), i] = 1
+
+                costs = self.costs.rvs(n_sets) + self.K.rvs() * incidence_matrix.sum(
+                    axis=0
+                )
+                if self.fix_sets:
+                    self.fixed_matrix = incidence_matrix
+                    self.fixed_costs = costs
+            else:
+                incidence_matrix = self.fixed_matrix
+                (_, n_sets) = incidence_matrix.shape
+                costs = self.fixed_costs * self.costs_jitter.rvs(n_sets)
+            return SetCoverData(
+                costs=costs.round(2),
+                incidence_matrix=incidence_matrix,
+            )
+
+        return [_sample() for _ in range(n_samples)]
+
+
+def build_setcover_model_gurobipy(data: Union[str, SetCoverData]) -> GurobiModel:
+    data = _read_setcover_data(data)
+    (n_elements, n_sets) = data.incidence_matrix.shape
+    model = gp.Model()
+    x = model.addMVar(n_sets, vtype=GRB.BINARY, name="x")
+    model.addConstr(data.incidence_matrix @ x >= np.ones(n_elements), name="eqs")
+    model.setObjective(data.costs @ x)
+    model.update()
+    return GurobiModel(model)
+
+
+def build_setcover_model_pyomo(
+    data: Union[str, SetCoverData],
+    solver="gurobi_persistent",
+) -> PyomoModel:
+    data = _read_setcover_data(data)
+    (n_elements, n_sets) = data.incidence_matrix.shape
+    model = pe.ConcreteModel()
+    model.sets = pe.Set(initialize=range(n_sets))
+    model.x = pe.Var(model.sets, domain=pe.Boolean, name="x")
+    model.eqs = pe.Constraint(pe.Any)
+    for i in range(n_elements):
+        model.eqs[i] = (
+            sum(data.incidence_matrix[i, j] * model.x[j] for j in range(n_sets)) >= 1
+        )
+    model.obj = pe.Objective(
+        expr=sum(data.costs[j] * model.x[j] for j in range(n_sets))
+    )
+    return PyomoModel(model, solver)
+
+
+def _read_setcover_data(data: Union[str, SetCoverData]) -> SetCoverData:
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, SetCoverData)
+    return data

miplearn/problems/setpack.py

@@ -0,0 +1,66 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from dataclasses import dataclass
+from typing import List, Union
+
+import gurobipy as gp
+import numpy as np
+from gurobipy.gurobipy import GRB
+from scipy.stats import uniform, randint
+from scipy.stats.distributions import rv_frozen
+
+from .setcover import SetCoverGenerator
+from miplearn.solvers.gurobi import GurobiModel
+from ..io import read_pkl_gz
+
+
+@dataclass
+class SetPackData:
+    costs: np.ndarray
+    incidence_matrix: np.ndarray
+
+
+class SetPackGenerator:
+    def __init__(
+        self,
+        n_elements: rv_frozen = randint(low=50, high=51),
+        n_sets: rv_frozen = randint(low=100, high=101),
+        costs: rv_frozen = uniform(loc=0.0, scale=100.0),
+        costs_jitter: rv_frozen = uniform(loc=-5.0, scale=10.0),
+        K: rv_frozen = uniform(loc=25.0, scale=0.0),
+        density: rv_frozen = uniform(loc=0.02, scale=0.00),
+        fix_sets: bool = True,
+    ) -> None:
+        self.gen = SetCoverGenerator(
+            n_elements=n_elements,
+            n_sets=n_sets,
+            costs=costs,
+            costs_jitter=costs_jitter,
+            K=K,
+            density=density,
+            fix_sets=fix_sets,
+        )
+
+    def generate(self, n_samples: int) -> List[SetPackData]:
+        return [
+            SetPackData(
+                s.costs,
+                s.incidence_matrix,
+            )
+            for s in self.gen.generate(n_samples)
+        ]
+
+
+def build_setpack_model(data: Union[str, SetPackData]) -> GurobiModel:
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, SetPackData)
+    (n_elements, n_sets) = data.incidence_matrix.shape
+    model = gp.Model()
+    x = model.addMVar(n_sets, vtype=GRB.BINARY, name="x")
+    model.addConstr(data.incidence_matrix @ x <= np.ones(n_elements))
+    model.setObjective(-data.costs @ x)
+    model.update()
+    return GurobiModel(model)

miplearn/problems/stab.py

@@ -1,93 +1,28 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
-from typing import List, Dict
 
+from dataclasses import dataclass
+from typing import List, Union
+
+import gurobipy as gp
 import networkx as nx
 import numpy as np
 import pyomo.environ as pe
+from gurobipy import GRB, quicksum
 from networkx import Graph
-from overrides import overrides
 from scipy.stats import uniform, randint
 from scipy.stats.distributions import rv_frozen
 
-from miplearn.instance.base import Instance
+from miplearn.io import read_pkl_gz
+from miplearn.solvers.gurobi import GurobiModel
+from miplearn.solvers.pyomo import PyomoModel
 
 
-class ChallengeA:
+@dataclass
-    def __init__(
+class MaxWeightStableSetData:
-        self,
+    graph: Graph
-        seed: int = 42,
+    weights: np.ndarray
-        n_training_instances: int = 500,
-        n_test_instances: int = 50,
-    ) -> None:
-        np.random.seed(seed)
-        self.generator = MaxWeightStableSetGenerator(
-            w=uniform(loc=100.0, scale=50.0),
-            n=randint(low=200, high=201),
-            p=uniform(loc=0.05, scale=0.0),
-            fix_graph=True,
-        )
-
-        np.random.seed(seed + 1)
-        self.training_instances = self.generator.generate(n_training_instances)
-
-        np.random.seed(seed + 2)
-        self.test_instances = self.generator.generate(n_test_instances)
-
-
-class MaxWeightStableSetInstance(Instance):
-    """An instance of the Maximum-Weight Stable Set Problem.
-
-    Given a graph G=(V,E) and a weight w_v for each vertex v, the problem asks for a stable
-    set S of G maximizing sum(w_v for v in S). A stable set (also called independent set) is
-    a subset of vertices, no two of which are adjacent.
-
-    This is one of Karp's 21 NP-complete problems.
-    """
-
-    def __init__(self, graph: Graph, weights: np.ndarray) -> None:
-        super().__init__()
-        self.graph = graph
-        self.weights = weights
-        self.nodes = list(self.graph.nodes)
-
-    @overrides
-    def to_model(self) -> pe.ConcreteModel:
-        model = pe.ConcreteModel()
-        model.x = pe.Var(self.nodes, domain=pe.Binary)
-        model.OBJ = pe.Objective(
-            expr=sum(model.x[v] * self.weights[v] for v in self.nodes),
-            sense=pe.maximize,
-        )
-        model.clique_eqs = pe.ConstraintList()
-        for clique in nx.find_cliques(self.graph):
-            model.clique_eqs.add(sum(model.x[v] for v in clique) <= 1)
-        return model
-
-    @overrides
-    def get_variable_features(self, names: np.ndarray) -> np.ndarray:
-        features = []
-        assert len(names) == len(self.nodes)
-        for i, v1 in enumerate(self.nodes):
-            assert names[i] == f"x[{v1}]".encode()
-            neighbor_weights = [0.0] * 15
-            neighbor_degrees = [100.0] * 15
-            for v2 in self.graph.neighbors(v1):
-                neighbor_weights += [self.weights[v2] / self.weights[v1]]
-                neighbor_degrees += [self.graph.degree(v2) / self.graph.degree(v1)]
-            neighbor_weights.sort(reverse=True)
-            neighbor_degrees.sort()
-            f = []
-            f += neighbor_weights[:5]
-            f += neighbor_degrees[:5]
-            f += [self.graph.degree(v1)]
-            features.append(f)
-        return np.array(features)
-
-    @overrides
-    def get_variable_categories(self, names: np.ndarray) -> np.ndarray:
-        return np.array(["default" for _ in names], dtype="S")
 
 
 class MaxWeightStableSetGenerator:
@@ -132,16 +67,50 @@ class MaxWeightStableSetGenerator:
         if fix_graph:
             self.graph = self._generate_graph()
 
-    def generate(self, n_samples: int) -> List[MaxWeightStableSetInstance]:
+    def generate(self, n_samples: int) -> List[MaxWeightStableSetData]:
-        def _sample() -> MaxWeightStableSetInstance:
+        def _sample() -> MaxWeightStableSetData:
             if self.graph is not None:
                 graph = self.graph
             else:
                 graph = self._generate_graph()
-            weights = self.w.rvs(graph.number_of_nodes())
+            weights = np.round(self.w.rvs(graph.number_of_nodes()), 2)
-            return MaxWeightStableSetInstance(graph, weights)
+            return MaxWeightStableSetData(graph, weights)
 
         return [_sample() for _ in range(n_samples)]
 
     def _generate_graph(self) -> Graph:
         return nx.generators.random_graphs.binomial_graph(self.n.rvs(), self.p.rvs())
+
+
+def build_stab_model_gurobipy(data: MaxWeightStableSetData) -> GurobiModel:
+    data = _read_stab_data(data)
+    model = gp.Model()
+    nodes = list(data.graph.nodes)
+    x = model.addVars(nodes, vtype=GRB.BINARY, name="x")
+    model.setObjective(quicksum(-data.weights[i] * x[i] for i in nodes))
+    for clique in nx.find_cliques(data.graph):
+        model.addConstr(quicksum(x[i] for i in clique) <= 1)
+    model.update()
+    return GurobiModel(model)
+
+
+def build_stab_model_pyomo(
+    data: MaxWeightStableSetData,
+    solver="gurobi_persistent",
+) -> PyomoModel:
+    data = _read_stab_data(data)
+    model = pe.ConcreteModel()
+    nodes = pe.Set(initialize=list(data.graph.nodes))
+    model.x = pe.Var(nodes, domain=pe.Boolean, name="x")
+    model.obj = pe.Objective(expr=sum([-data.weights[i] * model.x[i] for i in nodes]))
+    model.clique_eqs = pe.ConstraintList()
+    for clique in nx.find_cliques(data.graph):
+        model.clique_eqs.add(expr=sum(model.x[i] for i in clique) <= 1)
+    return PyomoModel(model, solver)
+
+
+def _read_stab_data(data: Union[str, MaxWeightStableSetData]) -> MaxWeightStableSetData:
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, MaxWeightStableSetData)
+    return data

miplearn/problems/tsp.py

@@ -1,118 +1,29 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
-from typing import List, Tuple, FrozenSet, Any, Optional, Dict
 
+from dataclasses import dataclass
+from typing import List, Tuple, Optional, Any, Union
+
+import gurobipy as gp
 import networkx as nx
 import numpy as np
-import pyomo.environ as pe
+from gurobipy import quicksum, GRB, tuplelist
-from overrides import overrides
 from scipy.spatial.distance import pdist, squareform
 from scipy.stats import uniform, randint
 from scipy.stats.distributions import rv_frozen
+import logging
 
-from miplearn.instance.base import Instance
+from miplearn.io import read_pkl_gz
-from miplearn.solvers.learning import InternalSolver
+from miplearn.solvers.gurobi import GurobiModel
-from miplearn.solvers.pyomo.base import BasePyomoSolver
+
-from miplearn.types import ConstraintName
+logger = logging.getLogger(__name__)
 
 
-class ChallengeA:
+@dataclass
-    def __init__(
+class TravelingSalesmanData:
-        self,
+    n_cities: int
-        seed: int = 42,
+    distances: np.ndarray
-        n_training_instances: int = 500,
-        n_test_instances: int = 50,
-    ) -> None:
-        np.random.seed(seed)
-        self.generator = TravelingSalesmanGenerator(
-            x=uniform(loc=0.0, scale=1000.0),
-            y=uniform(loc=0.0, scale=1000.0),
-            n=randint(low=350, high=351),
-            gamma=uniform(loc=0.95, scale=0.1),
-            fix_cities=True,
-            round=True,
-        )
-
-        np.random.seed(seed + 1)
-        self.training_instances = self.generator.generate(n_training_instances)
-
-        np.random.seed(seed + 2)
-        self.test_instances = self.generator.generate(n_test_instances)
-
-
-class TravelingSalesmanInstance(Instance):
-    """An instance ot the Traveling Salesman Problem.
-
-    Given a list of cities and the distance between each pair of cities, the problem
-    asks for the shortest route starting at the first city, visiting each other city
-    exactly once, then returning to the first city. This problem is a generalization
-    of the Hamiltonian path problem, one of Karp's 21 NP-complete problems.
-    """
-
-    def __init__(self, n_cities: int, distances: np.ndarray) -> None:
-        super().__init__()
-        assert isinstance(distances, np.ndarray)
-        assert distances.shape == (n_cities, n_cities)
-        self.n_cities = n_cities
-        self.distances = distances
-        self.edges = [
-            (i, j) for i in range(self.n_cities) for j in range(i + 1, self.n_cities)
-        ]
-
-    @overrides
-    def to_model(self) -> pe.ConcreteModel:
-        model = pe.ConcreteModel()
-        model.x = pe.Var(self.edges, domain=pe.Binary)
-        model.obj = pe.Objective(
-            expr=sum(model.x[i, j] * self.distances[i, j] for (i, j) in self.edges),
-            sense=pe.minimize,
-        )
-        model.eq_degree = pe.ConstraintList()
-        model.eq_subtour = pe.ConstraintList()
-        for i in range(self.n_cities):
-            model.eq_degree.add(
-                sum(
-                    model.x[min(i, j), max(i, j)]
-                    for j in range(self.n_cities)
-                    if i != j
-                )
-                == 2
-            )
-        return model
-
-    @overrides
-    def find_violated_lazy_constraints(
-        self,
-        solver: InternalSolver,
-        model: Any,
-    ) -> List[ConstraintName]:
-        selected_edges = [e for e in self.edges if model.x[e].value > 0.5]
-        graph = nx.Graph()
-        graph.add_edges_from(selected_edges)
-        violations = []
-        for c in list(nx.connected_components(graph)):
-            if len(c) < self.n_cities:
-                violations.append(",".join(map(str, c)).encode())
-        return violations
-
-    @overrides
-    def enforce_lazy_constraint(
-        self,
-        solver: InternalSolver,
-        model: Any,
-        violation: ConstraintName,
-    ) -> None:
-        assert isinstance(solver, BasePyomoSolver)
-        component = [int(v) for v in violation.decode().split(",")]
-        cut_edges = [
-            e
-            for e in self.edges
-            if (e[0] in component and e[1] not in component)
-            or (e[0] not in component and e[1] in component)
-        ]
-        constr = model.eq_subtour.add(expr=sum(model.x[e] for e in cut_edges) >= 2)
-        solver.add_constraint(constr)
 
 
 class TravelingSalesmanGenerator:
@@ -134,7 +45,7 @@ class TravelingSalesmanGenerator:
         distributions `n`, `x` and `y`. For each (unordered) pair of cities (i,j),
         the distance d[i,j] between them is set to:
 
-            d[i,j] = gamma[i,j] \sqrt{(x_i - x_j)^2 + (y_i - y_j)^2}
+            d[i,j] = gamma[i,j] \\sqrt{(x_i - x_j)^2 + (y_i - y_j)^2}
 
         where gamma is sampled from the provided probability distribution `gamma`.
 
@@ -180,8 +91,8 @@ class TravelingSalesmanGenerator:
             self.fixed_n = None
             self.fixed_cities = None
 
-    def generate(self, n_samples: int) -> List[TravelingSalesmanInstance]:
+    def generate(self, n_samples: int) -> List[TravelingSalesmanData]:
-        def _sample() -> TravelingSalesmanInstance:
+        def _sample() -> TravelingSalesmanData:
             if self.fixed_cities is not None:
                 assert self.fixed_n is not None
                 n, cities = self.fixed_n, self.fixed_cities
@@ -191,7 +102,7 @@ class TravelingSalesmanGenerator:
             distances = np.tril(distances) + np.triu(distances.T, 1)
             if self.round:
                 distances = distances.round()
-            return TravelingSalesmanInstance(n, distances)
+            return TravelingSalesmanData(n, distances)
 
         return [_sample() for _ in range(n_samples)]
 
@@ -199,3 +110,68 @@ class TravelingSalesmanGenerator:
         n = self.n.rvs()
         cities = np.array([(self.x.rvs(), self.y.rvs()) for _ in range(n)])
         return n, cities
+
+
+def build_tsp_model(data: Union[str, TravelingSalesmanData]) -> GurobiModel:
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, TravelingSalesmanData)
+
+    edges = tuplelist(
+        (i, j) for i in range(data.n_cities) for j in range(i + 1, data.n_cities)
+    )
+    model = gp.Model()
+
+    # Decision variables
+    x = model.addVars(edges, vtype=GRB.BINARY, name="x")
+
+    model._x = x
+    model._edges = edges
+    model._n_cities = data.n_cities
+
+    # Objective function
+    model.setObjective(quicksum(x[(i, j)] * data.distances[i, j] for (i, j) in edges))
+
+    # Eq: Must choose two edges adjacent to each node
+    model.addConstrs(
+        (
+            quicksum(x[min(i, j), max(i, j)] for j in range(data.n_cities) if i != j)
+            == 2
+            for i in range(data.n_cities)
+        ),
+        name="eq_degree",
+    )
+
+    def find_violations(model: GurobiModel) -> List[Any]:
+        violations = []
+        x = model.inner.cbGetSolution(model.inner._x)
+        selected_edges = [e for e in model.inner._edges if x[e] > 0.5]
+        graph = nx.Graph()
+        graph.add_edges_from(selected_edges)
+        for component in list(nx.connected_components(graph)):
+            if len(component) < model.inner._n_cities:
+                cut_edges = [
+                    e
+                    for e in model.inner._edges
+                    if (e[0] in component and e[1] not in component)
+                    or (e[0] not in component and e[1] in component)
+                ]
+                violations.append(cut_edges)
+        return violations
+
+    def fix_violations(model: GurobiModel, violations: List[Any], where: str) -> None:
+        for violation in violations:
+            constr = quicksum(model.inner._x[e[0], e[1]] for e in violation) >= 2
+            if where == "cb":
+                model.inner.cbLazy(constr)
+            else:
+                model.inner.addConstr(constr)
+        logger.info(f"tsp: added {len(violations)} subtour elimination constraints")
+
+    model.update()
+
+    return GurobiModel(
+        model,
+        find_violations=find_violations,
+        fix_violations=fix_violations,
+    )

miplearn/problems/uc.py

@@ -0,0 +1,201 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from dataclasses import dataclass
+from math import pi
+from typing import List, Optional, Union
+
+import gurobipy as gp
+import numpy as np
+from gurobipy import GRB, quicksum
+from scipy.stats import uniform, randint
+from scipy.stats.distributions import rv_frozen
+
+from miplearn.io import read_pkl_gz
+from miplearn.solvers.gurobi import GurobiModel
+
+
+@dataclass
+class UnitCommitmentData:
+    demand: np.ndarray
+    min_power: np.ndarray
+    max_power: np.ndarray
+    min_uptime: np.ndarray
+    min_downtime: np.ndarray
+    cost_startup: np.ndarray
+    cost_prod: np.ndarray
+    cost_fixed: np.ndarray
+
+
+class UnitCommitmentGenerator:
+    def __init__(
+        self,
+        n_units: rv_frozen = randint(low=1_000, high=1_001),
+        n_periods: rv_frozen = randint(low=72, high=73),
+        max_power: rv_frozen = uniform(loc=50, scale=450),
+        min_power: rv_frozen = uniform(loc=0.5, scale=0.25),
+        cost_startup: rv_frozen = uniform(loc=0, scale=10_000),
+        cost_prod: rv_frozen = uniform(loc=0, scale=50),
+        cost_fixed: rv_frozen = uniform(loc=0, scale=1_000),
+        min_uptime: rv_frozen = randint(low=2, high=8),
+        min_downtime: rv_frozen = randint(low=2, high=8),
+        cost_jitter: rv_frozen = uniform(loc=0.75, scale=0.5),
+        demand_jitter: rv_frozen = uniform(loc=0.9, scale=0.2),
+        fix_units: bool = False,
+    ) -> None:
+        self.n_units = n_units
+        self.n_periods = n_periods
+        self.max_power = max_power
+        self.min_power = min_power
+        self.cost_startup = cost_startup
+        self.cost_prod = cost_prod
+        self.cost_fixed = cost_fixed
+        self.min_uptime = min_uptime
+        self.min_downtime = min_downtime
+        self.cost_jitter = cost_jitter
+        self.demand_jitter = demand_jitter
+        self.fix_units = fix_units
+        self.ref_data: Optional[UnitCommitmentData] = None
+
+    def generate(self, n_samples: int) -> List[UnitCommitmentData]:
+        def _sample() -> UnitCommitmentData:
+            if self.ref_data is None:
+                T = self.n_periods.rvs()
+                G = self.n_units.rvs()
+
+                # Generate unit parameteres
+                max_power = self.max_power.rvs(G)
+                min_power = max_power * self.min_power.rvs(G)
+                max_power = max_power
+                min_uptime = self.min_uptime.rvs(G)
+                min_downtime = self.min_downtime.rvs(G)
+                cost_startup = self.cost_startup.rvs(G)
+                cost_prod = self.cost_prod.rvs(G)
+                cost_fixed = self.cost_fixed.rvs(G)
+                capacity = max_power.sum()
+
+                # Generate periodic demand in the range [0.4, 0.8] * capacity, with a peak every 12 hours.
+                demand = np.sin([i / 6 * pi for i in range(T)])
+                demand *= uniform(loc=0, scale=1).rvs(T)
+                demand -= demand.min()
+                demand /= demand.max() / 0.4
+                demand += 0.4
+                demand *= capacity
+            else:
+                T, G = len(self.ref_data.demand), len(self.ref_data.max_power)
+                demand = self.ref_data.demand * self.demand_jitter.rvs(T)
+                min_power = self.ref_data.min_power
+                max_power = self.ref_data.max_power
+                min_uptime = self.ref_data.min_uptime
+                min_downtime = self.ref_data.min_downtime
+                cost_startup = self.ref_data.cost_startup * self.cost_jitter.rvs(G)
+                cost_prod = self.ref_data.cost_prod * self.cost_jitter.rvs(G)
+                cost_fixed = self.ref_data.cost_fixed * self.cost_jitter.rvs(G)
+
+            data = UnitCommitmentData(
+                demand.round(2),
+                min_power.round(2),
+                max_power.round(2),
+                min_uptime,
+                min_downtime,
+                cost_startup.round(2),
+                cost_prod.round(2),
+                cost_fixed.round(2),
+            )
+
+            if self.ref_data is None and self.fix_units:
+                self.ref_data = data
+
+            return data
+
+        return [_sample() for _ in range(n_samples)]
+
+
+def build_uc_model(data: Union[str, UnitCommitmentData]) -> GurobiModel:
+    """
+    Models the unit commitment problem according to equations (1)-(5) of:
+
+        Bendotti, P., Fouilhoux, P. & Rottner, C. The min-up/min-down unit
+        commitment polytope. J Comb Optim 36, 1024-1058 (2018).
+        https://doi.org/10.1007/s10878-018-0273-y
+
+    """
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, UnitCommitmentData)
+
+    T = len(data.demand)
+    G = len(data.min_power)
+    D = data.demand
+    Pmin, Pmax = data.min_power, data.max_power
+    L = data.min_uptime
+    l = data.min_downtime
+
+    model = gp.Model()
+    is_on = model.addVars(G, T, vtype=GRB.BINARY, name="is_on")
+    switch_on = model.addVars(G, T, vtype=GRB.BINARY, name="switch_on")
+    prod = model.addVars(G, T, name="prod")
+
+    # Objective function
+    model.setObjective(
+        quicksum(
+            is_on[g, t] * data.cost_fixed[g]
+            + switch_on[g, t] * data.cost_startup[g]
+            + prod[g, t] * data.cost_prod[g]
+            for g in range(G)
+            for t in range(T)
+        )
+    )
+
+    # Eq 1: Minimum up-time constraint: If unit g is down at time t, then it
+    # cannot have start up during the previous L[g] periods.
+    model.addConstrs(
+        (
+            quicksum(switch_on[g, k] for k in range(t - L[g] + 1, t + 1)) <= is_on[g, t]
+            for g in range(G)
+            for t in range(L[g] - 1, T)
+        ),
+        name="eq_min_uptime",
+    )
+
+    # Eq 2: Minimum down-time constraint: Symmetric to the minimum-up constraint.
+    model.addConstrs(
+        (
+            quicksum(switch_on[g, k] for k in range(t - l[g] + 1, t + 1))
+            <= 1 - is_on[g, t - l[g] + 1]
+            for g in range(G)
+            for t in range(l[g] - 1, T)
+        ),
+        name="eq_min_downtime",
+    )
+
+    # Eq 3: Ensures that if unit g start up at time t, then the start-up variable
+    # must be one.
+    model.addConstrs(
+        (
+            switch_on[g, t] >= is_on[g, t] - is_on[g, t - 1]
+            for g in range(G)
+            for t in range(1, T)
+        ),
+        name="eq_startup",
+    )
+
+    # Eq 4: Ensures that demand is satisfied at each time period.
+    model.addConstrs(
+        (quicksum(prod[g, t] for g in range(G)) >= D[t] for t in range(T)),
+        name="eq_demand",
+    )
+
+    # Eq 5: Sets the bounds to the quantity of power produced by each unit.
+    model.addConstrs(
+        (Pmin[g] * is_on[g, t] <= prod[g, t] for g in range(G) for t in range(T)),
+        name="eq_prod_lb",
+    )
+    model.addConstrs(
+        (prod[g, t] <= Pmax[g] * is_on[g, t] for g in range(G) for t in range(T)),
+        name="eq_prod_ub",
+    )
+    model.update()
+
+    return GurobiModel(model)

miplearn/problems/vertexcover.py

@@ -0,0 +1,54 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from dataclasses import dataclass
+from typing import List, Union
+
+import gurobipy as gp
+import numpy as np
+from gurobipy import GRB, quicksum
+from networkx import Graph
+from scipy.stats import uniform, randint
+from scipy.stats.distributions import rv_frozen
+
+from .stab import MaxWeightStableSetGenerator
+from miplearn.solvers.gurobi import GurobiModel
+from ..io import read_pkl_gz
+
+
+@dataclass
+class MinWeightVertexCoverData:
+    graph: Graph
+    weights: np.ndarray
+
+
+class MinWeightVertexCoverGenerator:
+    def __init__(
+        self,
+        w: rv_frozen = uniform(loc=10.0, scale=1.0),
+        n: rv_frozen = randint(low=250, high=251),
+        p: rv_frozen = uniform(loc=0.05, scale=0.0),
+        fix_graph: bool = True,
+    ):
+        self._generator = MaxWeightStableSetGenerator(w, n, p, fix_graph)
+
+    def generate(self, n_samples: int) -> List[MinWeightVertexCoverData]:
+        return [
+            MinWeightVertexCoverData(s.graph, s.weights)
+            for s in self._generator.generate(n_samples)
+        ]
+
+
+def build_vertexcover_model(data: Union[str, MinWeightVertexCoverData]) -> GurobiModel:
+    if isinstance(data, str):
+        data = read_pkl_gz(data)
+    assert isinstance(data, MinWeightVertexCoverData)
+    model = gp.Model()
+    nodes = list(data.graph.nodes)
+    x = model.addVars(nodes, vtype=GRB.BINARY, name="x")
+    model.setObjective(quicksum(data.weights[i] * x[i] for i in nodes))
+    for (v1, v2) in data.graph.edges:
+        model.addConstr(x[v1] + x[v2] >= 1)
+    model.update()
+    return GurobiModel(model)

miplearn/solvers/__init__.py

@@ -1,48 +1,3 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-import sys
-from typing import Any, List, TextIO, cast, TypeVar, Optional, Sized
-
-logger = logging.getLogger(__name__)
-
-
-class _RedirectOutput:
-    def __init__(self, streams: List[Any]) -> None:
-        self.streams = streams
-
-    def write(self, data: Any) -> None:
-        for stream in self.streams:
-            stream.write(data)
-
-    def flush(self) -> None:
-        for stream in self.streams:
-            stream.flush()
-
-    def __enter__(self) -> Any:
-        self._original_stdout = sys.stdout
-        self._original_stderr = sys.stderr
-        sys.stdout = cast(TextIO, self)
-        sys.stderr = cast(TextIO, self)
-        return self
-
-    def __exit__(
-        self,
-        _type: Any,
-        _value: Any,
-        _traceback: Any,
-    ) -> None:
-        sys.stdout = self._original_stdout
-        sys.stderr = self._original_stderr
-
-
-T = TypeVar("T", bound=Sized)
-
-
-def _none_if_empty(obj: T) -> Optional[T]:
-    if len(obj) == 0:
-        return None
-    else:
-        return obj

miplearn/solvers/abstract.py

@@ -0,0 +1,70 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+
+from abc import ABC, abstractmethod
+from typing import Optional, Dict
+
+import numpy as np
+
+from miplearn.h5 import H5File
+
+
+class AbstractModel(ABC):
+    _supports_basis_status = False
+    _supports_sensitivity_analysis = False
+    _supports_node_count = False
+    _supports_solution_pool = False
+
+    @abstractmethod
+    def add_constrs(
+        self,
+        var_names: np.ndarray,
+        constrs_lhs: np.ndarray,
+        constrs_sense: np.ndarray,
+        constrs_rhs: np.ndarray,
+        stats: Optional[Dict] = None,
+    ) -> None:
+        pass
+
+    @abstractmethod
+    def extract_after_load(self, h5: H5File) -> None:
+        pass
+
+    @abstractmethod
+    def extract_after_lp(self, h5: H5File) -> None:
+        pass
+
+    @abstractmethod
+    def extract_after_mip(self, h5: H5File) -> None:
+        pass
+
+    @abstractmethod
+    def fix_variables(
+        self,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict] = None,
+    ) -> None:
+        pass
+
+    @abstractmethod
+    def optimize(self) -> None:
+        pass
+
+    @abstractmethod
+    def relax(self) -> "AbstractModel":
+        pass
+
+    @abstractmethod
+    def set_warm_starts(
+        self,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict] = None,
+    ) -> None:
+        pass
+
+    @abstractmethod
+    def write(self, filename: str) -> None:
+        pass

miplearn/solvers/gurobi.py

@@ -1,315 +1,216 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
-import logging
+from typing import Dict, Optional, Callable, Any, List
-import re
-import sys
-from io import StringIO
-from random import randint
-from typing import List, Any, Dict, Optional, TYPE_CHECKING
 
+import gurobipy as gp
+from gurobipy import GRB, GurobiError
 import numpy as np
-from overrides import overrides
+from scipy.sparse import lil_matrix
 
-from miplearn.instance.base import Instance
+from miplearn.h5 import H5File
-from miplearn.solvers import _RedirectOutput
-from miplearn.solvers.internal import (
-    InternalSolver,
-    LPSolveStats,
-    IterationCallback,
-    LazyCallback,
-    MIPSolveStats,
-    Variables,
-    Constraints,
-)
-from miplearn.solvers.pyomo.base import PyomoTestInstanceKnapsack
-from miplearn.types import (
-    SolverParams,
-    UserCutCallback,
-    Solution,
-)
-
-if TYPE_CHECKING:
-    import gurobipy
-
-logger = logging.getLogger(__name__)
 
 
-class GurobiSolver(InternalSolver):
+class GurobiModel:
-    """
+    _supports_basis_status = True
-    An InternalSolver backed by Gurobi's Python API (without Pyomo).
+    _supports_sensitivity_analysis = True
-
+    _supports_node_count = True
-    Parameters
+    _supports_solution_pool = True
-    ----------
-    params: Optional[SolverParams]
-        Parameters to pass to Gurobi. For example, `params={"MIPGap": 1e-3}`
-        sets the gap tolerance to 1e-3.
-    lazy_cb_frequency: int
-        If 1, calls lazy constraint callbacks whenever an integer solution
-        is found. If 2, calls it also at every node, after solving the
-        LP relaxation of that node.
-    """
 
     def __init__(
         self,
-        params: Optional[SolverParams] = None,
+        inner: gp.Model,
-        lazy_cb_frequency: int = 1,
+        find_violations: Optional[Callable] = None,
+        fix_violations: Optional[Callable] = None,
     ) -> None:
-        import gurobipy
+        self.fix_violations = fix_violations
+        self.find_violations = find_violations
+        self.inner = inner
+        self.violations_: Optional[List[Any]] = None
 
-        assert lazy_cb_frequency in [1, 2]
+    def add_constrs(
-        if params is None:
+        self,
-            params = {}
+        var_names: np.ndarray,
-        params["InfUnbdInfo"] = True
+        constrs_lhs: np.ndarray,
-        params["Seed"] = randint(0, 1_000_000)
+        constrs_sense: np.ndarray,
+        constrs_rhs: np.ndarray,
+        stats: Optional[Dict] = None,
+    ) -> None:
+        assert len(var_names.shape) == 1
+        nvars = len(var_names)
+        assert len(constrs_lhs.shape) == 2
+        nconstrs = constrs_lhs.shape[0]
+        assert constrs_lhs.shape[1] == nvars
+        assert constrs_sense.shape == (nconstrs,)
+        assert constrs_rhs.shape == (nconstrs,)
 
-        self.gp = gurobipy
+        gp_vars = [self.inner.getVarByName(var_name.decode()) for var_name in var_names]
-        self.instance: Optional[Instance] = None
+        self.inner.addMConstr(constrs_lhs, gp_vars, constrs_sense, constrs_rhs)
-        self.model: Optional["gurobipy.Model"] = None
-        self.params: SolverParams = params
-        self.cb_where: Optional[int] = None
-        self.lazy_cb_frequency = lazy_cb_frequency
-        self._dirty = True
-        self._has_lp_solution = False
-        self._has_mip_solution = False
 
-        self._varname_to_var: Dict[bytes, "gurobipy.Var"] = {}
+        if stats is not None:
-        self._cname_to_constr: Dict[str, "gurobipy.Constr"] = {}
+            if "Added constraints" not in stats:
-        self._gp_vars: List["gurobipy.Var"] = []
+                stats["Added constraints"] = 0
-        self._gp_constrs: List["gurobipy.Constr"] = []
+            stats["Added constraints"] += nconstrs
-        self._var_names: np.ndarray = np.empty(0)
-        self._constr_names: List[str] = []
-        self._var_types: np.ndarray = np.empty(0)
-        self._var_lbs: np.ndarray = np.empty(0)
-        self._var_ubs: np.ndarray = np.empty(0)
-        self._var_obj_coeffs: np.ndarray = np.empty(0)
 
-        if self.lazy_cb_frequency == 1:
+    def extract_after_load(self, h5: H5File) -> None:
-            self.lazy_cb_where = [self.gp.GRB.Callback.MIPSOL]
+        """
+        Given a model that has just been loaded, extracts static problem
+        features, such as variable names and types, objective coefficients, etc.
+        """
+        self.inner.update()
+        self._extract_after_load_vars(h5)
+        self._extract_after_load_constrs(h5)
+        h5.put_scalar("static_sense", "min" if self.inner.modelSense > 0 else "max")
+        h5.put_scalar("static_obj_offset", self.inner.objCon)
+
+    def extract_after_lp(self, h5: H5File) -> None:
+        """
+        Given a linear programming model that has just been solved, extracts
+        dynamic problem features, such as optimal LP solution, basis status,
+        etc.
+        """
+        self._extract_after_lp_vars(h5)
+        self._extract_after_lp_constrs(h5)
+        h5.put_scalar("lp_obj_value", self.inner.objVal)
+        h5.put_scalar("lp_wallclock_time", self.inner.runtime)
+
+    def extract_after_mip(self, h5: H5File) -> None:
+        """
+        Given a mixed-integer linear programming model that has just been
+        solved, extracts dynamic problem features, such as optimal MIP solution.
+        """
+        h5.put_scalar("mip_wallclock_time", self.inner.runtime)
+        h5.put_scalar("mip_node_count", self.inner.nodeCount)
+        if self.inner.status == GRB.INFEASIBLE:
+            return
+        gp_vars = self.inner.getVars()
+        gp_constrs = self.inner.getConstrs()
+        h5.put_array(
+            "mip_var_values",
+            np.array(self.inner.getAttr("x", gp_vars), dtype=float),
+        )
+        h5.put_array(
+            "mip_constr_slacks",
+            np.abs(np.array(self.inner.getAttr("slack", gp_constrs), dtype=float)),
+        )
+        h5.put_scalar("mip_obj_value", self.inner.objVal)
+        h5.put_scalar("mip_obj_bound", self.inner.objBound)
+        try:
+            h5.put_scalar("mip_gap", self.inner.mipGap)
+        except AttributeError:
+            pass
+        self._extract_after_mip_solution_pool(h5)
+
+    def fix_variables(
+        self,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict] = None,
+    ) -> None:
+        assert len(var_values.shape) == 1
+        assert len(var_values.shape) == 1
+        assert var_names.shape == var_values.shape
+
+        n_fixed = 0
+        for (var_idx, var_name) in enumerate(var_names):
+            var_val = var_values[var_idx]
+            if np.isfinite(var_val):
+                var = self.inner.getVarByName(var_name.decode())
+                var.vtype = "C"
+                var.lb = var_val
+                var.ub = var_val
+                n_fixed += 1
+        if stats is not None:
+            stats["Fixed variables"] = n_fixed
+
+    def optimize(self) -> None:
+        self.violations_ = []
+
+        def callback(m: gp.Model, where: int) -> None:
+            assert self.find_violations is not None
+            assert self.violations_ is not None
+            assert self.fix_violations is not None
+            if where == GRB.Callback.MIPSOL:
+                violations = self.find_violations(self)
+                self.violations_.extend(violations)
+                self.fix_violations(self, violations, "cb")
+
+        if self.fix_violations is not None:
+            self.inner.Params.lazyConstraints = 1
+            self.inner.optimize(callback)
         else:
-            self.lazy_cb_where = [
+            self.inner.optimize()
-                self.gp.GRB.Callback.MIPSOL,
-                self.gp.GRB.Callback.MIPNODE,
-            ]
 
-    @overrides
+    def relax(self) -> "GurobiModel":
-    def add_constraints(self, cf: Constraints) -> None:
+        return GurobiModel(self.inner.relax())
-        assert cf.names is not None
-        assert cf.senses is not None
-        assert cf.lhs is not None
-        assert cf.rhs is not None
-        assert self.model is not None
-        for i in range(len(cf.names)):
-            sense = cf.senses[i]
-            lhs = self.gp.quicksum(
-                self._varname_to_var[varname] * coeff for (varname, coeff) in cf.lhs[i]
-            )
-            if sense == b"=":
-                self.model.addConstr(lhs == cf.rhs[i], name=cf.names[i])
-            elif sense == b"<":
-                self.model.addConstr(lhs <= cf.rhs[i], name=cf.names[i])
-            elif sense == b">":
-                self.model.addConstr(lhs >= cf.rhs[i], name=cf.names[i])
-            else:
-                raise Exception(f"Unknown sense: {sense}")
-        self.model.update()
-        self._dirty = True
-        self._has_lp_solution = False
-        self._has_mip_solution = False
 
-    @overrides
+    def set_time_limit(self, time_limit_sec: float) -> None:
-    def are_callbacks_supported(self) -> bool:
+        self.inner.params.timeLimit = time_limit_sec
-        return True
 
-    @overrides
+    def set_warm_starts(
-    def are_constraints_satisfied(
         self,
-        cf: Constraints,
+        var_names: np.ndarray,
-        tol: float = 1e-5,
+        var_values: np.ndarray,
-    ) -> List[bool]:
+        stats: Optional[Dict] = None,
-        assert cf.names is not None
+    ) -> None:
-        assert cf.senses is not None
+        assert len(var_values.shape) == 2
-        assert cf.lhs is not None
+        (n_starts, n_vars) = var_values.shape
-        assert cf.rhs is not None
+        assert len(var_names.shape) == 1
-        assert self.model is not None
+        assert var_names.shape[0] == n_vars
-        result = []
+
-        for i in range(len(cf.names)):
+        self.inner.numStart = n_starts
-            sense = cf.senses[i]
+        for start_idx in range(n_starts):
-            lhs = sum(
+            self.inner.params.startNumber = start_idx
-                self._varname_to_var[varname].x * coeff
+            for (var_idx, var_name) in enumerate(var_names):
-                for (varname, coeff) in cf.lhs[i]
+                var_val = var_values[start_idx, var_idx]
+                if np.isfinite(var_val):
+                    var = self.inner.getVarByName(var_name.decode())
+                    var.start = var_val
+
+        if stats is not None:
+            stats["WS: Count"] = n_starts
+            stats["WS: Number of variables set"] = (
+                np.isfinite(var_values).mean(axis=0).sum()
             )
-            if sense == "<":
-                result.append(lhs <= cf.rhs[i] + tol)
-            elif sense == ">":
-                result.append(lhs >= cf.rhs[i] - tol)
-            else:
-                result.append(abs(cf.rhs[i] - lhs) <= tol)
-        return result
 
-    @overrides
+    def _extract_after_load_vars(self, h5: H5File) -> None:
-    def build_test_instance_infeasible(self) -> Instance:
+        gp_vars = self.inner.getVars()
-        return GurobiTestInstanceInfeasible()
+        for (h5_field, gp_field) in {
-
+            "static_var_names": "varName",
-    @overrides
+            "static_var_types": "vtype",
-    def build_test_instance_knapsack(self) -> Instance:
+        }.items():
-        return GurobiTestInstanceKnapsack(
+            h5.put_array(
-            weights=[23.0, 26.0, 20.0, 18.0],
+                h5_field, np.array(self.inner.getAttr(gp_field, gp_vars), dtype="S")
-            prices=[505.0, 352.0, 458.0, 220.0],
+            )
-            capacity=67.0,
+        for (h5_field, gp_field) in {
-        )
+            "static_var_upper_bounds": "ub",
-
+            "static_var_lower_bounds": "lb",
-    @overrides
+            "static_var_obj_coeffs": "obj",
-    def clone(self) -> "GurobiSolver":
+        }.items():
-        return GurobiSolver(
+            h5.put_array(
-            params=self.params,
+                h5_field, np.array(self.inner.getAttr(gp_field, gp_vars), dtype=float)
-            lazy_cb_frequency=self.lazy_cb_frequency,
-        )
-
-    @overrides
-    def fix(self, solution: Solution) -> None:
-        self._raise_if_callback()
-        for (varname, value) in solution.items():
-            if value is None:
-                continue
-            var = self._varname_to_var[varname]
-            var.vtype = self.gp.GRB.CONTINUOUS
-            var.lb = value
-            var.ub = value
-
-    @overrides
-    def get_constraint_attrs(self) -> List[str]:
-        return [
-            "basis_status",
-            "categories",
-            "dual_values",
-            "lazy",
-            "lhs",
-            "names",
-            "rhs",
-            "sa_rhs_down",
-            "sa_rhs_up",
-            "senses",
-            "slacks",
-            "user_features",
-        ]
-
-    @overrides
-    def get_constraints(
-        self,
-        with_static: bool = True,
-        with_sa: bool = True,
-        with_lhs: bool = True,
-    ) -> Constraints:
-        model = self.model
-        assert model is not None
-        assert model.numVars == len(self._gp_vars)
-
-        def _parse_gurobi_cbasis(v: int) -> str:
-            if v == 0:
-                return "B"
-            if v == -1:
-                return "N"
-            raise Exception(f"unknown cbasis: {v}")
-
-        gp_constrs = model.getConstrs()
-        constr_names = np.array(model.getAttr("constrName", gp_constrs), dtype="S")
-        lhs: Optional[List] = None
-        rhs, senses, slacks, basis_status = None, None, None, None
-        dual_value, basis_status, sa_rhs_up, sa_rhs_down = None, None, None, None
-
-        if with_static:
-            rhs = np.array(model.getAttr("rhs", gp_constrs), dtype=float)
-            senses = np.array(model.getAttr("sense", gp_constrs), dtype="S")
-            if with_lhs:
-                lhs = [None for _ in gp_constrs]
-                for (i, gp_constr) in enumerate(gp_constrs):
-                    expr = model.getRow(gp_constr)
-                    lhs[i] = [
-                        (self._var_names[expr.getVar(j).index], expr.getCoeff(j))
-                        for j in range(expr.size())
-                    ]
-
-        if self._has_lp_solution:
-            dual_value = np.array(model.getAttr("pi", gp_constrs), dtype=float)
-            basis_status = np.array(
-                [_parse_gurobi_cbasis(c) for c in model.getAttr("cbasis", gp_constrs)],
-                dtype="S",
             )
-            if with_sa:
-                sa_rhs_up = np.array(model.getAttr("saRhsUp", gp_constrs), dtype=float)
-                sa_rhs_down = np.array(
-                    model.getAttr("saRhsLow", gp_constrs), dtype=float
-                )
 
-        if self._has_lp_solution or self._has_mip_solution:
+    def _extract_after_load_constrs(self, h5: H5File) -> None:
-            slacks = np.array(model.getAttr("slack", gp_constrs), dtype=float)
+        gp_constrs = self.inner.getConstrs()
+        gp_vars = self.inner.getVars()
+        rhs = np.array(self.inner.getAttr("rhs", gp_constrs), dtype=float)
+        senses = np.array(self.inner.getAttr("sense", gp_constrs), dtype="S")
+        names = np.array(self.inner.getAttr("constrName", gp_constrs), dtype="S")
+        nrows, ncols = len(gp_constrs), len(gp_vars)
+        tmp = lil_matrix((nrows, ncols), dtype=float)
+        for (i, gp_constr) in enumerate(gp_constrs):
+            expr = self.inner.getRow(gp_constr)
+            for j in range(expr.size()):
+                tmp[i, expr.getVar(j).index] = expr.getCoeff(j)
+        lhs = tmp.tocoo()
 
-        return Constraints(
+        h5.put_array("static_constr_names", names)
-            basis_status=basis_status,
+        h5.put_array("static_constr_rhs", rhs)
-            dual_values=dual_value,
+        h5.put_array("static_constr_sense", senses)
-            lhs=lhs,
+        h5.put_sparse("static_constr_lhs", lhs)
-            names=constr_names,
-            rhs=rhs,
-            sa_rhs_down=sa_rhs_down,
-            sa_rhs_up=sa_rhs_up,
-            senses=senses,
-            slacks=slacks,
-        )
-
-    @overrides
-    def get_solution(self) -> Optional[Solution]:
-        assert self.model is not None
-        if self.cb_where is not None:
-            if self.cb_where == self.gp.GRB.Callback.MIPNODE:
-                return {
-                    v.varName.encode(): self.model.cbGetNodeRel(v)
-                    for v in self.model.getVars()
-                }
-            elif self.cb_where == self.gp.GRB.Callback.MIPSOL:
-                return {
-                    v.varName.encode(): self.model.cbGetSolution(v)
-                    for v in self.model.getVars()
-                }
-            else:
-                raise Exception(
-                    f"get_solution can only be called from a callback "
-                    f"when cb_where is either MIPNODE or MIPSOL"
-                )
-        if self.model.solCount == 0:
-            return None
-        return {v.varName.encode(): v.x for v in self.model.getVars()}
-
-    @overrides
-    def get_variable_attrs(self) -> List[str]:
-        return [
-            "names",
-            "basis_status",
-            "categories",
-            "lower_bounds",
-            "obj_coeffs",
-            "reduced_costs",
-            "sa_lb_down",
-            "sa_lb_up",
-            "sa_obj_down",
-            "sa_obj_up",
-            "sa_ub_down",
-            "sa_ub_up",
-            "types",
-            "upper_bounds",
-            "user_features",
-            "values",
-        ]
-
-    @overrides
-    def get_variables(
-        self,
-        with_static: bool = True,
-        with_sa: bool = True,
-    ) -> Variables:
-        model = self.model
-        assert model is not None
 
+    def _extract_after_lp_vars(self, h5: H5File) -> None:
         def _parse_gurobi_vbasis(b: int) -> str:
             if b == 0:
                 return "B"
@@ -320,399 +221,81 @@ class GurobiSolver(InternalSolver):
             elif b == -3:
                 return "S"
             else:
-                raise Exception(f"unknown vbasis: {basis_status}")
+                raise Exception(f"unknown vbasis: {b}")
 
-        basis_status: Optional[np.ndarray] = None
+        gp_vars = self.inner.getVars()
-        upper_bounds, lower_bounds, types, values = None, None, None, None
+        h5.put_array(
-        obj_coeffs, reduced_costs = None, None
+            "lp_var_basis_status",
-        sa_obj_up, sa_ub_up, sa_lb_up = None, None, None
+            np.array(
-        sa_obj_down, sa_ub_down, sa_lb_down = None, None, None
-
-        if with_static:
-            upper_bounds = self._var_ubs
-            lower_bounds = self._var_lbs
-            types = self._var_types
-            obj_coeffs = self._var_obj_coeffs
-
-        if self._has_lp_solution:
-            reduced_costs = np.array(model.getAttr("rc", self._gp_vars), dtype=float)
-            basis_status = np.array(
                 [
                     _parse_gurobi_vbasis(b)
-                    for b in model.getAttr("vbasis", self._gp_vars)
+                    for b in self.inner.getAttr("vbasis", gp_vars)
                 ],
                 dtype="S",
+            ),
+        )
+        for (h5_field, gp_field) in {
+            "lp_var_reduced_costs": "rc",
+            "lp_var_sa_obj_up": "saobjUp",
+            "lp_var_sa_obj_down": "saobjLow",
+            "lp_var_sa_ub_up": "saubUp",
+            "lp_var_sa_ub_down": "saubLow",
+            "lp_var_sa_lb_up": "salbUp",
+            "lp_var_sa_lb_down": "salbLow",
+            "lp_var_values": "x",
+        }.items():
+            h5.put_array(
+                h5_field,
+                np.array(self.inner.getAttr(gp_field, gp_vars), dtype=float),
             )
 
-            if with_sa:
+    def _extract_after_lp_constrs(self, h5: H5File) -> None:
-                sa_obj_up = np.array(
+        def _parse_gurobi_cbasis(v: int) -> str:
-                    model.getAttr("saobjUp", self._gp_vars),
+            if v == 0:
-                    dtype=float,
+                return "B"
-                )
+            if v == -1:
-                sa_obj_down = np.array(
+                return "N"
-                    model.getAttr("saobjLow", self._gp_vars),
+            raise Exception(f"unknown cbasis: {v}")
-                    dtype=float,
-                )
-                sa_ub_up = np.array(
-                    model.getAttr("saubUp", self._gp_vars),
-                    dtype=float,
-                )
-                sa_ub_down = np.array(
-                    model.getAttr("saubLow", self._gp_vars),
-                    dtype=float,
-                )
-                sa_lb_up = np.array(
-                    model.getAttr("salbUp", self._gp_vars),
-                    dtype=float,
-                )
-                sa_lb_down = np.array(
-                    model.getAttr("salbLow", self._gp_vars),
-                    dtype=float,
-                )
 
-        if model.solCount > 0:
+        gp_constrs = self.inner.getConstrs()
-            values = np.array(model.getAttr("x", self._gp_vars), dtype=float)
+        h5.put_array(
-
+            "lp_constr_basis_status",
-        return Variables(
+            np.array(
-            names=self._var_names,
+                [
-            upper_bounds=upper_bounds,
+                    _parse_gurobi_cbasis(c)
-            lower_bounds=lower_bounds,
+                    for c in self.inner.getAttr("cbasis", gp_constrs)
-            types=types,
+                ],
-            obj_coeffs=obj_coeffs,
+                dtype="S",
-            reduced_costs=reduced_costs,
+            ),
-            basis_status=basis_status,
+        )
-            sa_obj_up=sa_obj_up,
+        for (h5_field, gp_field) in {
-            sa_obj_down=sa_obj_down,
+            "lp_constr_dual_values": "pi",
-            sa_ub_up=sa_ub_up,
+            "lp_constr_sa_rhs_up": "saRhsUp",
-            sa_ub_down=sa_ub_down,
+            "lp_constr_sa_rhs_down": "saRhsLow",
-            sa_lb_up=sa_lb_up,
+        }.items():
-            sa_lb_down=sa_lb_down,
+            h5.put_array(
-            values=values,
+                h5_field,
+                np.array(self.inner.getAttr(gp_field, gp_constrs), dtype=float),
+            )
+        h5.put_array(
+            "lp_constr_slacks",
+            np.abs(np.array(self.inner.getAttr("slack", gp_constrs), dtype=float)),
         )
 
-    @overrides
+    def _extract_after_mip_solution_pool(self, h5: H5File) -> None:
-    def is_infeasible(self) -> bool:
+        gp_vars = self.inner.getVars()
-        assert self.model is not None
+        pool_var_values = []
-        return self.model.status in [self.gp.GRB.INFEASIBLE, self.gp.GRB.INF_OR_UNBD]
+        pool_obj_values = []
-
+        for i in range(self.inner.SolCount):
-    @overrides
+            self.inner.params.SolutionNumber = i
-    def remove_constraints(self, names: List[str]) -> None:
-        assert self.model is not None
-        constrs = [self.model.getConstrByName(n) for n in names]
-        self.model.remove(constrs)
-        self.model.update()
-
-    @overrides
-    def set_instance(
-        self,
-        instance: Instance,
-        model: Any = None,
-    ) -> None:
-        self._raise_if_callback()
-        if model is None:
-            model = instance.to_model()
-        assert isinstance(model, self.gp.Model)
-        self.instance = instance
-        self.model = model
-        self.model.update()
-        self._update()
-
-    @overrides
-    def set_warm_start(self, solution: Solution) -> None:
-        self._raise_if_callback()
-        self._clear_warm_start()
-        for (var_name, value) in solution.items():
-            var = self._varname_to_var[var_name]
-            if value is not None:
-                var.start = value
-
-    @overrides
-    def solve(
-        self,
-        tee: bool = False,
-        iteration_cb: Optional[IterationCallback] = None,
-        lazy_cb: Optional[LazyCallback] = None,
-        user_cut_cb: Optional[UserCutCallback] = None,
-    ) -> MIPSolveStats:
-        self._raise_if_callback()
-        assert self.model is not None
-        if iteration_cb is None:
-            iteration_cb = lambda: False
-        callback_exceptions = []
-
-        # Create callback wrapper
-        def cb_wrapper(cb_model: Any, cb_where: int) -> None:
             try:
-                self.cb_where = cb_where
+                pool_var_values.append(self.inner.getAttr("Xn", gp_vars))
-                if lazy_cb is not None and cb_where in self.lazy_cb_where:
+                pool_obj_values.append(self.inner.PoolObjVal)
-                    lazy_cb(self, self.model)
+            except GurobiError:
-                if user_cut_cb is not None and cb_where == self.gp.GRB.Callback.MIPNODE:
+                pass
-                    user_cut_cb(self, self.model)
+        h5.put_array("pool_var_values", np.array(pool_var_values))
-            except Exception as e:
+        h5.put_array("pool_obj_values", np.array(pool_obj_values))
-                logger.exception("callback error")
-                callback_exceptions.append(e)
-            finally:
-                self.cb_where = None
 
-        # Configure Gurobi
+    def write(self, filename: str) -> None:
-        if lazy_cb is not None:
+        self.inner.update()
-            self.params["LazyConstraints"] = 1
+        self.inner.write(filename)
-        if user_cut_cb is not None:
-            self.params["PreCrush"] = 1
-
-        # Solve problem
-        total_wallclock_time = 0
-        total_nodes = 0
-        streams: List[Any] = [StringIO()]
-        if tee:
-            streams += [sys.stdout]
-        self._apply_params(streams)
-        while True:
-            with _RedirectOutput(streams):
-                self.model.optimize(cb_wrapper)
-                self._dirty = False
-            if len(callback_exceptions) > 0:
-                raise callback_exceptions[0]
-            total_wallclock_time += self.model.runtime
-            total_nodes += int(self.model.nodeCount)
-            should_repeat = iteration_cb()
-            if not should_repeat:
-                break
-        self._has_lp_solution = False
-        self._has_mip_solution = self.model.solCount > 0
-
-        # Fetch results and stats
-        log = streams[0].getvalue()
-        ub, lb = None, None
-        sense = "min" if self.model.modelSense == 1 else "max"
-        if self.model.solCount > 0:
-            if self.model.modelSense == 1:
-                lb = self.model.objBound
-                ub = self.model.objVal
-            else:
-                lb = self.model.objVal
-                ub = self.model.objBound
-        ws_value = self._extract_warm_start_value(log)
-        return MIPSolveStats(
-            mip_lower_bound=lb,
-            mip_upper_bound=ub,
-            mip_wallclock_time=total_wallclock_time,
-            mip_nodes=total_nodes,
-            mip_sense=sense,
-            mip_log=log,
-            mip_warm_start_value=ws_value,
-        )
-
-    @overrides
-    def solve_lp(
-        self,
-        tee: bool = False,
-    ) -> LPSolveStats:
-        self._raise_if_callback()
-        streams: List[Any] = [StringIO()]
-        if tee:
-            streams += [sys.stdout]
-        self._apply_params(streams)
-        assert self.model is not None
-        for (i, var) in enumerate(self._gp_vars):
-            if self._var_types[i] == b"B":
-                var.vtype = self.gp.GRB.CONTINUOUS
-                var.lb = 0.0
-                var.ub = 1.0
-        with _RedirectOutput(streams):
-            self.model.optimize()
-            self._dirty = False
-        for (i, var) in enumerate(self._gp_vars):
-            if self._var_types[i] == b"B":
-                var.vtype = self.gp.GRB.BINARY
-        log = streams[0].getvalue()
-        self._has_lp_solution = self.model.solCount > 0
-        self._has_mip_solution = False
-        opt_value = None
-        if not self.is_infeasible():
-            opt_value = self.model.objVal
-        return LPSolveStats(
-            lp_value=opt_value,
-            lp_log=log,
-            lp_wallclock_time=self.model.runtime,
-        )
-
-    def _apply_params(self, streams: List[Any]) -> None:
-        assert self.model is not None
-        with _RedirectOutput(streams):
-            for (name, value) in self.params.items():
-                self.model.setParam(name, value)
-
-    def _clear_warm_start(self) -> None:
-        for var in self._varname_to_var.values():
-            var.start = self.gp.GRB.UNDEFINED
-
-    @staticmethod
-    def _extract(
-        log: str,
-        regexp: str,
-        default: Optional[str] = None,
-    ) -> Optional[str]:
-        value = default
-        for line in log.splitlines():
-            matches = re.findall(regexp, line)
-            if len(matches) == 0:
-                continue
-            value = matches[0]
-        return value
-
-    def _extract_warm_start_value(self, log: str) -> Optional[float]:
-        ws = self._extract(log, "MIP start with objective ([0-9.e+-]*)")
-        if ws is None:
-            return None
-        return float(ws)
-
-    def _get_value(self, var: Any) -> float:
-        assert self.model is not None
-        if self.cb_where == self.gp.GRB.Callback.MIPSOL:
-            return self.model.cbGetSolution(var)
-        elif self.cb_where == self.gp.GRB.Callback.MIPNODE:
-            return self.model.cbGetNodeRel(var)
-        elif self.cb_where is None:
-            return var.x
-        else:
-            raise Exception(
-                "get_value cannot be called from cb_where=%s" % self.cb_where
-            )
-
-    def _raise_if_callback(self) -> None:
-        if self.cb_where is not None:
-            raise Exception("method cannot be called from a callback")
-
-    def _update(self) -> None:
-        assert self.model is not None
-        gp_vars: List["gurobipy.Var"] = self.model.getVars()
-        gp_constrs: List["gurobipy.Constr"] = self.model.getConstrs()
-        var_names: np.ndarray = np.array(
-            self.model.getAttr("varName", gp_vars),
-            dtype="S",
-        )
-        var_types: np.ndarray = np.array(
-            self.model.getAttr("vtype", gp_vars),
-            dtype="S",
-        )
-        var_ubs: np.ndarray = np.array(
-            self.model.getAttr("ub", gp_vars),
-            dtype=float,
-        )
-        var_lbs: np.ndarray = np.array(
-            self.model.getAttr("lb", gp_vars),
-            dtype=float,
-        )
-        var_obj_coeffs: np.ndarray = np.array(
-            self.model.getAttr("obj", gp_vars),
-            dtype=float,
-        )
-        constr_names: List[str] = self.model.getAttr("constrName", gp_constrs)
-        varname_to_var: Dict[bytes, "gurobipy.Var"] = {}
-        cname_to_constr: Dict = {}
-        for (i, gp_var) in enumerate(gp_vars):
-            assert var_names[i] not in varname_to_var, (
-                f"Duplicated variable name detected: {var_names[i]}. "
-                f"Unique variable names are currently required."
-            )
-            if var_types[i] == b"I":
-                assert var_ubs[i] == 1.0, (
-                    "Only binary and continuous variables are currently supported. "
-                    f"Integer variable {var_names[i]} has upper bound {var_ubs[i]}."
-                )
-                assert var_lbs[i] == 0.0, (
-                    "Only binary and continuous variables are currently supported. "
-                    f"Integer variable {var_names[i]} has lower bound {var_ubs[i]}."
-                )
-                var_types[i] = b"B"
-            assert var_types[i] in [b"B", b"C"], (
-                "Only binary and continuous variables are currently supported. "
-                f"Variable {var_names[i]} has type {var_types[i]}."
-            )
-            varname_to_var[var_names[i]] = gp_var
-        for (i, gp_constr) in enumerate(gp_constrs):
-            assert constr_names[i] not in cname_to_constr, (
-                f"Duplicated constraint name detected: {constr_names[i]}. "
-                f"Unique constraint names are currently required."
-            )
-            cname_to_constr[constr_names[i]] = gp_constr
-        self._varname_to_var = varname_to_var
-        self._cname_to_constr = cname_to_constr
-        self._gp_vars = gp_vars
-        self._gp_constrs = gp_constrs
-        self._var_names = var_names
-        self._constr_names = constr_names
-        self._var_types = var_types
-        self._var_lbs = var_lbs
-        self._var_ubs = var_ubs
-        self._var_obj_coeffs = var_obj_coeffs
-
-    def __getstate__(self) -> Dict:
-        return {
-            "params": self.params,
-            "lazy_cb_where": self.lazy_cb_where,
-        }
-
-    def __setstate__(self, state: Dict) -> None:
-        self.params = state["params"]
-        self.lazy_cb_where = state["lazy_cb_where"]
-        self.instance = None
-        self.model = None
-        self.cb_where = None
-
-
-class GurobiTestInstanceInfeasible(Instance):
-    @overrides
-    def to_model(self) -> Any:
-        import gurobipy as gp
-        from gurobipy import GRB
-
-        model = gp.Model()
-        x = model.addVars(1, vtype=GRB.BINARY, name="x")
-        model.addConstr(x[0] >= 2)
-        model.setObjective(x[0])
-        return model
-
-
-class GurobiTestInstanceKnapsack(PyomoTestInstanceKnapsack):
-    """
-    Simpler (one-dimensional) knapsack instance, implemented directly in Gurobi
-    instead of Pyomo, used for testing.
-    """
-
-    def __init__(
-        self,
-        weights: List[float],
-        prices: List[float],
-        capacity: float,
-    ) -> None:
-        super().__init__(weights, prices, capacity)
-
-    @overrides
-    def to_model(self) -> Any:
-        import gurobipy as gp
-        from gurobipy import GRB
-
-        model = gp.Model("Knapsack")
-        n = len(self.weights)
-        x = model.addVars(n, vtype=GRB.BINARY, name="x")
-        z = model.addVar(vtype=GRB.CONTINUOUS, name="z", ub=self.capacity)
-        model.addConstr(
-            gp.quicksum(x[i] * self.weights[i] for i in range(n)) == z,
-            "eq_capacity",
-        )
-        model.setObjective(
-            gp.quicksum(x[i] * self.prices[i] for i in range(n)), GRB.MAXIMIZE
-        )
-        return model
-
-    @overrides
-    def enforce_lazy_constraint(
-        self,
-        solver: InternalSolver,
-        model: Any,
-        violation: str,
-    ) -> None:
-        x0 = model.getVarByName("x[0]")
-        model.cbLazy(x0 <= 0)

miplearn/solvers/internal.py

@@ -1,348 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-from abc import ABC, abstractmethod
-from dataclasses import dataclass
-from typing import Any, Optional, List, Tuple, TYPE_CHECKING
-
-import numpy as np
-
-from miplearn.instance.base import Instance
-from miplearn.types import (
-    IterationCallback,
-    LazyCallback,
-    UserCutCallback,
-    Solution,
-)
-
-logger = logging.getLogger(__name__)
-
-if TYPE_CHECKING:
-    from miplearn.features.sample import Sample
-
-
-@dataclass
-class LPSolveStats:
-    lp_log: Optional[str] = None
-    lp_value: Optional[float] = None
-    lp_wallclock_time: Optional[float] = None
-
-    def to_list(self) -> List[float]:
-        features: List[float] = []
-        for attr in ["lp_value", "lp_wallclock_time"]:
-            if getattr(self, attr) is not None:
-                features.append(getattr(self, attr))
-        return features
-
-
-@dataclass
-class MIPSolveStats:
-    mip_lower_bound: Optional[float] = None
-    mip_log: Optional[str] = None
-    mip_nodes: Optional[int] = None
-    mip_sense: Optional[str] = None
-    mip_upper_bound: Optional[float] = None
-    mip_wallclock_time: Optional[float] = None
-    mip_warm_start_value: Optional[float] = None
-
-
-@dataclass
-class Variables:
-    names: Optional[np.ndarray] = None
-    basis_status: Optional[np.ndarray] = None
-    lower_bounds: Optional[np.ndarray] = None
-    obj_coeffs: Optional[np.ndarray] = None
-    reduced_costs: Optional[np.ndarray] = None
-    sa_lb_down: Optional[np.ndarray] = None
-    sa_lb_up: Optional[np.ndarray] = None
-    sa_obj_down: Optional[np.ndarray] = None
-    sa_obj_up: Optional[np.ndarray] = None
-    sa_ub_down: Optional[np.ndarray] = None
-    sa_ub_up: Optional[np.ndarray] = None
-    types: Optional[np.ndarray] = None
-    upper_bounds: Optional[np.ndarray] = None
-    values: Optional[np.ndarray] = None
-
-
-@dataclass
-class Constraints:
-    basis_status: Optional[np.ndarray] = None
-    dual_values: Optional[np.ndarray] = None
-    lazy: Optional[np.ndarray] = None
-    lhs: Optional[List[List[Tuple[bytes, float]]]] = None
-    names: Optional[np.ndarray] = None
-    rhs: Optional[np.ndarray] = None
-    sa_rhs_down: Optional[np.ndarray] = None
-    sa_rhs_up: Optional[np.ndarray] = None
-    senses: Optional[np.ndarray] = None
-    slacks: Optional[np.ndarray] = None
-
-    @staticmethod
-    def from_sample(sample: "Sample") -> "Constraints":
-        return Constraints(
-            basis_status=sample.get_array("lp_constr_basis_status"),
-            dual_values=sample.get_array("lp_constr_dual_values"),
-            lazy=sample.get_array("static_constr_lazy"),
-            # lhs=sample.get_vector("static_constr_lhs"),
-            names=sample.get_array("static_constr_names"),
-            rhs=sample.get_array("static_constr_rhs"),
-            sa_rhs_down=sample.get_array("lp_constr_sa_rhs_down"),
-            sa_rhs_up=sample.get_array("lp_constr_sa_rhs_up"),
-            senses=sample.get_array("static_constr_senses"),
-            slacks=sample.get_array("lp_constr_slacks"),
-        )
-
-    def __getitem__(self, selected: List[bool]) -> "Constraints":
-        return Constraints(
-            basis_status=(
-                None if self.basis_status is None else self.basis_status[selected]
-            ),
-            dual_values=(
-                None if self.dual_values is None else self.dual_values[selected]
-            ),
-            names=(None if self.names is None else self.names[selected]),
-            lazy=(None if self.lazy is None else self.lazy[selected]),
-            lhs=self._filter(self.lhs, selected),
-            rhs=(None if self.rhs is None else self.rhs[selected]),
-            sa_rhs_down=(
-                None if self.sa_rhs_down is None else self.sa_rhs_down[selected]
-            ),
-            sa_rhs_up=(None if self.sa_rhs_up is None else self.sa_rhs_up[selected]),
-            senses=(None if self.senses is None else self.senses[selected]),
-            slacks=(None if self.slacks is None else self.slacks[selected]),
-        )
-
-    def _filter(
-        self,
-        obj: Optional[List],
-        selected: List[bool],
-    ) -> Optional[List]:
-        if obj is None:
-            return None
-        return [obj[i] for (i, selected_i) in enumerate(selected) if selected_i]
-
-
-class InternalSolver(ABC):
-    """
-    Abstract class representing the MIP solver used internally by LearningSolver.
-    """
-
-    @abstractmethod
-    def add_constraints(self, cf: Constraints) -> None:
-        """Adds the given constraints to the model."""
-        pass
-
-    @abstractmethod
-    def are_constraints_satisfied(
-        self,
-        cf: Constraints,
-        tol: float = 1e-5,
-    ) -> List[bool]:
-        """
-        Checks whether the current solution satisfies the given constraints.
-        """
-        pass
-
-    def are_callbacks_supported(self) -> bool:
-        """
-        Returns True if this solver supports native callbacks, such as lazy constraints
-        callback or user cuts callback.
-        """
-        return False
-
-    @abstractmethod
-    def build_test_instance_infeasible(self) -> Instance:
-        """
-        Returns an infeasible instance, for testing purposes.
-        """
-        pass
-
-    @abstractmethod
-    def build_test_instance_knapsack(self) -> Instance:
-        """
-        Returns an instance corresponding to the following MIP, for testing purposes:
-
-          maximize  505 x0 + 352 x1 + 458 x2 + 220 x3
-          s.t.      eq_capacity: z = 23 x0 + 26 x1 + 20 x2 + 18 x3
-                    x0, x1, x2, x3 binary
-                    0 <= z <= 67 continuous
-        """
-        pass
-
-    @abstractmethod
-    def clone(self) -> "InternalSolver":
-        """
-        Returns a new copy of this solver with identical parameters, but otherwise
-        completely unitialized.
-        """
-        pass
-
-    @abstractmethod
-    def fix(self, solution: Solution) -> None:
-        """
-        Fixes the values of a subset of decision variables. Missing values in the
-        solution indicate variables that should be left free.
-        """
-        pass
-
-    @abstractmethod
-    def get_solution(self) -> Optional[Solution]:
-        """
-        Returns current solution found by the solver.
-
-        If called after `solve`, returns the best primal solution found during
-        the search. If called after `solve_lp`, returns the optimal solution
-        to the LP relaxation. If no primal solution is available, return None.
-        """
-        pass
-
-    @abstractmethod
-    def get_constraint_attrs(self) -> List[str]:
-        """
-        Returns a list of constraint attributes supported by this solver. Used for
-        testing purposes only.
-        """
-
-        pass
-
-    @abstractmethod
-    def get_constraints(
-        self,
-        with_static: bool = True,
-        with_sa: bool = True,
-        with_lhs: bool = True,
-    ) -> Constraints:
-        pass
-
-    @abstractmethod
-    def get_variable_attrs(self) -> List[str]:
-        """
-        Returns a list of variable attributes supported by this solver. Used for
-        testing purposes only.
-        """
-        pass
-
-    @abstractmethod
-    def get_variables(
-        self,
-        with_static: bool = True,
-        with_sa: bool = True,
-    ) -> Variables:
-        """
-        Returns a description of the decision variables in the problem.
-
-        Parameters
-        ----------
-        with_static: bool
-            If True, include features that do not change during the solution process,
-            such as variable types and names. This parameter is used to reduce the
-            amount of duplicated data collected by LearningSolver. Features that do
-            not change are only collected once.
-        with_sa: bool
-            If True, collect sensitivity analysis information. For large models,
-            collecting this information may be expensive, so this parameter is useful
-            for reducing running times.
-        """
-        pass
-
-    @abstractmethod
-    def is_infeasible(self) -> bool:
-        """
-        Returns True if the model has been proved to be infeasible.
-        Must be called after solve.
-        """
-        pass
-
-    @abstractmethod
-    def remove_constraints(self, names: np.ndarray) -> None:
-        """
-        Removes the given constraints from the model.
-        """
-        pass
-
-    @abstractmethod
-    def set_instance(
-        self,
-        instance: Instance,
-        model: Any = None,
-    ) -> None:
-        """
-        Loads the given instance into the solver.
-
-        Parameters
-        ----------
-        instance: Instance
-            The instance to be loaded.
-        model: Any
-            The concrete optimization model corresponding to this instance
-            (e.g. JuMP.Model or pyomo.core.ConcreteModel). If not provided,
-            it will be generated by calling `instance.to_model()`.
-        """
-        pass
-
-    @abstractmethod
-    def set_warm_start(self, solution: Solution) -> None:
-        """
-        Sets the warm start to be used by the solver.
-
-        Only one warm start is supported. Calling this function when a warm start
-        already exists will remove the previous warm start.
-        """
-        pass
-
-    @abstractmethod
-    def solve(
-        self,
-        tee: bool = False,
-        iteration_cb: Optional[IterationCallback] = None,
-        lazy_cb: Optional[LazyCallback] = None,
-        user_cut_cb: Optional[UserCutCallback] = None,
-    ) -> MIPSolveStats:
-        """
-        Solves the currently loaded instance. After this method finishes,
-        the best solution found can be retrieved by calling `get_solution`.
-
-        Parameters
-        ----------
-        iteration_cb: IterationCallback
-            By default, InternalSolver makes a single call to the native `solve`
-            method and returns the result. If an iteration callback is provided
-            instead, InternalSolver enters a loop, where `solve` and `iteration_cb`
-            are called alternatively. To stop the loop, `iteration_cb` should return
-            False. Any other result causes the solver to loop again.
-        lazy_cb: LazyCallback
-            This function is called whenever the solver finds a new candidate
-            solution and can be used to add lazy constraints to the model. Only the
-            following operations within the callback are allowed:
-                - Querying the value of a variable
-                - Querying if a constraint is satisfied
-                - Adding a new constraint to the problem
-            Additional operations may be allowed by specific subclasses.
-        user_cut_cb: UserCutCallback
-            This function is called whenever the solver found a new integer-infeasible
-            solution and needs to generate cutting planes to cut it off.
-        tee: bool
-            If true, prints the solver log to the screen.
-        """
-        pass
-
-    @abstractmethod
-    def solve_lp(
-        self,
-        tee: bool = False,
-    ) -> LPSolveStats:
-        """
-        Solves the LP relaxation of the currently loaded instance. After this
-        method finishes, the solution can be retrieved by calling `get_solution`.
-
-        This method should not permanently modify the problem. That is, subsequent
-        calls to `solve` should solve the original MIP, not the LP relaxation.
-
-        Parameters
-        ----------
-        tee
-            If true, prints the solver log to the screen.
-        """
-        pass

miplearn/solvers/learning.py

@@ -1,458 +1,43 @@
 #  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
 #  Released under the modified BSD license. See COPYING.md for more details.
+from os.path import exists
+from tempfile import NamedTemporaryFile
+from typing import List, Any, Union
 
-import logging
+from miplearn.h5 import H5File
-import time
+from miplearn.io import _to_h5_filename
-import traceback
+from miplearn.solvers.abstract import AbstractModel
-from typing import Optional, List, Any, cast, Dict, Tuple
-
-from p_tqdm import p_map
-
-from miplearn.components.component import Component
-from miplearn.components.dynamic_lazy import DynamicLazyConstraintsComponent
-from miplearn.components.dynamic_user_cuts import UserCutsComponent
-from miplearn.components.objective import ObjectiveValueComponent
-from miplearn.components.primal import PrimalSolutionComponent
-from miplearn.features.extractor import FeaturesExtractor
-from miplearn.instance.base import Instance
-from miplearn.instance.picklegz import PickleGzInstance
-from miplearn.solvers import _RedirectOutput
-from miplearn.solvers.internal import InternalSolver
-from miplearn.solvers.pyomo.gurobi import GurobiPyomoSolver
-from miplearn.types import LearningSolveStats
-
-logger = logging.getLogger(__name__)
-
-
-class _GlobalVariables:
-    def __init__(self) -> None:
-        self.solver: Optional[LearningSolver] = None
-        self.instances: Optional[List[Instance]] = None
-        self.discard_outputs: bool = False
-
-
-# Global variables used for multiprocessing. Global variables are copied by the
-# operating system when the process forks. Local variables are copied through
-# serialization, which is a much slower process.
-_GLOBAL = [_GlobalVariables()]
-
-
-def _parallel_solve(
-    idx: int,
-) -> Tuple[Optional[LearningSolveStats], Optional[Instance]]:
-    solver = _GLOBAL[0].solver
-    instances = _GLOBAL[0].instances
-    discard_outputs = _GLOBAL[0].discard_outputs
-    assert solver is not None
-    assert instances is not None
-    try:
-        stats = solver.solve(
-            instances[idx],
-            discard_output=discard_outputs,
-        )
-        instances[idx].free()
-        return stats, instances[idx]
-    except Exception as e:
-        traceback.print_exc()
-        logger.exception(f"Exception while solving {instances[idx]}. Ignoring.")
-        return None, None
 
 
 class LearningSolver:
-    """
+    def __init__(self, components: List[Any], skip_lp=False):
-    Mixed-Integer Linear Programming (MIP) solver that extracts information
+        self.components = components
-    from previous runs and uses Machine Learning methods to accelerate the
+        self.skip_lp = skip_lp
-    solution of new (yet unseen) instances.
 
-    Parameters
+    def fit(self, data_filenames):
-    ----------
+        h5_filenames = [_to_h5_filename(f) for f in data_filenames]
-    components: List[Component]
+        for comp in self.components:
-        Set of components in the solver. By default, includes
+            comp.fit(h5_filenames)
-        `ObjectiveValueComponent`, `PrimalSolutionComponent`,
-        `DynamicLazyConstraintsComponent` and `UserCutsComponent`.
-    mode: str
-        If "exact", solves problem to optimality, keeping all optimality
-        guarantees provided by the MIP solver. If "heuristic", uses machine
-        learning more aggressively, and may return suboptimal solutions.
-    solver: Callable[[], InternalSolver]
-        A callable that constructs the internal solver. If None is provided,
-        use GurobiPyomoSolver.
-    use_lazy_cb: bool
-        If true, use native solver callbacks for enforcing lazy constraints,
-        instead of a simple loop. May not be supported by all solvers.
-    solve_lp: bool
-        If true, solve the root LP relaxation before solving the MIP. This
-        option should be activated if the LP relaxation is not very
-        expensive to solve and if it provides good hints for the integer
-        solution.
-    simulate_perfect: bool
-        If true, each call to solve actually performs three actions: solve
-        the original problem, train the ML models on the data that was just
-        collected, and solve the problem again. This is useful for evaluating
-        the theoretical performance of perfect ML models.
-    """
 
-    def __init__(
+    def optimize(self, model: Union[str, AbstractModel], build_model=None):
-        self,
+        if isinstance(model, str):
-        components: Optional[List[Component]] = None,
+            h5_filename = _to_h5_filename(model)
-        mode: str = "exact",
+            assert build_model is not None
-        solver: Optional[InternalSolver] = None,
+            model = build_model(model)
-        use_lazy_cb: bool = False,
-        solve_lp: bool = True,
-        simulate_perfect: bool = False,
-        extractor: Optional[FeaturesExtractor] = None,
-        extract_lhs: bool = True,
-        extract_sa: bool = True,
-    ) -> None:
-        if solver is None:
-            solver = GurobiPyomoSolver()
-        if extractor is None:
-            extractor = FeaturesExtractor(
-                with_sa=extract_sa,
-                with_lhs=extract_lhs,
-            )
-        assert isinstance(solver, InternalSolver)
-        self.components: Dict[str, Component] = {}
-        self.internal_solver: Optional[InternalSolver] = None
-        self.internal_solver_prototype: InternalSolver = solver
-        self.mode: str = mode
-        self.simulate_perfect: bool = simulate_perfect
-        self.solve_lp: bool = solve_lp
-        self.tee = False
-        self.use_lazy_cb: bool = use_lazy_cb
-        self.extractor = extractor
-        if components is not None:
-            for comp in components:
-                self._add_component(comp)
         else:
-            self._add_component(ObjectiveValueComponent())
+            h5_filename = NamedTemporaryFile().name
-            self._add_component(PrimalSolutionComponent(mode=mode))
+        stats = {}
-            self._add_component(DynamicLazyConstraintsComponent())
+        mode = "r+" if exists(h5_filename) else "w"
-            self._add_component(UserCutsComponent())
+        with H5File(h5_filename, mode) as h5:
-        assert self.mode in ["exact", "heuristic"]
+            model.extract_after_load(h5)
-
+            if not self.skip_lp:
-    def _solve(
+                relaxed = model.relax()
-        self,
+                relaxed.optimize()
-        instance: Instance,
+                relaxed.extract_after_lp(h5)
-        model: Any = None,
+            for comp in self.components:
-        discard_output: bool = False,
+                comp.before_mip(h5_filename, model, stats)
-        tee: bool = False,
+            model.optimize()
-    ) -> LearningSolveStats:
+            model.extract_after_mip(h5)
-
-        # Generate model
-        # -------------------------------------------------------
-        instance.load()
-        if model is None:
-            with _RedirectOutput([]):
-                model = instance.to_model()
-
-        # Initialize training sample
-        # -------------------------------------------------------
-        sample = instance.create_sample()
-
-        # Initialize stats
-        # -------------------------------------------------------
-        stats: LearningSolveStats = {}
-
-        # Initialize internal solver
-        # -------------------------------------------------------
-        self.tee = tee
-        self.internal_solver = self.internal_solver_prototype.clone()
-        assert self.internal_solver is not None
-        assert isinstance(self.internal_solver, InternalSolver)
-        self.internal_solver.set_instance(instance, model)
-
-        # Extract features (after-load)
-        # -------------------------------------------------------
-        logger.info("Extracting features (after-load)...")
-        initial_time = time.time()
-        self.extractor.extract_after_load_features(
-            instance, self.internal_solver, sample
-        )
-        logger.info(
-            "Features (after-load) extracted in %.2f seconds"
-            % (time.time() - initial_time)
-        )
-
-        callback_args = (
-            self,
-            instance,
-            model,
-            stats,
-            sample,
-        )
-
-        # Solve root LP relaxation
-        # -------------------------------------------------------
-        lp_stats = None
-        if self.solve_lp:
-            logger.debug("Running before_solve_lp callbacks...")
-            for component in self.components.values():
-                component.before_solve_lp(*callback_args)
-
-            logger.info("Solving root LP relaxation...")
-            lp_stats = self.internal_solver.solve_lp(tee=tee)
-            stats.update(cast(LearningSolveStats, lp_stats.__dict__))
-            assert lp_stats.lp_wallclock_time is not None
-            logger.info(
-                "LP relaxation solved in %.2f seconds" % lp_stats.lp_wallclock_time
-            )
-
-            logger.debug("Running after_solve_lp callbacks...")
-            for component in self.components.values():
-                component.after_solve_lp(*callback_args)
-
-            # Extract features (after-lp)
-            # -------------------------------------------------------
-            logger.info("Extracting features (after-lp)...")
-            initial_time = time.time()
-            self.extractor.extract_after_lp_features(
-                self.internal_solver, sample, lp_stats
-            )
-            logger.info(
-                "Features (after-lp) extracted in %.2f seconds"
-                % (time.time() - initial_time)
-            )
-
-        # Callback wrappers
-        # -------------------------------------------------------
-        def iteration_cb_wrapper() -> bool:
-            should_repeat = False
-            for comp in self.components.values():
-                if comp.iteration_cb(self, instance, model):
-                    should_repeat = True
-            return should_repeat
-
-        def lazy_cb_wrapper(
-            cb_solver: InternalSolver,
-            cb_model: Any,
-        ) -> None:
-            for comp in self.components.values():
-                comp.lazy_cb(self, instance, model)
-
-        def user_cut_cb_wrapper(
-            cb_solver: InternalSolver,
-            cb_model: Any,
-        ) -> None:
-            for comp in self.components.values():
-                comp.user_cut_cb(self, instance, model)
-
-        lazy_cb = None
-        if self.use_lazy_cb:
-            lazy_cb = lazy_cb_wrapper
-
-        user_cut_cb = None
-        if instance.has_user_cuts():
-            user_cut_cb = user_cut_cb_wrapper
-
-        # Before-solve callbacks
-        # -------------------------------------------------------
-        logger.debug("Running before_solve_mip callbacks...")
-        for component in self.components.values():
-            component.before_solve_mip(*callback_args)
-
-        # Solve MIP
-        # -------------------------------------------------------
-        logger.info("Solving MIP...")
-        mip_stats = self.internal_solver.solve(
-            tee=tee,
-            iteration_cb=iteration_cb_wrapper,
-            user_cut_cb=user_cut_cb,
-            lazy_cb=lazy_cb,
-        )
-        assert mip_stats.mip_wallclock_time is not None
-        logger.info("MIP solved in %.2f seconds" % mip_stats.mip_wallclock_time)
-        stats.update(cast(LearningSolveStats, mip_stats.__dict__))
-        stats["Solver"] = "default"
-        stats["Gap"] = self._compute_gap(
-            ub=mip_stats.mip_upper_bound,
-            lb=mip_stats.mip_lower_bound,
-        )
-        stats["Mode"] = self.mode
-
-        # Extract features (after-mip)
-        # -------------------------------------------------------
-        logger.info("Extracting features (after-mip)...")
-        initial_time = time.time()
-        for (k, v) in mip_stats.__dict__.items():
-            sample.put_scalar(k, v)
-        self.extractor.extract_after_mip_features(self.internal_solver, sample)
-        logger.info(
-            "Features (after-mip) extracted in %.2f seconds"
-            % (time.time() - initial_time)
-        )
-
-        # After-solve callbacks
-        # -------------------------------------------------------
-        logger.debug("Calling after_solve_mip callbacks...")
-        for component in self.components.values():
-            component.after_solve_mip(*callback_args)
-
-        # Flush
-        # -------------------------------------------------------
-        if not discard_output:
-            instance.flush()
 
         return stats
-
-    def solve(
-        self,
-        instance: Instance,
-        model: Any = None,
-        discard_output: bool = False,
-        tee: bool = False,
-    ) -> LearningSolveStats:
-        """
-        Solves the given instance. If trained machine-learning models are
-        available, they will be used to accelerate the solution process.
-
-        The argument `instance` may be either an Instance object or a
-        filename pointing to a pickled Instance object.
-
-        This method adds a new training sample to `instance.training_sample`.
-        If a filename is provided, then the file is modified in-place. That is,
-        the original file is overwritten.
-
-        If `solver.solve_lp_first` is False, the properties lp_solution and
-        lp_value will be set to dummy values.
-
-        Parameters
-        ----------
-        instance: Instance
-            The instance to be solved.
-        model: Any
-            The corresponding Pyomo model. If not provided, it will be created.
-        discard_output: bool
-            If True, do not write the modified instances anywhere; simply discard
-            them. Useful during benchmarking.
-        tee: bool
-            If true, prints solver log to screen.
-
-        Returns
-        -------
-        LearningSolveStats
-            A dictionary of solver statistics containing at least the following
-            keys: "Lower bound", "Upper bound", "Wallclock time", "Nodes",
-            "Sense", "Log", "Warm start value" and "LP value".
-
-            Additional components may generate additional keys. For example,
-            ObjectiveValueComponent adds the keys "Predicted LB" and
-            "Predicted UB". See the documentation of each component for more
-            details.
-        """
-        if self.simulate_perfect:
-            if not isinstance(instance, PickleGzInstance):
-                raise Exception("Not implemented")
-            self._solve(
-                instance=instance,
-                model=model,
-                tee=tee,
-            )
-            self.fit([instance])
-            instance.instance = None
-        return self._solve(
-            instance=instance,
-            model=model,
-            discard_output=discard_output,
-            tee=tee,
-        )
-
-    def parallel_solve(
-        self,
-        instances: List[Instance],
-        n_jobs: int = 4,
-        label: str = "Solve",
-        discard_outputs: bool = False,
-    ) -> List[LearningSolveStats]:
-        """
-        Solves multiple instances in parallel.
-
-        This method is equivalent to calling `solve` for each item on the list,
-        but it processes multiple instances at the same time. Like `solve`, this
-        method modifies each instance in place. Also like `solve`, a list of
-        filenames may be provided.
-
-        Parameters
-        ----------
-        discard_outputs: bool
-            If True, do not write the modified instances anywhere; simply discard
-            them instead. Useful during benchmarking.
-        label: str
-            Label to show in the progress bar.
-        instances: List[Instance]
-            The instances to be solved.
-        n_jobs: int
-            Number of instances to solve in parallel at a time.
-
-        Returns
-        -------
-        List[LearningSolveStats]
-            List of solver statistics, with one entry for each provided instance.
-            The list is the same you would obtain by calling
-            `[solver.solve(p) for p in instances]`
-        """
-        if n_jobs == 1:
-            return [self.solve(p) for p in instances]
-        else:
-            self.internal_solver = None
-            self._silence_miplearn_logger()
-            _GLOBAL[0].solver = self
-            _GLOBAL[0].instances = instances
-            _GLOBAL[0].discard_outputs = discard_outputs
-            results = p_map(
-                _parallel_solve,
-                list(range(len(instances))),
-                num_cpus=n_jobs,
-                desc=label,
-            )
-            results = [r for r in results if r[0]]
-            stats = []
-            for (idx, (s, instance)) in enumerate(results):
-                stats.append(s)
-                instances[idx] = instance
-            self._restore_miplearn_logger()
-            return stats
-
-    def fit(
-        self,
-        training_instances: List[Instance],
-        n_jobs: int = 1,
-    ) -> None:
-        if len(training_instances) == 0:
-            logger.warning("Empty list of training instances provided. Skipping.")
-            return
-        Component.fit_multiple(
-            list(self.components.values()),
-            training_instances,
-            n_jobs=n_jobs,
-        )
-
-    def _add_component(self, component: Component) -> None:
-        name = component.__class__.__name__
-        self.components[name] = component
-
-    def _silence_miplearn_logger(self) -> None:
-        miplearn_logger = logging.getLogger("miplearn")
-        self.prev_log_level = miplearn_logger.getEffectiveLevel()
-        miplearn_logger.setLevel(logging.WARNING)
-
-    def _restore_miplearn_logger(self) -> None:
-        miplearn_logger = logging.getLogger("miplearn")
-        miplearn_logger.setLevel(self.prev_log_level)
-
-    def __getstate__(self) -> Dict:
-        self.internal_solver = None
-        return self.__dict__
-
-    @staticmethod
-    def _compute_gap(ub: Optional[float], lb: Optional[float]) -> Optional[float]:
-        if lb is None or ub is None or lb * ub < 0:
-            # solver did not find a solution and/or bound
-            return None
-        elif abs(ub - lb) < 1e-6:
-            # avoid division by zero when ub = lb = 0
-            return 0.0
-        else:
-            # divide by max(abs(ub),abs(lb)) to ensure gap <= 1
-            return (ub - lb) / max(abs(ub), abs(lb))

miplearn/solvers/pyomo.py

@@ -0,0 +1,366 @@
+#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
+#  Copyright (C) 2020-2022, UChicago Argonne, LLC. All rights reserved.
+#  Released under the modified BSD license. See COPYING.md for more details.
+from numbers import Number
+from typing import Optional, Dict, List, Any
+
+import numpy as np
+import pyomo
+from pyomo.core import Objective, Var, Suffix
+from pyomo.core.base import _GeneralVarData
+from pyomo.core.expr.numeric_expr import SumExpression, MonomialTermExpression
+from scipy.sparse import coo_matrix
+
+from miplearn.h5 import H5File
+from miplearn.solvers.abstract import AbstractModel
+import pyomo.environ as pe
+
+
+class PyomoModel(AbstractModel):
+    def __init__(self, model: pe.ConcreteModel, solver_name: str = "gurobi_persistent"):
+        self.inner = model
+        self.solver_name = solver_name
+        self.solver = pe.SolverFactory(solver_name)
+        self.is_persistent = hasattr(self.solver, "set_instance")
+        if self.is_persistent:
+            self.solver.set_instance(model)
+        self.results = None
+        self._is_warm_start_available = False
+        if not hasattr(self.inner, "dual"):
+            self.inner.dual = Suffix(direction=Suffix.IMPORT)
+            self.inner.rc = Suffix(direction=Suffix.IMPORT)
+            self.inner.slack = Suffix(direction=Suffix.IMPORT)
+
+    def add_constrs(
+        self,
+        var_names: np.ndarray,
+        constrs_lhs: np.ndarray,
+        constrs_sense: np.ndarray,
+        constrs_rhs: np.ndarray,
+        stats: Optional[Dict] = None,
+    ) -> None:
+        variables = self._var_names_to_vars(var_names)
+        if not hasattr(self.inner, "added_eqs"):
+            self.inner.added_eqs = pe.ConstraintList()
+        for i in range(len(constrs_sense)):
+            lhs = sum([variables[j] * constrs_lhs[i, j] for j in range(len(variables))])
+            sense = constrs_sense[i]
+            rhs = constrs_rhs[i]
+            if sense == b"=":
+                eq = self.inner.added_eqs.add(lhs == rhs)
+            elif sense == b"<":
+                eq = self.inner.added_eqs.add(lhs <= rhs)
+            elif sense == b">":
+                eq = self.inner.added_eqs.add(lhs >= rhs)
+            else:
+                raise Exception(f"Unknown sense: {sense}")
+            self.solver.add_constraint(eq)
+
+    def _var_names_to_vars(self, var_names):
+        varname_to_var = {}
+        for var in self.inner.component_objects(Var):
+            for idx in var:
+                v = var[idx]
+                varname_to_var[v.name] = var[idx]
+        return [varname_to_var[var_name.decode()] for var_name in var_names]
+
+    def extract_after_load(self, h5: H5File) -> None:
+        self._extract_after_load_vars(h5)
+        self._extract_after_load_constrs(h5)
+        h5.put_scalar("static_sense", self._get_sense())
+
+    def extract_after_lp(self, h5: H5File) -> None:
+        self._extract_after_lp_vars(h5)
+        self._extract_after_lp_constrs(h5)
+        h5.put_scalar("lp_obj_value", self.results["Problem"][0]["Lower bound"])
+        h5.put_scalar("lp_wallclock_time", self._get_runtime())
+
+    def _get_runtime(self):
+        solver_dict = self.results["Solver"][0]
+        for key in ["Wallclock time", "User time"]:
+            if isinstance(solver_dict[key], Number):
+                return solver_dict[key]
+        raise Exception("Time unavailable")
+
+    def extract_after_mip(self, h5: H5File) -> None:
+        h5.put_scalar("mip_wallclock_time", self._get_runtime())
+        if self.results["Solver"][0]["Termination condition"] == "infeasible":
+            return
+        self._extract_after_mip_vars(h5)
+        self._extract_after_mip_constrs(h5)
+        if self._get_sense() == "max":
+            obj_value = self.results["Problem"][0]["Lower bound"]
+            obj_bound = self.results["Problem"][0]["Upper bound"]
+        else:
+            obj_value = self.results["Problem"][0]["Upper bound"]
+            obj_bound = self.results["Problem"][0]["Lower bound"]
+        h5.put_scalar("mip_obj_value", obj_value)
+        h5.put_scalar("mip_obj_bound", obj_bound)
+        h5.put_scalar("mip_gap", self._gap(obj_value, obj_bound))
+
+    def fix_variables(
+        self,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict] = None,
+    ) -> None:
+        variables = self._var_names_to_vars(var_names)
+        for (var, val) in zip(variables, var_values):
+            if np.isfinite(val):
+                var.fix(val)
+                self.solver.update_var(var)
+
+    def optimize(self) -> None:
+        if self.is_persistent:
+            self.results = self.solver.solve(
+                tee=True,
+                warmstart=self._is_warm_start_available,
+            )
+        else:
+            self.results = self.solver.solve(
+                self.inner,
+                tee=True,
+            )
+
+    def relax(self) -> "AbstractModel":
+        relaxed = self.inner.clone()
+        for var in relaxed.component_objects(Var):
+            for idx in var:
+                if var[idx].domain == pyomo.core.base.set_types.Binary:
+                    lb, ub = var[idx].bounds
+                    var[idx].setlb(lb)
+                    var[idx].setub(ub)
+                    var[idx].domain = pyomo.core.base.set_types.Reals
+        return PyomoModel(relaxed, self.solver_name)
+
+    def set_warm_starts(
+        self,
+        var_names: np.ndarray,
+        var_values: np.ndarray,
+        stats: Optional[Dict] = None,
+    ) -> None:
+        assert len(var_values.shape) == 2
+        (n_starts, n_vars) = var_values.shape
+        assert len(var_names.shape) == 1
+        assert var_names.shape[0] == n_vars
+        assert n_starts == 1, "Pyomo does not support multiple warm starts"
+        variables = self._var_names_to_vars(var_names)
+        for (var, val) in zip(variables, var_values[0, :]):
+            if np.isfinite(val):
+                var.value = val
+        self._is_warm_start_available = True
+
+    def _extract_after_load_vars(self, h5):
+        names: List[str] = []
+        types: List[str] = []
+        upper_bounds: List[float] = []
+        lower_bounds: List[float] = []
+        obj_coeffs: List[float] = []
+
+        obj = None
+        obj_offset = 0.0
+        obj_count = 0
+        for obj in self.inner.component_objects(Objective):
+            obj, obj_offset = self._parse_pyomo_expr(obj.expr)
+            obj_count += 1
+        assert obj_count == 1, f"One objective function expected; found {obj_count}"
+
+        for (i, var) in enumerate(self.inner.component_objects(pyomo.core.Var)):
+            for idx in var:
+                v = var[idx]
+
+                # Variable name
+                if idx is None:
+                    names.append(var.name)
+                else:
+                    names.append(var[idx].name)
+
+                # Variable type
+                if v.domain == pyomo.core.Binary:
+                    types.append("B")
+                elif v.domain in [
+                    pyomo.core.Reals,
+                    pyomo.core.NonNegativeReals,
+                    pyomo.core.NonPositiveReals,
+                    pyomo.core.NegativeReals,
+                    pyomo.core.PositiveReals,
+                ]:
+                    types.append("C")
+                else:
+                    raise Exception(f"unknown variable domain: {v.domain}")
+
+                # Variable upper/lower bounds
+                lb, ub = v.bounds
+                if lb is None:
+                    lb = -float("inf")
+                if ub is None:
+                    ub = float("Inf")
+                upper_bounds.append(float(ub))
+                lower_bounds.append(float(lb))
+
+                # Objective coefficients
+                if v.name in obj:
+                    obj_coeffs.append(obj[v.name])
+                else:
+                    obj_coeffs.append(0.0)
+
+        h5.put_array("static_var_names", np.array(names, dtype="S"))
+        h5.put_array("static_var_types", np.array(types, dtype="S"))
+        h5.put_array("static_var_lower_bounds", np.array(lower_bounds))
+        h5.put_array("static_var_upper_bounds", np.array(upper_bounds))
+        h5.put_array("static_var_obj_coeffs", np.array(obj_coeffs))
+        h5.put_scalar("static_obj_offset", obj_offset)
+
+    def _extract_after_load_constrs(self, h5):
+        names: List[str] = []
+        rhs: List[float] = []
+        senses: List[str] = []
+        lhs_row: List[int] = []
+        lhs_col: List[int] = []
+        lhs_data: List[float] = []
+
+        varname_to_idx = {}
+        for var in self.inner.component_objects(Var):
+            for idx in var:
+                varname = var.name
+                if idx is not None:
+                    varname = var[idx].name
+                varname_to_idx[varname] = len(varname_to_idx)
+
+        def _parse_constraint(c: pe.Constraint, row: int) -> None:
+            # Extract RHS and sense
+            has_ub = c.has_ub()
+            has_lb = c.has_lb()
+            assert (
+                (not has_lb) or (not has_ub) or c.upper() == c.lower()
+            ), "range constraints not supported"
+            if not has_ub:
+                senses.append(">")
+                rhs.append(float(c.lower()))
+            elif not has_lb:
+                senses.append("<")
+                rhs.append(float(c.upper()))
+            else:
+                senses.append("=")
+                rhs.append(float(c.upper()))
+
+            # Extract LHS
+            expr = c.body
+            if isinstance(expr, SumExpression):
+                for term in expr._args_:
+                    if isinstance(term, MonomialTermExpression):
+                        lhs_row.append(row)
+                        lhs_col.append(varname_to_idx[term._args_[1].name])
+                        lhs_data.append(float(term._args_[0]))
+                    elif isinstance(term, _GeneralVarData):
+                        lhs_row.append(row)
+                        lhs_col.append(varname_to_idx[term.name])
+                        lhs_data.append(1.0)
+                    else:
+                        raise Exception(f"Unknown term type: {term.__class__.__name__}")
+            elif isinstance(expr, _GeneralVarData):
+                lhs_row.append(row)
+                lhs_col.append(varname_to_idx[expr.name])
+                lhs_data.append(1.0)
+            else:
+                raise Exception(f"Unknown expression type: {expr.__class__.__name__}")
+
+        curr_row = 0
+        for (i, constr) in enumerate(
+            self.inner.component_objects(pyomo.core.Constraint)
+        ):
+            if len(constr) > 0:
+                for idx in constr:
+                    names.append(constr[idx].name)
+                    _parse_constraint(constr[idx], curr_row)
+                    curr_row += 1
+            else:
+                names.append(constr.name)
+                _parse_constraint(constr, curr_row)
+                curr_row += 1
+
+        lhs = coo_matrix((lhs_data, (lhs_row, lhs_col))).tocoo()
+        h5.put_sparse("static_constr_lhs", lhs)
+        h5.put_array("static_constr_names", np.array(names, dtype="S"))
+        h5.put_array("static_constr_rhs", np.array(rhs))
+        h5.put_array("static_constr_sense", np.array(senses, dtype="S"))
+
+    def _extract_after_lp_vars(self, h5):
+        rc = []
+        values = []
+        for var in self.inner.component_objects(Var):
+            for idx in var:
+                v = var[idx]
+                rc.append(self.inner.rc[v])
+                values.append(v.value)
+        h5.put_array("lp_var_reduced_costs", np.array(rc))
+        h5.put_array("lp_var_values", np.array(values))
+
+    def _extract_after_lp_constrs(self, h5):
+        dual = []
+        slacks = []
+        for constr in self.inner.component_objects(pyomo.core.Constraint):
+            for idx in constr:
+                c = constr[idx]
+                dual.append(self.inner.dual[c])
+                slacks.append(abs(self.inner.slack[c]))
+        h5.put_array("lp_constr_dual_values", np.array(dual))
+        h5.put_array("lp_constr_slacks", np.array(slacks))
+
+    def _extract_after_mip_vars(self, h5):
+        values = []
+        for var in self.inner.component_objects(Var):
+            for idx in var:
+                v = var[idx]
+                values.append(v.value)
+        h5.put_array("mip_var_values", np.array(values))
+
+    def _extract_after_mip_constrs(self, h5):
+        slacks = []
+        for constr in self.inner.component_objects(pyomo.core.Constraint):
+            for idx in constr:
+                c = constr[idx]
+                slacks.append(abs(self.inner.slack[c]))
+        h5.put_array("mip_constr_slacks", np.array(slacks))
+
+    def _parse_pyomo_expr(self, expr: Any):
+        lhs = {}
+        offset = 0.0
+        if isinstance(expr, SumExpression):
+            for term in expr._args_:
+                if isinstance(term, MonomialTermExpression):
+                    lhs[term._args_[1].name] = float(term._args_[0])
+                elif isinstance(term, _GeneralVarData):
+                    lhs[term.name] = 1.0
+                elif isinstance(term, Number):
+                    offset += term
+                else:
+                    raise Exception(f"Unknown term type: {term.__class__.__name__}")
+        elif isinstance(expr, _GeneralVarData):
+            lhs[expr.name] = 1.0
+        else:
+            raise Exception(f"Unknown expression type: {expr.__class__.__name__}")
+        return lhs, offset
+
+    def _gap(self, zp, zd, tol=1e-6):
+        # Reference: https://www.gurobi.com/documentation/9.5/refman/mipgap2.html
+        if abs(zp) < tol:
+            if abs(zd) < tol:
+                return 0
+            else:
+                return float("inf")
+        else:
+            return abs(zp - zd) / abs(zp)
+
+    def _get_sense(self):
+        for obj in self.inner.component_objects(Objective):
+            sense = obj.sense
+            if sense == pyomo.core.kernel.objective.minimize:
+                return "min"
+            elif sense == pyomo.core.kernel.objective.maximize:
+                return "max"
+            else:
+                raise Exception(f"Unknown sense: ${sense}")
+
+    def write(self, filename: str) -> None:
+        self.inner.write(filename, io_options={"symbolic_solver_labels": True})

miplearn/solvers/pyomo/__init__.py

@@ -1,3 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.

miplearn/solvers/pyomo/base.py

@@ -1,648 +0,0 @@
-#  MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
-#  Copyright (C) 2020-2021, UChicago Argonne, LLC. All rights reserved.
-#  Released under the modified BSD license. See COPYING.md for more details.
-
-import logging
-import re
-import sys
-from io import StringIO
-from typing import Any, List, Dict, Optional, Tuple
-
-import numpy as np
-import pyomo
-from overrides import overrides
-from pyomo import environ as pe
-from pyomo.core import Var, Suffix, Objective
-from pyomo.core.base import _GeneralVarData
-from pyomo.core.base.constraint import ConstraintList
-from pyomo.core.expr.numeric_expr import SumExpression, MonomialTermExpression
-from pyomo.opt import TerminationCondition
-from pyomo.opt.base.solvers import SolverFactory
-
-from miplearn.instance.base import Instance
-from miplearn.solvers import _RedirectOutput, _none_if_empty
-from miplearn.solvers.internal import (
-    InternalSolver,
-    LPSolveStats,
-    IterationCallback,
-    LazyCallback,
-    MIPSolveStats,
-    Variables,
-    Constraints,
-)
-from miplearn.types import (
-    SolverParams,
-    UserCutCallback,
-    Solution,
-)
-
-logger = logging.getLogger(__name__)
-
-
-class BasePyomoSolver(InternalSolver):
-    """
-    Base class for all Pyomo solvers.
-    """
-
-    def __init__(
-        self,
-        solver_factory: SolverFactory,
-        params: SolverParams,
-    ) -> None:
-        self.instance: Optional[Instance] = None
-        self.model: Optional[pe.ConcreteModel] = None
-        self.params = params
-        self._all_vars: List[pe.Var] = []
-        self._bin_vars: List[pe.Var] = []
-        self._is_warm_start_available: bool = False
-        self._pyomo_solver: SolverFactory = solver_factory
-        self._obj_sense: str = "min"
-        self._varname_to_var: Dict[bytes, pe.Var] = {}
-        self._cname_to_constr: Dict[str, pe.Constraint] = {}
-        self._termination_condition: str = ""
-        self._has_lp_solution = False
-        self._has_mip_solution = False
-        self._obj: Dict[str, float] = {}
-
-        for (key, value) in params.items():
-            self._pyomo_solver.options[key] = value
-
-    def add_constraint(
-        self,
-        constr: Any,
-    ) -> None:
-        assert self.model is not None
-        self._pyomo_solver.add_constraint(constr)
-        self._termination_condition = ""
-        self._has_lp_solution = False
-        self._has_mip_solution = False
-
-    @overrides
-    def add_constraints(self, cf: Constraints) -> None:
-        assert cf.names is not None
-        assert cf.senses is not None
-        assert cf.lhs is not None
-        assert cf.rhs is not None
-        assert self.model is not None
-        for (i, name) in enumerate(cf.names):
-            lhs = 0.0
-            for (varname, coeff) in cf.lhs[i]:
-                var = self._varname_to_var[varname]
-                lhs += var * coeff
-            if cf.senses[i] == b"=":
-                expr = lhs == cf.rhs[i]
-            elif cf.senses[i] == b"<":
-                expr = lhs <= cf.rhs[i]
-            elif cf.senses[i] == b">":
-                expr = lhs >= cf.rhs[i]
-            else:
-                raise Exception(f"Unknown sense: {cf.senses[i]}")
-            cl = pe.Constraint(expr=expr, name=name)
-            self.model.add_component(name.decode(), cl)
-            self._pyomo_solver.add_constraint(cl)
-            self._cname_to_constr[name] = cl
-        self._termination_condition = ""
-        self._has_lp_solution = False
-        self._has_mip_solution = False
-
-    @overrides
-    def are_callbacks_supported(self) -> bool:
-        return False
-
-    @overrides
-    def are_constraints_satisfied(
-        self,
-        cf: Constraints,
-        tol: float = 1e-5,
-    ) -> List[bool]:
-        assert cf.names is not None
-        assert cf.lhs is not None
-        assert cf.rhs is not None
-        assert cf.senses is not None
-        result = []
-        for (i, name) in enumerate(cf.names):
-            lhs = 0.0
-            for (varname, coeff) in cf.lhs[i]:
-                var = self._varname_to_var[varname]
-                lhs += var.value * coeff
-            if cf.senses[i] == "<":
-                result.append(lhs <= cf.rhs[i] + tol)
-            elif cf.senses[i] == ">":
-                result.append(lhs >= cf.rhs[i] - tol)
-            else:
-                result.append(abs(cf.rhs[i] - lhs) < tol)
-        return result
-
-    @overrides
-    def build_test_instance_infeasible(self) -> Instance:
-        return PyomoTestInstanceInfeasible()
-
-    @overrides
-    def build_test_instance_knapsack(self) -> Instance:
-        return PyomoTestInstanceKnapsack(
-            weights=[23.0, 26.0, 20.0, 18.0],
-            prices=[505.0, 352.0, 458.0, 220.0],
-            capacity=67.0,
-        )
-
-    @overrides
-    def fix(self, solution: Solution) -> None:
-        for (varname, value) in solution.items():
-            if value is None:
-                continue
-            var = self._varname_to_var[varname]
-            var.fix(value)
-            self._pyomo_solver.update_var(var)
-
-    @overrides
-    def get_constraints(
-        self,
-        with_static: bool = True,
-        with_sa: bool = True,
-        with_lhs: bool = True,
-    ) -> Constraints:
-        model = self.model
-        assert model is not None
-
-        names: List[str] = []
-        rhs: List[float] = []
-        lhs: List[List[Tuple[bytes, float]]] = []
-        senses: List[str] = []
-        dual_values: List[float] = []
-        slacks: List[float] = []
-
-        def _parse_constraint(c: pe.Constraint) -> None:
-            assert model is not None
-            if with_static:
-                # Extract RHS and sense
-                has_ub = c.has_ub()
-                has_lb = c.has_lb()
-                assert (
-                    (not has_lb) or (not has_ub) or c.upper() == c.lower()
-                ), "range constraints not supported"
-                if not has_ub:
-                    senses.append(">")
-                    rhs.append(float(c.lower()))
-                elif not has_lb:
-                    senses.append("<")
-                    rhs.append(float(c.upper()))
-                else:
-                    senses.append("=")
-                    rhs.append(float(c.upper()))
-
-                if with_lhs:
-                    # Extract LHS
-                    lhsc = []
-                    expr = c.body
-                    if isinstance(expr, SumExpression):
-                        for term in expr._args_:
-                            if isinstance(term, MonomialTermExpression):
-                                lhsc.append(
-                                    (
-                                        term._args_[1].name.encode(),
-                                        float(term._args_[0]),
-                                    )
-                                )
-                            elif isinstance(term, _GeneralVarData):
-                                lhsc.append((term.name.encode(), 1.0))
-                            else:
-                                raise Exception(
-                                    f"Unknown term type: {term.__class__.__name__}"
-                                )
-                    elif isinstance(expr, _GeneralVarData):
-                        lhsc.append((expr.name.encode(), 1.0))
-                    else:
-                        raise Exception(
-                            f"Unknown expression type: {expr.__class__.__name__}"
-                        )
-                    lhs.append(lhsc)
-
-            # Extract dual values
-            if self._has_lp_solution:
-                dual_values.append(model.dual[c])
-
-            # Extract slacks
-            if self._has_mip_solution or self._has_lp_solution:
-                slacks.append(model.slack[c])
-
-        for constr in model.component_objects(pyomo.core.Constraint):
-            if isinstance(constr, pe.ConstraintList):
-                for idx in constr:
-                    names.append(f"{constr.name}[{idx}]")
-                    _parse_constraint(constr[idx])
-            else:
-                names.append(constr.name)
-                _parse_constraint(constr)
-
-        return Constraints(
-            names=_none_if_empty(np.array(names, dtype="S")),
-            rhs=_none_if_empty(np.array(rhs, dtype=float)),
-            senses=_none_if_empty(np.array(senses, dtype="S")),
-            lhs=_none_if_empty(lhs),
-            slacks=_none_if_empty(np.array(slacks, dtype=float)),
-            dual_values=_none_if_empty(np.array(dual_values, dtype=float)),
-        )
-
-    @overrides
-    def get_constraint_attrs(self) -> List[str]:
-        return [
-            "dual_values",
-            "lhs",
-            "names",
-            "rhs",
-            "senses",
-            "slacks",
-        ]
-
-    @overrides
-    def get_solution(self) -> Optional[Solution]:
-        assert self.model is not None
-        if self.is_infeasible():
-            return None
-        solution: Solution = {}
-        for var in self.model.component_objects(Var):
-            for index in var:
-                if var[index].fixed:
-                    continue
-                solution[f"{var}[{index}]".encode()] = var[index].value
-        return solution
-
-    @overrides
-    def get_variables(
-        self,
-        with_static: bool = True,
-        with_sa: bool = True,
-    ) -> Variables:
-        assert self.model is not None
-
-        names: List[str] = []
-        types: List[str] = []
-        upper_bounds: List[float] = []
-        lower_bounds: List[float] = []
-        obj_coeffs: List[float] = []
-        reduced_costs: List[float] = []
-        values: List[float] = []
-
-        for (i, var) in enumerate(self.model.component_objects(pyomo.core.Var)):
-            for idx in var:
-                v = var[idx]
-
-                # Variable name
-                if idx is None:
-                    names.append(str(var))
-                else:
-                    names.append(f"{var}[{idx}]")
-
-                if with_static:
-                    # Variable type
-                    if v.domain == pyomo.core.Binary:
-                        types.append("B")
-                    elif v.domain in [
-                        pyomo.core.Reals,
-                        pyomo.core.NonNegativeReals,
-                        pyomo.core.NonPositiveReals,
-                        pyomo.core.NegativeReals,
-                        pyomo.core.PositiveReals,
-                    ]:
-                        types.append("C")
-                    else:
-                        raise Exception(f"unknown variable domain: {v.domain}")
-
-                    # Bounds
-                    lb, ub = v.bounds
-                    upper_bounds.append(float(ub))
-                    lower_bounds.append(float(lb))
-
-                    # Objective coefficient
-                    if v.name in self._obj:
-                        obj_coeffs.append(self._obj[v.name])
-                    else:
-                        obj_coeffs.append(0.0)
-
-                # Reduced costs
-                if self._has_lp_solution:
-                    reduced_costs.append(self.model.rc[v])
-
-                # Values
-                if self._has_lp_solution or self._has_mip_solution:
-                    values.append(v.value)
-
-        return Variables(
-            names=_none_if_empty(np.array(names, dtype="S")),
-            types=_none_if_empty(np.array(types, dtype="S")),
-            upper_bounds=_none_if_empty(np.array(upper_bounds, dtype=float)),
-            lower_bounds=_none_if_empty(np.array(lower_bounds, dtype=float)),
-            obj_coeffs=_none_if_empty(np.array(obj_coeffs, dtype=float)),
-            reduced_costs=_none_if_empty(np.array(reduced_costs, dtype=float)),
-            values=_none_if_empty(np.array(values, dtype=float)),
-        )
-
-    @overrides
-    def get_variable_attrs(self) -> List[str]:
-        return [
-            "names",
-            # "basis_status",
-            "categories",
-            "lower_bounds",
-            "obj_coeffs",
-            "reduced_costs",
-            # "sa_lb_down",
-            # "sa_lb_up",
-            # "sa_obj_down",
-            # "sa_obj_up",
-            # "sa_ub_down",
-            # "sa_ub_up",
-            "types",
-            "upper_bounds",
-            "user_features",
-            "values",
-        ]
-
-    @overrides
-    def is_infeasible(self) -> bool:
-        return self._termination_condition == TerminationCondition.infeasible
-
-    @overrides
-    def remove_constraints(self, names: List[str]) -> None:
-        assert self.model is not None
-        for name in names:
-            constr = self._cname_to_constr[name]
-            del self._cname_to_constr[name]
-            self.model.del_component(constr)
-            self._pyomo_solver.remove_constraint(constr)
-
-    @overrides
-    def set_instance(
-        self,
-        instance: Instance,
-        model: Any = None,
-    ) -> None:
-        if model is None:
-            model = instance.to_model()
-        assert isinstance(model, pe.ConcreteModel)
-        self.instance = instance
-        self.model = model
-        self.model.extra_constraints = ConstraintList()
-        self.model.dual = Suffix(direction=Suffix.IMPORT)
-        self.model.rc = Suffix(direction=Suffix.IMPORT)
-        self.model.slack = Suffix(direction=Suffix.IMPORT)
-        self._pyomo_solver.set_instance(model)
-        self._update_obj()
-        self._update_vars()
-        self._update_constrs()
-
-    @overrides
-    def set_warm_start(self, solution: Solution) -> None:
-        self._clear_warm_start()
-        count_fixed = 0
-        for (var_name, value) in solution.items():
-            if value is None:
-                continue
-            var = self._varname_to_var[var_name]
-            var.value = solution[var_name]
-            count_fixed += 1
-        if count_fixed > 0:
-            self._is_warm_start_available = True
-
-    @overrides
-    def solve(
-        self,
-        tee: bool = False,
-        iteration_cb: Optional[IterationCallback] = None,
-        lazy_cb: Optional[LazyCallback] = None,
-        user_cut_cb: Optional[UserCutCallback] = None,
-    ) -> MIPSolveStats:
-        assert lazy_cb is None, "callbacks are not currently supported"
-        assert user_cut_cb is None, "callbacks are not currently supported"
-        total_wallclock_time = 0
-        streams: List[Any] = [StringIO()]
-        if tee:
-            streams += [sys.stdout]
-        if iteration_cb is None:
-            iteration_cb = lambda: False
-        while True:
-            logger.debug("Solving MIP...")
-            with _RedirectOutput(streams):
-                results = self._pyomo_solver.solve(
-                    tee=True,
-                    warmstart=self._is_warm_start_available,
-                )
-            total_wallclock_time += results["Solver"][0]["Wallclock time"]
-            should_repeat = iteration_cb()
-            if not should_repeat:
-                break
-        log = streams[0].getvalue()
-        node_count = self._extract_node_count(log)
-        ws_value = self._extract_warm_start_value(log)
-        self._termination_condition = results["Solver"][0]["Termination condition"]
-        lb, ub = None, None
-        self._has_mip_solution = False
-        self._has_lp_solution = False
-        if not self.is_infeasible():
-            self._has_mip_solution = True
-            lb = results["Problem"][0]["Lower bound"]
-            ub = results["Problem"][0]["Upper bound"]
-        return MIPSolveStats(
-            mip_lower_bound=lb,
-            mip_upper_bound=ub,
-            mip_wallclock_time=total_wallclock_time,
-            mip_sense=self._obj_sense,
-            mip_log=log,
-            mip_nodes=node_count,
-            mip_warm_start_value=ws_value,
-        )
-
-    @overrides
-    def solve_lp(
-        self,
-        tee: bool = False,
-    ) -> LPSolveStats:
-        self._relax()
-        streams: List[Any] = [StringIO()]
-        if tee:
-            streams += [sys.stdout]
-        with _RedirectOutput(streams):
-            results = self._pyomo_solver.solve(tee=True)
-        self._termination_condition = results["Solver"][0]["Termination condition"]
-        self._restore_integrality()
-        opt_value = None
-        self._has_lp_solution = False
-        self._has_mip_solution = False
-        if not self.is_infeasible():
-            opt_value = results["Problem"][0]["Lower bound"]
-            self._has_lp_solution = True
-        return LPSolveStats(
-            lp_value=opt_value,
-            lp_log=streams[0].getvalue(),
-            lp_wallclock_time=results["Solver"][0]["Wallclock time"],
-        )
-
-    def _clear_warm_start(self) -> None:
-        for var in self._all_vars:
-            if not var.fixed:
-                var.value = None
-        self._is_warm_start_available = False
-
-    @staticmethod
-    def _extract(
-        log: str,
-        regexp: Optional[str],
-        default: Optional[str] = None,
-    ) -> Optional[str]:
-        if regexp is None:
-            return default
-        value = default
-        for line in log.splitlines():
-            matches = re.findall(regexp, line)
-            if len(matches) == 0:
-                continue
-            value = matches[0]
-        return value
-
-    def _extract_node_count(self, log: str) -> Optional[int]:
-        value = self._extract(log, self._get_node_count_regexp())
-        if value is None:
-            return None
-        return int(value)
-
-    def _extract_warm_start_value(self, log: str) -> Optional[float]:
-        value = self._extract(log, self._get_warm_start_regexp())
-        if value is None:
-            return None
-        return float(value)
-
-    def _get_node_count_regexp(self) -> Optional[str]:
-        return None
-
-    def _get_warm_start_regexp(self) -> Optional[str]:
-        return None
-
-    def _parse_pyomo_expr(self, expr: Any) -> Dict[str, float]:
-        lhs = {}
-        if isinstance(expr, SumExpression):
-            for term in expr._args_:
-                if isinstance(term, MonomialTermExpression):
-                    lhs[term._args_[1].name] = float(term._args_[0])
-                elif isinstance(term, _GeneralVarData):
-                    lhs[term.name] = 1.0
-                else:
-                    raise Exception(f"Unknown term type: {term.__class__.__name__}")
-        elif isinstance(expr, _GeneralVarData):
-            lhs[expr.name] = 1.0
-        else:
-            raise Exception(f"Unknown expression type: {expr.__class__.__name__}")
-        return lhs
-
-    def _relax(self) -> None:
-        for var in self._bin_vars:
-            lb, ub = var.bounds
-            var.setlb(lb)
-            var.setub(ub)
-            var.domain = pyomo.core.base.set_types.Reals
-            self._pyomo_solver.update_var(var)
-
-    def _restore_integrality(self) -> None:
-        for var in self._bin_vars:
-            var.domain = pyomo.core.base.set_types.Binary
-            self._pyomo_solver.update_var(var)
-
-    def _update_obj(self) -> None:
-        self._obj_sense = "max"
-        if self._pyomo_solver._objective.sense == pyomo.core.kernel.objective.minimize:
-            self._obj_sense = "min"
-
-    def _update_vars(self) -> None:
-        assert self.model is not None
-        self._all_vars = []
-        self._bin_vars = []
-        self._varname_to_var = {}
-        for var in self.model.component_objects(Var):
-            for idx in var:
-                varname = f"{var.name}[{idx}]".encode()
-                if idx is None:
-                    varname = var.name.encode()
-                self._varname_to_var[varname] = var[idx]
-                self._all_vars += [var[idx]]
-                if var[idx].domain == pyomo.core.base.set_types.Binary:
-                    self._bin_vars += [var[idx]]
-        for obj in self.model.component_objects(Objective):
-            self._obj = self._parse_pyomo_expr(obj.expr)
-            break
-
-    def _update_constrs(self) -> None:
-        assert self.model is not None
-        self._cname_to_constr.clear()
-        for constr in self.model.component_objects(pyomo.core.Constraint):
-            if isinstance(constr, pe.ConstraintList):
-                for idx in constr:
-                    self._cname_to_constr[f"{constr.name}[{idx}]"] = constr[idx]
-            else:
-                self._cname_to_constr[constr.name] = constr
-
-
-class PyomoTestInstanceInfeasible(Instance):
-    @overrides
-    def to_model(self) -> pe.ConcreteModel:
-        model = pe.ConcreteModel()
-        model.x = pe.Var([0], domain=pe.Binary)
-        model.OBJ = pe.Objective(expr=model.x[0], sense=pe.maximize)
-        model.eq = pe.Constraint(expr=model.x[0] >= 2)
-        return model
-
-
-class PyomoTestInstanceKnapsack(Instance):
-    """
-    Simpler (one-dimensional) Knapsack Problem, used for testing.
-    """
-
-    def __init__(
-        self,
-        weights: List[float],
-        prices: List[float],
-        capacity: float,
-    ) -> None:
-        super().__init__()
-        self.weights = weights
-        self.prices = prices
-        self.capacity = capacity
-        self.n = len(weights)
-
-    @overrides
-    def to_model(self) -> pe.ConcreteModel:
-        model = pe.ConcreteModel()
-        items = range(len(self.weights))
-        model.x = pe.Var(items, domain=pe.Binary)
-        model.z = pe.Var(domain=pe.Reals, bounds=(0, self.capacity))
-        model.OBJ = pe.Objective(
-            expr=sum(model.x[v] * self.prices[v] for v in items),
-            sense=pe.maximize,
-        )
-        model.eq_capacity = pe.Constraint(
-            expr=sum(model.x[v] * self.weights[v] for v in items) == model.z
-        )
-        return model
-
-    @overrides
-    def get_instance_features(self) -> np.ndarray:
-        return np.array(
-            [
-                self.capacity,
-                np.average(self.weights),
-            ]
-        )
-
-    @overrides
-    def get_variable_features(self, names: np.ndarray) -> np.ndarray:
-        return np.vstack(
-            [
-                [[self.weights[i], self.prices[i]] for i in range(self.n)],
-                [0.0, 0.0],
-            ]
-        )
-
-    @overrides
-    def get_variable_categories(self, names: np.ndarray) -> np.ndarray:
-        return np.array(
-            ["default" if n.decode().startswith("x") else "" for n in names],
-            dtype="S",
-        )