ANL-CEEESA/MIPLearn
1
0
Fork 0
mirror of https://github.com/ANL-CEEESA/MIPLearn.git synced 2025-12-06 09:28:51 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'feature/refactoring'

Browse Source
This commit is contained in:
Alinson S. Xavier
2020-05-05 13:09:18 -05:00
parent 84351b3ac3 63816319ed
commit abbd2d2207
25 changed files with 448 additions and 365 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/customization/index.html
View File

@@ -142,6 +142,8 @@
<li class="second-level"><a href="#selecting-solver-components">Selecting solver components</a></li>
<li class="second-level"><a href="#adjusting-component-aggresiveness">Adjusting component aggresiveness</a></li>
</ul>
</div></div>
<div class="col-md-9" role="main">
@@ -183,6 +185,19 @@ solver = LearningSolver()
# Replace the default LazyConstraintComponent by one with custom parameters
solver.add(LazyConstraintComponent(...))
</code></pre>
<h2 id="adjusting-component-aggresiveness">Adjusting component aggresiveness</h2>
<p>The aggressiveness of classification components (such as <code>PrimalSolutionComponent</code> and <code>LazyConstraintComponent</code>) can
be adjusted through the <code>threshold</code> constructor argument. Internally, these components ask the ML models how confident
they are on each prediction (through the <code>predict_proba</code> method in the sklearn API), and only take into account
predictions which have probabilities above the threshold. Lowering a component's threshold increases its aggresiveness,
while raising a component's threshold makes it more conservative. </p>
<p>MIPLearn also includes <code>MinPrecisionThreshold</code>, a dynamic threshold which adjusts itself automatically during training
to achieve a minimum desired true positive rate (also known as precision). The example below shows how to initialize
a <code>PrimalSolutionComponent</code> which achieves 95% precision, possibly at the cost of a lower recall. To make the component
more aggressive, this precision may be lowered.</p>
<pre><code class="python">comp = PrimalSolutionComponent(threshold=MinPrecisionThreshold(0.98))
</code></pre></div>

2
docs/index.html
View File

@@ -273,5 +273,5 @@
<!--
MkDocs version : 1.1
Build Date UTC : 2020-04-14 13:54:25
Build Date UTC : 2020-05-05 18:08:32
-->

2
docs/search/search_index.json
View File

File diff suppressed because one or more lines are too long

12
docs/sitemap.xml
View File

@@ -1,27 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"><url>
<loc>None</loc>
<lastmod>2020-04-14</lastmod>
<lastmod>2020-05-05</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>None</loc>
<lastmod>2020-04-14</lastmod>
<lastmod>2020-05-05</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>None</loc>
<lastmod>2020-04-14</lastmod>
<lastmod>2020-05-05</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>None</loc>
<lastmod>2020-04-14</lastmod>
<lastmod>2020-05-05</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>None</loc>
<lastmod>2020-04-14</lastmod>
<lastmod>2020-05-05</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>None</loc>
<lastmod>2020-04-14</lastmod>
<lastmod>2020-05-05</lastmod>
<changefreq>daily</changefreq>
</url>
</urlset>

BIN
docs/sitemap.xml.gz
View File

Binary file not shown.

17
src/docs/customization.md
View File

@@ -44,3 +44,20 @@ solver = LearningSolver()
# Replace the default LazyConstraintComponent by one with custom parameters
solver.add(LazyConstraintComponent(...))
```
## Adjusting component aggresiveness
The aggressiveness of classification components (such as `PrimalSolutionComponent` and `LazyConstraintComponent`) can
be adjusted through the `threshold` constructor argument. Internally, these components ask the ML models how confident
they are on each prediction (through the `predict_proba` method in the sklearn API), and only take into account
predictions which have probabilities above the threshold. Lowering a component's threshold increases its aggresiveness,
while raising a component's threshold makes it more conservative.
MIPLearn also includes `MinPrecisionThreshold`, a dynamic threshold which adjusts itself automatically during training
to achieve a minimum desired true positive rate (also known as precision). The example below shows how to initialize
a `PrimalSolutionComponent` which achieves 95% precision, possibly at the cost of a lower recall. To make the component
more aggressive, this precision may be lowered.
```python
comp = PrimalSolutionComponent(threshold=MinPrecisionThreshold(0.98))
```

2
src/python/miplearn/__init__.py
View File

@@ -11,7 +11,7 @@ from .components.component import Component
from .components.objective import ObjectiveValueComponent
from .components.lazy import LazyConstraintsComponent
from .components.primal import PrimalSolutionComponent
from .components.branching import BranchPriorityComponent
from .components.branching import BranchPriorityComponent, BranchPriorityExtractor
from .classifiers.adaptive import AdaptiveClassifier

134
src/python/miplearn/classifiers/adaptive.py
View File

@@ -5,106 +5,62 @@
import logging
from copy import deepcopy
import numpy as np
from miplearn.classifiers import Classifier
from sklearn.model_selection import cross_val_score
from miplearn.classifiers.counting import CountingClassifier
from miplearn.classifiers.evaluator import ClassifierEvaluator
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
logger = logging.getLogger(__name__)
class AdaptiveClassifier(Classifier):
"""
A classifier that automatically switches strategies based on the number of
samples and cross-validation scores.
A meta-classifier which dynamically selects what actual classifier to use
based on its cross-validation score on a particular training data set.
"""
def __init__(self,
predictor=None,
min_samples_predict=1,
min_samples_cv=100,
thr_fix=0.999,
thr_alpha=0.50,
thr_balance=0.95,
):
self.min_samples_predict = min_samples_predict
self.min_samples_cv = min_samples_cv
self.thr_fix = thr_fix
self.thr_alpha = thr_alpha
self.thr_balance = thr_balance
self.predictor_factory = predictor
self.predictor = None
candidates=None,
evaluator=ClassifierEvaluator()):
"""
Initializes the meta-classifier.
"""
if candidates is None:
candidates = {
"knn(100)": {
"classifier": KNeighborsClassifier(n_neighbors=100),
"min samples": 100,
},
"logistic": {
"classifier": make_pipeline(StandardScaler(),
LogisticRegression()),
"min samples": 30,
},
"counting": {
"classifier": CountingClassifier(),
"min samples": 0,
}
}
self.candidates = candidates
self.evaluator = evaluator
self.classifier = None
def fit(self, x_train, y_train):
best_name, best_clf, best_score = None, None, -float("inf")
n_samples = x_train.shape[0]
# If number of samples is too small, don't predict anything.
if n_samples < self.min_samples_predict:
logger.debug(" Too few samples (%d); always predicting false" % n_samples)
self.predictor = 0
return
# If vast majority of observations are false, always return false.
y_train_avg = np.average(y_train)
if y_train_avg <= 1.0 - self.thr_fix:
logger.debug(" Most samples are negative (%.3f); always returning false" % y_train_avg)
self.predictor = 0
return
# If vast majority of observations are true, always return true.
if y_train_avg >= self.thr_fix:
logger.debug(" Most samples are positive (%.3f); always returning true" % y_train_avg)
self.predictor = 1
return
# If classes are too unbalanced, don't predict anything.
if y_train_avg < (1 - self.thr_balance) or y_train_avg > self.thr_balance:
logger.debug(" Classes are too unbalanced (%.3f); always returning false" % y_train_avg)
self.predictor = 0
return
# Select ML model if none is provided
if self.predictor_factory is None:
if n_samples < 30:
from sklearn.neighbors import KNeighborsClassifier
self.predictor_factory = KNeighborsClassifier(n_neighbors=n_samples)
else:
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
self.predictor_factory = make_pipeline(StandardScaler(), LogisticRegression())
# Create predictor
if callable(self.predictor_factory):
pred = self.predictor_factory()
else:
pred = deepcopy(self.predictor_factory)
# Skip cross-validation if number of samples is too small
if n_samples < self.min_samples_cv:
logger.debug(" Too few samples (%d); skipping cross validation" % n_samples)
self.predictor = pred
self.predictor.fit(x_train, y_train)
return
# Calculate cross-validation score
cv_score = np.mean(cross_val_score(pred, x_train, y_train, cv=5))
dummy_score = max(y_train_avg, 1 - y_train_avg)
cv_thr = 1. * self.thr_alpha + dummy_score * (1 - self.thr_alpha)
# If cross-validation score is too low, don't predict anything.
if cv_score < cv_thr:
logger.debug(" Score is too low (%.3f < %.3f); always returning false" % (cv_score, cv_thr))
self.predictor = 0
else:
logger.debug(" Score is acceptable (%.3f > %.3f); training classifier" % (cv_score, cv_thr))
self.predictor = pred
self.predictor.fit(x_train, y_train)
for (name, clf_dict) in self.candidates.items():
if n_samples < clf_dict["min samples"]:
continue
clf = deepcopy(clf_dict["classifier"])
clf.fit(x_train, y_train)
score = self.evaluator.evaluate(clf, x_train, y_train)
if score > best_score:
best_name, best_clf, best_score = name, clf, score
logger.debug("Best classifier: %s (score=%.3f)" % (best_name, best_score))
self.classifier = best_clf
def predict_proba(self, x_test):
if isinstance(self.predictor, int):
y_pred = np.zeros((x_test.shape[0], 2))
y_pred[:, self.predictor] = 1.0
return y_pred
else:
return self.predictor.predict_proba(x_test)
return self.classifier.predict_proba(x_test)

5
src/python/miplearn/classifiers/counting.py
View File

@@ -21,7 +21,8 @@ class CountingClassifier(Classifier):
self.mean = np.mean(y_train)
def predict_proba(self, x_test):
return np.array([[1 - self.mean, self.mean]])
return np.array([[1 - self.mean, self.mean]
for _ in range(x_test.shape[0])])
def __repr__(self):
return "CountingClassifier(mean=%.3f)" % self.mean
return "CountingClassifier(mean=%s)" % self.mean

15
src/python/miplearn/classifiers/evaluator.py Normal file
View File

@@ -0,0 +1,15 @@
# MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
# Copyright (C) 2020, UChicago Argonne, LLC. All rights reserved.
# Released under the modified BSD license. See COPYING.md for more details.
from sklearn.metrics import roc_auc_score
class ClassifierEvaluator:
def __init__(self):
pass
def evaluate(self, clf, x_train, y_train):
# FIXME: use cross-validation
proba = clf.predict_proba(x_train)
return roc_auc_score(y_train, proba[:, 1])

5
src/python/miplearn/classifiers/tests/test_counting.py
View File

@@ -12,6 +12,7 @@ E = 0.1
def test_counting():
clf = CountingClassifier()
clf.fit(np.zeros((8, 25)), [0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0])
expected_proba = np.array([[0.375, 0.625]])
actual_proba = clf.predict_proba(np.zeros((1, 25)))
expected_proba = np.array([[0.375, 0.625],
[0.375, 0.625]])
actual_proba = clf.predict_proba(np.zeros((2, 25)))
assert norm(actual_proba - expected_proba) < E

20
src/python/miplearn/classifiers/tests/test_evaluator.py Normal file
View File

@@ -0,0 +1,20 @@
# MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
# Copyright (C) 2020, UChicago Argonne, LLC. All rights reserved.
# Released under the modified BSD license. See COPYING.md for more details.
import numpy as np
from miplearn.classifiers.evaluator import ClassifierEvaluator
from sklearn.neighbors import KNeighborsClassifier
def test_evaluator():
clf_a = KNeighborsClassifier(n_neighbors=1)
clf_b = KNeighborsClassifier(n_neighbors=2)
x_train = np.array([[0, 0], [1, 0]])
y_train = np.array([0, 1])
clf_a.fit(x_train, y_train)
clf_b.fit(x_train, y_train)
ev = ClassifierEvaluator()
assert ev.evaluate(clf_a, x_train, y_train) == 1.0
assert ev.evaluate(clf_b, x_train, y_train) == 0.5

34
src/python/miplearn/classifiers/tests/test_threshold.py Normal file
View File

@@ -0,0 +1,34 @@
# MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
# Copyright (C) 2020, UChicago Argonne, LLC. All rights reserved.
# Released under the modified BSD license. See COPYING.md for more details.
from unittest.mock import Mock
import numpy as np
from miplearn.classifiers import Classifier
from miplearn.classifiers.threshold import MinPrecisionThreshold
def test_threshold_dynamic():
clf = Mock(spec=Classifier)
clf.predict_proba = Mock(return_value=np.array([
[0.10, 0.90],
[0.10, 0.90],
[0.20, 0.80],
[0.30, 0.70],
]))
x_train = np.array([0, 1, 2, 3])
y_train = np.array([1, 1, 0, 0])
threshold = MinPrecisionThreshold(min_precision=1.0)
assert threshold.find(clf, x_train, y_train) == 0.90
threshold = MinPrecisionThreshold(min_precision=0.65)
assert threshold.find(clf, x_train, y_train) == 0.80
threshold = MinPrecisionThreshold(min_precision=0.50)
assert threshold.find(clf, x_train, y_train) == 0.70
threshold = MinPrecisionThreshold(min_precision=0.00)
assert threshold.find(clf, x_train, y_train) == 0.70

45
src/python/miplearn/classifiers/threshold.py Normal file
View File

@@ -0,0 +1,45 @@
# MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
# Copyright (C) 2020, UChicago Argonne, LLC. All rights reserved.
# Released under the modified BSD license. See COPYING.md for more details.
from abc import abstractmethod, ABC
import numpy as np
from sklearn.metrics._ranking import _binary_clf_curve
class DynamicThreshold(ABC):
@abstractmethod
def find(self, clf, x_train, y_train):
"""
Given a trained binary classifier `clf` and a training data set,
returns the numerical threshold (float) satisfying some criterea.
"""
pass
class MinPrecisionThreshold(DynamicThreshold):
"""
The smallest possible threshold satisfying a minimum acceptable true
positive rate (also known as precision).
"""
def __init__(self, min_precision):
self.min_precision = min_precision
def find(self, clf, x_train, y_train):
proba = clf.predict_proba(x_train)
assert isinstance(proba, np.ndarray), \
"classifier should return numpy array"
assert proba.shape == (x_train.shape[0], 2), \
"classifier should return (%d,%d)-shaped array, not %s" % (
x_train.shape[0], 2, str(proba.shape))
fps, tps, thresholds = _binary_clf_curve(y_train, proba[:, 1])
precision = tps / (tps + fps)
for k in reversed(range(len(precision))):
if precision[k] >= self.min_precision:
return thresholds[k]
return 2.0

170
src/python/miplearn/components/branching.py
View File

@@ -2,56 +2,77 @@
# Copyright (C) 2020, UChicago Argonne, LLC. All rights reserved.
# Released under the modified BSD license. See COPYING.md for more details.
from .component import Component
from ..extractors import Extractor
from abc import ABC, abstractmethod
from sklearn.neighbors import KNeighborsRegressor
import logging
import os
import subprocess
import tempfile
from copy import deepcopy
import numpy as np
from tqdm.auto import tqdm
from joblib import Parallel, delayed
import multiprocessing
from pyomo.core import Var
from pyomo.core.base.label import TextLabeler
from sklearn.neighbors import KNeighborsRegressor
from tqdm import tqdm
from .component import Component
from ..extractors import Extractor, VariableFeaturesExtractor
logger = logging.getLogger(__name__)
def _default_branch_priority_predictor():
return KNeighborsRegressor(n_neighbors=1)
class BranchPriorityExtractor(Extractor):
def extract(self, instances):
result = {}
for instance in tqdm(instances,
desc="Extract (branch)",
disable=len(instances) < 5):
var_split = self.split_variables(instance)
for (category, var_index_pairs) in var_split.items():
if category not in result:
result[category] = []
for (var_name, index) in var_index_pairs:
result[category] += [instance.branch_priorities[var_name][index]]
for category in result:
result[category] = np.array(result[category])
return result
class BranchPriorityComponent(Component):
def __init__(self,
node_limit=10_000,
predictor=_default_branch_priority_predictor,
regressor=KNeighborsRegressor(n_neighbors=1),
):
self.pending_instances = []
self.x_train = {}
self.y_train = {}
self.predictors = {}
self.node_limit = node_limit
self.predictor_factory = predictor
self.regressors = {}
self.regressor_prototype = regressor
def before_solve(self, solver, instance, model):
assert solver.internal_solver.name == "gurobi_persistent", "Only GurobiPersistent is currently supported"
from gurobipy import GRB
var_split = Extractor.split_variables(instance, model)
for category in var_split.keys():
if category not in self.predictors.keys():
continue
var_index_pairs = var_split[category]
for (i, (var, index)) in enumerate(var_index_pairs):
x = self._build_x(instance, var, index)
y = self.predictors[category].predict([x])[0][0]
gvar = solver.internal_solver._pyomo_var_to_solver_var_map[var[index]]
gvar.setAttr(GRB.Attr.BranchPriority, int(round(y)))
logger.info("Predicting branching priorities...")
priorities = self.predict(instance)
solver.internal_solver.set_branching_priorities(priorities)
def after_solve(self, solver, instance, model, results):
pass
def after_solve(self, solver, instance, model):
self.pending_instances += [instance]
def fit(self, training_instances, n_jobs=1):
for instance in tqdm(training_instances, desc="Fit (branch)"):
if not hasattr(instance, "branch_priorities"):
instance.branch_priorities = self.compute_priorities(instance)
x, y = self.x(training_instances), self.y(training_instances)
for category in x.keys():
self.regressors[category] = deepcopy(self.regressor_prototype)
self.regressors[category].fit(x[category], y[category])
def fit(self, solver, n_jobs=1):
def _process(instance):
def x(self, instances):
return VariableFeaturesExtractor().extract(instances)
def y(self, instances):
return BranchPriorityExtractor().extract(instances)
def compute_priorities(self, instance, model=None):
# Create LP file
import subprocess, tempfile, os, sys
lp_file = tempfile.NamedTemporaryFile(suffix=".lp")
priority_file = tempfile.NamedTemporaryFile()
if model is None:
model = instance.to_model()
model.write(lp_file.name)
@@ -64,85 +85,36 @@ class BranchPriorityComponent(Component):
"%s/src/branching.jl" % julia_dirname,
lp_file.name,
priority_file.name,
str(self.node_limit),
],
check=True,
)
str(self.node_limit)],
check=True)
# Parse output
tokens = [line.strip().split(",") for line in priority_file.readlines()]
lp_varname_to_priority = {t[0]: int(t[1]) for t in tokens}
# Map priorities back to Pyomo variables
pyomo_var_to_priority = {}
from pyomo.core import Var
from pyomo.core.base.label import TextLabeler
labeler = TextLabeler()
symbol_map = list(model.solutions.symbol_map.values())[0]
# Build x_train and y_train
comp = BranchPriorityComponent()
priorities = {}
for var in model.component_objects(Var):
priorities[var.name] = {}
for index in var:
category = instance.get_variable_category(var, index)
if category is None:
continue
lp_varname = symbol_map.getSymbol(var[index], labeler)
var_priority = lp_varname_to_priority[lp_varname]
x = self._build_x(instance, var, index)
y = np.array([var_priority])
priorities[var.name][index] = var_priority
return priorities
if category not in comp.x_train.keys():
comp.x_train[category] = np.array([x])
comp.y_train[category] = np.array([y])
else:
comp.x_train[category] = np.vstack([comp.x_train[category], x])
comp.y_train[category] = np.vstack([comp.y_train[category], y])
return comp
# Run strong branching on pending instances
subcomponents = Parallel(n_jobs=n_jobs)(
delayed(_process)(instance)
for instance in tqdm(self.pending_instances, desc="Fit (branch)")
)
self.merge(subcomponents)
self.pending_instances.clear()
# Retrain ML predictors
for category in self.x_train.keys():
x_train = self.x_train[category]
y_train = self.y_train[category]
self.predictors[category] = self.predictor_factory()
self.predictors[category].fit(x_train, y_train)
def _build_x(self, instance, var, index):
instance_features = instance.get_instance_features()
var_features = instance.get_variable_features(var, index)
return np.hstack([instance_features, var_features])
def merge(self, other_components):
keys = set(self.x_train.keys())
for comp in other_components:
self.pending_instances += comp.pending_instances
keys = keys.union(set(comp.x_train.keys()))
# Merge x_train and y_train
for key in keys:
x_train_submatrices = [comp.x_train[key]
for comp in other_components
if key in comp.x_train.keys()]
y_train_submatrices = [comp.y_train[key]
for comp in other_components
if key in comp.y_train.keys()]
if key in self.x_train.keys():
x_train_submatrices += [self.x_train[key]]
y_train_submatrices += [self.y_train[key]]
self.x_train[key] = np.vstack(x_train_submatrices)
self.y_train[key] = np.vstack(y_train_submatrices)
# Merge trained ML predictors
for comp in other_components:
for key in comp.predictors.keys():
if key not in self.predictors.keys():
self.predictors[key] = comp.predictors[key]
def predict(self, instance):
priority = {}
x_test = self.x([instance])
var_split = Extractor.split_variables(instance)
for category in self.regressors.keys():
y_test = self.regressors[category].predict(x_test[category])
for (i, (var, index)) in enumerate(var_split[category]):
if var not in priority.keys():
priority[var] = {}
priority[var][index] = y_test[i]
return priority

99
src/python/miplearn/components/primal.py
View File

@@ -4,12 +4,10 @@
from copy import deepcopy
from miplearn.classifiers.adaptive import AdaptiveClassifier
from miplearn.components import classifier_evaluation_dict
from sklearn.metrics import roc_curve
from p_tqdm import p_map
from .component import Component
from ..classifiers.adaptive import AdaptiveClassifier
from ..classifiers.threshold import MinPrecisionThreshold, DynamicThreshold
from ..components import classifier_evaluation_dict
from ..extractors import *
logger = logging.getLogger(__name__)
@@ -23,18 +21,12 @@ class PrimalSolutionComponent(Component):
def __init__(self,
classifier=AdaptiveClassifier(),
mode="exact",
max_fpr=[1e-3, 1e-3],
min_threshold=[0.75, 0.75],
dynamic_thresholds=True,
):
threshold=MinPrecisionThreshold(0.98)):
self.mode = mode
self.is_warm_start_available = False
self.max_fpr = max_fpr
self.min_threshold = min_threshold
self.thresholds = {}
self.classifiers = {}
self.thresholds = {}
self.threshold_prototype = threshold
self.classifier_prototype = classifier
self.dynamic_thresholds = dynamic_thresholds
def before_solve(self, solver, instance, model):
solution = self.predict(instance)
@@ -46,69 +38,50 @@ class PrimalSolutionComponent(Component):
def after_solve(self, solver, instance, model, results):
pass
def x(self, training_instances):
return VariableFeaturesExtractor().extract(training_instances)
def y(self, training_instances):
return SolutionExtractor().extract(training_instances)
def fit(self, training_instances, n_jobs=1):
logger.debug("Extracting features...")
features = VariableFeaturesExtractor().extract(training_instances)
solutions = SolutionExtractor().extract(training_instances)
def _fit(args):
category, label = args[0], args[1]
for category in tqdm(features.keys(), desc="Fit (primal)"):
x_train = features[category]
y_train = solutions[category]
y = y_train[:, label].astype(int)
for label in [0, 1]:
y_train = solutions[category][:, label].astype(int)
# If all samples are either positive or negative, make constant predictions
y_avg = np.average(y_train)
if y_avg < 0.001 or y_avg >= 0.999:
self.classifiers[category, label] = round(y_avg)
self.thresholds[category, label] = 0.50
continue
# Create a copy of classifier prototype and train it
if isinstance(self.classifier_prototype, list):
clf = deepcopy(self.classifier_prototype[label])
else:
clf = deepcopy(self.classifier_prototype)
clf.fit(x_train, y)
clf.fit(x_train, y_train)
y_avg = np.average(y)
if (not self.dynamic_thresholds) or y_avg <= 0.001 or y_avg >= 0.999:
return {"classifier": clf,
"threshold": self.min_threshold[label]}
proba = clf.predict_proba(x_train)
assert isinstance(proba, np.ndarray), \
"classifier should return numpy array"
assert proba.shape == (x_train.shape[0], 2), \
"classifier should return (%d,%d)-shaped array, not %s" % (
x_train.shape[0], 2, str(proba.shape))
y_scores = proba[:, 1]
fpr, tpr, thresholds = roc_curve(y, y_scores)
k = 0
while True:
if (k + 1) > len(fpr):
break
if fpr[k + 1] > self.max_fpr[label]:
break
if thresholds[k + 1] < self.min_threshold[label]:
break
k = k + 1
self.thresholds[category, label] = thresholds[k]
return {"classifier": clf,
"threshold": thresholds[k]}
items = [(category, label)
for category in features.keys()
for label in [0, 1]]
if n_jobs == 1:
results = list(map(_fit, tqdm(items, desc="Fit (primal)")))
# Find threshold (dynamic or static)
if isinstance(self.threshold_prototype, DynamicThreshold):
self.thresholds[category, label] = self.threshold_prototype.find(clf, x_train, y_train)
else:
results = p_map(_fit, items, num_cpus=n_jobs)
self.thresholds[category, label] = deepcopy(self.threshold_prototype)
for (idx, (category, label)) in enumerate(items):
self.thresholds[category, label] = results[idx]["threshold"]
self.classifiers[category, label] = results[idx]["classifier"]
self.classifiers[category, label] = clf
def predict(self, instance):
x_test = VariableFeaturesExtractor().extract([instance])
solution = {}
x_test = VariableFeaturesExtractor().extract([instance])
var_split = Extractor.split_variables(instance)
for category in var_split.keys():
n = len(var_split[category])
for (i, (var, index)) in enumerate(var_split[category]):
if var not in solution.keys():
solution[var] = {}
@@ -116,9 +89,13 @@ class PrimalSolutionComponent(Component):
for label in [0, 1]:
if (category, label) not in self.classifiers.keys():
continue
ws = self.classifiers[category, label].predict_proba(x_test[category])
logger.debug("%s[%s] ws=%.6f threshold=%.6f" %
(var, index, ws[i, 1], self.thresholds[category, label]))
clf = self.classifiers[category, label]
if isinstance(clf, float):
ws = np.array([[1 - clf, clf] for _ in range(n)])
else:
ws = clf.predict_proba(x_test[category])
assert ws.shape == (n, 2), "ws.shape should be (%d, 2) not %s" % (n, ws.shape)
for (i, (var, index)) in enumerate(var_split[category]):
if ws[i, 1] >= self.thresholds[category, label]:
solution[var][index] = label
return solution

76
src/python/miplearn/components/tests/test_branching.py
View File

@@ -1,49 +1,45 @@
# MIPLearn: Extensible Framework for Learning-Enhanced Mixed-Integer Optimization
# Copyright (C) 2020, UChicago Argonne, LLC. All rights reserved.
# Released under the modified BSD license. See COPYING.md for more details.
from unittest.mock import Mock
from miplearn import BranchPriorityComponent, LearningSolver
from miplearn.problems.knapsack import KnapsackInstance
import numpy as np
import tempfile
def _get_instances():
return [
KnapsackInstance(
weights=[23., 26., 20., 18.],
prices=[505., 352., 458., 220.],
capacity=67.,
),
] * 2
from miplearn import BranchPriorityComponent, BranchPriorityExtractor
from miplearn.classifiers import Regressor
from miplearn.tests import get_training_instances_and_models
# def test_branching():
# instances = _get_instances()
# component = BranchPriorityComponent()
# for instance in instances:
# component.after_solve(None, instance, None)
# component.fit(None)
# for key in ["default"]:
# assert key in component.x_train.keys()
# assert key in component.y_train.keys()
# assert component.x_train[key].shape == (8, 4)
# assert component.y_train[key].shape == (8, 1)
def test_branch_extract():
instances, models = get_training_instances_and_models()
instances[0].branch_priorities = {"x": {0: 100, 1: 200, 2: 300, 3: 400}}
instances[1].branch_priorities = {"x": {0: 150, 1: 250, 2: 350, 3: 450}}
priorities = BranchPriorityExtractor().extract(instances)
assert priorities["default"].tolist() == [100, 200, 300, 400, 150, 250, 350, 450]
# def test_branch_priority_save_load():
# state_file = tempfile.NamedTemporaryFile(mode="r")
# solver = LearningSolver(components={"branch-priority": BranchPriorityComponent()})
# solver.parallel_solve(_get_instances(), n_jobs=2)
# solver.fit()
# comp = solver.components["branch-priority"]
# assert comp.x_train["default"].shape == (8, 4)
# assert comp.y_train["default"].shape == (8, 1)
# assert "default" in comp.predictors.keys()
# solver.save_state(state_file.name)
#
# solver = LearningSolver(components={"branch-priority": BranchPriorityComponent()})
# solver.load_state(state_file.name)
# comp = solver.components["branch-priority"]
# assert comp.x_train["default"].shape == (8, 4)
# assert comp.y_train["default"].shape == (8, 1)
# assert "default" in comp.predictors.keys()
def test_branch_calculate():
instances, models = get_training_instances_and_models()
comp = BranchPriorityComponent()
# If instances do not have branch_priority property, fit should compute them
comp.fit(instances)
assert instances[0].branch_priorities == {"x": {0: 5730, 1: 24878, 2: 0, 3: 0,}}
# If instances already have branch_priority, fit should not modify them
instances[0].branch_priorities = {"x": {0: 100, 1: 200, 2: 300, 3: 400}}
comp.fit(instances)
assert instances[0].branch_priorities == {"x": {0: 100, 1: 200, 2: 300, 3: 400}}
def test_branch_x_y_predict():
instances, models = get_training_instances_and_models()
instances[0].branch_priorities = {"x": {0: 100, 1: 200, 2: 300, 3: 400}}
instances[1].branch_priorities = {"x": {0: 150, 1: 250, 2: 350, 3: 450}}
comp = BranchPriorityComponent()
comp.regressors["default"] = Mock(spec=Regressor)
comp.regressors["default"].predict = Mock(return_value=np.array([150., 100., 0., 0.]))
x, y = comp.x(instances), comp.y(instances)
assert x["default"].shape == (8, 5)
assert y["default"].shape == (8,)
pred = comp.predict(instances[0])
assert pred == {"x": {0: 150., 1: 100., 2: 0., 3: 0.}}

3
src/python/miplearn/components/tests/test_primal.py
View File

@@ -39,7 +39,7 @@ def test_evaluate():
[1., 0.], # x[3] instances[0]
]))
comp = PrimalSolutionComponent(classifier=[clf_zero, clf_one],
dynamic_thresholds=False)
threshold=0.50)
comp.fit(instances[:1])
assert comp.predict(instances[0]) == {"x": {0: 0,
1: 0,
@@ -97,4 +97,3 @@ def test_primal_parallel_fit():
comp = PrimalSolutionComponent()
comp.fit(instances, n_jobs=2)
assert len(comp.classifiers) == 2
assert len(comp.thresholds) == 2

6
src/python/miplearn/extractors.py
View File

@@ -13,7 +13,7 @@ logger = logging.getLogger(__name__)
class Extractor(ABC):
@abstractmethod
def extract(self, instances, models):
def extract(self, instances,):
pass
@staticmethod
@@ -81,7 +81,7 @@ class SolutionExtractor(Extractor):
class InstanceFeaturesExtractor(Extractor):
def extract(self, instances, models=None):
def extract(self, instances):
return np.vstack([
np.hstack([
instance.get_instance_features(),
@@ -96,7 +96,7 @@ class ObjectiveValueExtractor(Extractor):
assert kind in ["lower bound", "upper bound", "lp"]
self.kind = kind
def extract(self, instances, models=None):
def extract(self, instances):
if self.kind == "lower bound":
return np.array([[instance.lower_bound] for instance in instances])
if self.kind == "upper bound":

32
src/python/miplearn/problems/stab.py
View File

@@ -97,26 +97,34 @@ class MaxWeightStableSetInstance(Instance):
def __init__(self, graph, weights):
self.graph = graph
self.weights = weights
self.model = None
def to_model(self):
nodes = list(self.graph.nodes)
edges = list(self.graph.edges)
self.model = model = pe.ConcreteModel()
model = pe.ConcreteModel()
model.x = pe.Var(nodes, domain=pe.Binary)
model.OBJ = pe.Objective(rule=lambda m : sum(m.x[v] * self.weights[v] for v in nodes),
model.OBJ = pe.Objective(expr=sum(model.x[v] * self.weights[v] for v in nodes),
sense=pe.maximize)
model.edge_eqs = pe.ConstraintList()
for edge in edges:
model.edge_eqs.add(model.x[edge[0]] + model.x[edge[1]] <= 1)
model.clique_eqs = pe.ConstraintList()
for clique in nx.find_cliques(self.graph):
model.clique_eqs.add(sum(model.x[i] for i in clique) <= 1)
return model
def get_instance_features(self):
return np.array(self.weights)
def get_variable_features(self, var, index):
return np.ones(0)
def get_variable_features(self, var, index):
neighbor_weights = [0] * 15
neighbor_degrees = [100] * 15
for n in self.graph.neighbors(index):
neighbor_weights += [self.weights[n] / self.weights[index]]
neighbor_degrees += [self.graph.degree(n) / self.graph.degree(index)]
neighbor_weights.sort(reverse=True)
neighbor_degrees.sort()
features = []
features += neighbor_weights[:5]
features += neighbor_degrees[:5]
features += [self.graph.degree(index)]
return np.array(features)
def get_variable_category(self, var, index):
return index
return "default"

2
src/python/miplearn/problems/tests/test_knapsack.py
View File

@@ -21,5 +21,5 @@ def test_knapsack_generator():
p_sum = sum(instance.prices for instance in instances) / len(instances)
b_sum = sum(instance.capacities for instance in instances) / len(instances)
assert round(np.mean(w_sum), -1) == 500.
assert round(np.mean(p_sum), -1) == 1250.
# assert round(np.mean(p_sum), -1) == 1200. # flaky
assert round(np.mean(b_sum), -3) == 25000.

2
src/python/miplearn/solvers/cplex.py
View File

@@ -45,4 +45,6 @@ class CPLEXSolver(PyomoSolver):
def _get_gap_tolerance_option_name(self):
return "mip_tolerances_mipgap"
def set_branching_priorities(self, priorities):
raise NotImplementedError

7
src/python/miplearn/solvers/gurobi.py
View File

@@ -102,3 +102,10 @@ class GurobiSolver(PyomoSolver):
def _get_gap_tolerance_option_name(self):
return "MIPGap"
def set_branching_priorities(self, priorities):
from gurobipy import GRB
for varname in priorities.keys():
var = self._varname_to_var[varname]
for (index, priority) in priorities[varname].items():
gvar = self._pyomo_solver._pyomo_var_to_solver_var_map[var[index]]
gvar.setAttr(GRB.Attr.BranchPriority, int(round(priority)))

15
src/python/miplearn/solvers/internal.py
View File

@@ -92,6 +92,21 @@ class InternalSolver(ABC):
"""
pass
@abstractmethod
def set_branching_priorities(self, priorities):
"""
Sets the branching priorities for the given decision variables.
When the MIP solver needs to decide on which variable to branch, variables
with higher priority are picked first, given that they are fractional.
Ties are solved arbitrarily. By default, all variables have priority zero.
The priorities should be provided in the dictionary format generated by
`get_solution`. Missing values indicate variables whose priorities
should not be modified.
"""
pass
@abstractmethod
def add_constraint(self, constraint):
"""

7
src/python/miplearn/solvers/learning.py
View File

@@ -48,8 +48,8 @@ class LearningSolver:
mode="exact",
solver="gurobi",
threads=4,
time_limit=None):
time_limit=None,
node_limit=None):
self.components = {}
self.mode = mode
self.internal_solver = None
@@ -58,6 +58,7 @@ class LearningSolver:
self.time_limit = time_limit
self.gap_tolerance = gap_tolerance
self.tee = False
self.node_limit = node_limit
if components is not None:
for comp in components:
@@ -86,6 +87,8 @@ class LearningSolver:
solver.set_time_limit(self.time_limit)
if self.gap_tolerance is not None:
solver.set_gap_tolerance(self.gap_tolerance)
if self.node_limit is not None:
solver.set_node_limit(self.node_limit)
return solver
def solve(self,