"1. Install the Julia/JuMP version of MIPLearn\n",
"1. Install the Julia/JuMP version of MIPLearn\n",
"2. Model a simple optimization problem using JuMP\n",
"2. Model a simple optimization problem using JuMP\n",
"3. Generate training data and train the ML models\n",
"3. Generate training data and train the ML models\n",
"4. Use the ML models together with SCIP to solve new instances\n",
"4. Use the ML models together with Gurobi to solve new instances\n",
"\n",
"<div class=\"alert alert-info\">\n",
"Note\n",
" \n",
"In this tutorial, we use SCIP because it is more widely available than commercial MIP solvers. However, all the steps below should work for Gurobi, CPLEX or XPRESS, as long as you have a license for these solvers. The performance impact of MIPLearn may also change for different solvers.\n",
"Next, we create a function that converts this data structure into a concrete JuMP model. For more details on the JuMP syntax, see [the official JuMP documentation](https://jump.dev/JuMP.jl/stable/)."
"Next, we create a function that converts this data structure into a concrete JuMP model. For more details on the JuMP syntax, see [the official JuMP documentation](https://jump.dev/JuMP.jl/stable/)."
@ -190,7 +178,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 4,
"execution_count": 4,
"id": "1b8245e2",
"id": "a7c048e4",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -219,7 +207,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "d4b9bfa3",
"id": "5f10142e",
"metadata": {},
"metadata": {},
"source": [
"source": [
"At this point, we can already use JuMP and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators, using SCIP:"
"At this point, we can already use JuMP and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators, using SCIP:"
"Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
"Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
@ -274,7 +262,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "fdc17cc8",
"id": "f34e3d44",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Generating training data\n",
"## Generating training data\n",
@ -289,7 +277,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 6,
"execution_count": 6,
"id": "adec6906",
"id": "a498e1e1",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -317,433 +305,377 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "b1f4bdbd",
"id": "e33bb12c",
"metadata": {},
"metadata": {},
"source": [
"source": [
"In this example, for simplicity, only the demands change from one instance to the next. We could also have made the prices and the production limits random. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
"In this example, for simplicity, only the demands change from one instance to the next. We could also have randomized the costs, production limits or even the number of units. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
"\n",
"\n",
"Now we generate 100 instances of this problem, each one with 1,000 generators. We will use the first 90 instances for training, and the remaining 10 instances to evaluate SCIP's performance."
"Now we generate 500 instances of this problem, each one with 50 generators, and we use 450 of these instances for training. After generating the instances, we write them to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold in memory the entire training data, as well as the concrete JuMP models. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial. The code below generates the files `uc/train/00001.jld2`, `uc/train/00002.jld2`, etc., which contain the input data in [JLD2 format](https://github.com/JuliaIO/JLD2.jl)."
"Next, we write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.\n",
"Finally, we use `LearningSolver` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The optimal solutions, along with other useful training data, are stored in HDF5 files `uc/train/00001.h5`, `uc/train/00002.h5`, etc."
"\n",
"The code below generates the files `uc/train/000001.jld2`, `uc/train/000002.jld2`, etc., which contain the input data in [JLD2 format](https://github.com/JuliaIO/JLD2.jl)."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 8,
"execution_count": 8,
"id": "e5ad203b",
"id": "c341b12d",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
"using MIPLearn\n",
"solver = LearningSolver(gurobi)\n",
"MIPLearn.save(data[1:90], \"uc/train/\")\n",
"solve!(solver, train_files, build_uc_model);"
"MIPLearn.save(data[91:100], \"uc/test/\")\n",
"\n",
"using Glob\n",
"train_files = glob(\"uc/train/*.jld2\")\n",
"test_files = glob(\"uc/test/*.jld2\");"
]
},
{
"cell_type": "markdown",
"id": "a758917d",
"metadata": {},
"source": [
"Finally, we use `MIPLearn.LearningSolver` and `MIPLearn.solve!` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The `solve!` function can be used to solve either one or multiple instances, and requires: (i) the list of files containing the training data; and (ii) the function that converts the data structure into a concrete JuMP model:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "930b59cd",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"105.491488 seconds (94.05 M allocations: 3.632 GiB, 1.17% gc time, 0.56% compilation time)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Dual bound 1.98665e+07 is larger than the objective of the primal solution 1.98665e+07. The solution might not be optimal.\n"
"The macro `@time` shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, are stored by MIPLearn in `.h5` files, stored side-by-side with the original `.jld2` files."
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "67b5cc7c",
"id": "189b4f60",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Solving new instances\n",
"## Solving new instances\n",
"\n",
"\n",
"With training data in hand, we can now fit the ML models using `MIPLearn.fit!`, then solve the test instances with `MIPLearn.solve!`, as shown below:"
"With training data in hand, we can now fit the ML models using `MIPLearn.fit!`, then solve the test instances with `MIPLearn.solve!`, as shown below. The `tee=true` parameter asks MIPLearn to print the solver log to the screen."
"The trained MIP solver was able to solve all test instances in about 6 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the `fit!` line:"
"By examining the solve log above, specifically the line `Loaded user MIP start with objective...`, we can see that MIPLearn was able to construct an initial solution which turned out to be near optimal for the problem. Now let us repeat the code above, but using an untrained solver. Note that the `fit` line is omitted."
"Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances.\n",
"In the log above, the `MIP start` line is missing, and Gurobi had to start with a significantly inferior initial solution. The solver was still able to find the optimal solution at the end, but it required using its own internal heuristic procedures. In this example, because we solve very small optimization problems, there was almost no difference in terms of running time. For larger problems, however, the difference can be significant. See benchmarks for more details.\n",
"\n",
"\n",
"<div class=\"alert alert-info\">\n",
"<div class=\"alert alert-info\">\n",
"Note\n",
"Note\n",
" \n",
" \n",
"Note that is is not necessary to specify what ML models to use. MIPLearn, by default, will try a number of classical ML models and will choose the one that performs the best, based on k-fold cross validation. MIPLearn is also able to automatically collect features based on the MIP formulation of the problem and the solution to the LP relaxation, among other things, so it does not require handcrafted features. If you do want to customize the models and features, however, that is also possible, as we will see in a later tutorial.\n",
"In addition to partial initial solutions, MIPLearn is also able to predict lazy constraints, cutting planes and branching priorities. See the next tutorials for more details.\n",
"</div>"
"</div>\n",
]
},
{
"cell_type": "markdown",
"id": "dc1bc410",
"metadata": {},
"source": [
"## Understanding the acceleration\n",
"\n",
"\n",
"Let us go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP's performance. First, we are going to solve one of the test instances again, using the trained solver, but this time using the `tee=true` parameter, so that we can see SCIP's log:"
"<div class=\"alert alert-info\">\n",
]
"Note\n",
},
" \n",
{
"It is not necessary to specify what ML models to use. MIPLearn, by default, will try a number of classical ML models and will choose the one that performs the best, based on k-fold cross validation. MIPLearn is also able to automatically collect features based on the MIP formulation of the problem and the solution to the LP relaxation, among other things, so it does not require handcrafted features. If you do want to customize the models and features, however, that is also possible, as we will see in a later tutorial.\n",
"The log above is quite complicated if you have never seen it before, but the important line is the one starting with `feasible solution found [...] objective value 1.705169e+07`. This line indicates that MIPLearn was able to construct a warm start with value `1.705169e+07`. Using this warm start, SCIP then used the branch-and-cut method to either prove its optimality or to find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed optimal. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast. \n",
"## Accessing the solution\n",
"\n",
"\n",
"Let us now repeat the process, but using the untrained solver this time:"
"In the example above, we used `MIPLearn.solve` together with data files to solve both the training and the test instances. The optimal solutions were saved to HDF5 files in the train/test folders, and could be retrieved by reading theses files, but that is not very convenient. In the following example, we show how to build and solve a JuMP model entirely in-memory, using our trained solver."
"# Construct model using previously defined functions\n",
]
"data = random_uc_data(samples=1, n=50)[1]\n",
},
"model = build_uc_model(data)\n",
{
"cell_type": "markdown",
"id": "92d23037",
"metadata": {},
"source": [
"In this log file, notice how the previous line about warm starts is missing. Since no warm starts were provided, SCIP had to find an initial solution using its own internal heuristics, which are not specifically tailored for this problem. The initial solution found by SCIP's heuristics has value `2.335200e+07`, which is significantly worse than the one constructed by MIPLearn. SCIP then proceeded to improve this solution, by generating cutting planes and repeatedly running additional primal heuristics. In the end, it was able to find the optimal solution, as expected, but it took longer.\n",
"\n",
"\n",
"In summary, MIPLearn accelerated the solution process by constructing a high-quality initial solution. In the following tutorials, we will see other strategies that MIPLearn can use to accelerate MIP performance, besides warm starts."
"# Solve model\n",
]
"solve!(solver_ml, model)\n",
},
{
"cell_type": "markdown",
"id": "94f869d7",
"metadata": {},
"source": [
"## Accessing the solution\n",
"\n",
"In the example above, we used `MIPLearn.solve!` together with data files to solve both the training and the test instances. The solutions were saved to a `.h5` files in the train/test folders, and could be retrieved by reading theses files, but that is not very convenient. In this section we will use an easier method.\n",
"\n",
"\n",
"We can use the function `MIPLearn.load!` to obtain a regular JuMP model:"
"# Print part of the optimal solution\n",
]
"println(\"obj = \", objective_value(model))\n",
},
"println(\" x = \", round.(value.(model[:x][1:10])))\n",
{
"println(\" y = \", round.(value.(model[:y][1:10]), digits=2))"
"Defaulting to user installation because normal site-packages is not writeable\n",
"Requirement already satisfied: MIPLearn==0.2.0.dev13 in /home/axavier/.local/lib/python3.7/site-packages (0.2.0.dev13)\n",
"Requirement already satisfied: p-tqdm<2,>=1 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (1.3.3)\n",
"Requirement already satisfied: overrides<4,>=3 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (3.1.0)\n",
"Requirement already satisfied: scikit-learn<0.25,>=0.24 in /opt/anaconda3/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (0.24.1)\n",
"Requirement already satisfied: h5py<4,>=3 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (3.5.0)\n",
"Requirement already satisfied: matplotlib<4,>=3 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (3.4.1)\n",
"Requirement already satisfied: seaborn<0.12,>=0.11 in /opt/anaconda3/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (0.11.1)\n",
"Requirement already satisfied: python-markdown-math<0.9,>=0.8 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (0.8)\n",
"Requirement already satisfied: decorator<5,>=4 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (4.4.2)\n",
"Requirement already satisfied: mypy==0.790 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (0.790)\n",
"Requirement already satisfied: pandas<2,>=1 in /opt/anaconda3/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (1.2.3)\n",
"Requirement already satisfied: pytest<7,>=6 in /opt/anaconda3/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (6.2.3)\n",
"Requirement already satisfied: numpy<1.21,>=1 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (1.19.5)\n",
"Requirement already satisfied: tqdm<5,>=4 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (4.60.0)\n",
"Requirement already satisfied: pyomo<6,>=5 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (5.7.3)\n",
"Requirement already satisfied: networkx<3,>=2 in /home/axavier/.local/lib/python3.7/site-packages (from MIPLearn==0.2.0.dev13) (2.5.1)\n",
"Requirement already satisfied: typed-ast<1.5.0,>=1.4.0 in /opt/anaconda3/lib/python3.7/site-packages (from mypy==0.790->MIPLearn==0.2.0.dev13) (1.4.2)\n",
"Requirement already satisfied: typing-extensions>=3.7.4 in /opt/anaconda3/lib/python3.7/site-packages (from mypy==0.790->MIPLearn==0.2.0.dev13) (3.7.4.3)\n",
"Requirement already satisfied: mypy-extensions<0.5.0,>=0.4.3 in /home/axavier/.local/lib/python3.7/site-packages (from mypy==0.790->MIPLearn==0.2.0.dev13) (0.4.3)\n",
"Requirement already satisfied: cached-property in /home/axavier/.local/lib/python3.7/site-packages (from h5py<4,>=3->MIPLearn==0.2.0.dev13) (1.5.2)\n",
"Requirement already satisfied: pyparsing>=2.2.1 in /opt/anaconda3/lib/python3.7/site-packages (from matplotlib<4,>=3->MIPLearn==0.2.0.dev13) (2.4.7)\n",
"Requirement already satisfied: pillow>=6.2.0 in /opt/anaconda3/lib/python3.7/site-packages (from matplotlib<4,>=3->MIPLearn==0.2.0.dev13) (8.2.0)\n",
"Requirement already satisfied: python-dateutil>=2.7 in /opt/anaconda3/lib/python3.7/site-packages (from matplotlib<4,>=3->MIPLearn==0.2.0.dev13) (2.8.1)\n",
"Requirement already satisfied: cycler>=0.10 in /opt/anaconda3/lib/python3.7/site-packages (from matplotlib<4,>=3->MIPLearn==0.2.0.dev13) (0.10.0)\n",
"Requirement already satisfied: kiwisolver>=1.0.1 in /opt/anaconda3/lib/python3.7/site-packages (from matplotlib<4,>=3->MIPLearn==0.2.0.dev13) (1.3.1)\n",
"Requirement already satisfied: six in /opt/anaconda3/lib/python3.7/site-packages (from cycler>=0.10->matplotlib<4,>=3->MIPLearn==0.2.0.dev13) (1.15.0)\n",
"Requirement already satisfied: pathos in /home/axavier/.local/lib/python3.7/site-packages (from p-tqdm<2,>=1->MIPLearn==0.2.0.dev13) (0.2.7)\n",
"Requirement already satisfied: pytz>=2017.3 in /opt/anaconda3/lib/python3.7/site-packages (from pandas<2,>=1->MIPLearn==0.2.0.dev13) (2021.1)\n",
"Requirement already satisfied: PyUtilib>=6.0.0 in /home/axavier/.local/lib/python3.7/site-packages (from pyomo<6,>=5->MIPLearn==0.2.0.dev13) (6.0.0)\n",
"Requirement already satisfied: ply in /opt/anaconda3/lib/python3.7/site-packages (from pyomo<6,>=5->MIPLearn==0.2.0.dev13) (3.11)\n",
"Requirement already satisfied: attrs>=19.2.0 in /opt/anaconda3/lib/python3.7/site-packages (from pytest<7,>=6->MIPLearn==0.2.0.dev13) (20.3.0)\n",
"Requirement already satisfied: iniconfig in /opt/anaconda3/lib/python3.7/site-packages (from pytest<7,>=6->MIPLearn==0.2.0.dev13) (1.1.1)\n",
"Requirement already satisfied: packaging in /opt/anaconda3/lib/python3.7/site-packages (from pytest<7,>=6->MIPLearn==0.2.0.dev13) (20.9)\n",
"Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in /opt/anaconda3/lib/python3.7/site-packages (from pytest<7,>=6->MIPLearn==0.2.0.dev13) (0.13.1)\n",
"Requirement already satisfied: py>=1.8.2 in /opt/anaconda3/lib/python3.7/site-packages (from pytest<7,>=6->MIPLearn==0.2.0.dev13) (1.10.0)\n",
"Requirement already satisfied: toml in /opt/anaconda3/lib/python3.7/site-packages (from pytest<7,>=6->MIPLearn==0.2.0.dev13) (0.10.2)\n",
"Requirement already satisfied: importlib-metadata>=0.12 in /opt/anaconda3/lib/python3.7/site-packages (from pytest<7,>=6->MIPLearn==0.2.0.dev13) (3.10.0)\n",
"Requirement already satisfied: zipp>=0.5 in /opt/anaconda3/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<7,>=6->MIPLearn==0.2.0.dev13) (3.4.1)\n",
"Requirement already satisfied: Markdown>=3.0 in /opt/anaconda3/lib/python3.7/site-packages (from python-markdown-math<0.9,>=0.8->MIPLearn==0.2.0.dev13) (3.3.4)\n",
"Requirement already satisfied: nose in /opt/anaconda3/lib/python3.7/site-packages (from PyUtilib>=6.0.0->pyomo<6,>=5->MIPLearn==0.2.0.dev13) (1.3.7)\n",
"Requirement already satisfied: scipy>=0.19.1 in /opt/anaconda3/lib/python3.7/site-packages (from scikit-learn<0.25,>=0.24->MIPLearn==0.2.0.dev13) (1.6.2)\n",
"Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/anaconda3/lib/python3.7/site-packages (from scikit-learn<0.25,>=0.24->MIPLearn==0.2.0.dev13) (2.1.0)\n",
"Requirement already satisfied: joblib>=0.11 in /opt/anaconda3/lib/python3.7/site-packages (from scikit-learn<0.25,>=0.24->MIPLearn==0.2.0.dev13) (1.0.1)\n",
"Requirement already satisfied: multiprocess>=0.70.11 in /home/axavier/.local/lib/python3.7/site-packages (from pathos->p-tqdm<2,>=1->MIPLearn==0.2.0.dev13) (0.70.11.1)\n",
"Requirement already satisfied: ppft>=1.6.6.3 in /home/axavier/.local/lib/python3.7/site-packages (from pathos->p-tqdm<2,>=1->MIPLearn==0.2.0.dev13) (1.6.6.3)\n",
"Requirement already satisfied: dill>=0.3.3 in /home/axavier/.local/lib/python3.7/site-packages (from pathos->p-tqdm<2,>=1->MIPLearn==0.2.0.dev13) (0.3.3)\n",
"Requirement already satisfied: pox>=0.2.9 in /home/axavier/.local/lib/python3.7/site-packages (from pathos->p-tqdm<2,>=1->MIPLearn==0.2.0.dev13) (0.2.9)\n"
]
}
],
"source": [
"source": [
"!pip install MIPLearn==0.2.0.dev13"
"# !pip install MIPLearn==0.2.0.dev13"
]
]
},
},
{
{
@ -115,7 +62,7 @@
"id": "e8274543",
"id": "e8274543",
"metadata": {},
"metadata": {},
"source": [
"source": [
"In addition to MIPLearn itself, we will also install Gurobi 9.1, a state-of-the-art commercial MILP solver. To succesfully complete this step, you will need a valid Gurobi license."
"In addition to MIPLearn itself, we will also install Gurobi 9.5, a state-of-the-art commercial MILP solver. This step also install a demo license for Gurobi, which should able to solve the small optimization problems in this tutorial. A paid license is required for solving large-scale problems."
]
]
},
},
{
{
@ -128,14 +75,13 @@
"name": "stdout",
"name": "stdout",
"output_type": "stream",
"output_type": "stream",
"text": [
"text": [
"Defaulting to user installation because normal site-packages is not writeable\n",
"Looking in indexes: https://pypi.gurobi.com\n",
"Looking in indexes: https://pypi.gurobi.com\n",
"Requirement already satisfied: gurobipy==9.1.2 in /home/axavier/.local/lib/python3.7/site-packages (9.1.2)\n"
"Requirement already satisfied: gurobipy<9.6,>=9.5 in /opt/anaconda3/envs/miplearn/lib/python3.8/site-packages (9.5.1)\n"
"Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Python and Pyomo. We start by defining a class `UnitCommitmentInstance`, which holds all the input data. The class also contains a method `to_model` which constructs a concrete Pyomo model object:"
"Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Python and Pyomo. We start by defining a data class `UnitCommitmentData`, which holds all the input data."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 12,
"execution_count": 3,
"id": "b5bd2c58",
"id": "22a67170-10b4-43d3-8708-014d91141e73",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from dataclasses import dataclass\n",
"import numpy as np\n",
"\n",
"@dataclass\n",
"class UnitCommitmentData:\n",
" demand: float\n",
" pmin: np.ndarray\n",
" pmax: np.ndarray\n",
" cfix: np.ndarray\n",
" cvar: np.ndarray"
]
},
{
"cell_type": "markdown",
"id": "29f55efa-0751-465a-9b0a-a821d46a3d40",
"metadata": {},
"source": [
"Next, we write a `build_uc_model` function, which converts the input data into a concrete Pyomo model."
"print(\" x = \", [pe.value(model.x[i]) for i in range(3)])\n",
"solver.set_instance(model)\n",
"print(\" y = \", [pe.value(model.y[i]) for i in range(3)])\n"
"solver.solve()\n",
"print(\"obj =\", model.obj())\n",
"print(\"x =\", [model.x[i].value for i in range(3)])\n",
"print(\"y =\", [model.y[i].value for i in range(3)])"
]
]
},
},
{
{
@ -295,29 +257,29 @@
"\n",
"\n",
"Although Gurobi could solve the small example above in a fraction of a second, it gets slower for larger and more complex versions of the problem. If this is a problem that needs to be solved frequently, as it is often the case in practice, it could make sense to spend some time upfront generating a **trained** version of Gurobi, which can solve new instances (similar to the ones it was trained on) faster.\n",
"Although Gurobi could solve the small example above in a fraction of a second, it gets slower for larger and more complex versions of the problem. If this is a problem that needs to be solved frequently, as it is often the case in practice, it could make sense to spend some time upfront generating a **trained** version of Gurobi, which can solve new instances (similar to the ones it was trained on) faster.\n",
"\n",
"\n",
"In the following, we will use MIPLearn to train machine learning models that can be used to accelerate Gurobi's performance on a particular set of instances. More specifically, MIPLearn will train a model that is able to predict the optimal solution for instances that follow a given probability distribution, then it will provide this predicted solution to Gurobi as a warm start.\n",
"In the following, we will use MIPLearn to train machine learning models that is able to predict the optimal solution for instances that follow a given probability distribution, then it will provide this predicted solution to Gurobi as a warm start. Before we can train the model, we need to collect training data by solving a large number of instances. In real-world situations, we may construct these training instances based on historical data. In this tutorial, we will construct them using a random instance generator:"
"\n",
"Before we can train the model, we need to collect training data by solving a large number of instances. In real-world situations, we may construct these training instances based on historical data. In this tutorial, we will construct them using a random instance generator:"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 42,
"execution_count": 6,
"id": "5eb09fab",
"id": "5eb09fab",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
"from scipy.stats import uniform\n",
"from scipy.stats import uniform\n",
"from typing import List\n",
"import random\n",
"import random\n",
"\n",
"\n",
"def random_uc_instances(samples, n, seed=42):\n",
"In this example, for simplicity, only the demands change from one instance to the next. We could also have made the prices and the production limits random. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
"In this example, for simplicity, only the demands change from one instance to the next. We could also have randomized the costs, production limits or even the number of units. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
"\n",
"\n",
"Now we generate 100 instances of this problem, each one with 1,000 generators. We will use the first 90 instances for training, and the remaining 10 instances to evaluate performance."
"Now we generate 500 instances of this problem, each one with 50 generators, and we use 450 of these instances for training. After generating the instances, we write them to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold in memory the entire training data, as well as the concrete Pyomo models. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial. The code below generates the files `uc/train/00000.pkl.gz`, `uc/train/00001.pkl.gz`, etc., which contain the input data in compressed (gzipped) pickle format."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 43,
"execution_count": 7,
"id": "6156752c",
"id": "6156752c",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [],
{
"name": "stdout",
"output_type": "stream",
"text": [
"Using license file /home/axavier/gurobi.lic\n",
"Using gurobi.env file\n",
"Set parameter Threads to value 1\n",
"Read LP format model from file /tmp/tmpl37dgdq_.pyomo.lp\n",
"Next, we write these objects to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete Pyomo models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.\n",
"Finally, we use `LearningSolver` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The optimal solutions, along with other useful training data, are stored in HDF5 files `uc/train/00000.h5`, `uc/train/00001.h5`, etc."
"\n",
"The code below generates the files `uc/train/00001.h5`, `uc/train/00002.h5`, etc., which contain the input data in HDF5 format."
"train_instances = [FileInstance(f) for f in glob(\"uc/train/*.h5\")]\n",
"test_instances = [FileInstance(f) for f in glob(\"uc/test/*.h5\")]"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "b17af877",
"id": "2f24ee83",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Finally, we use `LearningSolver` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The solutions, and other useful training data, are stored in the same HDF5 files."
"## Solving test instances\n",
"\n",
"With training data in hand, we can now fit the ML models, using the `LearningSolver.fit` method, then solve the test instances with `LearningSolver.solve`, as shown below. The `tee=True` parameter asks MIPLearn to print the solver log to the screen."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 8,
"execution_count": 9,
"id": "7623f002",
"id": "c8385030",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
"name": "stderr",
"name": "stdout",
"output_type": "stream",
"output_type": "stream",
"text": [
"text": [
" 12%|█▏ | 11/89 [00:50<05:56, 4.57s/it]\n"
"Set parameter LogFile to value \"/tmp/tmpvbaqbyty.log\"\n",
]
"Set parameter QCPDual to value 1\n",
},
"Gurobi Optimizer version 9.5.1 build v9.5.1rc2 (linux64)\n",
{
"Thread count: 16 physical cores, 32 logical processors, using up to 1 threads\n",
"ename": "KeyboardInterrupt",
"Optimize a model with 101 rows, 100 columns and 250 nonzeros\n",
"\u001b[0;32m~/.conda/envs/miplearn/lib/python3.8/site-packages/Pyomo-5.7.3-py3.8-linux-x86_64.egg/pyomo/core/base/component.py\u001b[0m in \u001b[0;36mname\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 285\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 286\u001b[0m \u001b[0;34m\"\"\"Get the fully qualifed component name.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 287\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetname\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfully_qualified\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 288\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 289\u001b[0m \u001b[0;31m# Adding a setter here to help users adapt to the new\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"By examining the solve log above, specifically the line `Loaded user MIP start with objective...`, we can see that MIPLearn was able to construct an initial solution which turned out to be the optimal solution to the problem. Now let us repeat the code above, but using an untrained solver. Note that the `fit` line is omitted."
"\n",
"With training data in hand, we can now fit the ML models, using the `LearningSolver.fit` method, then solve the test instances with `MIPLearn.solve!`, as shown below:"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": null,
"execution_count": 10,
"id": "c8385030",
"id": "33d15d6c-6db4-477f-bd4b-fe8e84e5f023",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [
"source": [
"solver_ml = LearningSolver()\n",
"solver_ml.fit(train_instances)\n",
"for instance in tqdm(test_instances):\n",
" solver_ml.solve(instance)"
]
},
{
{
"cell_type": "markdown",
"name": "stdout",
"id": "8ae11854",
"output_type": "stream",
"metadata": {},
"text": [
"source": [
"Set parameter LogFile to value \"/tmp/tmp3uhhdurw.log\"\n",
"The trained MIP solver was able to solve all test instances in about 6 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the `fit!` line:"
"Set parameter QCPDual to value 1\n",
"Gurobi Optimizer version 9.5.1 build v9.5.1rc2 (linux64)\n",
"Thread count: 16 physical cores, 32 logical processors, using up to 1 threads\n",
"Optimize a model with 101 rows, 100 columns and 250 nonzeros\n",
"Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances.\n",
"In the log above, the `MIP start` line is missing, and Gurobi had to start with a significantly inferior initial solution. The solver was still able to find the optimal solution at the end, but it required using its own internal heuristic procedures. In this example, because we solve very small optimization problems, there was almost no difference in terms of running time. For larger problems, however, the difference can be significant. See benchmarks for more details.\n",
"\n",
"<div class=\"alert alert-info\">\n",
"Note\n",
" \n",
"In addition to partial initial solutions, MIPLearn is also able to predict lazy constraints, cutting planes and branching priorities. See the next tutorials for more details.\n",
"</div>\n",
"\n",
"\n",
"<div class=\"alert alert-info\">\n",
"<div class=\"alert alert-info\">\n",
"Note\n",
"Note\n",
" \n",
" \n",
"Note that is is not necessary to specify what ML models to use. MIPLearn, by default, will try a number of classical ML models and will choose the one that performs the best, based on k-fold cross validation. MIPLearn is also able to automatically collect features based on the MIP formulation of the problem and the solution to the LP relaxation, among other things, so it does not require handcrafted features. If you do want to customize the models and features, however, that is also possible, as we will see in a later tutorial.\n",
"It is not necessary to specify what ML models to use. MIPLearn, by default, will try a number of classical ML models and will choose the one that performs the best, based on k-fold cross validation. MIPLearn is also able to automatically collect features based on the MIP formulation of the problem and the solution to the LP relaxation, among other things, so it does not require handcrafted features. If you do want to customize the models and features, however, that is also possible, as we will see in a later tutorial.\n",
"</div>"
"</div>"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "dec6acd7",
"id": "eec97f06",
"metadata": {},
"metadata": {
"source": [
"tags": []
"## Understanding the acceleration\n",
"\n",
"Let us go a bit deeper and try to understand how exactly did MIPLearn accelerate Gurobi's performance. First, we are going to solve one of the test instances again, using the trained solver, but this time using the `tee=True` parameter, so that we can see Gurobi's log:"
]
},
},
{
"cell_type": "code",
"execution_count": null,
"id": "0d4df336",
"metadata": {},
"outputs": [],
"source": [
"source": [
"solver_ml.solve(test_instances[0], tee=True);"
"## Accessing the solution\n",
]
},
{
"cell_type": "markdown",
"id": "df5108b9",
"metadata": {},
"source": [
"The log above is quite complicated if you have never seen it before, but the important line is the one starting with `feasible solution found [...] objective value 1.705169e+07`. This line indicates that MIPLearn was able to construct a warm start with value `1.705169e+07`. Using this warm start, SCIP then used the branch-and-cut method to either prove its optimality or to find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed optimal. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast. \n",
"\n",
"\n",
"Let us now repeat the process, but using the untrained solver this time:"
"In the example above, we used `LearningSolver.solve` together with data files to solve both the training and the test instances. The optimal solutions were saved to HDF5 files in the train/test folders, and could be retrieved by reading theses files, but that is not very convenient. In the following example, we show how to build and solve a Pyomo model entirely in-memory, using our trained solver."
"In this log file, notice how the previous line about warm starts is missing. Since no warm starts were provided, SCIP had to find an initial solution using its own internal heuristics, which are not specifically tailored for this problem. The initial solution found by SCIP's heuristics has value `2.335200e+07`, which is significantly worse than the one constructed by MIPLearn. SCIP then proceeded to improve this solution, by generating cutting planes and repeatedly running additional primal heuristics. In the end, it was able to find the optimal solution, as expected, but it took longer.\n",
" x = [1.0, 1.0, 1.0, 1.0, 1.0]\n",
"\n",
" y = [1105176.593734543, 1891284.5155055337, 1708177.4224033852, 1438329.610189608, 535496.3347187206]\n"
"In summary, MIPLearn accelerated the solution process by constructing a high-quality initial solution. In the following tutorials, we will see other strategies that MIPLearn can use to accelerate MIP performance, besides warm starts."
]
]
},
}
{
],
"cell_type": "markdown",
"id": "eec97f06",
"metadata": {},
"source": [
"source": [
"## Accessing the solution\n",
"# Construct model using previously defined functions\n",
"data = random_uc_data(samples=1, n=50)[0]\n",
"model = build_uc_model(data)\n",
"\n",
"\n",
"In the example above, we used `MIPLearn.solve!` together with data files to solve both the training and the test instances. The solutions were saved to a `.h5` files in the train/test folders, and could be retrieved by reading theses files, but that is not very convenient. In this section we will use an easier method.\n",
"# Solve model using ML + Gurobi\n",
"solver_ml.solve(model)\n",
"\n",
"\n",
"We can use the function `MIPLearn.load!` to obtain a regular JuMP model:"
"# Print part of the optimal solution\n",
]
"print(\"obj =\", model.obj())\n",
},
"print(\" x =\", [model.x[i].value for i in range(5)])\n",
{
"print(\" y =\", [model.y[i].value for i in range(5)])"
