"We use SCIP in this tutorial because it is a fast and widely available noncommercial MIP solver. All the steps shown here also work for Gurobi, CPLEX and XPRESS, although the performance impact might be different.\n",
"In this tutorial, we use SCIP because it is more widely available than commercial MIP solvers. However, all the steps below should work for Gurobi, CPLEX or XPRESS, as long as you have a license for these solvers. The performance impact of MIPLearn may also change for different solvers.\n",
" \n",
"</div>\n",
"</div>\n",
"\n",
"\n",
"<div class=\"alert alert-warning\">\n",
"<div class=\"alert alert-warning\">\n",
@ -33,10 +32,10 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "1f59417f",
"id": "8b97258c",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Installing MIPLearn\n",
"## Installation\n",
"\n",
"\n",
"MIPLearn is available in two versions:\n",
"MIPLearn is available in two versions:\n",
"\n",
"\n",
@ -49,14 +48,16 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 1,
"execution_count": 1,
"id": "1ddeeb8e",
"id": "2dbeacbc",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
"name": "stderr",
"name": "stderr",
"output_type": "stream",
"output_type": "stream",
"text": [
"text": [
"Path `/home/axavier/Packages/MIPLearn.jl/dev` exists and looks like the correct package. Using existing path.\n",
"To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the **unit commitment problem,** a practical optimization problem solved daily by electric grid operators around the world. \n",
"To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the **unit commitment problem,** a practical optimization problem solved daily by electric grid operators around the world. \n",
"\n",
"\n",
"Suppose that you work at a utility company, and that it is your job to decide which electrical generators should be online at a certain hour of the day, and how much power should each generator produce. More specifically, assume that your company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs your company $c^\\text{fixed}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing, and costs nothing. You also know that the total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts). To minimize the costs to your company, which generators should be online, and how much power should they produce?\n",
"Suppose that you work at a utility company, and that it is your job to decide which electrical generators should be online at a certain hour of the day, as well as how much power should each generator produce. More specifically, assume that your company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs your company $c^\\text{fix}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing and costs nothing. You also know that the total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts). To minimize the costs to your company, which generators should be online, and how much power should they produce?\n",
"\n",
"\n",
"This simple problem be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem we need to solve is given by:\n",
"This simple problem can be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem is then given by:\n",
"\n",
"\n",
"$$\n",
"$$\n",
"\\begin{align}\n",
"\\begin{align}\n",
@ -155,17 +150,17 @@
" \n",
" \n",
"Note\n",
"Note\n",
" \n",
" \n",
"We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem. See the benchmark sections for more details.\n",
"We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem. See benchmarks for more details.\n",
" \n",
" \n",
"</div>\n",
"</div>\n",
"\n",
"\n",
"Next, let us convert this abstract mathematical formulation into a concrete optimization model, using the Julia and the JuMP modeling language. We start by defining a data structure that holds all input data:"
"Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Julia and JuMP. We start by defining a data structure that holds all the input data."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 3,
"execution_count": 3,
"id": "ec7dbab4",
"id": "b12d6483",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -180,16 +175,16 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "c8f6a5b8",
"id": "55cdb64b",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Next, we create a function that converts this data into a concrete JuMP model:"
"Next, we create a function that converts this data structure into a concrete JuMP model. For more details on the JuMP syntax, see [the official JuMP documentation](https://jump.dev/JuMP.jl/stable/)."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 4,
"execution_count": 4,
"id": "14e84c92",
"id": "1e38a266",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -218,7 +213,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "f647734f",
"id": "d28c4d5a",
"metadata": {},
"metadata": {},
"source": [
"source": [
"At this point, we can already use JuMP and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators, using SCIP:"
"At this point, we can already use JuMP and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators, using SCIP:"
@ -227,7 +222,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 5,
"execution_count": 5,
"id": "b2abe5e2",
"id": "9ff9f05c",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -242,7 +237,6 @@
],
],
"source": [
"source": [
"using SCIP\n",
"using SCIP\n",
"using Printf\n",
"\n",
"\n",
"model = build_uc_model(\n",
"model = build_uc_model(\n",
" UnitCommitmentData(\n",
" UnitCommitmentData(\n",
@ -266,7 +260,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "5be976f5",
"id": "345de591",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
"Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
@ -274,7 +268,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "96a1f952",
"id": "eb8904ef",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Generating training data\n",
"## Generating training data\n",
@ -289,7 +283,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 6,
"execution_count": 6,
"id": "353e6199",
"id": "7298bb0d",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -317,7 +311,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "2140968d",
"id": "c1feed43",
"metadata": {},
"metadata": {},
"source": [
"source": [
"In this example, for simplicity, only the demands change from one instance to the next. We could also have made the prices and the production limits random. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
"In this example, for simplicity, only the demands change from one instance to the next. We could also have made the prices and the production limits random. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
@ -328,7 +322,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 7,
"execution_count": 7,
"id": "1bb24909",
"id": "61d43994",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -339,34 +333,33 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "96bc0e42",
"id": "3fdeb8cd",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Next, we will write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.\n",
"Next, we write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.\n",
"\n",
"\n",
"The code below generates the files `uc/train/000001.jld2`, `uc/train/000002.jld2`, etc."
"The code below generates the files `uc/train/000001.jld2`, `uc/train/000002.jld2`, etc., which contain the input data in [JLD2 format](https://github.com/JuliaIO/JLD2.jl)."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 8,
"execution_count": 8,
"id": "8ec476b1",
"id": "31b48701",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
"using MIPLearn\n",
"using MIPLearn\n",
"using Glob\n",
"\n",
"MIPLearn.save(data[1:90], \"uc/train/\")\n",
"MIPLearn.save(data[1:90], \"uc/train/\")\n",
"MIPLearn.save(data[91:100], \"uc/test/\")\n",
"MIPLearn.save(data[91:100], \"uc/test/\")\n",
"\n",
"\n",
"using Glob\n",
"train_files = glob(\"uc/train/*.jld2\")\n",
"train_files = glob(\"uc/train/*.jld2\")\n",
"test_files = glob(\"uc/test/*.jld2\");"
"test_files = glob(\"uc/test/*.jld2\");"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "5d53a783",
"id": "5cecea59",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Finally, we use `MIPLearn.LearningSolver` and `MIPLearn.solve!` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The `solve!` function can be used to solve either one or multiple instances, and requires: (i) the list of files containing the training data; and (ii) the function that converts the data structure into a concrete JuMP model:"
"Finally, we use `MIPLearn.LearningSolver` and `MIPLearn.solve!` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The `solve!` function can be used to solve either one or multiple instances, and requires: (i) the list of files containing the training data; and (ii) the function that converts the data structure into a concrete JuMP model:"
@ -375,14 +368,14 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 9,
"execution_count": 9,
"id": "514a3b3a",
"id": "60732af0",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
"name": "stdout",
"name": "stdout",
"output_type": "stream",
"output_type": "stream",
"text": [
"text": [
"101.279699 seconds (93.52 M allocations: 3.599 GiB, 1.23% gc time, 0.52% compilation time)\n"
"103.808547 seconds (93.52 M allocations: 3.604 GiB, 1.19% gc time, 0.52% compilation time)\n"
]
]
},
},
{
{
@ -401,33 +394,33 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "72eb09f4",
"id": "bbc7ad82",
"metadata": {},
"metadata": {},
"source": [
"source": [
"The macro `@time` shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, is stored by MIPLearn in `.h5` files, stored side-by-side with the original `.jld2` files."
"The macro `@time` shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, are stored by MIPLearn in `.h5` files, stored side-by-side with the original `.jld2` files."
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "90406b90",
"id": "73379180",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Solving new instances\n",
"## Solving new instances\n",
"\n",
"\n",
"Now that we have training data, we can fit the ML models using `MIPLearn.fit!`, then solve the test instances with `MIPLearn.solve!`, as shown below:"
"With training data in hand, we can now fit the ML models using `MIPLearn.fit!`, then solve the test instances with `MIPLearn.solve!`, as shown below:"
"The trained MIP solver was able to solve all test instances in about 5 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the `fit!` line:"
"The trained MIP solver was able to solve all test instances in about 6 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the `fit!` line:"
"Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances, or about twice as long.\n",
"Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances.\n",
"\n",
"\n",
"<div class=\"alert alert-info\">\n",
"<div class=\"alert alert-info\">\n",
"Note\n",
"Note\n",
@ -480,18 +473,18 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "569f7c7a",
"id": "af451e87",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Understanding the acceleration\n",
"## Understanding the acceleration\n",
"\n",
"\n",
"Let us know go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP's performance. First, we are going to solve one of the training instances again, using the trained solver, but this time using the `tee=true` parameter, so that we can see SCIP's log:"
"Let us go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP's performance. First, we are going to solve one of the test instances again, using the trained solver, but this time using the `tee=true` parameter, so that we can see SCIP's log:"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 12,
"execution_count": 12,
"id": "46739739",
"id": "0c675452",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -538,7 +531,7 @@
"presolved problem has 2000 variables (1000 bin, 0 int, 0 impl, 1000 cont) and 2001 constraints\n",
"presolved problem has 2000 variables (1000 bin, 0 int, 0 impl, 1000 cont) and 2001 constraints\n",
" 2000 constraints of type <varbound>\n",
" 2000 constraints of type <varbound>\n",
" 1 constraints of type <linear>\n",
" 1 constraints of type <linear>\n",
"Presolving Time: 0.10\n",
"Presolving Time: 0.11\n",
"transformed 1/1 original solutions to the transformed problem space\n",
"transformed 1/1 original solutions to the transformed problem space\n",
"\n",
"\n",
" time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl. \n",
" time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl. \n",
@ -560,18 +553,18 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "9cdc02d0",
"id": "ff0b6858",
"metadata": {},
"metadata": {},
"source": [
"source": [
"The log above is quite complicated if you have never seen it before, but the important line in the one starting with `feasible solution found [...] objective value 1.705169e+07`. This line indicates that MIPLearn was able to construct a warm start with value `1.705169e+07`. Using this warm start, SCIP then proceeded with the branch-and-cut process to either prove its optimality or find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed optimal and terminated. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast. \n",
"The log above is quite complicated if you have never seen it before, but the important line is the one starting with `feasible solution found [...] objective value 1.705169e+07`. This line indicates that MIPLearn was able to construct a warm start with value `1.705169e+07`. Using this warm start, SCIP then used the branch-and-cut method to either prove its optimality or to find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed optimal. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast. \n",
"\n",
"\n",
"Let us now do the same thing again, but using the untrained solver this time:"
"Let us now repeat the process, but using the untrained solver this time:"
"In this log file, notice how the line we saw before is now missing; SCIP needs to find an initial solution using its own internal heuristics. The solution SCIP initially found has value `2.335200e+07`, which is significantly worse than the one MIPLearn constructed before. SCIP then proceeds to improve this solution by generating a number of cutting planes and repeatedly running primal heuristics. In the end, it is able to find the optimal solution, as expected, but it takes longer."
"In this log file, notice how the previous line about warm starts is missing. Since no warm starts were provided, SCIP had to find an initial solution using its own internal heuristics, which are not specifically tailored for this problem. The initial solution found by SCIP's heuristics has value `2.335200e+07`, which is significantly worse than the one constructed by MIPLearn. SCIP then proceeded to improve this solution, by generating cutting planes and repeatedly running additional primal heuristics. In the end, it was able to find the optimal solution, as expected, but it took longer.\n",
"\n",
"In summary, MIPLearn accelerated the solution process by constructing a high-quality initial solution. In the following tutorials, we will see other strategies that MIPLearn can use to accelerate MIP performance, besides warm starts."
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "36fb5f02",
"id": "ab9c1ff4",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Accessing the solution\n",
"## Accessing the solution\n",
@ -678,7 +673,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 14,
"execution_count": 14,
"id": "f62f28b4",
"id": "79759e87",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -710,7 +705,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "d5722dcf",
"id": "b096dcf9",
"metadata": {},
"metadata": {},
"source": [
"source": [
"We can then solve this model as before, with `MIPLearn.solve!`:"
"We can then solve this model as before, with `MIPLearn.solve!`:"
"We use SCIP in this tutorial because it is a fast and widely available noncommercial MIP solver. All the steps shown here also work for Gurobi, CPLEX and XPRESS, although the performance impact might be different.\n",
"In this tutorial, we use SCIP because it is more widely available than commercial MIP solvers. However, all the steps below should work for Gurobi, CPLEX or XPRESS, as long as you have a license for these solvers. The performance impact of MIPLearn may also change for different solvers.\n",
" \n",
"</div>\n",
"</div>\n",
"\n",
"\n",
"<div class=\"alert alert-warning\">\n",
"<div class=\"alert alert-warning\">\n",
@ -33,10 +32,10 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "1f59417f",
"id": "8b97258c",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Installing MIPLearn\n",
"## Installation\n",
"\n",
"\n",
"MIPLearn is available in two versions:\n",
"MIPLearn is available in two versions:\n",
"\n",
"\n",
@ -49,14 +48,16 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 1,
"execution_count": 1,
"id": "1ddeeb8e",
"id": "2dbeacbc",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
"name": "stderr",
"name": "stderr",
"output_type": "stream",
"output_type": "stream",
"text": [
"text": [
"Path `/home/axavier/Packages/MIPLearn.jl/dev` exists and looks like the correct package. Using existing path.\n",
"To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the **unit commitment problem,** a practical optimization problem solved daily by electric grid operators around the world. \n",
"To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the **unit commitment problem,** a practical optimization problem solved daily by electric grid operators around the world. \n",
"\n",
"\n",
"Suppose that you work at a utility company, and that it is your job to decide which electrical generators should be online at a certain hour of the day, and how much power should each generator produce. More specifically, assume that your company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs your company $c^\\text{fixed}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing, and costs nothing. You also know that the total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts). To minimize the costs to your company, which generators should be online, and how much power should they produce?\n",
"Suppose that you work at a utility company, and that it is your job to decide which electrical generators should be online at a certain hour of the day, as well as how much power should each generator produce. More specifically, assume that your company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs your company $c^\\text{fix}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing and costs nothing. You also know that the total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts). To minimize the costs to your company, which generators should be online, and how much power should they produce?\n",
"\n",
"\n",
"This simple problem be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem we need to solve is given by:\n",
"This simple problem can be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem is then given by:\n",
"\n",
"\n",
"$$\n",
"$$\n",
"\\begin{align}\n",
"\\begin{align}\n",
@ -155,17 +150,17 @@
" \n",
" \n",
"Note\n",
"Note\n",
" \n",
" \n",
"We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem. See the benchmark sections for more details.\n",
"We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem. See benchmarks for more details.\n",
" \n",
" \n",
"</div>\n",
"</div>\n",
"\n",
"\n",
"Next, let us convert this abstract mathematical formulation into a concrete optimization model, using the Julia and the JuMP modeling language. We start by defining a data structure that holds all input data:"
"Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Julia and JuMP. We start by defining a data structure that holds all the input data."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 3,
"execution_count": 3,
"id": "ec7dbab4",
"id": "b12d6483",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -180,16 +175,16 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "c8f6a5b8",
"id": "55cdb64b",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Next, we create a function that converts this data into a concrete JuMP model:"
"Next, we create a function that converts this data structure into a concrete JuMP model. For more details on the JuMP syntax, see [the official JuMP documentation](https://jump.dev/JuMP.jl/stable/)."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 4,
"execution_count": 4,
"id": "14e84c92",
"id": "1e38a266",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -218,7 +213,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "f647734f",
"id": "d28c4d5a",
"metadata": {},
"metadata": {},
"source": [
"source": [
"At this point, we can already use JuMP and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators, using SCIP:"
"At this point, we can already use JuMP and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators, using SCIP:"
@ -227,7 +222,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 5,
"execution_count": 5,
"id": "b2abe5e2",
"id": "9ff9f05c",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -242,7 +237,6 @@
],
],
"source": [
"source": [
"using SCIP\n",
"using SCIP\n",
"using Printf\n",
"\n",
"\n",
"model = build_uc_model(\n",
"model = build_uc_model(\n",
" UnitCommitmentData(\n",
" UnitCommitmentData(\n",
@ -266,7 +260,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "5be976f5",
"id": "345de591",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
"Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
@ -274,7 +268,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "96a1f952",
"id": "eb8904ef",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Generating training data\n",
"## Generating training data\n",
@ -289,7 +283,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 6,
"execution_count": 6,
"id": "353e6199",
"id": "7298bb0d",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -317,7 +311,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "2140968d",
"id": "c1feed43",
"metadata": {},
"metadata": {},
"source": [
"source": [
"In this example, for simplicity, only the demands change from one instance to the next. We could also have made the prices and the production limits random. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
"In this example, for simplicity, only the demands change from one instance to the next. We could also have made the prices and the production limits random. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
@ -328,7 +322,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 7,
"execution_count": 7,
"id": "1bb24909",
"id": "61d43994",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
@ -339,34 +333,33 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "96bc0e42",
"id": "3fdeb8cd",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Next, we will write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.\n",
"Next, we write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.\n",
"\n",
"\n",
"The code below generates the files `uc/train/000001.jld2`, `uc/train/000002.jld2`, etc."
"The code below generates the files `uc/train/000001.jld2`, `uc/train/000002.jld2`, etc., which contain the input data in [JLD2 format](https://github.com/JuliaIO/JLD2.jl)."
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 8,
"execution_count": 8,
"id": "8ec476b1",
"id": "31b48701",
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
"using MIPLearn\n",
"using MIPLearn\n",
"using Glob\n",
"\n",
"MIPLearn.save(data[1:90], \"uc/train/\")\n",
"MIPLearn.save(data[1:90], \"uc/train/\")\n",
"MIPLearn.save(data[91:100], \"uc/test/\")\n",
"MIPLearn.save(data[91:100], \"uc/test/\")\n",
"\n",
"\n",
"using Glob\n",
"train_files = glob(\"uc/train/*.jld2\")\n",
"train_files = glob(\"uc/train/*.jld2\")\n",
"test_files = glob(\"uc/test/*.jld2\");"
"test_files = glob(\"uc/test/*.jld2\");"
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "5d53a783",
"id": "5cecea59",
"metadata": {},
"metadata": {},
"source": [
"source": [
"Finally, we use `MIPLearn.LearningSolver` and `MIPLearn.solve!` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The `solve!` function can be used to solve either one or multiple instances, and requires: (i) the list of files containing the training data; and (ii) the function that converts the data structure into a concrete JuMP model:"
"Finally, we use `MIPLearn.LearningSolver` and `MIPLearn.solve!` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The `solve!` function can be used to solve either one or multiple instances, and requires: (i) the list of files containing the training data; and (ii) the function that converts the data structure into a concrete JuMP model:"
@ -375,14 +368,14 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 9,
"execution_count": 9,
"id": "514a3b3a",
"id": "60732af0",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
"name": "stdout",
"name": "stdout",
"output_type": "stream",
"output_type": "stream",
"text": [
"text": [
"101.279699 seconds (93.52 M allocations: 3.599 GiB, 1.23% gc time, 0.52% compilation time)\n"
"103.808547 seconds (93.52 M allocations: 3.604 GiB, 1.19% gc time, 0.52% compilation time)\n"
]
]
},
},
{
{
@ -401,33 +394,33 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "72eb09f4",
"id": "bbc7ad82",
"metadata": {},
"metadata": {},
"source": [
"source": [
"The macro `@time` shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, is stored by MIPLearn in `.h5` files, stored side-by-side with the original `.jld2` files."
"The macro `@time` shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, are stored by MIPLearn in `.h5` files, stored side-by-side with the original `.jld2` files."
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "90406b90",
"id": "73379180",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Solving new instances\n",
"## Solving new instances\n",
"\n",
"\n",
"Now that we have training data, we can fit the ML models using `MIPLearn.fit!`, then solve the test instances with `MIPLearn.solve!`, as shown below:"
"With training data in hand, we can now fit the ML models using `MIPLearn.fit!`, then solve the test instances with `MIPLearn.solve!`, as shown below:"
"The trained MIP solver was able to solve all test instances in about 5 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the `fit!` line:"
"The trained MIP solver was able to solve all test instances in about 6 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the `fit!` line:"
"Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances, or about twice as long.\n",
"Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances.\n",
"\n",
"\n",
"<div class=\"alert alert-info\">\n",
"<div class=\"alert alert-info\">\n",
"Note\n",
"Note\n",
@ -480,18 +473,18 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "569f7c7a",
"id": "af451e87",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Understanding the acceleration\n",
"## Understanding the acceleration\n",
"\n",
"\n",
"Let us know go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP's performance. First, we are going to solve one of the training instances again, using the trained solver, but this time using the `tee=true` parameter, so that we can see SCIP's log:"
"Let us go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP's performance. First, we are going to solve one of the test instances again, using the trained solver, but this time using the `tee=true` parameter, so that we can see SCIP's log:"
]
]
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 12,
"execution_count": 12,
"id": "46739739",
"id": "0c675452",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -538,7 +531,7 @@
"presolved problem has 2000 variables (1000 bin, 0 int, 0 impl, 1000 cont) and 2001 constraints\n",
"presolved problem has 2000 variables (1000 bin, 0 int, 0 impl, 1000 cont) and 2001 constraints\n",
" 2000 constraints of type <varbound>\n",
" 2000 constraints of type <varbound>\n",
" 1 constraints of type <linear>\n",
" 1 constraints of type <linear>\n",
"Presolving Time: 0.10\n",
"Presolving Time: 0.11\n",
"transformed 1/1 original solutions to the transformed problem space\n",
"transformed 1/1 original solutions to the transformed problem space\n",
"\n",
"\n",
" time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl. \n",
" time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl. \n",
@ -560,18 +553,18 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "9cdc02d0",
"id": "ff0b6858",
"metadata": {},
"metadata": {},
"source": [
"source": [
"The log above is quite complicated if you have never seen it before, but the important line in the one starting with `feasible solution found [...] objective value 1.705169e+07`. This line indicates that MIPLearn was able to construct a warm start with value `1.705169e+07`. Using this warm start, SCIP then proceeded with the branch-and-cut process to either prove its optimality or find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed optimal and terminated. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast. \n",
"The log above is quite complicated if you have never seen it before, but the important line is the one starting with `feasible solution found [...] objective value 1.705169e+07`. This line indicates that MIPLearn was able to construct a warm start with value `1.705169e+07`. Using this warm start, SCIP then used the branch-and-cut method to either prove its optimality or to find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed optimal. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast. \n",
"\n",
"\n",
"Let us now do the same thing again, but using the untrained solver this time:"
"Let us now repeat the process, but using the untrained solver this time:"
"In this log file, notice how the line we saw before is now missing; SCIP needs to find an initial solution using its own internal heuristics. The solution SCIP initially found has value `2.335200e+07`, which is significantly worse than the one MIPLearn constructed before. SCIP then proceeds to improve this solution by generating a number of cutting planes and repeatedly running primal heuristics. In the end, it is able to find the optimal solution, as expected, but it takes longer."
"In this log file, notice how the previous line about warm starts is missing. Since no warm starts were provided, SCIP had to find an initial solution using its own internal heuristics, which are not specifically tailored for this problem. The initial solution found by SCIP's heuristics has value `2.335200e+07`, which is significantly worse than the one constructed by MIPLearn. SCIP then proceeded to improve this solution, by generating cutting planes and repeatedly running additional primal heuristics. In the end, it was able to find the optimal solution, as expected, but it took longer.\n",
"\n",
"In summary, MIPLearn accelerated the solution process by constructing a high-quality initial solution. In the following tutorials, we will see other strategies that MIPLearn can use to accelerate MIP performance, besides warm starts."
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "36fb5f02",
"id": "ab9c1ff4",
"metadata": {},
"metadata": {},
"source": [
"source": [
"## Accessing the solution\n",
"## Accessing the solution\n",
@ -678,7 +673,7 @@
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 14,
"execution_count": 14,
"id": "f62f28b4",
"id": "79759e87",
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [
{
{
@ -710,7 +705,7 @@
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"id": "d5722dcf",
"id": "b096dcf9",
"metadata": {},
"metadata": {},
"source": [
"source": [
"We can then solve this model as before, with `MIPLearn.solve!`:"
"We can then solve this model as before, with `MIPLearn.solve!`:"
<p>We use SCIP in this tutorial because it is a fast and widely available noncommercial MIP solver. All the steps shown here also work for Gurobi, CPLEX and XPRESS, although the performance impact might be different.</p>
<p>In this tutorial, we use SCIP because it is more widely available than commercial MIP solvers. However, all the steps below should work for Gurobi, CPLEX or XPRESS, as long as you have a license for these solvers. The performance impact of MIPLearn may also change for different solvers.</p>
</div>
</div>
<divclass="admonition warning">
<divclass="admonition warning">
<pclass="admonition-title">Warning</p>
<pclass="admonition-title">Warning</p>
<p>MIPLearn is still in early development stage. If run into any bugs or issues, please submit a bug report in our GitHub repository. Comments, suggestions and pull requests are also very welcome!</p>
<p>MIPLearn is still in early development stage. If run into any bugs or issues, please submit a bug report in our GitHub repository. Comments, suggestions and pull requests are also very welcome!</p>
</div>
</div>
</div>
</div>
<divclass="section"id="Installing-MIPLearn">
<divclass="section"id="Installation">
<h2><spanclass="section-number">1.2. </span>Installing MIPLearn<aclass="headerlink"href="#Installing-MIPLearn" title="Permalink to this headline">¶</a></h2>
<h2><spanclass="section-number">1.2. </span>Installation<aclass="headerlink"href="#Installation" title="Permalink to this headline">¶</a></h2>
<p>MIPLearn is available in two versions:</p>
<p>MIPLearn is available in two versions:</p>
<ulclass="simple">
<ulclass="simple">
<li><p>Python version, compatible with the Pyomo modeling language,</p></li>
<li><p>Python version, compatible with the Pyomo modeling language,</p></li>
<spanclass="ansi-green-intense-fg ansi-bold"> No Changes</span> to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Project.toml`
<spanclass="ansi-green-intense-fg ansi-bold"> No Changes</span> to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Project.toml`
<spanclass="ansi-green-intense-fg ansi-bold"> No Changes</span> to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Manifest.toml`
<spanclass="ansi-green-intense-fg ansi-bold"> No Changes</span> to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Manifest.toml`
@ -517,10 +519,10 @@ Path `/home/axavier/Packages/MIPLearn.jl/dev` exists and looks like the correct
</div>
</div>
<p>In addition to MIPLearn itself, we will also install a few other packages that are required for this tutorial:</p>
<p>In addition to MIPLearn itself, we will also install a few other packages that are required for this tutorial:</p>
<ulclass="simple">
<ulclass="simple">
<li><p><code class="docutils literal notranslate"><spanclass="pre">SCIP</span></code>, a non-commercial mixed-integer programming solver</p></li>
<li><p><a class="reference external"href="https://www.scipopt.org/">SCIP</a>, one of the fastest non-commercial MIP solvers currently available</p></li>
<li><p><code class="docutils literal notranslate"><spanclass="pre">JuMP</span></code>, an open-source modeling language for Julia</p></li>
<li><p><a class="reference external"href="https://jump.dev/">JuMP</a>, an open source modeling language for Julia</p></li>
<li><p><code class="docutils literal notranslate"><spanclass="pre">Distributions</span></code>, a statistics package that we will use to generate random inputs</p></li>
<li><p><a class="reference external"href="https://github.com/JuliaStats/Distributions.jl">Distributions.jl</a>, a statistics package that we will use to generate random inputs</p></li>
<li><p><code class="docutils literal notranslate"><spanclass="pre">Glob</span></code>, a package that retrieves all files in a directory matching a certain pattern</p></li>
<li><p><a class="reference external"href="https://github.com/vtjnash/Glob.jl">Glob.jl</a>, a package that retrieves all files in a directory matching a certain pattern</p></li>
<h2><spanclass="section-number">1.3. </span>Modeling a simple optimization problem<aclass="headerlink"href="#Modeling-a-simple-optimization-problem"title="Permalink to this headline">¶</a></h2>
<h2><spanclass="section-number">1.3. </span>Modeling a simple optimization problem<aclass="headerlink"href="#Modeling-a-simple-optimization-problem"title="Permalink to this headline">¶</a></h2>
<p>To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the <strong>unit commitment problem,</strong> a practical optimization problem solved daily by electric grid operators around the world.</p>
<p>To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the <strong>unit commitment problem,</strong> a practical optimization problem solved daily by electric grid operators around the world.</p>
<p>Suppose that you work at a utility company, and that it is your job to decide which electrical generators should be online at a certain hour of the day, and how much power should each generator produce. More specifically, assume that your company owns <spanclass="math notranslate nohighlight">\(n\)</span> generators, denoted by <spanclass="math notranslate nohighlight">\(g_1, \ldots, g_n\)</span>. Each generator can either be online or offline. An online generator <spanclass="math notranslate nohighlight">\(g_i\)</span> can produce between <spanclass="math notranslate nohighlight">\(p^\text{min}_i\)</span> to <spanclass="math notranslate nohighlight">\(p^\text{max}_i\)</span> megawatts of power, and it costs your
<p>Suppose that you work at a utility company, and that it is your job to decide which electrical generators should be online at a certain hour of the day, as well as how much power should each generator produce. More specifically, assume that your company owns <spanclass="math notranslate nohighlight">\(n\)</span> generators, denoted by <spanclass="math notranslate nohighlight">\(g_1, \ldots, g_n\)</span>. Each generator can either be online or offline. An online generator <spanclass="math notranslate nohighlight">\(g_i\)</span> can produce between <spanclass="math notranslate nohighlight">\(p^\text{min}_i\)</span> to <spanclass="math notranslate nohighlight">\(p^\text{max}_i\)</span> megawatts of power, and it costs
company <spanclass="math notranslate nohighlight">\(c^\text{fixed}_i + c^\text{var}_i y_i\)</span>, where <spanclass="math notranslate nohighlight">\(y_i\)</span> is the amount of power produced. An offline generator produces nothing, and costs nothing. You also know that the total amount of power to be produced needs to be exactly equal to the total demand <spanclass="math notranslate nohighlight">\(d\)</span> (in megawatts). To minimize the costs to your company, which generators should be online, and how much power should they produce?</p>
your company <spanclass="math notranslate nohighlight">\(c^\text{fix}_i + c^\text{var}_i y_i\)</span>, where <spanclass="math notranslate nohighlight">\(y_i\)</span> is the amount of power produced. An offline generator produces nothing and costs nothing. You also know that the total amount of power to be produced needs to be exactly equal to the total demand <spanclass="math notranslate nohighlight">\(d\)</span> (in megawatts). To minimize the costs to your company, which generators should be online, and how much power should they produce?</p>
<p>This simple problem be modeled as a <em>mixed-integer linear optimization</em> problem as follows. For each generator <spanclass="math notranslate nohighlight">\(g_i\)</span>, let <spanclass="math notranslate nohighlight">\(x_i \in \{0,1\}\)</span> be a decision variable indicating whether <spanclass="math notranslate nohighlight">\(g_i\)</span> is online, and let <spanclass="math notranslate nohighlight">\(y_i \geq 0\)</span> be a decision variable indicating how much power does <spanclass="math notranslate nohighlight">\(g_i\)</span> produce. The problem we need to solve is given by:</p>
<p>This simple problem can be modeled as a <em>mixed-integer linear optimization</em> problem as follows. For each generator <spanclass="math notranslate nohighlight">\(g_i\)</span>, let <spanclass="math notranslate nohighlight">\(x_i \in \{0,1\}\)</span> be a decision variable indicating whether <spanclass="math notranslate nohighlight">\(g_i\)</span> is online, and let <spanclass="math notranslate nohighlight">\(y_i \geq 0\)</span> be a decision variable indicating how much power does <spanclass="math notranslate nohighlight">\(g_i\)</span> produce. The problem is then given by:</p>
<p>We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem. See the benchmark sections for more details.</p>
<p>We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem. See benchmarks for more details.</p>
</div>
</div>
<p>Next, let us convert this abstract mathematical formulation into a concrete optimization model, using the Julia and the JuMP modeling language. We start by defining a data structure that holds all input data:</p>
<p>Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Julia and JuMP. We start by defining a data structure that holds all the input data.</p>
<p>Next, we create a function that converts this data into a concrete JuMP model:</p>
<p>Next, we create a function that converts this data structure into a concrete JuMP model. For more details on the JuMP syntax, see <aclass="reference external"href="https://jump.dev/JuMP.jl/stable/">the official JuMP documentation</a>.</p>
<p>Next, we will write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.</p>
<p>Next, we write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.</p>
<p>The code below generates the files <codeclass="docutils literal notranslate"><spanclass="pre">uc/train/000001.jld2</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">uc/train/000002.jld2</span></code>, etc., which contain the input data in <aclass="reference external"href="https://github.com/JuliaIO/JLD2.jl">JLD2 format</a>.</p>
101.279699 seconds (93.52 M allocations: 3.599 GiB, 1.23% gc time, 0.52% compilation time)
103.808547 seconds (93.52 M allocations: 3.604 GiB, 1.19% gc time, 0.52% compilation time)
</pre></div></div>
</pre></div></div>
</div>
</div>
<divclass="nboutput nblast docutils container">
<divclass="nboutput nblast docutils container">
@ -757,11 +751,11 @@ obj = 1320.0
WARNING: Dual bound 1.98665e+07 is larger than the objective of the primal solution 1.98665e+07. The solution might not be optimal.
WARNING: Dual bound 1.98665e+07 is larger than the objective of the primal solution 1.98665e+07. The solution might not be optimal.
</pre></div></div>
</pre></div></div>
</div>
</div>
<p>The macro <codeclass="docutils literal notranslate"><spanclass="pre">@time</span></code> shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, is stored by MIPLearn in <codeclass="docutils literal notranslate"><spanclass="pre">.h5</span></code> files, stored side-by-side with the original <codeclass="docutils literal notranslate"><spanclass="pre">.jld2</span></code> files.</p>
<p>The macro <codeclass="docutils literal notranslate"><spanclass="pre">@time</span></code> shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, are stored by MIPLearn in <codeclass="docutils literal notranslate"><spanclass="pre">.h5</span></code> files, stored side-by-side with the original <codeclass="docutils literal notranslate"><spanclass="pre">.jld2</span></code> files.</p>
</div>
</div>
<divclass="section"id="Solving-new-instances">
<divclass="section"id="Solving-new-instances">
<h2><spanclass="section-number">1.5. </span>Solving new instances<aclass="headerlink"href="#Solving-new-instances"title="Permalink to this headline">¶</a></h2>
<h2><spanclass="section-number">1.5. </span>Solving new instances<aclass="headerlink"href="#Solving-new-instances"title="Permalink to this headline">¶</a></h2>
<p>Now that we have training data, we can fit the ML models using <codeclass="docutils literal notranslate"><spanclass="pre">MIPLearn.fit!</span></code>, then solve the test instances with <codeclass="docutils literal notranslate"><spanclass="pre">MIPLearn.solve!</span></code>, as shown below:</p>
<p>With training data in hand, we can now fit the ML models using <codeclass="docutils literal notranslate"><spanclass="pre">MIPLearn.fit!</span></code>, then solve the test instances with <codeclass="docutils literal notranslate"><spanclass="pre">MIPLearn.solve!</span></code>, as shown below:</p>
@ -778,10 +772,10 @@ WARNING: Dual bound 1.98665e+07 is larger than the objective of the primal solut
</div>
</div>
<divclass="output_area docutils container">
<divclass="output_area docutils container">
<divclass="highlight"><pre>
<divclass="highlight"><pre>
5.693951 seconds (9.33 M allocations: 334.689 MiB, 1.62% gc time)
5.951264 seconds (9.33 M allocations: 334.657 MiB, 1.51% gc time)
</pre></div></div>
</pre></div></div>
</div>
</div>
<p>The trained MIP solver was able to solve all test instances in about 5 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the <codeclass="docutils literal notranslate"><spanclass="pre">fit!</span></code> line:</p>
<p>The trained MIP solver was able to solve all test instances in about 6 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the <codeclass="docutils literal notranslate"><spanclass="pre">fit!</span></code> line:</p>
@ -797,10 +791,10 @@ WARNING: Dual bound 1.98665e+07 is larger than the objective of the primal solut
</div>
</div>
<divclass="output_area docutils container">
<divclass="output_area docutils container">
<divclass="highlight"><pre>
<divclass="highlight"><pre>
9.829350 seconds (8.17 M allocations: 278.008 MiB, 0.47% gc time)
10.390325 seconds (8.17 M allocations: 278.042 MiB, 0.89% gc time)
</pre></div></div>
</pre></div></div>
</div>
</div>
<p>Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances, or about twice as long.</p>
<p>Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances.</p>
<divclass="admonition note">
<divclass="admonition note">
<pclass="admonition-title">Note</p>
<pclass="admonition-title">Note</p>
<p>Note that is is not necessary to specify what ML models to use. MIPLearn, by default, will try a number of classical ML models and will choose the one that performs the best, based on k-fold cross validation. MIPLearn is also able to automatically collect features based on the MIP formulation of the problem and the solution to the LP relaxation, among other things, so it does not require handcrafted features. If you do want to customize the models and features, however, that is also possible, as
<p>Note that is is not necessary to specify what ML models to use. MIPLearn, by default, will try a number of classical ML models and will choose the one that performs the best, based on k-fold cross validation. MIPLearn is also able to automatically collect features based on the MIP formulation of the problem and the solution to the LP relaxation, among other things, so it does not require handcrafted features. If you do want to customize the models and features, however, that is also possible, as
@ -809,7 +803,7 @@ we will see in a later tutorial.</p>
<h2><spanclass="section-number">1.6. </span>Understanding the acceleration<aclass="headerlink"href="#Understanding-the-acceleration"title="Permalink to this headline">¶</a></h2>
<h2><spanclass="section-number">1.6. </span>Understanding the acceleration<aclass="headerlink"href="#Understanding-the-acceleration"title="Permalink to this headline">¶</a></h2>
<p>Let us know go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP’s performance. First, we are going to solve one of the training instances again, using the trained solver, but this time using the <codeclass="docutils literal notranslate"><spanclass="pre">tee=true</span></code> parameter, so that we can see SCIP’s log:</p>
<p>Let us go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP’s performance. First, we are going to solve one of the test instances again, using the trained solver, but this time using the <codeclass="docutils literal notranslate"><spanclass="pre">tee=true</span></code> parameter, so that we can see SCIP’s log:</p>
presolved problem has 2000 variables (1000 bin, 0 int, 0 impl, 1000 cont) and 2001 constraints
presolved problem has 2000 variables (1000 bin, 0 int, 0 impl, 1000 cont) and 2001 constraints
2000 constraints of type <varbound>
2000 constraints of type <varbound>
1 constraints of type <linear>
1 constraints of type <linear>
Presolving Time: 0.10
Presolving Time: 0.11
transformed 1/1 original solutions to the transformed problem space
transformed 1/1 original solutions to the transformed problem space
time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl.
time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl.
@ -879,9 +873,9 @@ Gap : 0.01 %
</pre></div></div>
</pre></div></div>
</div>
</div>
<p>The log above is quite complicated if you have never seen it before, but the important line in the one starting with <codeclass="docutils literal notranslate"><spanclass="pre">feasible</span><spanclass="pre">solution</span><spanclass="pre">found</span><spanclass="pre">[...]</span><spanclass="pre">objective</span><spanclass="pre">value</span><spanclass="pre">1.705169e+07</span></code>. This line indicates that MIPLearn was able to construct a warm start with value <codeclass="docutils literal notranslate"><spanclass="pre">1.705169e+07</span></code>. Using this warm start, SCIP then proceeded with the branch-and-cut process to either prove its optimality or find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was
<p>The log above is quite complicated if you have never seen it before, but the important line is the one starting with <codeclass="docutils literal notranslate"><spanclass="pre">feasible</span><spanclass="pre">solution</span><spanclass="pre">found</span><spanclass="pre">[...]</span><spanclass="pre">objective</span><spanclass="pre">value</span><spanclass="pre">1.705169e+07</span></code>. This line indicates that MIPLearn was able to construct a warm start with value <codeclass="docutils literal notranslate"><spanclass="pre">1.705169e+07</span></code>. Using this warm start, SCIP then used the branch-and-cut method to either prove its optimality or to find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed
indeed optimal and terminated. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast.</p>
optimal. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast.</p>
<p>Let us now do the same thing again, but using the untrained solver this time:</p>
<p>Let us now repeat the process, but using the untrained solver this time:</p>
<p>In this log file, notice how the line we saw before is now missing; SCIP needs to find an initial solution using its own internal heuristics. The solution SCIP initially found has value <codeclass="docutils literal notranslate"><spanclass="pre">2.335200e+07</span></code>, which is significantly worse than the one MIPLearn constructed before. SCIP then proceeds to improve this solution by generating a number of cutting planes and repeatedly running primal heuristics. In the end, it is able to find the optimal solution, as expected, but it takes longer.</p>
<p>In this log file, notice how the previous line about warm starts is missing. Since no warm starts were provided, SCIP had to find an initial solution using its own internal heuristics, which are not specifically tailored for this problem. The initial solution found by SCIP’s heuristics has value <codeclass="docutils literal notranslate"><spanclass="pre">2.335200e+07</span></code>, which is significantly worse than the one constructed by MIPLearn. SCIP then proceeded to improve this solution, by generating cutting planes and repeatedly running additional primal
heuristics. In the end, it was able to find the optimal solution, as expected, but it took longer.</p>
<p>In summary, MIPLearn accelerated the solution process by constructing a high-quality initial solution. In the following tutorials, we will see other strategies that MIPLearn can use to accelerate MIP performance, besides warm starts.</p>
</div>
</div>
<divclass="section"id="Accessing-the-solution">
<divclass="section"id="Accessing-the-solution">
<h2><spanclass="section-number">1.7. </span>Accessing the solution<aclass="headerlink"href="#Accessing-the-solution"title="Permalink to this headline">¶</a></h2>
<h2><spanclass="section-number">1.7. </span>Accessing the solution<aclass="headerlink"href="#Accessing-the-solution"title="Permalink to this headline">¶</a></h2>
