\n",
"Note\n",
" \n",
- "We use SCIP in this tutorial because it is a fast and widely available noncommercial MIP solver. All the steps shown here also work for Gurobi, CPLEX and XPRESS, although the performance impact might be different.\n",
- " \n",
+ "In this tutorial, we use SCIP because it is more widely available than commercial MIP solvers. However, all the steps below should work for Gurobi, CPLEX or XPRESS, as long as you have a license for these solvers. The performance impact of MIPLearn may also change for different solvers.\n",
"
\n",
"\n",
"\n",
@@ -33,10 +32,10 @@
},
{
"cell_type": "markdown",
- "id": "1f59417f",
+ "id": "8b97258c",
"metadata": {},
"source": [
- "## Installing MIPLearn\n",
+ "## Installation\n",
"\n",
"MIPLearn is available in two versions:\n",
"\n",
@@ -49,14 +48,16 @@
{
"cell_type": "code",
"execution_count": 1,
- "id": "1ddeeb8e",
+ "id": "2dbeacbc",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Path `/home/axavier/Packages/MIPLearn.jl/dev` exists and looks like the correct package. Using existing path.\n",
+ "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m git-repo `https://github.com/ANL-CEEESA/MIPLearn.jl.git`\n",
+ "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m registry at `~/.julia/registries/General`\n",
+ "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m git-repo `https://github.com/JuliaRegistries/General.git`\n",
"\u001b[32m\u001b[1m Resolving\u001b[22m\u001b[39m package versions...\n",
"\u001b[32m\u001b[1m No Changes\u001b[22m\u001b[39m to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Project.toml`\n",
"\u001b[32m\u001b[1m No Changes\u001b[22m\u001b[39m to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Manifest.toml`\n"
@@ -65,40 +66,35 @@
],
"source": [
"using Pkg\n",
- "Pkg.develop(PackageSpec(path=\"/home/axavier/Packages/MIPLearn.jl/dev\"))"
+ "Pkg.add(PackageSpec(url=\"https://github.com/ANL-CEEESA/MIPLearn.jl.git\"))"
]
},
{
"cell_type": "markdown",
- "id": "de7ab489",
+ "id": "b2f449e7",
"metadata": {},
"source": [
"In addition to MIPLearn itself, we will also install a few other packages that are required for this tutorial:\n",
"\n",
- "- `SCIP`, a non-commercial mixed-integer programming solver\n",
- "- `JuMP`, an open-source modeling language for Julia\n",
- "- `Distributions`, a statistics package that we will use to generate random inputs\n",
- "- `Glob`, a package that retrieves all files in a directory matching a certain pattern"
+ "- [**SCIP**](https://www.scipopt.org/), one of the fastest non-commercial MIP solvers currently available\n",
+ "- [**JuMP**](https://jump.dev/), an open source modeling language for Julia\n",
+ "- [**Distributions.jl**](https://github.com/JuliaStats/Distributions.jl), a statistics package that we will use to generate random inputs\n",
+ "- [**Glob.jl**](https://github.com/vtjnash/Glob.jl), a package that retrieves all files in a directory matching a certain pattern"
]
},
{
"cell_type": "code",
"execution_count": 2,
- "id": "29d29925",
+ "id": "68f99568",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m registry at `~/.julia/registries/General`\n",
- "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m git-repo `https://github.com/JuliaRegistries/General.git`\n",
"\u001b[32m\u001b[1m Resolving\u001b[22m\u001b[39m package versions...\n",
"\u001b[32m\u001b[1m No Changes\u001b[22m\u001b[39m to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Project.toml`\n",
- "\u001b[32m\u001b[1m No Changes\u001b[22m\u001b[39m to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Manifest.toml`\n",
- "\u001b[32m\u001b[1mPrecompiling\u001b[22m\u001b[39m project...\n",
- "\u001b[32m ✓ \u001b[39mMIPLearn\n",
- "1 dependency successfully precompiled in 10 seconds (96 already precompiled)\n"
+ "\u001b[32m\u001b[1m No Changes\u001b[22m\u001b[39m to `~/Packages/MIPLearn/dev/docs/jump-tutorials/Manifest.toml`\n"
]
}
],
@@ -109,13 +105,12 @@
" PackageSpec(name=\"JuMP\", version=\"0.21\"),\n",
" PackageSpec(name=\"Distributions\", version=\"0.25\"),\n",
" PackageSpec(name=\"Glob\", version=\"1\"),\n",
- "])\n",
- "using Revise"
+ "])"
]
},
{
"cell_type": "markdown",
- "id": "88074d87",
+ "id": "51e09fc9",
"metadata": {},
"source": [
"
\n",
@@ -129,16 +124,16 @@
},
{
"cell_type": "markdown",
- "id": "78482747",
+ "id": "18c300c4",
"metadata": {},
"source": [
"## Modeling a simple optimization problem\n",
"\n",
"To illustrate how can MIPLearn be used, we will model and solve a small optimization problem related to power systems optimization. The problem we discuss below is a simplification of the **unit commitment problem,** a practical optimization problem solved daily by electric grid operators around the world. \n",
"\n",
- "Suppose that you work at a utility company, and that it is your job to decide which electrical generators should be online at a certain hour of the day, and how much power should each generator produce. More specifically, assume that your company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs your company $c^\\text{fixed}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing, and costs nothing. You also know that the total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts). To minimize the costs to your company, which generators should be online, and how much power should they produce?\n",
+ "Suppose that you work at a utility company, and that it is your job to decide which electrical generators should be online at a certain hour of the day, as well as how much power should each generator produce. More specifically, assume that your company owns $n$ generators, denoted by $g_1, \\ldots, g_n$. Each generator can either be online or offline. An online generator $g_i$ can produce between $p^\\text{min}_i$ to $p^\\text{max}_i$ megawatts of power, and it costs your company $c^\\text{fix}_i + c^\\text{var}_i y_i$, where $y_i$ is the amount of power produced. An offline generator produces nothing and costs nothing. You also know that the total amount of power to be produced needs to be exactly equal to the total demand $d$ (in megawatts). To minimize the costs to your company, which generators should be online, and how much power should they produce?\n",
"\n",
- "This simple problem be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem we need to solve is given by:\n",
+ "This simple problem can be modeled as a *mixed-integer linear optimization* problem as follows. For each generator $g_i$, let $x_i \\in \\{0,1\\}$ be a decision variable indicating whether $g_i$ is online, and let $y_i \\geq 0$ be a decision variable indicating how much power does $g_i$ produce. The problem is then given by:\n",
"\n",
"$$\n",
"\\begin{align}\n",
@@ -155,17 +150,17 @@
" \n",
"Note\n",
" \n",
- "We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem. See the benchmark sections for more details.\n",
+ "We use a simplified version of the unit commitment problem in this tutorial just to make it easier to follow. MIPLearn can also handle realistic, large-scale versions of this problem. See benchmarks for more details.\n",
" \n",
"
\n",
"\n",
- "Next, let us convert this abstract mathematical formulation into a concrete optimization model, using the Julia and the JuMP modeling language. We start by defining a data structure that holds all input data:"
+ "Next, let us convert this abstract mathematical formulation into a concrete optimization model, using Julia and JuMP. We start by defining a data structure that holds all the input data."
]
},
{
"cell_type": "code",
"execution_count": 3,
- "id": "ec7dbab4",
+ "id": "b12d6483",
"metadata": {},
"outputs": [],
"source": [
@@ -180,16 +175,16 @@
},
{
"cell_type": "markdown",
- "id": "c8f6a5b8",
+ "id": "55cdb64b",
"metadata": {},
"source": [
- "Next, we create a function that converts this data into a concrete JuMP model:"
+ "Next, we create a function that converts this data structure into a concrete JuMP model. For more details on the JuMP syntax, see [the official JuMP documentation](https://jump.dev/JuMP.jl/stable/)."
]
},
{
"cell_type": "code",
"execution_count": 4,
- "id": "14e84c92",
+ "id": "1e38a266",
"metadata": {},
"outputs": [],
"source": [
@@ -218,7 +213,7 @@
},
{
"cell_type": "markdown",
- "id": "f647734f",
+ "id": "d28c4d5a",
"metadata": {},
"source": [
"At this point, we can already use JuMP and any mixed-integer linear programming solver to find optimal solutions to any instance of this problem. To illustrate this, let us solve a small instance with three generators, using SCIP:"
@@ -227,7 +222,7 @@
{
"cell_type": "code",
"execution_count": 5,
- "id": "b2abe5e2",
+ "id": "9ff9f05c",
"metadata": {},
"outputs": [
{
@@ -242,7 +237,6 @@
],
"source": [
"using SCIP\n",
- "using Printf\n",
"\n",
"model = build_uc_model(\n",
" UnitCommitmentData(\n",
@@ -266,7 +260,7 @@
},
{
"cell_type": "markdown",
- "id": "5be976f5",
+ "id": "345de591",
"metadata": {},
"source": [
"Running the code above, we found that the optimal solution for our small problem instance costs \\$1320. It is achieve by keeping generators 2 and 3 online and producing, respectively, 60 MW and 40 MW of power."
@@ -274,7 +268,7 @@
},
{
"cell_type": "markdown",
- "id": "96a1f952",
+ "id": "eb8904ef",
"metadata": {},
"source": [
"## Generating training data\n",
@@ -289,7 +283,7 @@
{
"cell_type": "code",
"execution_count": 6,
- "id": "353e6199",
+ "id": "7298bb0d",
"metadata": {},
"outputs": [],
"source": [
@@ -317,7 +311,7 @@
},
{
"cell_type": "markdown",
- "id": "2140968d",
+ "id": "c1feed43",
"metadata": {},
"source": [
"In this example, for simplicity, only the demands change from one instance to the next. We could also have made the prices and the production limits random. The more randomization we have in the training data, however, the more challenging it is for the machine learning models to learn solution patterns.\n",
@@ -328,7 +322,7 @@
{
"cell_type": "code",
"execution_count": 7,
- "id": "1bb24909",
+ "id": "61d43994",
"metadata": {},
"outputs": [],
"source": [
@@ -339,34 +333,33 @@
},
{
"cell_type": "markdown",
- "id": "96bc0e42",
+ "id": "3fdeb8cd",
"metadata": {},
"source": [
- "Next, we will write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.\n",
+ "Next, we write these data structures to individual files. MIPLearn uses files during the training process because, for large-scale optimization problems, it is often impractical to hold the entire training data, as well as the concrete JuMP models, in memory. Files also make it much easier to solve multiple instances simultaneously, potentially even on multiple machines. We will cover parallel and distributed computing in a future tutorial.\n",
"\n",
- "The code below generates the files `uc/train/000001.jld2`, `uc/train/000002.jld2`, etc."
+ "The code below generates the files `uc/train/000001.jld2`, `uc/train/000002.jld2`, etc., which contain the input data in [JLD2 format](https://github.com/JuliaIO/JLD2.jl)."
]
},
{
"cell_type": "code",
"execution_count": 8,
- "id": "8ec476b1",
+ "id": "31b48701",
"metadata": {},
"outputs": [],
"source": [
"using MIPLearn\n",
- "using Glob\n",
- "\n",
"MIPLearn.save(data[1:90], \"uc/train/\")\n",
"MIPLearn.save(data[91:100], \"uc/test/\")\n",
"\n",
+ "using Glob\n",
"train_files = glob(\"uc/train/*.jld2\")\n",
"test_files = glob(\"uc/test/*.jld2\");"
]
},
{
"cell_type": "markdown",
- "id": "5d53a783",
+ "id": "5cecea59",
"metadata": {},
"source": [
"Finally, we use `MIPLearn.LearningSolver` and `MIPLearn.solve!` to solve all the training instances. `LearningSolver` is the main component provided by MIPLearn, which integrates MIP solvers and ML. The `solve!` function can be used to solve either one or multiple instances, and requires: (i) the list of files containing the training data; and (ii) the function that converts the data structure into a concrete JuMP model:"
@@ -375,14 +368,14 @@
{
"cell_type": "code",
"execution_count": 9,
- "id": "514a3b3a",
+ "id": "60732af0",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- "101.279699 seconds (93.52 M allocations: 3.599 GiB, 1.23% gc time, 0.52% compilation time)\n"
+ "103.808547 seconds (93.52 M allocations: 3.604 GiB, 1.19% gc time, 0.52% compilation time)\n"
]
},
{
@@ -401,33 +394,33 @@
},
{
"cell_type": "markdown",
- "id": "72eb09f4",
+ "id": "bbc7ad82",
"metadata": {},
"source": [
- "The macro `@time` shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, is stored by MIPLearn in `.h5` files, stored side-by-side with the original `.jld2` files."
+ "The macro `@time` shows us how long did the code take to run. We can see that SCIP was able to solve all training instances in about 2 minutes. The solutions, and other useful training data, are stored by MIPLearn in `.h5` files, stored side-by-side with the original `.jld2` files."
]
},
{
"cell_type": "markdown",
- "id": "90406b90",
+ "id": "73379180",
"metadata": {},
"source": [
"## Solving new instances\n",
"\n",
- "Now that we have training data, we can fit the ML models using `MIPLearn.fit!`, then solve the test instances with `MIPLearn.solve!`, as shown below:"
+ "With training data in hand, we can now fit the ML models using `MIPLearn.fit!`, then solve the test instances with `MIPLearn.solve!`, as shown below:"
]
},
{
"cell_type": "code",
"execution_count": 10,
- "id": "e4de94db",
+ "id": "e045d644",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- " 5.693951 seconds (9.33 M allocations: 334.689 MiB, 1.62% gc time)\n"
+ " 5.951264 seconds (9.33 M allocations: 334.657 MiB, 1.51% gc time)\n"
]
}
],
@@ -439,23 +432,23 @@
},
{
"cell_type": "markdown",
- "id": "247c1087",
+ "id": "d8de7b26",
"metadata": {},
"source": [
- "The trained MIP solver was able to solve all test instances in about 5 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the `fit!` line:"
+ "The trained MIP solver was able to solve all test instances in about 6 seconds. To see that ML is being helpful here, let us repeat the code above, but remove the `fit!` line:"
]
},
{
"cell_type": "code",
"execution_count": 11,
- "id": "62061b12",
+ "id": "cf2a989e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- " 9.829350 seconds (8.17 M allocations: 278.008 MiB, 0.47% gc time)\n"
+ " 10.390325 seconds (8.17 M allocations: 278.042 MiB, 0.89% gc time)\n"
]
}
],
@@ -466,10 +459,10 @@
},
{
"cell_type": "markdown",
- "id": "8ea5c423",
+ "id": "e100b25d",
"metadata": {},
"source": [
- "Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances, or about twice as long.\n",
+ "Without the help of the ML models, SCIP took around 10 seconds to solve the same test instances.\n",
"\n",
"
\n",
"Note\n",
@@ -480,18 +473,18 @@
},
{
"cell_type": "markdown",
- "id": "569f7c7a",
+ "id": "af451e87",
"metadata": {},
"source": [
"## Understanding the acceleration\n",
"\n",
- "Let us know go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP's performance. First, we are going to solve one of the training instances again, using the trained solver, but this time using the `tee=true` parameter, so that we can see SCIP's log:"
+ "Let us go a bit deeper and try to understand how exactly did MIPLearn accelerate SCIP's performance. First, we are going to solve one of the test instances again, using the trained solver, but this time using the `tee=true` parameter, so that we can see SCIP's log:"
]
},
{
"cell_type": "code",
"execution_count": 12,
- "id": "46739739",
+ "id": "0c675452",
"metadata": {},
"outputs": [
{
@@ -538,7 +531,7 @@
"presolved problem has 2000 variables (1000 bin, 0 int, 0 impl, 1000 cont) and 2001 constraints\n",
" 2000 constraints of type \n",
" 1 constraints of type \n",
- "Presolving Time: 0.10\n",
+ "Presolving Time: 0.11\n",
"transformed 1/1 original solutions to the transformed problem space\n",
"\n",
" time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl. \n",
@@ -560,18 +553,18 @@
},
{
"cell_type": "markdown",
- "id": "9cdc02d0",
+ "id": "ff0b6858",
"metadata": {},
"source": [
- "The log above is quite complicated if you have never seen it before, but the important line in the one starting with `feasible solution found [...] objective value 1.705169e+07`. This line indicates that MIPLearn was able to construct a warm start with value `1.705169e+07`. Using this warm start, SCIP then proceeded with the branch-and-cut process to either prove its optimality or find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed optimal and terminated. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast. \n",
+ "The log above is quite complicated if you have never seen it before, but the important line is the one starting with `feasible solution found [...] objective value 1.705169e+07`. This line indicates that MIPLearn was able to construct a warm start with value `1.705169e+07`. Using this warm start, SCIP then used the branch-and-cut method to either prove its optimality or to find an even better solution. Very quickly, however, SCIP proved that the solution produced by MIPLearn was indeed optimal. It was able to do this without generating a single cutting plane or running any other heuristics; it could tell the optimality by the root LP relaxation alone, which was very fast. \n",
"\n",
- "Let us now do the same thing again, but using the untrained solver this time:"
+ "Let us now repeat the process, but using the untrained solver this time:"
]
},
{
"cell_type": "code",
"execution_count": 13,
- "id": "555af477",
+ "id": "1aa9230e",
"metadata": {},
"outputs": [
{
@@ -642,7 +635,7 @@
"L 0.7s| 1 | 0 | 1707 | - | alns| 0 |2000 |2001 |2015 | 14 | 11 | 0 | 0 | 1.705038e+07 | 1.705178e+07 | 0.01%| unknown\n",
"\n",
"SCIP Status : solving was interrupted [gap limit reached]\n",
- "Solving Time (sec) : 0.67\n",
+ "Solving Time (sec) : 0.68\n",
"Solving Nodes : 1\n",
"Primal Bound : +1.70517823853380e+07 (13 solutions)\n",
"Dual Bound : +1.70503798271962e+07\n",
@@ -657,15 +650,17 @@
},
{
"cell_type": "markdown",
- "id": "72a52d26",
+ "id": "9417bb85",
"metadata": {},
"source": [
- "In this log file, notice how the line we saw before is now missing; SCIP needs to find an initial solution using its own internal heuristics. The solution SCIP initially found has value `2.335200e+07`, which is significantly worse than the one MIPLearn constructed before. SCIP then proceeds to improve this solution by generating a number of cutting planes and repeatedly running primal heuristics. In the end, it is able to find the optimal solution, as expected, but it takes longer."
+ "In this log file, notice how the previous line about warm starts is missing. Since no warm starts were provided, SCIP had to find an initial solution using its own internal heuristics, which are not specifically tailored for this problem. The initial solution found by SCIP's heuristics has value `2.335200e+07`, which is significantly worse than the one constructed by MIPLearn. SCIP then proceeded to improve this solution, by generating cutting planes and repeatedly running additional primal heuristics. In the end, it was able to find the optimal solution, as expected, but it took longer.\n",
+ "\n",
+ "In summary, MIPLearn accelerated the solution process by constructing a high-quality initial solution. In the following tutorials, we will see other strategies that MIPLearn can use to accelerate MIP performance, besides warm starts."
]
},
{
"cell_type": "markdown",
- "id": "36fb5f02",
+ "id": "ab9c1ff4",
"metadata": {},
"source": [
"## Accessing the solution\n",
@@ -678,7 +673,7 @@
{
"cell_type": "code",
"execution_count": 14,
- "id": "f62f28b4",
+ "id": "79759e87",
"metadata": {},
"outputs": [
{
@@ -710,7 +705,7 @@
},
{
"cell_type": "markdown",
- "id": "d5722dcf",
+ "id": "b096dcf9",
"metadata": {},
"source": [
"We can then solve this model as before, with `MIPLearn.solve!`:"
@@ -719,7 +714,7 @@
{
"cell_type": "code",
"execution_count": 15,
- "id": "e49f9e60",
+ "id": "1b668c12",
"metadata": {},
"outputs": [
{