# 8. Learning Solver — MIPLearn 0.3

# 8. Learning Solver

On previous pages, we discussed various components of the MIPLearn framework, including training data collectors, feature extractors, and individual machine learning components. In this page, we introduce LearningSolver, the main class of the framework which integrates all the aforementioned components into a cohesive whole. Using LearningSolver involves three steps: (i) configuring the solver; (ii) training the ML components; and (iii) solving new MIP instances. In the following, we describe each of these steps, then conclude with a complete runnable example.

## 8.1. Configuring the solver

LearningSolver is composed by multiple individual machine learning components, each targeting a different part of the solution process, or implementing a different machine learning strategy. This architecture allows strategies to be easily enabled, disabled or customized, making the framework flexible. By default, no components are provided and LearningSolver is equivalent to a traditional MIP solver. To specify additional components, the `components` constructor argument may be used:

    solver = LearningSolver(
        components=[
            comp1,
            comp2,
            comp3,
        ]
    )

In this example, three components `comp1`, `comp2` and `comp3` are provided. The strategies implemented by these components are applied sequentially when solving the problem. For example, `comp1` and `comp2` could fix a subset of decision variables, while `comp3` constructs a warm start for the remaining problem.

## 8.2. Training and solving new instances

Once a solver is configured, its ML components need to be trained. This can be achieved by the `solver.fit` method, as illustrated below. The method accepts a list of HDF5 files and trains each individual component sequentially. Once the solver is trained, new instances can be solved using `solver.optimize`. The method returns a dictionary of statistics collected by each component, such as the number of variables fixed.

    # Build instances
    train_data = ...
    test_data = ...

    # Collect training data
    bc = BasicCollector()
    bc.collect(train_data, build_model)

    # Build solver
    solver = LearningSolver(...)

    # Train components
    solver.fit(train_data)

    # Solve a new test instance
    stats = solver.optimize(test_data[0], build_model)

## 8.3. Complete example

In the example below, we illustrate the usage of LearningSolver by building instances of the Traveling Salesman Problem, collecting training data, training the ML components, then solving a new instance.

    [3]:

    import random

    import numpy as np
    from scipy.stats import uniform, randint
    from sklearn.linear_model import LogisticRegression

    from miplearn.classifiers.minprob import MinProbabilityClassifier
    from miplearn.classifiers.singleclass import SingleClassFix
    from miplearn.collectors.basic import BasicCollector
    from miplearn.components.primal.actions import SetWarmStart
    from miplearn.components.primal.indep import IndependentVarsPrimalComponent
    from miplearn.extractors.AlvLouWeh2017 import AlvLouWeh2017Extractor
    from miplearn.io import write_pkl_gz
    from miplearn.problems.tsp import (
        TravelingSalesmanGenerator,
        build_tsp_model,
    )
    from miplearn.solvers.learning import LearningSolver

    # Set random seed to make example reproducible.
    random.seed(42)
    np.random.seed(42)

    # Generate a few instances of the traveling salesman problem.
    data = TravelingSalesmanGenerator(
        n=randint(low=10, high=11),
        x=uniform(loc=0.0, scale=1000.0),
        y=uniform(loc=0.0, scale=1000.0),
        gamma=uniform(loc=0.90, scale=0.20),
        fix_cities=True,
        round=True,
    ).generate(50)

    # Save instance data to data/tsp/00000.pkl.gz, data/tsp/00001.pkl.gz, ...
    all_data = write_pkl_gz(data, "data/tsp")

    # Split train/test data
    train_data = all_data[:40]
    test_data = all_data[40:]

    # Collect training data
    bc = BasicCollector()
    bc.collect(train_data, build_tsp_model, n_jobs=4)

    # Build learning solver
    solver = LearningSolver(
        components=[
            IndependentVarsPrimalComponent(
                base_clf=SingleClassFix(
                    MinProbabilityClassifier(
                        base_clf=LogisticRegression(),
                        thresholds=[0.95, 0.95],
                    ),
                ),
                extractor=AlvLouWeh2017Extractor(),
                action=SetWarmStart(),
            )
        ]
    )

    # Train ML models
    solver.fit(train_data)

    # Solve a test instance
    solver.optimize(test_data[0], build_tsp_model)

    Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)

    CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]
    Thread count: 16 physical cores, 32 logical processors, using up to 32 threads

    Optimize a model with 10 rows, 45 columns and 90 nonzeros
    Model fingerprint: 0x6ddcd141
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [4e+01, 1e+03]
      Bounds range     [1e+00, 1e+00]
      RHS range        [2e+00, 2e+00]
    Presolve time: 0.00s
    Presolved: 10 rows, 45 columns, 90 nonzeros

    Iteration    Objective       Primal Inf.    Dual Inf.      Time
           0    6.3600000e+02   1.700000e+01   0.000000e+00      0s
          15    2.7610000e+03   0.000000e+00   0.000000e+00      0s

    Solved in 15 iterations and 0.00 seconds (0.00 work units)
    Optimal objective  2.761000000e+03
    Set parameter LazyConstraints to value 1
    Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)

    CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]
    Thread count: 16 physical cores, 32 logical processors, using up to 32 threads

    Optimize a model with 10 rows, 45 columns and 90 nonzeros
    Model fingerprint: 0x74ca3d0a
    Variable types: 0 continuous, 45 integer (45 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [4e+01, 1e+03]
      Bounds range     [1e+00, 1e+00]
      RHS range        [2e+00, 2e+00]

    User MIP start produced solution with objective 2796 (0.00s)
    Loaded user MIP start with objective 2796

    Presolve time: 0.00s
    Presolved: 10 rows, 45 columns, 90 nonzeros
    Variable types: 0 continuous, 45 integer (45 binary)

    Root relaxation: objective 2.761000e+03, 14 iterations, 0.00 seconds (0.00 work units)

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0 2761.00000    0    - 2796.00000 2761.00000  1.25%     -    0s
         0     0     cutoff    0      2796.00000 2796.00000  0.00%     -    0s

    Cutting planes:
      Lazy constraints: 3

    Explored 1 nodes (16 simplex iterations) in 0.01 seconds (0.00 work units)
    Thread count was 32 (of 32 available processors)

    Solution count 1: 2796

    Optimal solution found (tolerance 1.00e-04)
    Best objective 2.796000000000e+03, best bound 2.796000000000e+03, gap 0.0000%

    User-callback calls 110, time in user-callback 0.00 sec

    [3]:

    {'WS: Count': 1, 'WS: Number of variables set': 41.0}

    [ ]:

7. Primal Components 9. Benchmark Problems