math-opt: add CPLEX solver backend#5125
Open
sschnug wants to merge 21 commits intogoogle:mainfrom
Open
Conversation
This commit adds dynamic loading support for CPLEX following the Gurobi/Xpress pattern. Filesystem search-paths are provided for Win/Linux/Mac. Additionally CPLEXDIR env-var is given higher priority. CPLEX versions supported are 20.10, 22.10, 22.11 and 22.12. These files have been manually ported and there is no corresponding code-generation script available for now.
This commit adds layer 2 and 3 of CPLEX-specific code following the three-layer approach of other solvers: - layer 1: dynamic loader (previous commit) -> cplex_environment.h/cc - layer 2: RAII wrapper -> g_cplex.h/cc - layer 3: math-opt interface -> cplex_solver.h/cc Additionally: add cplex-proto for solver-specific parameters. This commit is NOT self-contained and won't compile without follow-up commits related to solver-registration.
Adds SOLVER_TYPE_CPLEX to parameters.proto, CplexParameters C++ struct, and Python bindings (SolverType.CPLEX, SolveParameters, init_arguments). CMakeLists.txt did NOT get any USE_CPLEX filter as this math-opt work is based on dynamic-loading and should not be disabled on systems missing CPLEX install. This is different from other solvers due to NOT porting cplex in the legacy linear_solver framework to dynamic loading (like gurobi did).
… missing supports_best_bound_limit check + discard of solver-specific params Test-suite creates Model model on stack, builds the model, calls Solve(), and returns SolveResult. The SolveResult contains references to variable/constraint data owned by the Model. When the function returns, Model is destroyed → dangling references. ASAN catches this. The fix: Move Model to each caller's scope so it outlives SolveResult. The function takes Model& instead.
Due to some "special" solver-behaviour we need to add some cplex-based branching to some of those tests as already being done by other solvers although CPLEX is now (sadly) leading in this regard. Those exceptions are documented in every case.
This solves three issues:x - high prio: use-after-free on partial channel attachment failure - medium prio: duplicate output when message callback is registered - low prio: CPXPARAM_ParamDisplay leaks across Solve() calls
…case leading to async data
… of infeasibility
…code in MIP branch
…a as negative values are valid
…thon API (proto/c++ uses int32/int64 for safety)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
UPDATE: 14.04.26
This PR adds support for the commercial solver IBM CPLEX in MathOpt, following the discussion in #4748.
Scope
This is an initial implementation targeting the most common LP/MIP workflows. CPLEX supports more features than this PR currently exposes — extending coverage is planned as future work.
Supported
Not yet supported
Caveats / Unsupported by CPLEX
best_bound_limitsolve parameter (explicitly rejected)FIRST_ORDERLP algorithm (explicitly rejected)objective_limitrequires CPLEX 21.1.0 or newer (will return an error for CPLEX 20.1.0)Implementation
Architecture
The implementation follows the existing three-layer pattern used by Gurobi, GScip, Xpress et al.:
third_party_solvers/cplex_environment.{h,cc}dlopen-based loading with filesystem searchsolvers/cplex/g_cplex.{h,cc}solvers/cplex_solver.{h,cc}SolverInterfaceAdditionally:
solvers/cplex.protofor solver-specific parameters.The solver layer is intentionally structured (and ordered) like the Gurobi implementation, enabling diff-based comparison when adding features in the future.
Notable differences to other solver backend implementations
Unlike Gurobi, gSCIP, and Xpress, CPLEX requires message callbacks (channels) to be explicitly detached and screen output (
CPXPARAM_ScreenOutput) disabled before extracting solve results.This workaround is necessary because, unlike Gurobi—which provides a mechanism to check if an attribute like
node_countis available before querying it (e.g.,IsAttrAvailable)—CPLEX will emit internal diagnostic errors (e.g., "CPLEX Error 1217: No solution exists.") to the message callback and stderr when querying statistics that happen to be inapplicable to the current solve state.While there is likely a way to avoid these errors in a perfect world by querying only valid attributes, doing so reliably would require reverse-engineering CPLEX's internal state machine to know exactly which attributes are valid at any given time. Disabling output during the extraction phase ensures that valid solve-time messages are captured properly, while cleanly suppressing these noisy, expected errors during result extraction.
CMake note on
USE_CPLEXUnlike Gurobi and Xpress, the legacy
linear_solverbackend for CPLEX (cplex_interface.cc) was never ported to dynamic loading. It still links directly against the CPLEX library and requires CPLEX headers at build time. Thus, the CMake flagUSE_CPLEX=ONimplies a hard build-time dependency on CPLEX.MathOpt's new CPLEX backend, however, does use dynamic loading (
dlopen/LoadLibrary) and requires no build-time dependencies. To ensure users can build and use MathOpt's CPLEX backend without needing the CPLEX headers required bylinear_solver, the MathOpt CPLEX wrappers are always compiled unconditionally. They are intentionally not gated byUSE_CPLEXinmath_opt/solvers/CMakeLists.txt(this differs fromUSE_GUROBIandUSE_XPRESS, which do gate their respective MathOpt wrappers because theirlinear_solvercounterparts also use dynamic loading and don't force header dependencies).The CPLEX test target (
cplex_solver_test.cc), however, is still gated byUSE_CPLEX=ONsince the tests require a functional CPLEX installation to pass.Testing
Shared test-suite bug fixes
While adapting the shared test infrastructure, three pre-existing bugs were discovered and fixed:
LPForIterationLimit/SolveForGapLimit:Modelcreated on the stack inside a helper,SolveResultreturned with dangling references. Fix: caller owns theModel, helper takesModel&.BestBoundLimitMinimize: test ran unconditionally but requiressupports_best_bound_limit.IncompleteIpSolve:SolveArgumentswas constructed from scratch, discardingGetParam().parameters(solver-specific settings).CPLEX test coverage
All applicable shared test suites are instantiated in
cplex_solver_test.cc. Several tests require CPLEX-specific skip guards due to solver behavior quirks on small models (e.g., presolver solving before barrier starts, zero-iteration warm re-solves, root-only node counts reported as 0). Each exception is documented inline.Validation performed
--config=ci(all math_opt + third_party_solvers)--config=asan(all math_opt + third_party_solvers)USE_CPLEX=ON(CPLEX test binary)USE_CPLEX=OFFNote: MacOS CPLEX dynamic library search paths have not been validated (author currently lacks access to this environment but is working on it).
Future work
Higher priority (author's personal opinion)
CPXaddmipstarts)Lower priority (author's personal opinion)
cplex_environment.{h,cc}(CPLEX API is stable, manual approach is adequate for now)Unrelated follow-up work
linear_solver/cplex_interface.ccuses removedortools/base/logging.h— does not compile withUSE_CPLEX=ONinit_arguments.pyis missing Xpress Python init argument wiring (XpressInitializerProto.extract_namescannot be set)