[BUG] Segfault in nightly test for lp #560
Description
Describe the bug
Test python/cuopt/cuopt/tests/linear_programming/test_lp_solver.py::test_heuristics_only Setting parameter mip_heuristics_only to true
is failing.
Complete log is below.
python/cuopt/cuopt/tests/linear_programming/test_lp_solver.py::test_heuristics_only Setting parameter mip_heuristics_only to true
Setting parameter time_limit to 3.000000e+01
cuOpt version: 25.12.0, git hash: 3221e7c, host arch: x86_64, device archs: 70-real,75-real,80-real,86-real,90a-real,100f-real,120a-real,120
CPU: AMD EPYC 9554 64-Core Processor, threads (physical/logical): 12/12, RAM: 29.45 GiB
CUDA 12.9, device: NVIDIA L4 (ID 0), VRAM: 21.96 GiB
CUDA device UUID: ffffffa74affffff95ffffffb9-747b-4b61
Solving a problem with 884 constraints, 6805 variables (2306 integers), and 34965 nonzeros
Problem scaling:
Objective coefficents range: [1e+00, 1e+00]
Constraint matrix coefficients range: [1e+00, 1e+03]
Constraint rhs / bounds range: [1e+00, 2e+01]
Variable bounds range: [0e+00, 1e+00]
Original problem: 884 constraints, 6805 variables, 34965 nonzeros
Calling Papilo presolver
Presolve status: reduced the problem
Presolve removed: 402 constraints, 545 variables, 9485 nonzeros
Presolved problem: 482 constraints, 6260 variables, 25480 nonzeros
Papilo presolve time: 0.093346
Objective offset 0.000000 scaling_factor 1.000000
Running presolve!
After trivial presolve: 482 constraints, 6260 variables, objective offset 0.000000.
Fatal Python error: Segmentation fault
Current thread 0x000074548eb41b80 (most recent call first):
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/cuopt/linear_programming/solver/solver.py", line 110 in Solve
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/cuopt/utilities/exception_handler.py", line 36 in func
File "/__w/cuopt/cuopt/python/cuopt/cuopt/tests/linear_programming/test_lp_solver.py", line 656 in test_heuristics_only
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_hooks.py", line 512 in __call__
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/python.py", line 1792 in runtest
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_hooks.py", line 512 in __call__
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/runner.py", line 262 in <lambda>
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/runner.py", line 341 in from_call
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/runner.py", line 222 in call_and_report
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/runner.py", line 133 in runtestprotocol
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_hooks.py", line 512 in __call__
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/main.py", line 350 in pytest_runtestloop
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_hooks.py", line 512 in __call__
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/main.py", line 325 in _main
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/main.py", line 271 in wrap_session
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/main.py", line 318 in pytest_cmdline_main
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pluggy/_hooks.py", line 512 in __call__
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/config/__init__.py", line 169 in main
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/_pytest/config/__init__.py", line 192 in console_main
File "/pyenv/versions/3.11.14/lib/python3.11/site-packages/pytest/__main__.py", line 5 in <module>
File "<frozen runpy>", line 88 in _run_code
File "<frozen runpy>", line 198 in _run_module_as_main
Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, cuopt.linear_programming.cuopt_mps_parser.parser_wrapper, cuopt.linear_programming.data_model.data_model_wrapper, data_model.data_model_wrapper, cuopt_mps_parser.parser_wrapper, yaml._yaml, numba.core.typeconv._typeconv, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.typing.builtins.itertools, numba.cpython.builtins.math, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, numba.mviewbuf, cuda.bindings._bindings.cydriver, cuda.bindings.cydriver, cuda.bindings.driver, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime, cuda.core.experimental._utils.cuda_utils, cuda.core.experimental._memoryview, cuda.core.experimental._context, cuda.core.experimental._event, cuda.core.experimental._dlpack, cuda.core.experimental._stream, cuda.core.experimental._kernel_arg_handler, numba.core.typing.cmathdecl.cmath, numba.types.itertools, cuda.bindings._internal.utils, cuda.bindings._internal.nvjitlink, cuopt.linear_programming.internals.internals, pylibraft.common.cuda, rmm.pylibrmm.cuda_stream, rmm.pylibrmm.stream, rmm.pylibrmm.helper, rmm.pylibrmm.memory_resource._memory_resource, rmm.pylibrmm.memory_resource.experimental, rmm.pylibrmm.device_buffer, rmm.librmm._logger, rmm.pylibrmm.logger, pylibraft.common.handle, cupy_backends.cuda._softlink, cupy_backends.cuda.api._runtime_enum, cupy_backends.cuda.api.runtime, cupy._util, cupy.cuda.device, fastrlock.rlock, cupy.cuda.memory_hook, cupy_backends.cuda.stream, cupy.cuda.graph, cupy.cuda.stream, cupy_backends.cuda.api._driver_enum, cupy_backends.cuda.api.driver, cupy.cuda.memory, cupy._core.internal, cupy._core._carray, cupy.cuda.texture, cupy.cuda.function, cupy_backends.cuda.libs.nvrtc, cupy.cuda.pinned_memory, cupy.cuda.common, cupy.cuda.cub, cupy_backends.cuda.libs.nvtx, cupy.cuda.thrust, cupy._core._dtype, cupy._core._scalar, cupy._core._accelerator, cupy._core._memory_range, cupy._core._fusion_thread_local, cupy._core._kernel, cupy._core._routines_manipulation, cupy._core._routines_binary, cupy._core._optimize_config, cupy._core._cub_reduction, cupy._core._reduction, cupy._core._routines_math, cupy._core._routines_indexing, cupy._core._routines_linalg, cupy._core._routines_logic, cupy._core._routines_sorting, cupy._core._routines_statistics, cupy._core.dlpack, cupy._core.flags, cupy._core.core, cupy._core._fusion_variable, cupy._core._fusion_trace, cupy._core._fusion_kernel, cupy._core.new_fusion, cupy._core.fusion, cupy._core.raw, cupy.fft._cache, cupy.fft._callback, cupy.random._bit_generator, cupy.lib._polynomial, pyarrow.lib, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, nvtx._lib.lib, nvtx._lib.profiler, pylibcudf.libcudf.types, pylibcudf.types, pylibcudf.libcudf.aggregation, pylibcudf.aggregation, pylibcudf.gpumemoryview, pylibcudf.utils, pylibcudf._interop_helpers, pylibcudf.table, pylibcudf.filling, pylibcudf.traits, pylibcudf.column, pylibcudf.scalar, pylibcudf.libcudf.binaryop, pylibcudf.binaryop, pylibcudf.column_factories, pylibcudf.concatenate, pylibcudf.contiguous_split, pylibcudf.libcudf.copying, pylibcudf.copying, pylibcudf.libcudf.datetime, pylibcudf.datetime, pylibcudf.experimental._join_streams, pylibcudf.libcudf.expressions, pylibcudf.expressions, pylibcudf.groupby, pylibcudf.hashing, pylibcudf.interop, pylibcudf.io.datasource, pylibcudf.libcudf.io.json, pylibcudf.libcudf.io.types, pylibcudf.io.types, pylibcudf.io.avro, pylibcudf.io.csv, pylibcudf.io.json, pylibcudf.io.orc, pylibcudf.io.parquet, pylibcudf.io.parquet_metadata, pylibcudf.io.text, pylibcudf.io.timezone, pylibcudf.join, pylibcudf.json, pylibcudf.libcudf.labeling, pylibcudf.labeling, pylibcudf.libcudf.lists.combine, pylibcudf.libcudf.lists.contains, pylibcudf.lists, pylibcudf.merge, pylibcudf.null_mask, pylibcudf.nvtext.byte_pair_encode, pylibcudf.nvtext.deduplicate, pylibcudf.nvtext.edit_distance, pylibcudf.nvtext.generate_ngrams, pylibcudf.nvtext.jaccard, pylibcudf.nvtext.minhash, pylibcudf.nvtext.ngrams_tokenize, pylibcudf.nvtext.normalize, pylibcudf.nvtext.replace, pylibcudf.libcudf.nvtext.stemmer, pylibcudf.nvtext.stemmer, pylibcudf.nvtext.tokenize, pylibcudf.nvtext.wordpiece_tokenize, pylibcudf.partitioning, pylibcudf.prefetch, pylibcudf.quantiles, pylibcudf.libcudf.reduce, pylibcudf.reduce, pylibcudf.libcudf.replace, pylibcudf.replace, pylibcudf.reshape, pylibcudf.rolling, pylibcudf.libcudf.round, pylibcudf.round, pylibcudf.search, pylibcudf.sorting, pylibcudf.libcudf.stream_compaction, pylibcudf.stream_compaction, pylibcudf.strings.attributes, pylibcudf.libcudf.strings.char_types, pylibcudf.strings.capitalize, pylibcudf.strings.case, pylibcudf.strings.char_types, pylibcudf.libcudf.strings.combine, pylibcudf.strings.combine, pylibcudf.strings.regex_program, pylibcudf.strings.contains, pylibcudf.strings.convert.convert_booleans, pylibcudf.strings.convert.convert_datetime, pylibcudf.strings.convert.convert_durations, pylibcudf.strings.convert.convert_fixed_point, pylibcudf.strings.convert.convert_floats, pylibcudf.strings.convert.convert_integers, pylibcudf.strings.convert.convert_ipv4, pylibcudf.strings.convert.convert_lists, pylibcudf.strings.convert.convert_urls, pylibcudf.strings.extract, pylibcudf.strings.find, pylibcudf.strings.find_multiple, pylibcudf.strings.findall, pylibcudf.strings.padding, pylibcudf.libcudf.strings.regex_flags, pylibcudf.strings.regex_flags, pylibcudf.strings.repeat, pylibcudf.strings.replace, pylibcudf.strings.replace_re, pylibcudf.strings.reverse, pylibcudf.libcudf.strings.side_type, pylibcudf.strings.side_type, pylibcudf.strings.slice, pylibcudf.strings.split.partition, pylibcudf.strings.split.split, pylibcudf.strings.strip, pylibcudf.libcudf.strings.translate, pylibcudf.strings.translate, pylibcudf.strings.wrap, pylibcudf.transform, pylibcudf.transpose, pylibcudf.libcudf.unary, pylibcudf.unary, pylibcudf.utilities, numba.cpython.mathimpl.math, numba.cpython.mathimpl.sys, cudf._lib.strings_udf, pyarrow._feather, cuopt.linear_programming.solver.solver_parameters, cuopt.linear_programming.solver.solver_wrapper, cuopt.routing.utils_wrapper, cuopt.routing.vehicle_routing_wrapper, cuopt.distance_engine.waypoint_matrix_wrapper, numba.cpython.numbers.math, numba.cpython.hashing.math, numba.cpython.hashing.sys, cuda.bindings.cynvjitlink, cuda.bindings.nvjitlink (total: 288)
timeout: the monitored command dumped core
ci/test_wheel_cuopt.sh: line 77: 2797 Segmentation fault RAPIDS_DATASET_ROOT_DIR=./datasets timeout 30m python -m pytest --verbose --capture=no ./python/cuopt/cuopt/tests/
Error:
Environment details (please complete the following information):
Additional context
Add any other context about the problem here.