If compile SCIPOpt suite using git repository of every projects, output does not come out to console in real time

[adover134]

Describe the bug
#887 (comment)
This is extension from the link.

I installed Ubuntu server 24.04.
I downloaded SCIPsuite tarball from the following link.
https://www.scipopt.org/index.php#download

I unzipped tarball, then removed gcg, soplex, papilo, zimpl, scip folders.
Then I cloned up-to-date version of each folders from git hub.

I installed Ipopt by cloning from git hub.
This is because MUMPS was required by PySCIPOpt.

After installing Ipopt, I installed SCIPsuite with TPI=tny option.
Then I installed PySCIPOpt by cloning git repository.

After successful(?) installation, I tried to solve my model.
However, when I was not using redirectOutput(), the console did not showed output in real time.

To Reproduce
I used master branch of each projects given by scipopt.

Expected behavior
The output should be printed on terminal (console) in real time while not using redirectOutput().

Screenshots

System

• OS: Ubuntu
• Version: 24.04.1 LTS
• SCIP version: 10.0.0
• How did you install pyscipopt? cloned from git repository

Additional context
I used m.setParam('display/dualbound/active', 0).
However, it worked as m.setParam('display/nnodes/active', 0) and also not showing completion percentage.
This one is more easy to fix, right?

[Joao-Dionisio]

Hey, @adover134! Wait, is the output not showing in your screenshot? Or is that you only showing the desired output? In your code snippet you are using redirectOutput.

[adover134]

Hello, @Joao-Dionisio.
If not use redirectOutput(), then The console does not show something important; just save the file per some minutes. I cannot take the picture of it right now, because I am running the model.
The purpose of uploading screenshot is, I wanted to show you that the output does not shows number of nodes and completion percentage but showing dualbounds when I deactivated only showing dualbound.

[Joao-Dionisio]

I understand you not wanting to share your code on a public forum, but would it be possible to invite me to a google colab or something, where I would be able to run the code? My email is up201606210[at]up[dot]pt

You could then kick me out as soon as you want, of course

[adover134]

The console I am saying is the terminal which is running the jupyter notebook. And I think this problem is not related to model. This happens with any model on the same environment. You do not need the specific model to test it.

[adover134]

Before I send or not, I want to try some other models once again. I will send it if it happens on some more models.

[Opt-Mucca]

I haven't used Jupyter for some time, but I would say that it is a feature of the notebook. Take this with a grain of salt though.
By default the output should be directed to the terminal, i.e., where the jupyter notebook is called from. It would then be jupyter's responsibility on whether they capture this information and redirect it to the output of the cell. Maybe this isn't even possible? By calling redirectOutput you're redirecting it to Python's buffer, which the notebook obviously knows how to handle. I don't currently see this as an issue unless someone can correct my understanding.

[adover134]

@Opt-Mucca, sorry that I saw it right now. I will explain what I was saying.
First, this is the example image of right output when not setting any SCIP parameters and not using redirectOutput(). I took this from terminal of windows OS using Jupyter. This was using SCIP version 9.1.0.

As you can see in this image, output is normally printed on terminal even using Jupyter.
However, images in the previous issues I wrote, which are took from Ubuntu, shows no table output.
You can check the First image from here.
#887 (comment)
Almost half of a day was passed, but not showed any output on terminal.
This can be critical for people who want to solve a bit huge MILP model using SCIP.
Since Jupyter normally runs on local server, if redirectOutput() is necessary, it will be hard to solve huge model due to maximum memory of web browser.

Also you can see that this table shows many columns.
This was when using optimize(), not solveConcurrent().
When using solveConcurrent(), it shows only 5 columns; time, memory, dualboud, primalboud and gap.
I did not set any parameters to hide columns.

[Opt-Mucca]

I think I need a summary of the problem. I will write what I understand currently.

• If you do not use redirectOutput then the output when using Jupyter does not automatically appear, but rather is output in the terminal where Jupyter is called from.
• The above does not happen on your Ubuntu 20 machine?
• You are concerned about memory issues with redirectOutput. I don't think this is an issue. The problem size is not a concern here, but rather the potential log file size. Does the output appear periodically, or is it all dumped when the solver terminates? If the latter then maybe this is an issue if you have very little memory and lower the print frequency. I will not devote time to this for now.
• The output when using solveConcurrent only shows 5 columns because it is aggregating multiple concurrent solves. Therefore, it would be misleading to mention the current separation round and number of found conflicts etc, because these would be different over the different solving processes. The only information shown is time (universal), the current best bounds, and the total memory. This is not a bug. We have added a warning to solveConcurrent because of the amount of issues users have with this very experimental function.

[adover134]

@Opt-Mucca, I will answer what happened on your questions.

• When I do not use redirectOutput then the expected output is what you wrote; output should be in the terminal where Jupyter is called from. However, one thing was missing; output should be printed in real time. However, output printed after terminated or after optimized.
• This happened on any Ubuntu machine from 20 to 24.
• As answered on first, output dumped when the solver terminates. However, as the image I uploaded on If compile SCIP with option of TPI=tny while using whole SCIPsuite tarball, seems it makes crash #887 (comment), the environment had sufficient memory. Also, frequency was slow enough; display/freq was almost 2000000.

If this is not enough, please let me know what information I need to check.

[Joao-Dionisio]

Hey @adover134! Can you check if the current master branch fixes your problem? It seems #929 could help.

[DominikKamp]

I guess this can only have an effect when using redirectOutput(). The real time issue might be due to the output buffer not being flushed by SCIP because of performance. You could try to flush the terminal stdout manually when desired.

[Joao-Dionisio]

Closing due to inactivity and Dominik's comment seeming helpful.