Describe the bug
All the key/values are getting print here. But when the value is a huge list, the print/log is too big
https://github.com/Project-MONAI/MONAI/blob/dev/monai/bundle/scripts.py#L84
Looks like the pprint.pformat doesn't handle large lists correct.
To Reproduce
Steps to reproduce the behavior:
- You can use any monai bundle and pass the list of image/value pairs to the train/validation dataset
- Run single gpu training
Expected behavior
Smaller (first few elements and ...) log for the list; And if possible len of the list. Same should be in case of dict.
Screenshots
Currently it prints something huge.. like
2022-12-16 00:30:00,174 - INFO - > validate#dataset#data: [{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_039_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_039_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_039_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_040_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_040_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_040_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_041_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_041_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_041_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_042_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_042_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_042_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_043_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_043_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_043_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_044_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_044_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_044_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_045_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_045_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_045_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_046_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_046_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_046_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_047_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_047_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_047_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_048_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_048_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_048_type_map.npy'}]
Environment
Ensuring you use the relevant python executable, please paste the output of:
================================
Printing MONAI config...
================================
MONAI version: 1.1.0rc2
Numpy version: 1.23.4
Pytorch version: 1.13.0+cu116
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: d8652642d7ae849ccf7f53cbb483344682199043
MONAI __file__: /localhome/sachi/.local/lib/python3.10/site-packages/monai/__init__.py
Optional dependencies:
Pytorch Ignite version: 0.4.10
Nibabel version: 4.0.2
scikit-image version: 0.19.3
Pillow version: 9.0.1
Tensorboard version: 2.10.1
gdown version: 4.5.3
TorchVision version: 0.14.0+cu116
tqdm version: 4.64.1
lmdb version: 1.3.0
psutil version: 5.9.3
pandas version: 1.5.1
einops version: 0.5.0
transformers version: 4.21.3
mlflow version: 2.0.1
pynrrd version: 0.4.3
For details about installing the optional dependencies, please visit:
https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
================================
Printing system config...
================================
System: Linux
Linux version: Ubuntu 22.04 LTS
Platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Processor: x86_64
Machine: x86_64
Python version: 3.10.6
Process name: python
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Open files: []
Num physical CPUs: 16
Num logical CPUs: 32
Num usable CPUs: 32
CPU usage (%): [5.4, 5.4, 5.1, 6.5, 7.1, 5.4, 6.4, 5.7, 5.1, 6.8, 5.4, 5.1, 6.1, 6.8, 5.4, 5.1, 6.7, 4.7, 6.1, 5.4, 5.1, 6.1, 5.4, 5.1, 5.1, 4.7, 5.4, 5.1, 5.7, 5.4, 4.7, 99.3]
CPU freq. (MHz): 1062
Load avg. in last 1, 5, 15 mins (%): [8.5, 3.7, 3.9]
Disk usage (%): 4.7
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 251.6
Available memory (GB): 236.0
Used memory (GB): 13.5
================================
Printing GPU config...
================================
Num GPUs: 2
Has CUDA: True
CUDA version: 11.6
cuDNN enabled: True
cuDNN version: 8302
Current device: 0
Library compiled for CUDA architectures: ['sm_37', 'sm_50', 'sm_60', 'sm_70', 'sm_75', 'sm_80', 'sm_86']
GPU 0 Name: NVIDIA A40
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 84
GPU 0 Total memory (GB): 44.4
GPU 0 CUDA capability (maj.min): 8.6
GPU 1 Name: NVIDIA A40
GPU 1 Is integrated: False
GPU 1 Is multi GPU board: False
GPU 1 Multi processor count: 84
GPU 1 Total memory (GB): 44.4
GPU 1 CUDA capability (maj.min): 8.6
Additional context
Add any other context about the problem here.
Describe the bug
All the key/values are getting print here. But when the value is a huge list, the print/log is too big
https://github.com/Project-MONAI/MONAI/blob/dev/monai/bundle/scripts.py#L84
Looks like the pprint.pformat doesn't handle large lists correct.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Smaller (first few elements and ...) log for the list; And if possible len of the list. Same should be in case of dict.
Screenshots
Currently it prints something huge.. like
Environment
Ensuring you use the relevant python executable, please paste the output of:
Additional context
Add any other context about the problem here.