hoshi-hiyouga fcf1a3df62 Merge pull request #6152 from hiyouga/hiyouga/add_num_proc_in_data_load ...

[data] add num_proc in load_dataset

Former-commit-id: d8258ba7e792d5f17ae80d5e8b303e8fa820f162

2024-11-27 00:16:15 +08:00

..

processors

support rank0 logger

2024-11-02 18:31:04 +08:00

__init__.py

fix mixed mm inputs and rlhf-v

2024-09-01 20:52:47 +08:00

aligner.py

support rank0 logger

2024-11-02 18:31:04 +08:00

collator.py

fix mllama cross_mask

2024-11-26 15:56:58 +00:00

data_utils.py

support rank0 logger

2024-11-02 18:31:04 +08:00

formatter.py

use pre-commit

2024-10-29 09:07:46 +00:00

loader.py

fix #6149

2024-11-26 16:03:02 +00:00

mm_plugin.py

fix mllama cross_mask

2024-11-26 15:56:58 +00:00

parser.py

use pre-commit

2024-10-29 09:07:46 +00:00

preprocess.py

fix mixed mm inputs and rlhf-v

2024-09-01 20:52:47 +08:00

template.py

fix inputs

2024-11-23 18:26:02 +00:00

tool_utils.py

use pre-commit

2024-10-29 09:07:46 +00:00