Issues

• Add github dataset from redpajama

  kojimano/Megatron-DeepSpeed-ABCI#26 · Filter by author kuriyan1204 opened on May 10, 2023
  1

• Debug data processing pipelines

  kojimano/Megatron-DeepSpeed-ABCI#25 · Filter by author kuriyan1204 opened on May 10, 2023
  2

  

• Tokenizer周りについて

  kojimano/Megatron-DeepSpeed-ABCI#24 · Filter by author keisks opened on May 10, 2023

• Dataset Preprocessing Validation

  kojimano/Megatron-DeepSpeed-ABCI#21 · Filter by author kojimano opened on May 9, 2023

• General Preprocessing Pipeline

  datasets
  kojimano/Megatron-DeepSpeed-ABCI#20 · Filter by author kojimano opened on May 7, 2023

• instruction tuningについて

  kojimano/Megatron-DeepSpeed-ABCI#13 · Filter by author keisks opened on Apr 27, 2023
  4

• evaluation_jglue

  kojimano/Megatron-DeepSpeed-ABCI#2 · Filter by author kojimano opened on Apr 18, 2023
  2