how to prepare the training data to train starcoder?

[wwngh1233]

please tell me hot wo generate the following files:

WEIGHTS_TRAIN=/fsx/loubna/code/bigcode-data-mix/data/train_data_paths.txt.tmp

WEIGHTS_VALID=/fsx/loubna/code/bigcode-data-mix/data/valid_data_paths.txt.tmp

[KOVVURISATYANARAYANAREDDY]

Hello @wwngh1233 are you able to get these files? how to prepare these files?

[L1aoXingyu]

same questions