Training data

[BlessedTatonka]

Thank you for your great work!
I was wondering if there is any approximate timeline for the release of the training data?
Looking forward to your response.

[YanshekWoo]

Thank you for your great work! I was wondering if there is any approximate timeline for the release of the training data? Looking forward to your response.

Apologies for the delayed response.
We have already open-sourced the pre-training data: HIT-TMG/KaLM-embedding-pretrain-data. The open-sourcing of the fine-tuning data may require further consideration, particularly regarding the synthetic data generated by large language models (LLMs).
In the meantime, you can refer to the data list provided in our technical report for your own processing.

[pass-lin]

Thank you for your great work! I was wondering if there is any approximate timeline for the release of the training data? Looking forward to your response.

Apologies for the delayed response. We have already open-sourced the pre-training data: HIT-TMG/KaLM-embedding-pretrain-data. The open-sourcing of the fine-tuning data may require further consideration, particularly regarding the synthetic data generated by large language models (LLMs). In the meantime, you can refer to the data list provided in our technical report for your own processing.

你好，请问数据集是纯英文的吗

[YanshekWoo]

你好，请问数据集是纯英文的吗

有中文和其他语言。
预训练数据的具体语言列表可以参照技术报告 的Table 8

[pass-lin]

你好，请问数据集是纯英文的吗

有中文和其他语言。 预训练数据的具体语言列表可以参照技术报告 的Table 8

你好，我阅读了源代码和数据，发现数据集里有个“relevance”参数在你们训练的时候是不是没有被用到

[YanshekWoo]

你好，我阅读了源代码和数据，发现数据集里有个“relevance”参数在你们训练的时候是不是没有被用到

“relevance” 训练时没有使用。

该参数由bge-m3编码得到，最初是考虑用于进行数据过滤（选取分数高的部分数据）。
实际测试一版效果一般（没有变好但也没太差），因此开源的该模型版本是使用的未做过滤的预训练数据。
这里你们可以根据实际需求看该字段是否有用。