Implement other sampling methods

[certik]

Currently only the "greedy" sampling is implemented (the token with the highest probability is selected).

Implement other sampling methods, some options are:

• top-p
• top-k
• temperature (here is an example how it could be done: Daniyal branch jaymody/picoGPT#19)
• categorical sampling

[matteo-grella]

Hi @certik!

In RWKV.f90, we've already implemented several sampling methods, including:

• Temperature
• Top-p
• Frequency/Presence Penalty
• Speculative Decoding (this is still experimental)
• Classifier-Free Guidance (currently a work in progress)

Hope this helps in expanding the sampling methods for your project!

[certik]

@matteo-grella awesome, excellent! Thanks for the comment. I'll have a look.