Not sure how it should be implemented, but take all lines that are uncommon but within a certain edit distance of a commonly occuring line, and replace the uncommon line with the common line.
Also, it may be wise to have some human oversight into this process while it happens.
Not sure how it should be implemented, but take all lines that are uncommon but within a certain edit distance of a commonly occuring line, and replace the uncommon line with the common line.
Also, it may be wise to have some human oversight into this process while it happens.