Commit f4e57b99 authored by hoepfl's avatar hoepfl
Browse files

Adding option to check language in corpus data

Unfortunately, fasttext cannot natively detect sme, so it is only possible to check if a line matches either no, nn (the two versions of norwegian) or en.
If this is the case, it can be supposed that the line contains potentially a significant amount of text in this language.
parents
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment