There is no one way to determine whether the coherence score is good or bad. We calculate the global coherence of the topic in the same way as for the UMass coherence. In the original paper, those probabilities were estimated from the entire corpus of over two million English Wikipedia articles using a 10-words sliding window. Where is probability of seeing word in the sliding window and is probability of appearing words and together in the sliding window. Similarly, as for the UMass score, we define the UCI coherence between words and as

Therefore, if both words and appeared in the document but they’re not together in one sliding window, we don’t count as they appeared together. It means that if our sliding window has a size of 10, for one particular word, we observe only 10 words before and after the word. Instead of calculating how often two words appear in the document, we calculate the word co-occurrence using a sliding window. T his coherence score is based on sliding windows and the pointwise mutual information of all word pairs using top words by occurrence. Basically, it means we want that each document to have as few as possible articles, and each word belongs to as few as possible topics. We can do the whole process of training or maximizing probability using Gibbs sampling, where the general idea is to make each document and each word as monochromatic as possible. Where and define Dirichlet distributions, and define multinomial distributions, is the vector with topics of all words in all documents, is the vector with all words in all documents, number of documents, number of topics and number of words.

Maximize the probability of creating the same documents.įollowing that, the algorithm above is mathematically defined as.Sampling words and creating a document – initialize the Dirichlet distribution of topics in the word’s space and choose words, for each of the previously sampled topics, from the multinomial distribution of words over topics.Sampling topics – initialize the Dirichlet distribution of documents in the topic’s space and choose topics from multinomial distribution of topics over a document.

Semantic coherence examples