Page Summary
-
Dot product considers both angle and magnitude of vectors, making vectors with larger magnitudes more similar despite the angle between them.
-
Cosine similarity only considers the angle between vectors, disregarding their magnitudes.
-
Switching from cosine to dot product for similarity calculation makes popular items (with larger vector magnitudes) more similar to all other items.
-
Switching back from dot product to cosine similarity normalizes the effect of magnitude, making popular items less biased towards similarity with all other items.
In the image above, if you want
b
to be more similar to a
than to c
, which measure should you pick?Dot product
Correct! The dot product is proportional to both the cosine and the
lengths of vectors. So even though the cosine is higher for the angle
formed by
b
and c
than for the angle formed
by b
and a
, the greater vector length of a
makes the dot product of a
and b
greater than the dot product of b
and c
.Cosine
The cosine depends only on the angle between vectors, and the smaller
angle \(\theta_{bc}\) makes \(\cos(\theta_{bc})\) larger than
\(\cos(\theta_{ab})\).
Euclidean distance
The distance \(\vec{bc}\) is smaller than \(\vec{ab}\), leaving
b
closer to c
than to a
.You are calculating similarity for music videos. The length of the
embedding vectors of music videos is proportional to their popularity. You
switch from cosine to dot product to calculate similarity. How does
similarity between music videos change?
Popular videos become more similar
to all videos in general.
Since the dot product is affected by the lengths of both vectors, the
large vector length of popular videos will make them more similar to all
videos.
Popular videos only become more similar
to other popular videos.
Recall that the dot product is calculated as \(|a||b|\cos(\theta)\).
Assuming
a
is a popular music video, we know its
embedding length, \(|a|\), is larger than that of unpopular videos.
The larger length increases the
dot product irrespective of the value of \(|b|\). Hence, popular videos
become more similar to all other videos, not just other popular videos.Popular videos become less similar
to each other than to
less popular videos.
Since dot product increases with vector length, and popular videos have
high vector length, the similarity measure will increase, not decrease.
No change.
Dot product is affected by vector length. The high vector length of
popular videos will change the (previously cosine) similarity measure.
Given the same setup as the previous question, suppose you switch back
to cosine from dot product. How does similarity between music videos change?
Popular videos become less similar
to all videos.
Because cosine is not affected by vector length, the large vector
length of popular video embeddings does not contribute to similarity.
Thus, switching to cosine from dot product reduces the similarity
between popular videos and all other videos.
Popular videos become more similar
to all videos.
Cosine is not affected by the vector lengths, so switching from dot
product will cause the similarities for all popular videos to decrease.
Popular videos become more similar
only to each other.
Cosine is not affected by the vector lengths, so switching from dot
product will cause the similarities for all popular videos to decrease.
No change.
Since cosine is not affected by vector length, while dot product is,
switching from dot product to cosine will change similarities.
