Distributional semantic models (DSMs) generally
require sufficient examples for a word
to learn a high quality representation. This is
in stark contrast with human who can guess
the meaning of a word from one or a few
referents only. In this paper, we propose
Mem2Vec, a memory based embedding learning
method capable of acquiring high quality
word representations from fairly limited context.
Our method directly adapts the representations
produced by a DSM with a longterm
memory to guide its guess of a novel word.
Based on a pre-trained embedding space, the
proposed method delivers impressive performance
on two challenging few-shot word similarity
tasks. Embeddings learned with our
method also lead to considerable improvements
over strong baselines on NER and sentiment
classification.