Attention is not quite all you need

Whilst working on MacGraph, a neural network that answers questions using knowledge graphs, I came across a problem: How to tell if something’s not present in a list. In this article, we’ll show you our solution, the focus signal, and show how it performs reliably across a range of different datasets and architectures.

Read the full article

November 11, 2018