Schema2QA: Answering Complex Queries on the Structured Web with a Neural Model

Building a question-answering virtual assistant skill currently requires large annotated datasets, which are too expensive to acquire for all but the largest companies. This paper proposes Schema2QA, an open-source toolkit that can build a Q&A skill from a database schema, requiring just a few manual annotations on each field. The key concept is to cover the space of possible compound queries on the database with millions of domain-specific questions synthesized with the help of a corpus of generic query templates. The synthesized data and a small paraphrase set are used to train a novel neural network based on the BERT pretrained model. We apply Schema2QA to two different Schema.org domains, restaurants and people, and show that the skills we built can answer complex queries accurately. Our skills achieve an overall accuracy between 74% and 80% on crowdsourced questions for the domains. Any websites with Schema.org metadata in these domains can use Schema2QA to build their website-specific skill automatically. We show that with transfer learning from the restaurant to the hotel domain, we can achieve a 65% accuracy on crowdsourced questions with no manual effort. On the restaurant domain, Schema2QA outperforms the best commercial assistant by 21% on a set of diverse crowdsourced questions, while achieving equally good accuracy on questions commonly asked to existing virtual assistants.