The library is being designed to be simple to use and support a wide range of language and cognitive skills.
This page describes the core concepts of the library.
The basic system just has a parser, a composer, and an executor. Everything else is done by (external) modules. This makes it lightweight, extendible, and easier to understand and test. Some modules are linked rather directly to the system, but the interaction is minimal, and uses interfaces.
The library uses Entity Unification Grammar, a formalism I developed to make semantic composition easy. Easier, that is, than Montague grammar and Feature Structure Unification, in the sense that the cognitive load of developing a new rule is lower: the number of semantic operations is low, and picking the right operator is usually an obvious choice.
It is explained here.
Historically there are two ways to do syntactic analysis and semantic composition. The first analyses the sentence using a large amount of code that has a strong learning curve. The other uses syntactic rewriting rules, but the semantic composition rules are hard to understand and apply.
This library uses syntactic rewrite rules and has aimed to make the semantic composition as easy as possible. The developer attaches semantic forms to syntactic tree nodes that are just the most basic logical representation.
One of the ways to keep the composition simple is to leave quantification out of the composition process, and offer a built-in function that transforms the structure into a quantified form. The composition, which is done manually is therefore easier, while the transformation is done by the function.
Where most systems try to create a single SQL statement to execute a sentence, this system breaks the query down to a series of select statements and executes these. The result is not SELECT COUNT(*) FROM .. JOIN .. JOIN .. GROUP BY ...; rather all database queries involve only a single table, a few simple selects and some simple where clauses: SELECT a, b, c FROM t WHERE a = 1 anc c = 2.
Advantages
- Creating an adapter to a new database is simple
- Custom application logic can be included during the query
Disadvantages
- Doing the work in the application is always slower than letting the database do the work
The advantages may not seem to weigh up against the disadvantages. But consider this. Many natural language sentences can't be solved by a single database query, and require multiple interactions between data access and data processing. That's where this approach shines.
The disadvantage of accessing the database in an inefficient way is tackled by optimizer functions (OptimizerModule).
The meaning of questions, commands and statements are fundamentally different.
Questions require quantifier functions that execute during the answering process. They also need to be optimized for speed or they easily may become slow quickly. They may create clarification questions, but other than that they return immediately.
Commands have a different means of quantification. They often don't return immediately, because the command needs to be executed, and this takes real time. The response to the question is given before the command is executed in full. Also, the command may require planning before execution.
Statements have yet another implementation of quantification. A statement may be stored as a fact or as a rule. The rule may have exceptions. The fact may conflict with known facts.
This system creates an intermediate semantic form that is then transformed into a form suitable to each of these sentence types.
I continue with the lesser insights.
Many historical systems model agreement in syntax rules: person, number and gender agreement. The subject and the verb must have the same value for each of these features. But if we take into account that most people won't enter sentences like "How many blocks is on the table", and at the same time admit that the meaning of such a sentence is still the same as the grammatical one, you understand that we can do without.
Agreement is very important to decide between grammatical and ungrammatical sentences, and hence for what we call generation or production, but it is much less important for sentence analysis.
The demos shown here have all been implemented without regard for agreement.