Some behaviors of ensembles operating in open-ended, changing environments cannot be completely specified during their design. Instead, the ensemble has to learn from experience and reason about novel situations as they arise. Using this approach, initial behaviors are learned by the ensemble from simulations of likely environments during design time; at runtime these behaviors are improved as the ensemble learns how its actual environment differs from expectations.

Iliad is a framework for this style of learning and reasoning. It supports deep learning and hierarchical reinforcement learning, predicate-logic reasoning with integrated support for constraint processing, inference in Bayesian networks, and heuristic planning.

Iliad’s input language is called Poem. In Poem, programmers can leave choices of actions or values partially unspecified and indicate which learning or reasoning mechanisms should resolve the non-determinism of each choice. Therefore developers can either establish fixed behaviors, indicate design-time preferences or simply state the possible actions. Iliad will optimize these choices either by reasoning or by learning from feedback provided by the environment. Given sufficient knowledge or training, the actions determined by Iliad will converge to those with the highest expected value for the environment in which the ensemble is operating.

For example, in the rescue scenario, robots may combine map-based navigation planning with learning and reasoning about the environment to update inaccurate design-time assumptions with data gained at run time. This often leads to significantly better initial performance when compared to solutions solely based on learning; but even if the initial knowledge is highly inaccurate the learning mechanism will cause the behavior to improve as the robots obtain information about their environment at run time. In the depicted example, the design data was randomly initialized; in spite of this the performance of the system approached the optimal performance after a relatively short time.

Iliad can either be used as a knowledge repository for SCEL or as a stand-alone reasoner. Poem code can also be generated from executable parts of SOTA/GEM specifications written in the Prototype Verification System (PVS) specification language.

Further Information:

• User Guide
• Download