Clustering and generalization of abstract structures in reinforcement learning and musicality

Humans are remarkably adept at generalizing knowledge between experiences in a way that can be difficult for computers. Previous computational models and data suggest that rather than learning about each individual context, humans build latent abstract structures and learn to link these structures to arbitrary contexts, facilitating generalization, but with a cost in efficiency of initial learning. In these models, task structures that are more popular across contexts are likely to be reused in new contexts. Neural signatures of such structure learning are predictive across individuals of the ability to transfer knowledge to new situations. However, these models predict that structures are either re-used as a whole or created from scratch, prohibiting the ability to generalize constituent parts of learned structures. This contrasts with ecological settings, where task structures can be decomposed into constituent parts and reused in a compositional fashion. Moreover in many situations people can transfer structures that they have learned to entirely new situations, by analogy, even when surface aspects of the transition and reward functions change. I will present novel computational models across levels (from neural networks to bayesian formulations) that address how agents and humans can learn and generalize such abstract and compositional structure. Throughout, I will give examples of how such computations can allow a musician to learn to compositionally transfer musical scales and rhythms within and across instruments.