Learning manipulation skills can be viewed from three perspectives: skill acquisition, skill adaptation and skill generalization. In skill acquisition the challenge is to build a robust primitive skill. Skill acquisition is often done by transferring a skill from a human to the system. The process, called programming by demonstration or imitation learning, is usually solved as a supervised learning problem. The transferred skills can further be adapted to new contexts, for example by reinforcement learning. Finally, the ultimate goal of learning is to provide generalizable skills that can be applied in different contexts. I will review our recent work that aims to build these generalizable skills through a combination supervised and reinforcement learning.