If you had a robot who always tied your shoes for you, would you ever have learned how to tie your shoes yourself? What about if that same agent did all your arithmetic and all your writing, and eventually shaped all your decisions? The promise of AI is fraught with ethical questions that strike at the very heart of what it means to be human and to act as a moral agent in society. It reveals a fundamental tension between what AI can do and what AI should do. In the modern world, that tension is growing.