Memory Lapses or Confusion? When It’s Time to Add Oversight While Maintaining Autonomy