Can upper-level managers see the performance reviews of all the employees under them?

Yes, they can. Managers can see the performance reviews of their direct reports from the Team Dashboard, which can be found in the Performance menu.