PRISM-VO is a pure optimization-based sparse and direct visual odometry framework for focused plenoptic cameras that tightly integrates plenoptic geometry into a photometric bundle adjustment formulation.