In Jupyter notebook, you can convert a cell into a code cell, markdown cell or a raw cell. You need a markdown cell for superscripts and subscripts. A markdown cell can be created by selecting a cell then pressing the Esc key followed by the M key.
Then you can input the following code that uses latex with markdown to represent sub/super-scripts:
For Latex subscript:
For Latex superscript:
You can find more detailed examples here.
Hope this answer helps.