I suppose it may come down to the tools you’re using. If you can base64 encode without it making things noticeably less performant, this would likely be the easiest option as your tools could treat this data just like everything else and the client-side can then decode it (knowing that the field is base64-encoded jpeg).
Otherwise if you’re doing this OOB, maybe it’s best not to combine it with the JSON API interface (i.e. /person/1/photo) and provide it a new namespace as to avoid confusion in the future between the spec and special use-cases.
That said, what seems more enticing to me is a JSON API extension. If there was a binary extension, you could return the result and notify the client in the header that some binary format will be coming back. This would have to be well-formed and specified, of course, but may be the most seamless in the end.
Anyway, these are all the interesting engineering questions in the first place 
I am a bit curious to see how you handle this in the end anyway (to date, I don’t think I’ve encountered this use-case yet), so be sure to report back!
Similarly, this is the latest discussion I could find on the matter. In short, base64 is the recommendation to remain spec compliant.