The battery will NOT work. It almost certainly uses a 3.6v li-ion cell, probably in the 500-1000mah range. Not at all usable for a ps1 portable.
Mini DVDs: No. No DVD reading laser (Or other hardware or software support)= No DVD
UMD: Not with the stock hardware. If you somehow (Very unlikely, but play along) replaced the ps1 drive with a UMD drive... Sure, it would work. Only one problem: There is no such thing as a blank UMD. Also, there are no UMD burners in existence. In addition, the format is not well understood. So, while possible (Barely) in theory, it is realistically impossible.
@Blackbox dev: PSP tv out needs progressive scan. I looked into exactly what you are describing, but on a bigger scale. The psone display is not all that much bigger than the PSP one.
Offtopic: I used some of the parts from a UMD drive to try to make an uber small gamecube disc drive... I made a fair bit of progress. Not using the laser/encoder/complicated UMD stuff, just the disc holding system, slot, and hinge.
