jPSXdec is a cross-platform PlayStation 1 media decoder/converter.
Get the latest version here.

Wednesday, September 24, 2008

How to encode quality PlayStation 1 video

Update: I have since discovered this technique may work, but videos would need to be encoded as mpeg2 due to PlayStation videos having 2 more bits of quality than mpeg1 can handle. Unfortunately that starts getting tricky as mpeg2 has a different chroma subsampling position. In short, this technique has a lot of subtle caveats that should be considered. A future post may be made to explore the issues.

Recently I've run across a couple of groups trying to replace the video of a PlayStation 1 game. As we have to do the same thing with S.E. Lain, I thought I would share some insight on how it might be done with the highest quality output.

Since there are many things that go into encoding mpeg type images (DCT + quantization being the least of it), that task is best left to programs that know how to do it well, such as ffmpeg. It also allows you to tweak those options, and use its advanced features to get the best results.

So here is how you could do it, broken down into 6 ridiculously complicated steps.

  1. Convert your video into yuv4mpeg2 video (or perhaps an AVI with a YCbCr codec), but using the PSX specific rgb->YCbCr conversion.
  2. Feed the YCbCr video into ffmpeg at a mpeg1 allowed fps and create a mpeg1 movie with only I-frames (-intra). This should be done at a variety of different quality levels (-qscale from 1 to n).
  3. Parse the I-frames out of the movie, and parse each frame's macro-blocks
  4. Convert the mpeg1 VLCs to PSX VLCs
  5. Do this for every mpeg1 quality and pick the one that fits best within the amount of space available to each frame
  6. Multiplex the frames and construct all the sectors, including the correct frame headers and the sectors' ECE/EDC codes

If the new video is mostly just the old video with some changes (e.g. subtitles) then quality can improved tremendously with this variation.

  1. Use some method to determine which macro blocks to replace (manually picking them, or performing a diff on lossless video data, or a fuzzy diff on lossy video data)
  2. Only replace those macro blocks with new ones of the same qscale
  3. If the replaced data makes the frame too big, then either replace the entire frame with an ffmpeg created frame that will fit (qscale will probably be bigger), or remove some quality in the frame to make it smaller
  4. Multiplex the frames and construct all the sectors

Alternatively have ffmpeg write an AVI using MJPG codec, then parse the JPEG frames and convert the JPEG VLCs to PSX VLCs (I'm not sure how to get ffmpeg to produce quality variations with the MJPG codec).

Encoding video by the steps above will likely have better results than if you used Sony's official PS1 SDK video encoder (and will also reduce the likelihood of serious legal trouble and C&D letters).

Here is a possible ffmpeg command-line for step 2 above.

ffmpeg -i source.y4m -vcodec mpeg1video -r 30 -qscale #  \
-trellis 1 -intra -dc 10 -dct faan -debug dct_coeff \
-vstats_file qscale#.log out-qscale#.m1v

-i source.y4m : yuv4mpeg2 input file
-vcodec mpeg1video : write mpeg1
-r 30 : use 30 frames/sec
(just to use something
compatible with mpeg1)
-qscale # : chosen qscale
-trellis 1 : trellis quantization
does this even apply?
-intra : only write I-frames
-dc 10 : use 10 bits of precision
for the DC values?
(not sure if this is helpful)
-dct faan : use a floating-point
discrete-cosine-transform
for best quality
-debug dct_coeff : forgot what this does
-vstats_file qscale#.log : write a log of useful info
out-qscale#.m1v : output file, video only

Monday, July 14, 2008

zOMG psx.lain.pl is down!

And it has been for quite awhile. I thought it was just temporary, but finally learned yesterday that the good fellow that was nice enough to host the page has chosen to close the account. We are grateful for the time we could use the server.

Don't worry, we still have all data and work done so far. Unfortunately we're currently without a host, and the project is still stuck at around 14% completion.

I personally would like to thank everyone that has contributed thus far...

Hikari - Translated dialogs from video sequences to Polish.
phm - Extracted the media with jpsxdec and setup the wiki. Retranslated video sequences dialogs from Polish to English. Advertised for translators, and coordinated with their changes.
MercuryTW - Provided some translations from a translator of his own (Keigo). Edited/corrected translated files.
stalker-kun - Provided hosting for the wiki.
otakufish - Provided some translations.
toruvinn - Provided nice domain.
farhan - Ruthlessly proofread random translations, utilising his qualifications in the field of Applied Pedantry with Grammar Nazism.
fishy - Proofread translations, created the public project page.
arc - Provided some translations.
Jossos - Provided some translations.
utakata - corrected a Japanese transcription.
Najica - Provided some translations.
Quibbage - Created subtitle files for videos.

...and everyone else (sorry if I've missed your name here)--thank you all very much.

jPSXdec development has been pretty quiet as well. I should submit the bug that fails to decode some of the Lain videos, or just commit the fix. The next things to do with jPSXdec require some major changes, and things have been busy (my toon won't hit level 60 by itself ;). I'm still here and interested in the project and related topics. I don't plan on that changing anytime soon.

Sunday, May 4, 2008

Sonic

AVI
Fixed the obvious problems with AVI writing, but still problems persist.
* My Windows Quicktime install must be bad because it crashes from like everything (but Mac Quicktime plays fine)
* My Linux mplayer must be bad because it plays all movies stretched, one way or another
* All programs on my lappy crash from my movies except Windows Media Player
* My Linux Totem still plays it with green garbage

So I think I may be doing MJPG slightly wrong, but I'm not sure what. Plus I finally figured out why I could never find any documentation on the MJPG codec: "there is no document that defines a single exact format that is universally recognized as a complete specification of “Motion JPEG” for use in all contexts" Wikipedia

Native decoder
Got the native decoder working great on Windows, Linux and Mac. CMake has made cross-platform building a breeze :D It's really exciting to see the decoding plow through dozens of frames per second.

Indeed, native decoding is fast...

Java is faster
The heck?? Contrary to what I thought I saw in my initial tests, my swift Java decoding module is significantly faster than using the native decoder. How? Not sure, but I suspect it's due to all the overhead needed when calling a JNI function (such as changing big-endian to little-endian). It already melts through 150 sectors/second like a hot knife through butter--but since it spends half its time with garbage collection, could it be faster with some object pooling?

Tuesday, April 22, 2008

v0.34

I just needed to get something for a public release. It was the re-implementation of the command-line that kept hanging me up (it was boring and I wanted to work on other things ;). I admit in my rush to just get something released, I sorta skimped on the code commenting. :P

Testing

Before releasing, I randomly tested jpsxdec on the dozen or so games that my friends let me borrow. As a result I threw in some last-minute changes to handle a couple new movie types. 'Last minute' means these may not even work right (I look forward to finding a way to save variable frame rate movies ^^).

I ran some quick tests on my Linux Mint machine. Compiz was often rendering the progress window without controls, so there was no way to close the program! And there's an annoying JFileChooser 'All Files' filter bug... :P (just wrote a work-around a moment ago).

My tests on a Mac were a little more promising. Rendering was slick--even looked better than Windows rendering.

After seeing how jpsxdec worked on Linux, I was worried about the quality of jpsxdec--then I saw iTunes for Windows and was feeling pretty good again.

AVI writing is faulty

Seems my AVI writer is getting something wrong because it is crashing or otherwise playing poorly on Linux by VLC ('this avi is broken'), mplayer (plays upside-down, stretched, or just crashes), Totem player (green gunk on the screen). On Windows and Mac, Quicktime is having problems (plays video too slowly or crashes). Real Player also crashes the good crash.

Speed

Since much of the core decoding work for jpsxdec is finished, I started looking for something a little more exciting to work on. So, following MrVacBob's suggestion, I began writing a crazy fast Java decoder. After doing everything I could think of, and examining the work of previous coders, I came to the conclusion that a real-time PSX Java decoder may be possible, but the resulting code would be so ugly, long, and annoying, it just isn't worth it. I hope to complete a faster Java decoder that'll cut the decoding time by maybe half, but also make a small, cross-platform native library will be fast enough for real-time playback.

My friends keep dropping games on me--more than I can keep up with. It takes so long to rip games that I went ahead and threw together raw cd reading on Windows. It's pretty buggy, and I'm sure I'm doing something wrong, but it mostly worked. If ever I can get it more stable, I'll include it. For my Linux coding friends, the JNI Java class is in the source if you feel like writing a Linux version of the lib :)

Licenses

The LGPL of the Swing libraries included with jpsxdec don't require that I distribute the source code of the libraries (I haven't changed them, and I am linking them separately). But since jpsxdec is GPL, does that require me to include the LGPL sources with the distribution? I really didn't want the hassle of it, but just to be safe I included the source with the pre-combiled binaries together in the zip files.

Thursday, March 13, 2008

Incoming

Been wanting to get some stats on jpsxdec code for a long time (Google code doesn't have such features), finally ran across StatSVN and took a quick look at the results.


Now running over 20,000 lines of code, and almost 100 files. Compared to Oct 21, 2007's 7,000 lines, that's 13,000 LOC over 144 days, for a measly 90 lines per day.

Anyway, I'm guesstimating next week will be the next jpsxdec release. It will have some exciting features:
  • Saving to AVI, both uncompressed and MJPG compressed your mileage may vary
  • A very nice dialog box providing the many decoding options available it's arrite
  • Final Fantasy 9 finally working properly done!
  • Chrono Cross is also working! mostly done!
  • Cleaned up command-line done!
  • and lots of delicious code documentation :)

Monday, February 25, 2008

lain? lain!

The Serial Experiments Lain PSX Game Translation Project is really coming along. A few more wonderful translators have generously offered some help. We have roughly 7% of the game translated already.

Here are some exciting fruits of everyone's efforts so far (actual screenshots).



Sunday, February 17, 2008

nihongo niban

Much awesomeness has transpired recently. But first, the dull ramblings of jpsxdec status.

Looked up how to encode uncompressed (RGB) AVI files. It's not too complicated--Microsoft and other sites have usable documentation. Even found a Java class to get me started. But I got really excited when the hidden art of MJPG encoding was uncovered. It is so ridiculously easy, I can understand why no one ever describes how to do it.* Of course if we're gonna write AVI files, we need to know the frame-rate. Just a simple 400 lines of code is all that took.* Needed to also change the overall code design to a 'push' architecture with listeners. This had the additional bonus of pulling the actual file writing to the top-level interface--right where it belongs. :D

* /sarcasm

But overall things are looking pretty good. For the next release I want to add a new popup window showing the many decoding options.

Also in the future I want to allow for piping out uncompressed AVI to a program like ffmpeg or mencoder. For Windows I'll also include the infamous Video for Windows. Still need to add raw cd reading.

But while merrily carrying on with all this, out of nowhere came a fishy fellow with the powers of language transformation! He generously offered to spend some time with the S.E. Lain game. Sooo... efforts on jpsxdec are delayed as the Serial Experiments Lain PSX Game Translation Project commences (SELPGTP?).

But with 5.5 hours of audio, he has his work cut out for him. Do you share the powers of language transformation? We could really use your help.

On the technical side, we are working on adding the eventual translation into the game. And I present to you an actual screen shot based on an earlier idea.

Saturday, February 2, 2008

Amalgamation

Been busy with a bunch of different aspects of jpsxdec. Only the GUI and Code Design will be in the next release (v0.31 today!). The rest will show up in releases after that.

GUI
Even though it's a pretty lame GUI now, I hope it will drop the bar of entry a little.


Code Design
I had difficulty deciding on highest-level interface between the GUI/CLI and the logic. So I set it aside and worked on other things.

PSX YUV->yuv4mpeg2 conversion matrix
This will allow jpsxdec to output more accurate .yuv image sequences.

Frame Rate Calculation
I thought it was a good idea to guess the exact FF7 frame rate algorithm via trial-and-error. 8 hours later I finally saw that FF7 frame lengths do actually follow a pattern, but a ridiculous one. I hoped it would come to a frame rate of exactly 15/100.1% (half the rate of NTSC). That is close, but there was no way I could guess the underlying algorithm of this...


if (iMovieType == WACKED_NTSC1 && iThisFrame == 101)
iThisFrameLength = 11;
if (iMovieType == WACKED_NTSC2 && iThisFrame == 100)
iThisFrameLength = 11;
else if (iThisFrame == 200)
iThisFrameLength = 11;
else if (iThisFrame >= 203 && iThisFrame <= 298) {
if (((iThisFrame - 203) % 4) == 0)
iThisFrameLength = 11;
else if (((iThisFrame - 203) % 4) == 1)
iThisFrameLength = 9;
else
iThisFrameLength = 10;
}
else if (iThisFrame == 299)
iThisFrameLength = 11;
else if (iThisFrame >= 303 && iThisFrame <= 398) {
if (((iThisFrame - 303) % 4) == 0)
iThisFrameLength = 11;
else if (((iThisFrame - 303) % 4) == 1)
iThisFrameLength = 9;
else
iThisFrameLength = 10;
}
else if (iThisFrame == 399)
iThisFrameLength = 11;
else if (iThisFrame >= 402 && iThisFrame < 497) {
if (((iThisFrame - 402) % 4) == 0)
iThisFrameLength = 11;
else if (((iThisFrame - 402) % 4) == 2)
iThisFrameLength = 9;
else
iThisFrameLength = 10;
}
else if (iThisFrame == 498)
iThisFrameLength = 11;
else if (iThisFrame == 501)
iThisFrameLength = 11;
else if (iThisFrame >= 504 && iThisFrame <= 600) {
if (((iThisFrame - 504) % 4) == 0)
iThisFrameLength = 9;
else if (((iThisFrame - 504) % 4) == 2)
iThisFrameLength = 11;
else
iThisFrameLength = 10;
}
else
iThisFrameLength = 10;

Raw CD reading (Windows)
Made a native program to read raw CD sectors via DeviceIoControl. It's a simple interactive prompt that accepts commands to open/close a CD and read sectors. I even setup a Java program to communicate with the native program via stdin/out. Unfortunately it was slow, and was very vulnerable to deadlock while reading from stdin. I also started adding ASPI for raw CD reading, but there seems to be a lot more to using that API.

Encoding via VFW (Video for Windows)
I worked out a little program to convert a series of raw image and audio data into an AVI, using the oh-so-familiar Video for Windows popup. I wonder if it can handle accepting all the audio at once, then all the video at once....


JNI
Even though I said it would be a pain in the butt, I guess it's really not so bad. It's kinda like translating VB6 COM code into C++. This will likely speed up raw CD reading, and save me from communicating via stdin/out.

Code Design
Finally got back to this again. Since no solution was particularly attractive, I went with the simplest one. This will only use one optional callback listener (for progress status).

However, I have begun to think that the 'pull' architecture is not the best approach to begin with. A 'push' architecture is more applicable to this situation: as sectors are read from the disc, they need to be 'pushed' into the appropriate streams. Listeners can be added to collect the streams of interest. If jpsxdec was to ever play media in real time (not likely), this would be the necessary design. In any case, the 'push' approach would make decoding a little faster because there is less seek time.

Sunday, January 20, 2008

PSX YCbCr to yuv4mpeg2

PSX YCbCr

Given Y, Cb, and Cr color range of [-128, 127].
                              PSX
[ 1 0 1.402 ] [ Y + 128 ] [ r ]
[ 1 -0.3437 -0.7143 ] * [ Cb ] = [ g ]
[ 1 1.772 0 ] [ Cr ] [ b ]

yuv4mpeg2 (i.e. Rec.601) YCbCr

Given Y color range of [16, 235] and Cb,Cr color range of [16, 240].
                              Rec.601
[ 1.164 0 1.59 ] [ Y - 16 ] [ r ]
[ 1.164 -0.391 -0.813 ] * [ Cb - 128 ] = [ g ]
[ 1.164 2.018 0 ] [ Cr - 128 ] [ b ]

To convert a PSX YCbCr color to a Rec.601 YCbCr color:

[ 1 -3415973/13224846875 1242172/13224846875 ]
[ 0 105814197/105798775 -5608/105798775 ]
[ 0 19492/105798775 105791687/105798775 ]


Answer: yuv4mpeg2^-1 * PSX =
[ 1 3415973/13225888625 -1242172/13225888625 ]
[ 0 105791687/105807109 5608/105807109 ]
[ 0 -19492/105807109 105814197/105807109 ]


Rec601_YCbCr =
PSX
[Y + 128] [ 16]
Rec601_Matrix^-1 * PSX_Matrix * [Cb ] + [128]
[Cr ] [128]
Where
Rec601_Matrix^-1 * PSX_Matrix =

[250/291 -488509/2660418030 -82738/1330209015]
[0 4014411/4571165 164/4571165]
[0 3673/27426990 8031459/9142330]

Thanks to toruvinn for refreshing me on matrix math, and this site and Maxima for saving me from lots of raw calculations.

Update 27Feb2010: Fixed.
Update 23Mar2010: Really fixed.

Sunday, January 6, 2008

HATSUHINODE

Hope all had a pleasant new year. As we enter 2008, it is interesting to note that this year marks the 10th anniversary of the Serial Experiments Lain anime series, along with the Playstation game.

Now that holidays are over, I finally managed to release the latest compiled executable on the download page. No changes to the program interface (same wonderful command-line that you're used to ;), but it has all the latest internal goodness that I hope actually works--most notable is it now [edit] mostly supports FF8 (it would help to fully test it before making such calims :P).