Close the world, Open the nExt.

After watching and enjoying the Serial Experiments Lain anime series, my interest was piqued by something I saw on the disc extras.

A Lain Playstation game.

And I had to own it.

With the help of my Japanese co-worker during my trip to Japan, I managed to acquire a copy all of my own.

Jen's site was accurate when it said there was very little info about the game. Following her lead, I wanted to change that. I wanted to decode the video for all to see. The audio could be ripped just fine, but the video refused to decode. Endless Googles later and I was still left wanting.

While toying with the game in an emulator, it dawned on me that if I couldn't decode the video off the disc, maybe I could let the game do it for me. I got the open source P.E.Op.S. GPU plug-in and went to work grabbing the video frames before anything was drawn over them. Soon I had nearly all the media from disc 1 sitting on my hard drive.

However, I ran into a snag. The decoded audio from the disc wasn't lining up with the video frames I had captured.

I had big plans, but was burning out and couldn't ever find a reason for this discrepancy. The project was abandoned for another day.

Two years later...

Finding myself still stumped over the audio/video mismatching, I veered a different direction. Could I possibly learn how to decode PSX videos myself?

Another endless bout of Googles later and I learned about JPEG and MPEG encoding. Soon I had my own I-frame decoder. It was the Q-gears source code that finally pushed the decoder into aligning with PSX data. It was done. I was seeing video frames from games I'd never seen before, and without ever touching someone else's decoder.

I could do what existing decoders could do, but what of Lain? What is with all this video data? It doesn't add up to anything sensible. Again I hit a snag. The only way to know what the data means is to see what the game is doing, and how could I do that?

I was just about ready to settle with my mis-matched audio and video frames, but I couldn't leave well enough alone. I ran across the pSX emulator that provides debugging capabilities. So over a three day weekend I swam through a sea of R3051 assembly code. By the end I knew how it is all done.

Outraged at the lack of information and source code about the PSX, I decided to take my little decoder to the big time and share everything I have learned over the last few months.

And thus is the jPSXdec.

Open Sauce

As mentioned in the project goals, the code will be open source. And I want to make sure this source code stays open and free (unlike so many hacking tools before this one). As such, I plan on releasing jPSXdec under GPL v3 GPL v2 (for better compatibility).

Initially, I thought simply providing the source compressed in an archive on this site would be sufficient. I've mentioned a few ways people could help with the development if they were interested. I kept these very modular, so incorporating them would be fairly easy. I also didn't think the source code would be very large.

But there has been a request to setup a code repository instead. I've also realized the size of the source code is already non-trivial (almost 7000 lines of code), and will only grow larger (especially if a GUI is added).

I'm all for setting up a code repository, however, I don't know anything about existing hosting sites (except SF), nor do I have any experience setting up or maintaining such a site. I wish I could list some criteria, but I'm not even sure what I'll need. If anyone has any thoughts/info/suggestions/insight, I'd be interested in hearing them.

Update: After carefully weighing the different options, I found that Google Code was the best choice after all (thanks phm).

I will also be releasing two tidbits of documentation:

• documentation about the code, describing how the different classes are organized and used.
• a lengthy text file describing the STR file format and decoding process (nearly complete).

While I'm somewhat familiar with the GPL license, I've never had any experience with licensing documentation. It seems there are two recommended options: GNU FDL and FreeBSD Documentation License. After reading a little about them, I'm not sure which to choose. Again, if anyone has any thoughts, please share.

Update: Went with MIT license.

Thanks!

Links

Numerous links were followed during the development of this program. All of these were valid at the time I followed them. I'm just throwing these in here for now. I'll clean it up later.

PSX Tools
http://hitmen.c02.at/html/psx_tools.html
http://members.fortunecity.com/zlasher/files/toolz.html
http://www.psxforum.com/utilities.php
http://www.geocities.com/anapan8/cv/

Decoding links
http://jcatki.no-ip.org/cdxa/
http://www.momentaryfascinations.com/technology/the.spyro.soundtracks.html
http://forums.qhimm.com/index.php?topic=6473.0
http://forums.qhimm.com/index.php?topic=6140.0
http://q-gears.svn.sourceforge.net/viewvc/q-gears/trunk/src/common/movie/decoders/
http://wiki.multimedia.cx/index.php?title=PlayStation_Motion_Decoder
http://osdir.com/ml/video.xine.devel/2003-02/msg00179.html
http://osdir.com/ml/video.xine.devel/2003-02/msg00186.html
http://samples.mplayerhq.hu/game-formats/psx-str/
http://www.koders.com/c/fidF23C1EAFCEAF84CA539927A01093D37D9695722A.aspx

MPEG1 links
http://bmrc.berkeley.edu/research/mpeg/faq/mpeggeneral.html
http://www.chiariglione.org/ride/inside_MPEG-1/inside_MPEG-1.htm
http://www.andrewduncan.ws/MPEG/MPEG-1_Picts.html
http://www.geocities.com/xhelmboyx/quicktime/formats/mpeg-layout.txt
http://www.faqs.org/faqs/mpeg-faq/
http://www.wotsit.org/list.asp?search=mpeg
http://guru.multimedia.cx/the-mpeg124-and-h26123-idct/
http://www.uow.edu.au/~nabg/MPEG/IDCT.html

Other Info
http://forum.digital-digest.com/showthread.php?t=17898
http://www.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=000tS4
http://groups.google.com/group/alt.games.sony-playstation/browse_thread/thread/3bb9cc7ce10bbb58/02098464ed19f81a?hl=ened19f81a
http://www.megagames.com/psx/psx_copy_patch_linux.shtml
http://rpgd.emulationworld.com/klarth/

Goals and Roadmap

I have some simple goals for this project.

• Cross-platform
• Open source
• Very well documented
• Very clear, direct, non-obfuscated, and well documented source code (specifically the video decoding portion)
• More stable than existing decoders
• Capable of decoding all types of audio and video that existing decoders can, plus more
• Highest decoding quality possible

The program is written in Java because it helps meet many of these goals.

Unfortunately, the goals to make the source code as clear as possible, and highest decoding quality, have the side effect of making the decoder rather slow. Right now it decodes between 0.5-4 frames per second on my 2.8Ghz machine. The only way I plan on improving the speed is by using a disgustingly obfuscated IDCT. Since that is by far the slowest part of the decoder, it could help a lot.


The first version will simply be a command-line tool. It will decode video into a series of images, and the audio into a wav file. It will handle standard STR files, plus a few special formats: Final Fantasy VII, Final Fantasy VIII (delayed to 2nd version), and Serial Experiments Lain.

The next version I plan to include raw CD reading (for Windows), fix bugs, and maybe add some more special game handling. I think I also want to add raw file copying off CDs.

Following that, I hope to add a GUI to the program. I'm thinking of duplicating much of PsxMC's interface.

I'm not sure what to do about encoding the movies into another movie format. Trying to use Video for Windows in Java would be a pain in the butt (read JNI), and it also wouldn't be cross platform. My current thought is to do what jMencode does: provide a GUI for open-source encoding programs. It appears mencoder can encode using the computer's Video for Windows codecs. But I know how much everyone likes, and is familiar with how Video for Windows works. So currently I don't have a clear plan of what to do.

Want to help? Come up with a GUI, an encoding method, or raw CD reading for Linux/Mac!

teh h4x0rz

0xyyuu0038rrrr0000
yy = Luminance quantization scale
uu = Chrominance quantization scale
rrrr = number of run length codes in frame

v3 DC Coefficients style
11s                     (0, 1)
011s                    (0, 2)
0100 s                  (1, 1)
0101 s                  (0, 3)
0010 1s                 (0, 4)
0011 0s                 (2, 1)
0011 1s                 (0, 5)
0001 00s                (0, 6)
0001 01s                (3, 1)
0001 10s                (1, 2)
0001 11s                (0, 7)
0000 100s               (0, 8)
0000 101s               (4, 1)
0000 110s               (0, 9)
0000 111s               (5, 1)
0010 0000 s             (0, 10)
0010 0001 s             (0, 11)
0010 0010 s             (1, 3)
0010 0011 s             (6, 1)
0010 0100 s             (0, 12)
0010 0101 s             (0, 13)
0010 0110 s             (7, 1)
0010 0111 s             (0, 14)
0000 0010 00s           (0, 15)
0000 0010 01s           (2, 2)
0000 0010 10s           (8, 1)
0000 0010 11s           (1, 4)
0000 0011 00s           (0, 16)
0000 0011 01s           (0, 17)
0000 0011 10s           (9, 1)
0000 0011 11s           (0, 18)
0000 0001 0000 s        (0, 19)
0000 0001 0001 s        (1, 5)
0000 0001 0010 s        (0, 20)
0000 0001 0011 s        (10, 1)
0000 0001 0100 s        (0, 21)
0000 0001 0101 s        (3, 2)
0000 0001 0110 s        (12, 1)
0000 0001 0111 s        (0, 23)
0000 0001 1000 s        (0, 22)
0000 0001 1001 s        (11, 1)
0000 0001 1010 s        (0, 24)
0000 0001 1011 s        (0, 28)
0000 0001 1100 s        (0, 25)
0000 0001 1101 s        (1, 6)
0000 0001 1110 s        (2, 3)
0000 0001 1111 s        (0, 27)
0000 0000 1000 0s       (0, 26)
0000 0000 1000 1s       (13, 1)
0000 0000 1001 0s       (0, 29)
0000 0000 1001 1s       (1, 7)
0000 0000 1010 0s       (4, 2)
0000 0000 1010 1s       (0, 31)
0000 0000 1011 0s       (0, 30)
0000 0000 1011 1s       (14, 1)
0000 0000 1100 0s       (0, 32)
0000 0000 1100 1s       (0, 33)
0000 0000 1101 0s       (1, 8)
0000 0000 1101 1s       (0, 35)
0000 0000 1110 0s       (0, 34)
0000 0000 1110 1s       (5, 2)
0000 0000 1111 0s       (0, 36)
0000 0000 1111 1s       (0, 37)
0000 0000 0100 00s      (2, 4)
0000 0000 0100 01s      (1, 9)
0000 0000 0100 10s      (1, 24)
0000 0000 0100 11s      (0, 38)
0000 0000 0101 00s      (15, 1)
0000 0000 0101 01s      (0, 39)
0000 0000 0101 10s      (3, 3)
0000 0000 0101 11s      (7, 3)
0000 0000 0110 00s      (0, 40)
0000 0000 0110 01s      (0, 41)
0000 0000 0110 10s      (0, 42)
0000 0000 0110 11s      (0, 43)
0000 0000 0111 00s      (1, 10)
0000 0000 0111 01s      (0, 44)
0000 0000 0111 10s      (6, 2)
0000 0000 0111 11s      (0, 45)
0000 0000 0010 000s     (0, 47)
0000 0000 0010 001s     (0, 46)
0000 0000 0010 010s     (16, 1)
0000 0000 0010 011s     (2, 5)
0000 0000 0010 100s     (0, 48)
0000 0000 0010 101s     (1, 11)
0000 0000 0010 110s     (0, 49)
0000 0000 0010 111s     (0, 51)
0000 0000 0011 000s     (0, 50)
0000 0000 0011 001s     (7, 2)
0000 0000 0011 010s     (0, 52)
0000 0000 0011 011s     (4, 3)
0000 0000 0011 100s     (0, 53)
0000 0000 0011 101s     (17, 1)
0000 0000 0011 110s     (1, 12)
0000 0000 0011 111s     (0, 55)
0000 0000 0001 0000 s   (0, 54)
0000 0000 0001 0001 s   (0, 56)
0000 0000 0001 0010 s   (0, 57)
0000 0000 0001 0011 s   (21, 1)
0000 0000 0001 0100 s   (0, 58)
0000 0000 0001 0101 s   (3, 4)
0000 0000 0001 0110 s   (1, 13)
0000 0000 0001 0111 s   (23, 1)
0000 0000 0001 1000 s   (8, 2)
0000 0000 0001 1001 s   (0, 59)
0000 0000 0001 1010 s   (2, 6)
0000 0000 0001 1011 s   (19, 1)
0000 0000 0001 1100 s   (0, 60)
0000 0000 0001 1101 s   (9, 2)
0000 0000 0001 1110 s   (24, 1)
0000 0000 0001 1111 s   (18, 1)
0000 01                 MPEG1 escape style
10                      EOB