Hello everyone in the editing forum. :)

First off, I should say that, for the time being, I am going under the assumption that Pandemic / LucasArts will *not* be releasing any editing tools. Certainly not in the near future. Until I hear a specific confirmation along those lines, I don't think it's wise to assume they ever will.

Last night I started looking through a bunch of the *.lvl files. Here's what I've been able to figure out so far.

Each *.lvl file contains a series of what I'll call "chunks". In general, it doesn't seem to matter what order the chunks are in; although with some specific files (such as core.lvl or mission.lvl), it's definitely possible that order matters.

Integer values are stored with the least-significant byte first.

At the beginning of every *.lvl file is the 4-byte character string "ucfb". This is followed by a 4-byte integer indicating the size of the file (actually, the size minus 8; since it doesn't include this 8 bytes worth of header data).

Each chunk stores a particular kind of data, and has 8 bytes worth of header information. The first 4 bytes is a character string indicating the chunk type. The next 4 bytes is a 32-bit integer indicating the size of the chunk. In most chunks, there appear to be sub-chunks that I'll refer to as "sections". Each type of chunk seems to expect specific types of sections, often in a specific order.

Some of the chunk types I know of so far are as follows:

tex_ = Textures (can contain many)
skel = Unknown
modl = Model information
gmod = More model information?
coll = Collision information?
expc = Explosion information?
scr_ = Script?

I put together a really simple program that extracts some basic information from *.lvl files. You can see a (very preliminary) sample here:
Load\bes1.lvl (http://personal.riley-man.com/swbf/r3-lvl-bes1.lvl.html)
The above is the Load/bes1.lvl file, which contains the big bitmap image you see when the Bespin Platforms map loads. (ie. when it zooms from the galaxy, to the system, to the terrain).

Since textures/bitmaps are typically the easiest thing to decipher, I'm starting there. I know a little about how they're stored, such as pixel dimensions. My plan is to extract the pixel data and throw it into individual BMP files. At first, they'll probably just be grayscale. Hopefully I'll be able to determine color information sometime soon. Most of the textures seem to be 16-bit color, so I don't know if the R-G-B bits are 5-6-5 or 6-4-6 or something else.

What I do know about the tex_ chunk is they expect a NAME, INFO, and one or more FMT_ sections. From there, it's organized rather strange. Each FMT_ section contains an INFO (yes, another one) and one or more FACE sections. Each FACE section seems to contain one or more LVL_ sections. And each LVL_section contains an INFO (yet again!) and a BODY section. The BODY section is where the pixel data is actually stored.

What I hope to come of this is that, assuming editing tools never come, we can eventually build new maps and insert them into the game. If editing tools do come, then what the hey -- it's always good to know the file structure! ;)

This could be a nice step torward image changes , skybox and floor tile modding and mainlt repositioning objects will move along alot faster if we can crack the model position part. Please keep us informed of anything new you find.

Good work! :) A tool to extract even that information would be a big help - any chance of releasing it? ;)

Are the sounds in the sound .lvls divided into chunks? If so, this tool could tell us where they all are so it might be a lot easier to make a custom sound set.

whoa thats good stuff :eek:

Thanks guys. :)
Feel free to join in digging through the hex code to figure out how more of the various chunks / sections work. ;)

Yes, I'll be releasing the extracting tool once I've figuring out some more things. For now it's only written in Java. I'll give out the source code so that people can rewrite it in their preferred language. I did it in Java only because I work in Java all day every day.

I ran it against Sound/geo.lvl. It didn't produce any errors, but the chunk names are strange. So to answer that question, I *think* they are. Heh. It looks like the important stuff to look at in the sound files is near the end of each file.

I figured out a bit more about textures. The pixel data is stored in "DXT1" or "DXT3" format; the DXT stands for DirectX Texture. Find out more here (you might need to use Internet Explorer for this link):
MSDN DirectX Texture (DXT1) (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/graphics/ProgrammingGuide/GettingStarted/Direct3DTextures/compressed/alphatextures.asp)

It's a very ... interesting format. :p

Using this info and working through Sound/hot.lvl, I discovered it has three chunks. There are obviously more sounds than this on hoth which makes me think that the middle chunk is some kind of compressed file (it is considerably larger than any of the other chunks). The last chunk, which is tiny in comparison, looks to me to be a form of reference to the compressed file. It repeatedly has the marker DATA, followed by a byte 0D and 3 empty bytes, followed by up to 12 bytes of data, then another 4 empty bytes before the next DATA marker. There are also SCOP markers at various points throughtout this chunk whose function I can't determine.

One difference I noticed that may account for your odd chunk names when you ran the extract program on a sound file is that there are an extra 8 bytes at the start of the file before the first chunk. The first 4 bytes I can only guess are a marker of some kind (since they are repeated at the next chunk), and the last 4 bytes are the size of the chunk, including the normal marker headers (emo_ and the following 4 byte chunk size). Possibly some kind of 'chunk within a chunk'?

This is what it produces for Sound\hot.lvl:
Sound/hot.lvl (http://personal.riley-man.com/swbf/r3-lvl-hot.lvl.html)

Looking at it in UltraEdit, it does look like they're a "chunk within a chunk". The funny "!№[" marker always has a "emo_" directly inside of it.

Re: The DATA markers you're seeing there... The 0D 00 00 00 is indicating that 13 bytes of data follow. It's just that each chunk / section must be aligned to a 4-byte boundary, so you get three zeros at the end that act as padding.

eg. in Sound\hot.lvl, at 04D2F850h, you get:
44 41 54 41 0D 00 00 00 <-- DATA marker, with 13 bytes to follow:
B1 4A 32 BD 01 00 00 80 3F 00 00 00 00 00 00 00 <-- first 13 bytes are the actual data, the last 3 are just for padding

My thinking right now is that the planet-named files in \Sound\ might just contain the music selections. The cw.lvl and gcw.lvl might contain weapon sounds, vehicle sounds, etc. There's also a common.bnk file that seems to have the same chunk-style format -- why it has a different extension (bnk? Is that like the Bink from RAD Game Tools?) I don't know.


I was only partially right about textures in my above post. It actually stores any Direct3D surface texture, as stored in the DDS file format. I've been out of touch with DirectX since version 5, so I hadn't heard of DDS. ;) They store all kinds of different textures, including mip-maps and bump-maps. It also seems like it can store stuff like alpha and luminance information.

I dug around the 'net for info on this. Microsoft has a tool called DXTex (that comes with the newest DirectX SDK) that can be used to edit DDS files. Their MSDN web site has technical information on the file format as well:
DDS File Format (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/graphics/reference/DDSFileReference/ddsfileformat.asp)
I guess today's video cards have hardware that accepts and works with the various DDS-style textures directly.

It's starting to look more and more that, while the way the *.lvl files are glued together is probably proprietary, these files actually contain alot of standard format data at their core. Who knows... maybe we could even use some of the tools already existing for Battlefield 1942, and then have some simple Battlefront-specific tools that extract the data from those and put it into the *.lvl chunk format.

I have a thought about the skel heading. My guess is that it tells the engine what joint sets, or skeletons, to load for each character model, to apply animation to. It seems pretty clear Pandemic used only a couple of generic motion capture files for the character moverment, and that generic motion has to be applied to a particularly sized and spaced set of joints for each character.
Also, what you may be looking at in the massively subdivided texture sections is not just the bitmap texture, but also the bitmaps which define specular shading, bump mapping, shaders, and the like. Most often, the specular map and bump maps are separate texture files, and need to be combined and layered with the basic colour texture.

Thanks for the thoughts skellington. :) It somehow seems appropriate that someone with your handle should comment on the skel chunk. ;)

I'd already started to figure out all that about textures, once I looked over the Direct3D surface texture information on MSDN.

I put up a complete description of the tex_ chunk here:
tex_ Chunk Information (http://personal.riley-man.com/swbf/editing/swbf-lvl-tex.html)
I'm planning on putting up a page like this for each chunk type that I figure out.

Things are a bit slow going, as I have limited time to work on the LVL file extractor program I started writing. The program currently searches the LVL files thoroughly for chunks and sub-chunks. Often a little too thoroughly, as it occassionally "finds" sub-chunks that aren't actually sub-chunks. But I'm doing it this way so that we can analyze the output fully.

I don't know if I should release the program just yet, until we know more about -- at the very least -- the most common chunk types. Instead, I've posted the HTML output for a small selection of files.

mission.lvl (http://personal.riley-man.com/swbf/files/r3-lvl-mission.lvl.html)

core.lvl (Zipped) (http://personal.riley-man.com/swbf/files/r3-lvl-core.lvl.zip)

BES/bes1.lvl (Zipped) (http://personal.riley-man.com/swbf/files/BES/r3-lvl-bes1.lvl.zip)

FPM/ALL/allxwing.lvl (http://personal.riley-man.com/swbf/files/FPM/ALL/r3-lvl-allxwing.lvl.html)

FPM/IMP/impatat.lvl (http://personal.riley-man.com/swbf/files/FPM/IMP/r3-lvl-impatat.lvl.html)

Load/load.lvl (http://personal.riley-man.com/swbf/files/Load/r3-lvl-load.lvl.html)

Load/bes1.lvl (http://personal.riley-man.com/swbf/files/Load/r3-lvl-bes1.lvl.html)

Load/bes2.lvl (http://personal.riley-man.com/swbf/files/Load/r3-lvl-bes1.lvl.html)

Isn't that DDS texture format that was also used to store Max Payne bitmaps?
I used to have a DDS plugin for Adobe Photoshop to view and edit DDS bitmaps, if that helps...

I don't have Max Payne, but I wouldn't be the least bit surprised if it uses the DDS format. I came across the plug-in you mentioned when I was looking over the modding tools people use for Battlefield 1942.

Hey there Riley, Do u want a Gmail account

Well, I dont know if Im on the right track, but I have been able to produce semi-results with this great info you have posted. Going off of the lvl for textures, i have manages to create a very simple program(i still enter most locs) which extracts the basic dds. Using load/bes1.lvl, i am able to create this image:

http://img92.exs.cx/img92/2892/bes1.gif)

I honestly dont know if this is correct, but it seems pretty straight forward and may be on the right track.

ps. the image is zoomed in a bit and compressed, origonally 256 by 128 pxls.

Originally posted by Nimlot
Well, I dont know if Im on the right track, but I have been able to produce semi-results with this great info you have posted. Going off of the lvl for textures, i have manages to create a very simple program(i still enter most locs) which extracts the basic dds. Using load/bes1.lvl, i am able to create this image:

http://img92.exs.cx/img92/2892/bes1.gif)

I honestly dont know if this is correct, but it seems pretty straight forward and may be on the right track.

ps. the image is zoomed in a bit and compressed, origonally 256 by 128 pxls.

I could say u are going in the right direction to have a semi-result. its a start, great job,

Please show more results as you find them...this may be the first step in a real mod of the models/and skybox etc... I posted the cap of the load screen from your app over the load screen. These are a match... I would like to see what you using to show these...do you have an app ready?

http://www.gametoast.com/files/screenshots/match.gif)

Good stuff, Nimlot! I was hoping someone would start working on a texture extraction program. :)

I didn't post it here yet (I posted on this the gametoast.com forums), but I've been building a more complete editing reference web site here: SW:BF LVL Reference (http://personal.riley-man.com/swbf/editing/)

Have a look over the tex_ page (http://personal.riley-man.com/swbf/editing/swbf-lvl-tex.html) and see if there's anything new there that might help. I've marked a number of values in the INFO sections as "unknown". Any idea what some of those unknowns are?

Edit: Niteshift, thanks but I think I have all the e-mail accounts I can handle. ;)

Originally posted by Niteshift
Hey there Riley, Do u want a Gmail account

Ever heard of http://www.lucasforums.com/swbattlefront/sendpm.gif?) :roleyess:

Originally posted by TK-8252
Ever heard of http://www.lucasforums.com/swbattlefront/sendpm.gif?) :roleyess:

TK, i was going to give him a GMail account for all the work he is doing in the modding community, not so i could pm him

has anyone tested if increasing both the filesize and chunk size info to add (or decrease) information, so we wont have to keep everything the same size, and we dont have to do anthing but check the number of bytes we've added (or detracted)?

this should be merged and/or stickied...

LVL files definitely don't have to remain the exact size that they currently are. It would be possible, for example, to swap in a new texture that's bigger or smaller. The file would just have to be reconstructed, and the file size marker would have to be changed.

I've updated my LVL File Reference site, and will continue to do so in the days and weeks ahead. See my signature. :)

I've posted a big update on my reference site regarding models. I have a fair bit figured out, but there's still a ways to go. Anyone here who is knowledgable about the Direct3D *.x file format, or 3D models in general, might be able to make more sense of certain things than I can.

modl chunk reference information (http://personal.riley-man.com/swbf/editing/swbf-lvl-modl.html)

i think MTRL (material?) could be the reference to the texture for the model

SPHR (sphere?)...what models did it appear on? if it only appeared on one model, thin my guess would be the droideka's shield

Take another look. There's a section called TNAM that references the texture name. Each mesh segment within a model can apparently have its own texture.

MTRL may indeed mean "material", but if it does, it isn't a reference to a particular texture. It almost always seems to just store a single 32-bit integer (usually a value of 1).

All the models have a SPHR section. At least all the ones that I've looked at. This is just an educated guess, but it might just be a vector indicating the bounding box of the model, used as an early-out for collision detection.

LVL files definitely don't have to remain the exact size that they currently are. It would be possible, for example, to swap in a new texture that's bigger or smaller. The file would just have to be reconstructed, and the file size marker would have to be changed.

Have you successfully enlarged mission.lvl by 1 or more bytes?

Your .dds findings were the basics needed to make the Bviewer come together for Nimlot. I'm pondering a model and mission file editor being formed into this eventually...

any idea which of those chunks have the bumpmapping data? (not the RTYP, but the actual bumpmap) could the MTRL be a flag for what material this chunk is for reference to sounds, etc? (1 being common because they would be referencing a generic, faction trooper pain noise)

Have you successfully enlarged mission.lvl by 1 or more bytes?

Not the mission.lvl file, no. I haven't been working on that file, to be honest. All my deciphering so far has been on the map files, Load files, and FPM files.

I'm looking at it now, and there is one thing about it that is puzzling:
http://personal.riley-man.com/swbf/mission-lvl.gif)

There's an "lvl_" chunk, set to size 0x1770 (6000 bytes). I'm guessing that "lvl_" just means "embedded *.lvl file". Sort of a look-inside-me-for-the-actual-chunk.

Immediately following that is what looks like a chunk marker - the "kЅ†". But it's a different 4-byte code after every "lvl_" chunk. Directly inside that is the "scr_" section.

In order to know how to add in new missions into the game, I think we'd have to figure out what those funny chunk markers are all about. Are they just arbitrary? Do they act as a reference to something else? Perhaps a hash-map key to something? I've tried reading them as integers and floating-point values -- neither seems to come out with any kind of meaningful pattern.

any idea which of those chunks have the bumpmapping data?

Individual textures (in the "tex_" chunks) can store bump-maps. The *.dds file format supports all kinds of different surface textures, from standard RGB-based pixel data to bump-maps to cubic-environment-maps. I'm not sure how widely used bump-maps are in Battlefront.

Available texture formats in *.dds files (http://msdn.microsoft.com/archive/default.asp?url=/archive/en-us/directx9_c/directx/graphics/reference/d3d/enums/d3dformat.asp)
Scroll down to where it says "Signed Formats".

I'm currently revising my code and actually added support for the MIP maps (it was only throwing out the chunk, now it exports in correct DDS MIP structure). But, I am stumped on a second file format, which most skins seem to have a second face. The format in Hex is 17 00 00 00, and I am having a hard time finding any info for this. I was thinking it would be the bump map for the skin, but it doesnt make sense why it wouldnt be in the same DXT1 or DXT3 format. Anyone have any ideas, Riley? :)

A day or so ago I updated the tex_ chunk page on my site with a complete description of how to properly extract textures: http://personal.riley-man.com/swbf/editing/swbf-lvl-tex.html)
Just scroll down. :)

For the textures that have multiple pixel formats (ie. more than one FMT_ section), you'll have to extract each into its own *.dds file.

The link in my previous post ("Available texture formats in *.dds files") lists all the pixel formats. ;) The 17 00 00 00 is equivalent to decimal 23, which means it's in 16-bit R5-G6-B5 format.

My guess is that they often have multiple pixel formats for compatibility with all video cards. Some video cards might not have built-in support for decompressing the DXT# pixel format on the fly. So the game code would only load the pixel format it needs (probably preferring DXT, since it's compressed and can thus load more of them into the video card's memory).

Just posting some updates...

Information on the storage structure of several additional chunk types has been added to my site. Also, on the gametoast.com forums, there has been a fair bit of progress made on how Scripts (using the Lua programming language (http://www.lua.org/)) are used by Battlefront.

Keep looking in those two places for the most recent discoveries. And if anyone has anything else to add, we are, of course, always looking forward to any new editing information you might be aware of!