The Bitmap Part of the Icon Story

Making a 32x32 bitmap is a simple enough task, and then getting a hex dump for the file from Quickview Plus is also easy enough . . . unfortunately, Quickview won't print the dump and so we have to do OCR or make a gif of the result.

Making it a graphics file is not too bad an idea since we can then use colors to show the layout of the file itself, and that is most interesting. If we make a number of bitmap files of icon sized graphics, and we make each one a separate color of the 16 that Windows icons use, we can determine that the bulk of each file will be a hex number representing the color we chose, and surely as night turns to day at dawn, that is what happens. The results are shown in the table below:
 
 

Now what in the world has happened to the rendering of the bitmap file which makes the thing split in the middle and have pieces everywhere? Well, if we use some Yankee ingenuity, we might lop off the section on the left and switch it over to the right and paste it on, and see what we come up with:

Why goodness me, if we flip it:

we're right back to our original layout, are we not? Conclusion? A bit map file must begin showing its pixels at the lower left hand corner, running across through the width of the graphic, and then going to the next row up, from there showing each column in turn, until the thing is finished. The cutting and pasting was likely a result of the way the Quickview dump came out rather than the makeup of the file itself. It looks like the first seven rows (of 32 columns per row) plus 14 columns describe the methodology, the fact that it is a bitmap graphics file, the palette, and who knows what else, where the last 32 rows actually comprise 4-bit descriptors of the color of the 1024 pixels in the file. Since the file gets made and consists of 630 bytes, one can make a guess that 512 of those bytes break down into 4-bit indexes of the color of the individual pixels, and thus 118 bytes are left to define and describe the type of file and how it is to be read.

A standard Windows icon file contains 766 bytes, 512 of which are precisely and exactly the same as the 512 in a bitmap, so we can now deduce that it takes 254 bytes for Windows to figure out how to open and use an icon (extension .ico) file. Of course, 118 of those 254 may be the same thing which describes a bitmap, but so far I am unable to verify that . . . the dumps don't LOOK the same in the beginning stages, but I'm not through with it yet . . . someone out there KNOWS what the icon file comprises, and I can guarantee one thing: he isn't telling me.

The next thing to do is to take this bitmap file and run it through IconForge and create an icon of it; when we get that done, we can then compare the two files, bitmap and icon, and see what the differences are, at least with respect to the way that Quickview presents a hexdump of them. I think I can also continue the cut and paste routine with the icon file and see how it looks that way as well . . . did that, and when you cut, paste, and then flip, the icon file is the same red, green, yellow, blue across the top as the bitmap file . . . let's take a look at the 118 and 254 OTHER bytes in the file, or at least the hex dumps that would likely BE those bytes . . .

It is obvious that the colors of the pixels for both the bitmap and the icon files start in row 8 (0070Hex) with the beginning of the 5's, but what to make of the first 236 numbers in the bitmap file on the top and the first 254 numbers in the icon file? Are these pairs which comprise 8 bit bytes, or are they indexed some way which is understood by graphics software and which identifies the file type, the palette, the background color, or some other parameters as well? What happens before the first 55 in the row seems similar: there is a 10 in the fourth row after which then ensues
nine sets of zeroes before an 80 (Hex?) in the bitmap, and thirteen sets of zeroes prior to an 80 in the icon file . . . after which every bit or byte seems to be the same in the two files. It sort of begs one to leap to the conclusion that at least the 9 double zeroes and what follow until the first 55 are bitmap color oriented, and would always be the same, for that file, but let's go on now and take the rest of the dump which is not the same . . . for comparison's sake we can look at this in real print instead of graphics:
 
 

42	4d	76	02	00	00	00	00	00	00	76	00	00	00	28	00
00	00	20	00	00	00	20	00	00	00	01	00	04	00	00	00
00	00	00	02	00	00	00	00	00	00	00	00	00	00	10	00
00	00	10
00	00	01	00	01	00	20	20	10	00	00	00	00	00	e8	02
00	00	16	00	00	00	28	00	00	00	20	00	00	00	40	00
00	00	01	00	04	00	00	00	00	00	80	02	00	00	00	00
00	00	00	00	00	00	10	00	00	00	00	00	00	00	00	00

I have no idea what any of this means, but someone had to agree on a format or protocol somewhere some time, and this is what has to be in the files to be readable as bitmaps and icons . . .

Maybe I'd know a lot more about it if I knew just what Quickview did to GET this dump. My concept is that they just read the file one byte at a time and report it out in text or in hex as the user chooses, and the text would be whatever ANSI characters the bytes represent, and the hex pairs would be whatever the eight bits cause them to be. Somewhere there is formatting for this, but I don't know where . . .

Leaving no stone unturned, perhaps if we make a text file called qvtest.dat which comprises no more than four lines of 62 characters plus a carriage return and line feed, we might see what Quickview does with that file in a hex dump and make some new assumptions about its use. The file will essentially be like the following text:

In theory that should test the system, and indeed it does and it clears up what the funny little stuff on the right hand side of the dump comprises as well. I leave it to the reader to make all the connections, sufficing it to say that Quickview behaves exactly as predicted, taking each byte in turn and reporting its hexidecimal equivalent, where 30-39(h) are the numerals 0 through 9, 0D(h) is a carriage return, 0A(h) is a line feed, and 5A(h) is the letter capital Zee.

So we carry on realizing that in the beginning of the bitmap file there is the hex representation for BMv?, or 42 4d 76 02, 42(h) being B, 4d(h) being M, 76(h) being a small "v," and 02 being who knows what---it represents a large bullet in ANSI code, but a guess is that once the Windows OS "sees" the "BM" it goes into bitmap reading mode and the rest of the characters follow some arcane indexing scheme devised to keep people from understanding it. At least we can identify the color codes, and we can take solace in the idea that if they wanted us to be able to understand it, they'd have made it simple and straightforward instead of the way they made it.

Indeed, I have found an incredible item among the gems at a website which fell out of a book called "Graphics File Formats," by Murray and Van Ryper---the book is out of print, but the website is used for upgrades of various file types, and it is a nice find. It is located at  http://www.ora.com/centers/gff/index.htm and on that site I found that there is no standard for either icons or bitmaps in Windows, even though Microsoft will let you know how they are made if you buy their "developers kit" material on cd-ROM. It seems that there is a "map" of the file structure, fully half of which comprises fields or words or bytes which are no longer used and require zeroes or more properly nulls in order for the file to work properly . . . typically, Microsoft holds back something as well, because the NUMBERS mentioned here previously which correspond to the 16 colors, 0 for black, F for white, 9 for red, etc., are not called out in the file structure, but apparently are created from some internal operating system "knowledge" which directs the system to color this pixel green and that one blue, etc. Oh, the COLORS are shown as four-byte "words" or fields which are rgb (red green blue) numbers and some OTHER number which may or may not be intensity, who can say since THEY do not, but the order of the colors has nothing to do with the numbers assigned to them.

In fact, the order of the colors shown in the file seems to be entirely related to a "bubble" sorting algorithm, except for the fact that C0C0C0 alone is out of place . . . in any event, I now have a much better picture of how the files, bitmap and icon, actually are constructed . . . once I have a complete picture of the various fields or bytes in the file, I'll report that here. Meantime, all I wished to impart was the certain knowledge that there is no written standard specification for bitmap or Windows icon files. People who make them must figure it out for themselves and hope for the best.

Now I have made an amazing discovery, or more properly a discovery of an amazing facet of the bitmap-icon story. A difference between bitmap and icon files is also seen in a trailing series of 128 bytes in the icon file which follows right after the 512 bytes which describe (as 4-bit representations) the pixel by pixel color map of the icon. This trailer in a standard icon with no "transparent" pixels is always a bunch of zeroes, but when the icon is created to "blend" with the screen background, leaving out the icon's background and picking up what is on the monitor BEHIND the icon, then the trailing 128 bytes are used to tell the operating system which pixels to use and which to mask so that your nice icon will only show up as the initials "DOS" for the icon which is a shortcut to a DOS prompt, and indeed the inside of the "D" and the "O" will also take on the background on screen rather than from the file. Here is a before and after of a gif of this icon, which isn't the same file of course, but gives the same effect.

Now how can 128 little bytes tell you which pixels to fire and which not, when it has already taken 512 to lay out the entire 32x32 grid of pixels for the icon? As you have surmised, the 512 are split into half bytes which then can be used to represent all 1024 pixels (or think of them as squares, if you like), but how to use 128 to say what to do with 1024? Here is the amazing feat of the designer of the icon algorithm: he uses each bit in each byte to signal the Operating System whether to fire or not the pixel named, and thus each of the bits will be either 0 or 1, and if 0 the pixel fires as shown above in the 4-bit color map, and if 1 it does not fire but shows the background behind the icon.

Is that amazing or not? I have no idea how the individual bytes are actually placed in the file; that would be a story unto itself. I can place them or change them myself by entering the file as a binary and just telling the byte to be whatever ascii (developed by ANSI) character I require to get a given bit structure.

So what is this all about? What am I up to? Why am I learning all this obscure and arcane stuff about bitmaps and icons? What I want to be able to do is understand the icon file structure well enough that I can create an icon file from scratch as it were, though I don't mind starting off with a bitmap made in MSPaint or Paint Shop Pro to get the ball rolling . . . we have seen how the bitmap file uses 512 bytes to show the various colors of each pixel of the 1024 in a 32x32 bitmap as half bytes, and I can just make an ascii string which will do all of that, but since I have Paint Shop Pro, it will be easier to start with a bmp file from there and modify it to change it into an ico file which will be acceptable to the Windows operating system as a desktop icon.

I intend fully to go through the whole process right here on this page, so that everyone in the world can create an icon without buying an inexpensive software package with which to do so; not that I'm trying to take business away from Cursor Arts of Bend, Oregon, just that I want to explore the unknown and take away some of the mystery of this machine called a computer which does far more than just compute.

The rest will be coming soon, I promise, unless Jose intervenes . . .

Getting started, I made an icon for a friend, the initials are "TC," and that can stand for whatever you like, including Tony Cohelo or Thad Cochran. It's a simple icon, looks like this, more or less, since this is a gif rather than an icon . . .

First we can look at the entire file of 766 bytes as 766 separate bytes, both as ascii representations and as hexadecimal representations, and we find that view of it at this site here.

That doesn't tell you very much unless you have a scorecard with which to actually look at the file in some kind of compressed representation, such as the ones above, so here will be the TC.ico file as a gif from the QuickView hex view of the file; I won't cut and paste this one, leaving the built in bias or offset as it was in the original.

I believe you can see the "TC" showing up there in the hex "C" characters, (remember that icons and bitmaps start at the bottom left of the design, as if it were a graph whose beginning point is x=0 and y=0 at the bottom left of the positive quadrant, and thus the real drawing would be a "flip" of the hex dump above--i.e. bring the bottom left up to top left), which oddly enough represent RED this time, a mystery to me since in previous icons and bitmaps red was represented by hex 9's, but then you see in the last 128 bytes the interesting series of hex numbers which are apparently meaningless. Not so. One must persevere, and one can develop yet another bitmap within the bitmap which represents the mask by using only the single bit in each byte as an either/or switch which lets the pixel color in the icon show or lets the background behind the 32x32 square show through. That is what we will explore next with a simple 32 square 1 and 0 representation of each byte in that set of 128. Again, I hope you can see the "TC" show in the numbers.
 
 

I can't explain the apparent anomaly in the fourth rows from top and bottom in the "C" part of this mask, I suppose that it is a bug or perhaps a compression device in the icon creating software which is Icon Forge from CursorArts. It certainly doesn't show in the actual icon, and it logically shouldn't be there in that single-0 section, but there it is, and I didn't make it up. Another possibility is that in using QuickBASIC to "read" the file, there is a misread which got by the MS censors. No, QB doesn't print the zeroes in red, but html does, and I used that device to highlight the outline of the letters in the bit rendering.

So far I haven't said much about the PALETTE that shows up in both the bitmap and icon files. In a 16-color palette, there will be four bytes for each possible color whether used or not, and as I noted earlier the order seems not to be regulated by the hex numbers used to represent a color, nor even are the four bytes each in real ascending or descending order, and different icon and different bitmap files will change those orders slightly as well. This is the second time I have alluded to the palette numbers, and I will treat it with more respect in this section.

The palette apparently is composed of 64 bytes, 4 each for each color in the palette. Either the first three bytes or the last three are the rgb numbers for the color being described, and will comprise 0, 128, and 255 primarily, with only one offshoot in one gray color of 192,192,192 showing up, all the other 15 being combinations of 0 (0hex), 128 (80hex), and 255 (FFhex). For those who haven't worked with rgb, the system isn't so much at blending colors but in arranging digital algorithms to represent the spectrum, and thus the color red in a rgb numeric description will be 255-0-0, meaning 255 red, 0 green, and 0 blue; similarly green would be 0-255-0, and blue 0-0-255. Where it gets tricky is in combinations of rgb NOT represented by even distributions, but that is beyond our scope here; what we will look at is the 16 colors represented in our files. To that end the following table will show the colors and their attendant rgb numbers in hex, and you can pick out that same arrangement in the palette section of any 16-color bitmap or icon file.
 
 

And this is how it looks in the file itself, with an extra pair of zeroes which I said MAY be an intensity spot, who could possibly say except the person who devised this scheme?

I see here that if you begin with black at "0" and count up to red, THIS time the red is shown in the palette in the "C" position. Remember that this icon was made with the same software from Cursorarts which made the other one in which the red was shown as a hex "9" but the first time I used a bitmap for a "starter." I think I ought to write the makers of ImageForge icon construction software and see how they handle this . . . if you put the wrong palette map into place, your colors will be all messed up, won't they?

Next we get to the "headers" of the bitmap and icon files, and take them apart byte by byte to see what, if anything, we can conclude about the entries and their nature. Before that deep understanding however, I have something to ask and something to tell . . . I found a clue in the bitmap file dump, and I wonder if anyone can find one in the icon file dumps . . . remember the start of the dump of the bitmap file showed the first four bytes to be 42, 4D, 76, and 02? Well, 42 is B (for "bit"), 4D is M(for "map"), 76(h) is 118 decimal and byte number 118 is where the map starts (or the "offset" as some nerds call it), and finally the 02 is the number of pixels per byte in the "map" portion of the file which indeed begins at byte 118 of the file.

Well, that's what I had to tell, and now I ask you if you can make any sense out of the icon file . . . there is one E8(h) in there, and a stray 40(h) along with the obvious 20(h) for width and height and the 10(h) which is no doubt "number of colors." What I am looking for is the offset here if there be one, and perhaps somewhere in the file is something which tells how many bytes there are in the thing, but I don't even know why that information would be worth anything since you can tell how long a file is by using the "LEN" (n) statement or other means; in older icon files there were often more than one icon, also, which meant that the OS should find the one which suits it best, a monochrome, a 4-bit, or I'm not sure what else . . . maybe some of them were 32x64, if I recall correctly.

Clear the decks. We've seen the 64 byte palette, we've seen the 512 byte map of the colors, and we've seen the 128 byte "transparency" scheme at the end of the file. What is left of the 766 ought to be 62 bytes of something which is necessary for the Operating System to know so it can successfully depict an icon. We have already heard that many of those bytes are truly not necessary for anything and are always zero, no matter what else happens in the icon itself, so now we can examine the bytes, words, and double words of the "system" of an icon file.
 
 

There you have it; create a 766 byte file with the above characteristics and you will have created an icon file which Windows can use on your desktop. The fact that we haven't any idea what most of the bytes in the "header" portion of the file comprise is testimony to the efficacy of Microsoft in hiding its information even from developers who need it to make software which will work with the Windows Operating System. I'm not satisfied with the "speculations" in the above, but there is little I can do but that. There are no specifications that I can find which outline the requirements for a real icon file, and so we can make one which works by using numbers we don't understand so as to create the final result we want.

An analysis of the file structure would show that a large percent of the information contained in the file is totally unnecessary and always has been . . . the fact that you have a palette and a bitmap and a pixels per byte value automatically shows the size and shape of the icon to anyone opening the file. What would the number of colors BE if the palette contained 16? The offset position of the start of the bitmap would be nice to have, but I can't find it in the file anywhere . . . in short, the entire .ico file is a conglomeration of old and unused information which no longer applies, and we've been through three new generations of Windows and no one has seen fit to upgrade or update it.

Appropos of nothing, I leave you with a 16 color bitmap of the Windows icon colors, and wish you a happy day: