Binary files don't have "chunks". I'm more than a little confused by that. A binary file is a string of bytes that span the length of the file. The only time I've seen "chunks" was in a human readable format.
Typically a binary file will be formatted to have a header with offsets to the various data and that header will be immediately followed by the file data. What a binary (hexadecimal) string represents to the file loader is entirely defined by the file format.
Here is a binary file. It is a simple 5x5 pixel TGA file. The first 18 bytes are the file header and the image data starts at byte 19 (the 00 immediately following the 08).
00 00 02 00 00 00 00 00 00 00 00 00 05 00 05 00 20 08 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF 00 00 00 FF
The "02" in the header tells the loader that this is an uncompressed, true-color image.
The first "05 00" defines a 16-bit word that tells the loader the width of the image; In this case, 5 pixels.
The second "05 00" defines a 16-bit word that tells the loader the height of the image.
The "20" that follows is 32 in hexadecimal, telling the loader that each pixel in the pixel data is 32-bits (4 bytes).
The "08" is an encoded byte, whose bits tell the loader if the image is flipped as well as the number of alpha channel bits.
The other bytes that I did not explain have special meaning as well, in certain cases, but are unused in this file.
The image data is stored as BGRA (blue, green, red, alpha) and in this case, each pixel has the values "00 00 00 FF".
If you are seeing anything other than hexadecimal in a file, it isn't a binary file.