10.08.09

asc-gzip/.xfd compression

Posted in General at 4:24 pm by Steven

If you should ever find yourself in the position of having to figure out how to compress XFDL files with the asc-gzip encoding, and I don’t wish it on you, here’s Python code to do it. Obviously, you’ll need some imports and error handling and optimization.

This thread gave me pointers to figure this out; Bryan was just a little off because he was using a gzip library rather than a zlib one.

# compress according to wacky XFDL compression scheme
def compress(fc):
    CHUNK_SIZE = 60000
    out = ''
    for i in range(0, len(fc), CHUNK_SIZE):
        chunk = fc[i:i + CHUNK_SIZE]
        chunklen = len(chunk)
        compressedchunk = zlib.compress(chunk)
        compressedchunklen = len(compressedchunk)
        out += chr(compressedchunklen / 256)
        out += chr(compressedchunklen % 256)
        out += chr(chunklen / 256)
        out += chr(chunklen % 256)
        out += compressedchunk

    f = StringIO.StringIO()
    f.write('application/x-xfdl;content-encoding="asc-gzip"\n')
    b64 = base64.standard_b64encode(out)

    for i in range(0, len(b64), 76):
        f.write(b64[i:i+76])
        f.write('\r\n')
    ret = f.getvalue()
    f.close()
    return ret

3 Comments »

  1. anna said,

    October 21, 2009 at 12:11 pm

    ahhh, code beautiful code! Remember the night I “watched you write code”? OO

  2. Scott Stafford said,

    December 10, 2009 at 11:08 am

    Thanks for your post. Of course, I needed the opposite, I had one I needed to decompress. So I backwarded your algorithm and here is the result:

    def decompress(fc):
    fc2 = fc.splitlines(True)
    fc3 = "".join(fc2[1:]) # could verify that it's asc-gzip here if we wanted to...
    unb64 = base64.standard_b64decode(fc3)

    ctr = 0
    ret = []
    while 1:
        if ctr == len(unb64): break

        ccltop = ord(unb64[ctr])
        ctr += 1
        cclbottom = ord(unb64[ctr])
        ctr += 1
        compressedchunklen = ccltop * 256 + cclbottom

        cltop = ord(unb64[ctr])
        ctr += 1
        clbottom = ord(unb64[ctr])
        ctr += 1
        chunklen = cltop * 256 + clbottom
        #~ print compressedchunklen, chunklen

        compressedchunk = unb64[ctr:ctr+compressedchunklen]
        ctr += compressedchunklen

        chunk = zlib.decompress(compressedchunk)
        assert(len(chunk) ==  chunklen)
        ret.append(chunk)

    return "".join(ret)

  3. Steven’s weblog » asc-gzip/.xfd decompression said,

    December 10, 2009 at 11:30 am

    [...] to post code for asc-gzip/.xfd decompression to go with my asc-gzip/.xfd compression code. See this comment. I’m also reposting it here because the comment formatting is a little more bad than the [...]

Leave a Comment