10.08.09

asc-gzip/.xfd compression

Posted in General at 4:24 pm by Steven

If you should ever find yourself in the position of having to figure out how to compress XFDL files with the asc-gzip encoding, and I don’t wish it on you, here’s Python code to do it. Obviously, you’ll need some imports and error handling and optimization.

This thread gave me pointers to figure this out; Bryan was just a little off because he was using a gzip library rather than a zlib one.

# compress according to wacky XFDL compression scheme
def compress(fc):
    CHUNK_SIZE = 60000
    out = ''
    for i in range(0, len(fc), CHUNK_SIZE):
        chunk = fc[i:i + CHUNK_SIZE]
        chunklen = len(chunk)
        compressedchunk = zlib.compress(chunk)
        compressedchunklen = len(compressedchunk)
        out += chr(compressedchunklen / 256)
        out += chr(compressedchunklen % 256)
        out += chr(chunklen / 256)
        out += chr(chunklen % 256)
        out += compressedchunk

    f = StringIO.StringIO()
    f.write('application/x-xfdl;content-encoding="asc-gzip"\n')
    b64 = base64.standard_b64encode(out)

    for i in range(0, len(b64), 76):
        f.write(b64[i:i+76])
        f.write('\r\n')
    ret = f.getvalue()
    f.close()
    return ret

3 Comments »

  1. anna said,

    October 21, 2009 at 12:11 pm

    ahhh, code beautiful code! Remember the night I “watched you write code”? OO

  2. Scott Stafford said,

    December 10, 2009 at 11:08 am

    Thanks for your post. Of course, I needed the opposite, I had one I needed to decompress. So I backwarded your algorithm and here is the result:

    def decompress(fc):
        fc2 = fc.splitlines(True)
        fc3 = "".join(fc2[1:]) # could verify that it's asc-gzip here if we wanted to...
        unb64 = base64.standard_b64decode(fc3)
    
        ctr = 0
        ret = []
        while 1:
            if ctr == len(unb64): break
    
            ccltop = ord(unb64[ctr])
            ctr += 1
            cclbottom = ord(unb64[ctr])
            ctr += 1
            compressedchunklen = ccltop * 256 + cclbottom
    
            cltop = ord(unb64[ctr])
            ctr += 1
            clbottom = ord(unb64[ctr])
            ctr += 1
            chunklen = cltop * 256 + clbottom
            #~ print compressedchunklen, chunklen
    
            compressedchunk = unb64[ctr:ctr+compressedchunklen]
            ctr += compressedchunklen
    
            chunk = zlib.decompress(compressedchunk)
            assert(len(chunk) ==  chunklen)
            ret.append(chunk)
    
        return "".join(ret)
    
  3. Steven’s weblog » asc-gzip/.xfd decompression said,

    December 10, 2009 at 11:30 am

    [...] to post code for asc-gzip/.xfd decompression to go with my asc-gzip/.xfd compression code. See this comment. I’m also reposting it here because the comment formatting is a little more bad than the [...]

Leave a Comment