PHP - Uncompress and Base64 Decode Download On-the-Fly

[wraithdu]

Why you might ask? Well it seems to be a popular practice for web hosts to scan their servers with multi-brand AV products, similar to VirusTotal and Jotti. Inevitably a good number of these crappy AV engines detect every AutoIt script as a false positive. This has resulted in my domain being suspended TWICE until I could work it out with support (kudos to Dreamhost on at least having decent support). Regardless it's a PITA and takes time. Not to mention I'm sure they keep track of this kind of thing, and eventually no amount of email persuasion is going to get me back online.

My first solutions was to pack all my downloads into password protected encrypted zip files. This kept the AV clowns away, but it's annoying for users and just plain looks bad. So I thought about it a bit more lately and came up with a solution. I have to give some credit to UEZ and his File to Base64 String Code Generator for the idea. I decided to store the files base64 encoded and gzip compressed, then use PHP to uncompress and decode them on the fly. This way the AV scanners just see some some gzip compressed text files, and users get the real deal on their end.

First I wrote a short script using Ward's base64 and Zlib machine code UDFs to create the files. It adds a dummy header to the file to confuse anything that might try to base64 decode it, and encodes the actual file size into the file name for the PHP script. Additionally Ward's code breaks the encoded string into 76 character lines for standards compliance, but it also makes it look more like a regular text file.

expandcollapse popup

#NoTrayIcon
#AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6
#AutoIt3Wrapper_res_requestedExecutionLevel=asInvoker
#AutoIt3Wrapper_Run_Obfuscator=y
#Obfuscator_Parameters=/so
#AutoIt3Wrapper_Run_After=del "%scriptdir%%scriptfile%_Obfuscated.au3"
#AutoIt3Wrapper_Change2CUI=y
#include <_Base64.au3>
#include <_ZLIB.au3>

_Main()

Func _Main()
    If $CmdLine[0] <> 1 Then Exit
    Local $SrcFile = $CmdLine[1]
    If $SrcFile = "" Or (Not FileExists($SrcFile)) Then Exit

    Local $BufferSize = 0x80000
    Local $SrcFileSize = FileGetSize($SrcFile)
    Local $SrcFileHandle = FileOpen($SrcFile, 16)
    Local $DestFile = $SrcFile & ".b64"
    Local $DestFileHandle = FileOpen($DestFile, 2 + 16)

    ConsoleWrite("Encoding..." & @CRLF & @CRLF & $SrcFile & @CRLF & "--> " & $DestFile & @CRLF & @CRLF)

    Local $timer = TimerInit()

    ; write header
    FileWrite($DestFileHandle, "thisismyboomstick" & @LF)

    Local $State = _Base64EncodeInit()
    For $i = 1 To Ceiling($SrcFileSize / $BufferSize)
        ConsoleWrite(".")
        FileWrite($DestFileHandle, _Base64EncodeData($State, FileRead($SrcFileHandle, $BufferSize)))
    Next
    ConsoleWrite(" Finishing..." & @CRLF & @CRLF)
    FileWrite($DestFileHandle, _Base64EncodeEnd($State))

    FileClose($SrcFileHandle)
    FileClose($DestFileHandle)

    $SrcFile = $DestFile
    $DestFile = $CmdLine[1] & "." & $SrcFileSize & ".b64gz"

    ConsoleWrite("Compressing..." & @CRLF & @CRLF & $SrcFile & @CRLF & "--> " & $DestFile & @CRLF & @CRLF)

    _ZLIB_GZFileCompress($SrcFile, $DestFile)

    ConsoleWrite("Done... " & Round(TimerDiff($timer) / 1000, 4) & " sec" & @CRLF & @CRLF)
EndFunc

I already had a PHP click counter / download script in place, so I modified the end of it to server these special files. The $url variable holds the relative or full URL of the target file. The PHP code must strip the dummy header and line endings from the uncompressed data before base64 decoding it.

expandcollapse popup

<?php

if (file_exists($url)) {
    if (substr_compare($url, ".b64gz", -6, 6, true)) {
        // normal file
        /* Redirect to the link URL */
        Header('Location: '.$url);
    } else {
        // b64 encoded, gz compressed file
        $base = basename($url, ".b64gz");
        // get file size for Content-Length from name
        $size = end(explode(".", $base));
        // remove the size
        $fname = basename($base, ".".$size);

        header('Content-Description: File Transfer');
        header('Content-Type: application/octet-stream');
        header('Content-Disposition: attachment; filename='.$fname);
        header('Content-Length: '.$size);
        header('Content-Transfer-Encoding: binary');
        header('Expires: 0');
        header('Cache-Control: must-revalidate');
        header('Pragma: public');

        ob_clean();
        flush();

        // i'm gonna put this right here... the gz file commands that make working with gzipped
        // data transparent to the user are freakin awesome

        // each line of 76 chars has a trailing LF that must be stripped
        // meaning we need a chunksize that is a multiple of 77
        // and the resulting stripped string must be Mod 4 and Mod 3 to decode
        // tl;dr, 4K ~ 4158, 8K ~ 8085
        $chunk = 8085;
        $head = strlen("somethingclevereatmen");

        $f = gzopen($url, 'rb');
        if (!$f) die();

        // strip leading header first
        gzread($f, $head);

        while (!feof($f)) {
            echo base64_decode(str_replace("n", "", gzread($f, $chunk)));
            ob_flush();
            flush();
        }

        gzclose($f);
    }
}

exit();
?>

Voila. Cheapo AV scanners are thusly defeated and end users are none the wiser. Bonus - the data files are kept in a secured directory where only PHP can access them, preventing direct downloads.

Edited September 25, 2012 by wraithdu

[wraithdu]

<rant>Ugh, is there a way to stop the forum from mangling code tags and removing the space / tab formatting? Makes editing posts with code utterly painful.</rant>

[guinness]

Use the basic editor by selecting the 'toggle editing mode' button (top left hand corner.)

Nice article.

[wraithdu]

Use the basic editor by selecting the 'toggle editing mode' button (top left hand corner.)

Yup that works for creating the post, but the formatting gets blown away as soon as you hit the edit link to change anything. Doesn't matter if you turn off formatting or switch to the full editor :-( Edited September 25, 2012 by wraithdu

[guinness]

I use Opera and always use the basic editor, even when editing a post.

[wraithdu]

I use Opera and always use the basic editor, even when editing a post.

Totally unrelated to my OP, but I solved the editor problem, it was a cookie issue. Signed out, removed AutoIt cookies, and now it remembers my 'Toggle editing mode' preference. Hooray.