Buca Bay - Always nice

Dua tiko noqu toa loaloa, na yacana ko… laga mai…

Secure password hashing and storage in PHP

November7

Everyone knows you should never store passwords as plain text, right?

Recently I’ve come across a lot of bad advice on how to store passwords securely. I thought I’d share a bit of my research into the subject. Like most web developers, I’m not a cryptography expert. However, this should not be the excuse we use when we make a database of thousands of passwords available to hackers. It is quite simple to save passwords in a reasonably secure manner.

Main points: secure password hashing

  1. Force your users to use secure passwords
  2. Do not use 2 way encryption, save passwords as hashes
  3. Salt each password with a unique salt
  4. Hash the password and salt together multiple times (1000 or more)
  5. Use an adaptive hashing technique

Cryptographic Hashes

Passwords should never be stored in their original form, instead a cryptographic hash must be stored. A hash function is a function that performs a one way operation on an input producing an output of fixed length called a digest.

A popular hash function is MD5. An example in PHP:

$hash = md5('test'); // 098f6bcd4621d373cade4e832627b4f6

From the hash, the original text cannot be computed, however, knowing the original text you can easily compute the hash. This is the basis of saving passwords securely.

To verify the password, we see if it hashes to the same hash as the one we have stored.

if (md5($password) == $hash) { /* password is valid */ }

Notice how we never store the actual password, only the hash. We also don’t need to be able to decrypt the hash, we only have to verify it given the password supplied by the user.

Note I’m only using MD5 for example, since it is well known. You probably want to use a more secure hash such as SHA256 or Whirlpool as MD5 and SHA1 have a few problems (don’t ask me what they are, I don’t know).

Key Based Encryption

I’ve come across developers advising the use of a key based encryption. However, this is not a secure way of storing passwords. You’re only deferring the task of having to hide the password, to having to hide the secret key.

Another argument is that you need to be able to retrieve the password if the user forgets it. If we backtrack to why we use passwords, we realize that allowing the user to gain access to their account is the concern, not retrieving their password. So instead, send the user a unique key (token) through email or some other authenticated resource, so the user can generate a new password.

Precomputation Attacks - Rainbow Tables and salts

Now, the problem with just hashing a password, is that users normally use very easy to remember passwords, which are thus very easy to guess. Due to this, there are many databases available that map all the possible passwords made up of up of alpha-numeric characters in upper and lower case, as well as a lot of special characters. There are usually in the form of rainbow tables. Most do not have matches for passwords longer then 14 characters however due to the amount of computation required to generate these tables.

The way to beat rainbow tables or any other precomputation attack is to use a salt. A salt is a randomly generated input that you add to the password before hashing. This is done so that the input becomes long enough that generating a table that includes it is computationally infeasible.

So our hash function becomes:

$salt = 'f39ae656bc79a9b2398890bb4';
$hash = md5('test'.$salt); // 1db36e73cf2e68df3640fb1052e801da 

Now even if the user supplies a very easy to guess password, such as “love”, it still cannot be looked up in a rainbow table because the table would have to hold the whole input, “lovef39ae656bc79a9b2398890bb4″. No public rainbow tables have that many characters in them.

Use a unique salt for each password

If you use the same salt for each password (a common practice in PHP applications) is a very bad idea. It allows the attacker to crack all your passwords in one go using brute force. Imagine a database with a million passwords. Each guess becomes one million times more likely to match one of those passwords. Using a unique salt for each password prevents this.

Does salt size matter?

Another piece of advice I’ve come across is that the size of the salt does not matter. Using common sense this is not correct. A salt such as “52e801da” will still generate the input “love52e801da” which will be in rainbow tables.

Early Unix user passwords had a vulnerability that was partly due to the size of the salt.

Brute Force Attacks on passwords

Now even though we have mitigated precomputation attacks such as rainbow tables, we still have one issue. We have no way to hide the salt. Since we cannot hide the salt securely, we must assume it is visible. If the salt is visible, the attacker still cannot use a rainbow table, however, they can just guess the password by simply guessing passwords and then hashing them with the visible salt, then comparing the result to the stored hash. This is called a brute force attack.

A brute force attack in it’s simplest form, is to try every combination of passwords in sequence. However, sophisticated forms will use precomputated data such as the word combinations in a dictionary (dictionary attack) combined with statistics to find the password “love” very quickly.

Better Passwords - Key Stretching and Adaptive Hashing

Now the first way to combat this is to force your users to have a password at minimum 6 characters and consist of numbers as well as alphabetic characters in both upper and lower case. This is probably the best thing you could do as you have made the range of possible passwords large enough that a brute force will take a while.

Now you need to slow down the brute force. One way to do this is by using key strengthening. This is the process of hashing multiple times, in order to make the hashing process longer. Hash functions such as MD5, SHA1 are very fast. This is not a good thing for you, since the attacker can do numerous hashes very quickly. However, if you hash 1000 times, then the attacker has to also hash each possible password the same number of times, making their brute force attack 1000 times slower.

Now it is up to you to figure out how many times you want to hash the password. Computational power is increasing every day, so you may also want to increase the number of times you hash the password as time progresses. There are hash functions that take this into account such as MD5 crypt and BCrypt. A lot of why this is needed is explained at this Matasano security article.

PHP implementation of secure password hashing

Now I know this has been a rather long article, so here is what I have come up with that incorporates these methods of mitigating these password cracking techniques.

/**
 * Generate cryptographic Hashes for passwords
 *
 * Features:
 * 	Harderned against precomputation attacks like rainbow tables (using unique salts)
 * 	Harderned against brute force and dictionary attacks (using key stretching and optional secret key)
 *
 *  http://en.wikipedia.org/wiki/Password_cracking
 *
 *  Note: for PHP4 and lower, just remove the "public static" before function declaration
 *
 * @author gabe@fijiwebdesign.com
 * @link http://www.fijiwebdesign.com/
 * @version $Id$
 */
class Password_Hash
{

	/**
	 * Generate the Hash
	 * @return String
	 * @param $password String
	 * @param $salt String[optional]
	 * @param $iterations Int[optional]
	 * @param $secret String[optional]
	 */
	public static function generate($password, $salt = null, $iterations = 10000, $hash_function = 'sha1', $secret = '')
	{
		$salt or $salt = self::generateToken();
		$hashes = array();
		$hash = $password;
		// hash a sequence of hashes, each hash depends on the last one, so any implementation must hash each one individually
		$i = $iterations;
		while(--$i)
		{
			$hash = $hash_function($hash.$salt.$secret);
		}
		return implode(':', array($hash, $iterations, $hash_function, $salt));
	}

	/**
	 * Verify a password meets a hash
	 * @return Bool
	 * @param $password String
	 * @param $hash String
	 * @param $secret String[optional]
	 */
	public static function verify($password, $hash, $secret = '')
	{
		list($_hash, $iterations, $hash_function, $salt) = explode(':', $hash);
		return ($hash == self::generate($password, $salt, $iterations, $hash_function, $secret));
	}

	/**
	 * Generate a random hex based token
	 * @return String
	 * @param $length Int[optional]
	 */
	public static function generateToken($length = 40)
	{
		$token = array();
		for( $i = 0; $i < $length; ++$i )
		{
			$token[] =	dechex( mt_rand(0, 15) );
		}
		return implode('', $token);
	}

}

Example using SHA1 with default of 10000 iterations.

// generating the hash
$password = 'test';
$hash = Password_Hash::generate($password);

// verifying a password
$result = Password_Hash::verify($password, $hash);

// dump results
var_dump($hash, $result);

Example using whirlpool as the hash function, a 128 length salt as well as a secret.

// define our custom hash function
function whirlpool($str) {
	return hash('whirlpool', $str);
}

$password = 'test';
$salt = password_Hash::generateToken(128);
$secret = password_Hash::generateToken(128);
$iterations = 10000;

// generate hash
$hash = Password_Hash::generate($password, $salt, $iterations, 'whirlpool', $secret);

// verify
$result = Password_Hash::verify($password, $hash, $secret);

// dump results
var_dump($result);

Base conversion in JavaScript

September4

I just realized recently that you can convert between number bases in JavaScript using the built in method Object.prototype.toString() and parseInt().

Math.base = function(n, to, from) {
     return parseInt(n, from || 10).toString(to);
}

For example, to convert from decimal to hex, or hex to decimal:

// convert the decimal 10 to hex
Math.base(10, 16); // 'a'

// convert the hex 'a' to decimal
Math.base('a', 10, 16); // 10
// or
Math.base(0xA, 10); // 10

Or from hex or dec to binary:

// convert the decimal 10 to binary
Math.base(10, 2); // '1010'

// convert the hex 'a' to decimal
Math.base('a', 2, 16); // 1010

This should work for bases between 2 and 36. ie: the number of characters from 0-9a-z.

Edit: changed base() to Math.base() for better namespace

I’ve also added base conversion in PHP up to a radix of 255.

Fun with JavaScript bookmarks

July15

Here are some random JavaScript sinippets I wrote for bookmarks.

How they work is when you click the bookmark, the JavaScript is executed. So you can actually do a bit of programming inside a browser bookmark.

How to save JavaScript bookmarks:

  1. Copy the Bookmarks JavaScript code
  2. Right Click on the bookmark toolbar
  3. Choose “new bookmark” or the equivalent on your browser
  4. Fill in a name of choice
  5. In the location/url field paste the javascript code
Here is the list:
Translator:

javascript:window.location='http://www.google.com/translate_c?u='+window.location
Calculator:
javascript:void(alert(eval(prompt('Calculate:'))))
Binary Convertor:
javascript:str='';i=0;c=prompt('Encode in Binary:');while(n=c.charCodeAt(i)){ b = '';while(n>0) { b = (n&1 ? '1' : '0')+b; n >>= 1; } i++;str+=b+' ' } void(alert(str))

Real-time Concurrency Control and Version Control

July11

Almost every software developer has used some sort of source code version control system. You’re probably familiar with the acronyms GIT, SVN or the now aging CVS. Here is a list of version control systems.

Version control is a form of concurrency control with the benefit of revisions. Todays major versioning systems work by creating a copy of a resource, and allowing a user to work on that copy. The user then checks their modified copy back to the central source code repository. If the copy at the repository has changed, then the user must merge the two changed copies together.

Since each user is working on a remote copy, there is no real time collaboration involved at all. Each user must edit their copy individually without input from the other users. The collaboration is somewhat chunked, instead of a smooth integration of two or more users inputs.

I’ve been waiting for the day when real time concurrency control is integrated with source code version control. Just lately Google launched Google Wave, which essentially implements real time concurrency control in document editing. A while before that, Appjet Launched their Etherpad product, which implemented real time concurrency control in document and JavaScript source code editing.

Neither of these currently implement source code version control. However, they are proofs that real time concurrency control can be achievable over the web (HTTP) using Operantional Transformation (OT).

Now the missing link is to tie together a real time concurrency control system, with a version control system. Imagine checking out a document with a number of colleges, editing it concurrently in real time, and checking it back into version control.

The model of source code versioning will change dramatically with integrated real-time concurrency control. Checking out a copy may be a thing of the past. You can just join a a group that is editing a live copy, and revisions can be saved automatically for individuals and/or the group as a whole.

Add your own thoughts here…

PHP Object Cache

June25

The last two days I’ve been writing an Object cache in PHP as part of a larger project. I released it today as open source so it will help those doing something similar and get some that helpful feedback open source offers.

PHP Object Cache is a Memory Object cache, implemented with PHP Sockets. It runs on PHP4 or PHP5+ and requires your PHP build to have sockets enabled.

The project is a web (browser) based chat system called Joomla Ajax Chat. The initial development of the chat was 3 and a half years ago, so many of what worked then (HTTP Polling and AJAX) is becoming old school now and not as efficient as what is possible with browsers today.

One of the new features is implementing comet like HTTP, which essentially means keeping the HTTP connection open for as long as you can. XMPP defined a specification for this called BOSH for their Instant Messaging Protocol to work over HTTP. With a good implementation, Comet can be very efficient.

Our problem is that we have to implement Comet on regular ol’ Apache, Lighty, Nginx, IIS servers running different versions and builds of PHP. The code will have to run on the average shared hosting, dedicated servers to cloud based solutions. The other problem is it has to run on top of Joomla, and other CMSs. A call to a minimal Joomla page alone, is around 5-6 database reads and one or two writes and about 5-6Mb of ram. It wouldn’t take much to crash a shared hosting account trying to implement comet on top of that.

So the solution would have to be an object cache of some sort, even if it means a file based cache (last resort). Thus, PHP Object Cache, which hopefully will work for the percentage of shared server that allow sockets. The idea it to take the most intensive IO processes, such as session management, chat events and put them into the Object cache. When everyone online has viewed what they need from the cache, flush that bit to the database or some other persistent storage if needed.

Since the browser is able to do more as time goes on, hopefully php on old shared servers can keep up.

Backup and export Appjet Applications

June10

Appjet recently anounced that they are closing down. They however offer an open source download that allows you to run the appjet engine on your own server.

JGate has gone to the intiative of cloning the Appjet service. This allows you to transfer your apps from Appjet to JGate, including your storage and libraries.

Backup your Appjet Applications

For those needing to backup your applications, I’ve written a backup application that will take all your published applications and create a single compressed download in Zip or Gzip format.

http://export.appjet.net/

This app is actually cloned from two other apps. It has to contact a PHP server on the background in order to create the compressed files. I couldn’t find any JavaScript implementations of ZIP and Gzip readily available.

Using the Application is simple, just input your username and click on “backup my apps” and it will prompt a download after a few seconds.

Export your Appjet Storage

To export your storage you need to import an Appjet Library.

http://lib-export.appjet.net/
http://apps.jgate.de/lib-export

This library exists both on Appjet and JGate, and if you are hosting your own Appjet server then you need to include that library.

Here are the steps to export your appjet storage:

  • Import lib-export into both apps ie: import(’lib-export’);
  • Show the Admin Panel. On appjet.net, you can do this via the preview window. In JGate y ou need to include showExportAdminPanel(); after your import command and view the app in regular mode.
  • First set a password on both apps.
  • Then in the app you want to export from, turn off the admin panel. ie: remove the line showExportAdminPanel();
  • In the app you want to import to, click on the “import” link in the admin panel.
  • Fill in the details, make sure the URL does not include a trailing slash. eg: http://example-app.appjet.net
  • Submit the form.
You can also view the JSON representation of your root storage object (storage) by going to the URL: http://appname.appjet.net/export?password=mypass where appname and mypass are substituted for your own.

Other uses of lib-export:

If you’re interested in how it works, check out the source for lib-export.
Since appjet uses object storage instead of a traditional relational database, you can serialize the storage Objects into regular objects, and then into JSON, for export.
You can also do the opposite and unserialize JSON into native JavaScript Objects, and native JavaScript objects into Appjet storage Objects. (StorageObject and StorableCollection instances).
This can be useful if you want to storage random JavaScript Objects without knowing their type. It should even store your custom JavaScript instances, though I haven’t tried it.
eg: storing native and custom Objects
import('storage');
import('lib-export');
// native object
storage.date = storablelizeObjects(new Date());
function MyCustomObject() {
   this.name = 'my custom object';
}
// custom object
storage.customObj = storablelizeObjects(new MyCustomObject());
eg: storing Arrays
import('storage');
import('lib-export');
storage.myArray = storablelizeObjects(['hi', 'bye']);
eg: Store anything
import('storage');
import('lib-export');
storage.myArray = storablelizeObjects({
   'myarr': ['hi', 'bye', {"another object": "value"}],
   'myObj': new MyCustomObject(),
   'date': new Date(),
   'url': wget('http://example.com')
});
Convert Storage Object to native Objects
import('storage');
import('lib-export');
var myObj = objectizeStorables(storage.myObj);
Export your Storage Objects as JSON
import('storage');
import('lib-export');
var myObj_str = serializeStorables(storage.myObj);
Note that properties resolved from Object.prototype is not stored. Obviously, using StorableCollection() and StorableObject() will be faster then casting native Objects to Storables using lib-export, though the latter is more convenient for storing objects of unknown types.

Notes

The library lib-export is provided without any guarantees. Please do not use it if you do not take full responsibility for any outcome. ie: It works for me, but I cannot guarantee that it will work for your app.
Please note that the export lib is still in development, so there may be some changes to it which I will document here.

Appjet service ends

June2

Appjet, a web based javascript application development platform, sent out an email to members today that they are closing down the service.

According to the email, they are closing their Appjet service because of the success of Etherpad - which is a real time document collaboration service that they built on Appjet. I think what they really mean is, appjet isn’t making any money and has become a liability. In that case, they just didn’t find a good way to capitalize on what they have, because it is quite awesome.

Appjet has been the only server side JavaScript development platform that offered a web based editor so you could create applications with just a connection to the internet, nothing else. There are many server side JavaScript alternatives, and many JavaScript Application development Platforms, none had the simplicity Appjet had. Nor is there a platform that encourages the sharing of libraries amongst developers like Appjet.

I have quite a few applications running on Appjet. One the fetches updates from freelance sites and posts it to a twitter account, freelance_jobs. Another pings websites and keeps a log of their uptime.Yet another acts as a proxy allowing RSS feeds to be fetched cross domain as a JSON callback. Theres even one that allows to to cheat at facebook’s wordtwist game by guessing all the possible combinations of words available for that round. There’s more then 10 of these apps I wrote and they were all hosted freely by Appjet. These were all developed in my spare time and at first I thought of them only as fun apps, but a few of them have become quite useful.

Now comes the porting of these apps to a new location. The great thing is that the appjet engine is available for download.

I’m considering installing this on my own server, or just rewriting everything in PHP.

Incoming mail with PHP Mime Mail Parser

May13

I’ve just added the ability to parse mime mail from standard input into PHP Mime Mail Parser. This allows you to receive and parse email in PHP efficiently and effortlessly. 

To pipe email to PHP, follow one of these articles:

Evolt - Incoming Mail and PHP
DevPapers - Incoming Mail and PHP
DevArticles - Incoming Mail and PHP

Then for your PHP code, get the PHP Mime Mail Parser, and use:

<?php
// include the mime-mail-parser class
require_once('MimeMailParser.class.php');

// instantiate
$Parser = new MimeMailParser();
// read the email from stdin
$Parser->setStream(STDIN);

// get the email parts
$to = $Parser->getHeader('to');
$delivered_to = $Parser->getHeader('delivered-to');
$from = $Parser->getHeader('from');
$subject = $Parser->getHeader('subject');
$text = $Parser->getMessageBody('text');
$html = $Parser->getMessageBody('html');
$attachments = $Parser->getAttachments();

?>

Monster - 3d in JavaScript

April24

I was reading the Qt blog’s on Webkit when I came aross this post on the chrome experiment Monster.

The experiment is written by Dean McNamee and is a JavaScript 3d rendering engine. There is a great article on it at the google code blog.

It renders a cube that turns into a sphere then a monster. All with just JavaScript, no Flash or other plugins.

I tried launching the demo (click to open in a new window) in Chrome, Safari and Firefox. Chrome was the only browser that handled the animation well. It didn’t even use more then a few MB of memory to do it. Safari climbed to about 200MB of memory and didn’t render the more intensive parts of the animation. Firefox was slow in rendering and also took up to 200MB of memory. This just shows how vastly superior Chrome’s Javascript engine (V8) is. You’d also have to take into account the implementation of the Canvas element, but the Javascript execution is what really matters.

The code for Monster isn’t released yet. Would be cool to play around with it when it is.

A PHP Mime Mail parser using MailParse Extension

April21

Ever tried parsing Mime Mail? Not for the faint hearted, I assure you. The great thing is that PHP has an extension for parsing Mime Messages called MailParse. The bad news is that using this extension is probably just as hard as writing your own Mime Parser. 

Fortunately with a bit of help I’ve put together a Mime Mail Parser Class that wraps the MailParse extension functions making it simple, efficient and fast to parse mime mail in PHP. 

Why another Mime Mail parser? Well for two main reasons. 

1) Pure PHP implementations are slow and inefficient compared to MailParse.

2) MailParse is too hard to use, and is not OO. 

Therefore welcome to MimeMailParser.

Here is a an example that shows how easy it is to parse raw mime mail using MimeMailParser:

<?php

require_once('MimeMailParser.class.php');

$path = 'path/to/mail.txt';
$Parser = new MimeMailParser();
$Parser->setPath($path);

$to = $Parser->getHeader('to');
$from = $Parser->getHeader('from');
$subject = $Parser->getHeader('subject');
$text = $Parser->getMessageBody('text');
$html = $Parser->getMessageBody('html');
//$attachments = $Parser->getAttachments();
$attachments = $Parser->getAttachmentsAsStreams();

?>

You can find the source code here for your enjoyment.

Secure PHP Programming for Web Developers

February12

Security in PHP is the same as any server side programming language, they are all vulnerable to the same attacks.

PHP has a history of being vulnerable mainly because of its popularity.

1) More non-security aware developers use PHP then any other language, so there code has flaws almost 100% of the time.
2) There are more sites written in PHP then any other server side language, thus more chances to find security holes amongst so many.

PHP has had a rep of having security problems in the PHP core itself, but this has improved greatly. Much of which can be attributed to the Hardened-PHP project.
http://www.hardened-php.net/suhosin/

If you are using a shared host, ask if they have PHP4 with the suhosin patch or PHP5 or higher. I’m not sure if PHP5 still needs Suhosin but it seems many large sites aren’t using the two together so I believe PHP5 has a better security then PHP4 natively.

As a developer I think there are just about 4 or so main security vulnerabilities to keep in mind when coding.

1) XSS (Cross Site Scripting)

This is the most common vulnerability in any website. It is estimated that around 70% of websites have an XSS vulenerability.
http://en.wikipedia.org/wiki/Cross-site_scripting

A simple example in PHP:

<?php

echo $_GET['username'];

?>

What happens is the PHP echo’s a variable passed in from HTTP (in this case a GET parameter). If a user typed in the browser URL:

site.com/example.php?username=<script>alert('document.cookie')</script>

They would see the cookies saved for their session. An attacker can make a user click a link that will also retrieve these cookies from JavaScript, and send it to them - without the user knowing.

To prevent it:

<?php

echo htmlentities($_GET['username'], ENT_QUOTES, 'UTF-8');

?>[

This will turn any HTML into HTML entities. You also have to specify the encoding you used for the page (in this case UTF-8). The reason is so PHP codes not mangle the character encoding, which can also result in XSS.

I must say here, that you should never use your own PHP filtering functions for HTML, or any other "cleansing" of user input. Most likely you will miss something that an attacker will use.

2) XSRF - Cross Site Request Forgery

This is similar to XSS and just as common or maybe even more common. It is when a website fails to protect it's users from being used by 3rd parties without their knowledge.
http://en.wikipedia.org/wiki/Cross-site_request_forgery

And example of this in PHP is a simple comment form.

<form action="submit.php">

<textarea name="comment">
<input type="submit" value="Post Comment" />

</form>

Imagine the comment form is only available for logged in users. Now an attacker can just send an already logged in user the URL:

site.com/submit.php?comment=I hacked you&submit=Post Comment

So when the logged in user clicks on that link, they have posted the comment without knowing. This can even be done in a hidden frame, so the user never see's it.

So the attacker is using the user's already authenticated session (privileges) to do his/her bidding.

Preventing XSRF:

<form action="submit.php">

<textarea name="comment">
<input type="submit" value="Post Comment" />
<input type="key" value="some_random_value" />

</form>

Notice the new <input /> called "key". It will contain a random value remembered by PHP. This random value should be saved, and be unique for every form that is sensitive.

This way, the attacker would not be able to make the user post something on their behalf, since they don't know the value of "key".

(This only works if you don't have an XSS vulnerability of the page itself, as that can lead to the attacker knowing what the value of "key" is)

3) SQL injection

SQL injection is when an attacker manages to manipulate any SQL database queries in your website in a way you didn't intend.
http://en.wikipedia.org/wiki/SQL_injection

Example:

<?php

$query = "SELECT * FROM users where password = '".$_GET['password']."'";
$result = mysql_query($query);

?>

Because the $_GET['password'] can be anything the attacker wishes to put in the URL, they could craft a URL like:


site.com/login.php?username=joe&password=nothing' or 1

Notice the ‘ in the value for the parameter “password”

This will make your sql query:

SELECT * FROM users where password = 'nothing' or 1

This would make it return the first user instead of the user with password = “nothing” since “or 1″ is always true.

Preventing SQL injection:

< ?php

$query = "SELECT * FROM users where password = '".mysql_real_escape_string($_GET['password'])."'";
$result = mysql_query($query);

?>

the function mysql_real_escape_string() will prevent any SQL injection by escaping any character that would otherwise terminate the string.

4) Remote File inclusion

Remote file inclusion is when an attacker can include remote file into your PHP code. This is the most dangerous attack, as it allows the attacker to execute arbitrary code on your PHP server.
http://en.wikipedia.org/wiki/Remote_File_Inclusion

eg:

<?php

include('/pages/'.$_GET['page'].'.php');

?>

With this code the developer is hoping to have a URL such as:

site.com/pages.php?page=home

And this would include the file:

/pages/home.php

However, any attacker can now place a URL such as:


site.com/pages.php?page=../../passwords.txt

And it would reveal the contents of the file passwords.txt

Or they could use it to include a remove file from their server, if the URL wrappers are enabled for file includes (which is common).

It is best not to have any user input in the files to include. However, if you think it is beneficial to your PHP website, then make sure you have a predefined list of files that can be included.

For example:

<?php

if (in_array($_GET['page'], array('home', 'contact', 'links', 'about')) {
    include('/pages/'.$_GET['page'].'.php');
}

?>

With these in mind, you should be able to keep your PHP site secure from most attacks. Now in one paragraph:

When ever you write a PHP page, make sure all input from users (HTTP) is escaped according to the the following rules, HTML just be entity encoded with htmlentities(), XML with htmlspecialchars(), SQL must be escaped by mysql_real_escape_string(). User input should not be used for file inclusions at all, but if you do, then make sure it is from a predefined list of possible files. All forms must contain a secure, random, key that is used to authenticate it only once so that there can be no subsequent posts of the same form, or posts from other sources.

That actually covered all the points above. Can you see how simple it is? It is just keeping those in mind at all times while coding that is the hard part.

Get Satisfaction and W3C Compliance

February9

We recently have tried using Get Satisfaction to complement our support efforts at Fiji Web Design. It is a very interesting concept, using and open support channel that is transparent, viewable by anyone.

What GetSatisfaction offers is just the platform for providing support. The actual support is carried out in their open message boards, by whoever wants to pitch in.

Today I noticed that our site was not passing W3C xHTML1.0 standards validation. It appears that the cut and paste codes provided by GetSatisfaction are not xHTML compliant. The issue lies in the embedding of the <style> tag in the <body> section.

The solution is to either load the code in the <head> section, or a rewrite, which is what I did.

The actual snippet looks something like this:





The xHTML valid version would be:



Now the JavaScript is encapsulated in comments, so special characters like < or & will not cause validation errors. The <style> declaration is also moved to JavaScript, so the JavaScirpt writes it to the document instead.

The results is a great simple to use service while keeping your website W3C compliant.

Client Side Application development for Web Developers

January31

Coming from a web development background, you would probably find using either Adobe AIR, or XUL the simplest to understand and require the least development. Both are developed to mimic browser based development, which is very high level. You work mostly with XML, and would use EcmaScript implementations you are familiar with like Javascript in XUL, or Actionscript in Air.

XUL is XML markup for UIs and is the platform on which Firefox is built. You just need XULRunner, from Mozilla, which is the runtime. XUL allows you to write the UI in XML, then use JavaScript for scripting. XULRunner acts like a mini firefox, providing the same environment you would see in Firefox Extensions or the actual Firefox runtime environment. The JavaScript has an interface to Java, C++, Python, ActiveX, etc. Storage can be XML files or SQLite.
You can embed HTML in XUL, as well as the Flash Player.

So basically you could set up your desktop application to work like a webpage, and just edit HTML and create links, forms etc. (links are relative and internal to your app, no server needed unless you add a http protocol based link) Do some JavaScript when you need a bit of UI animation or storage access etc. If you wanted flashy animations, just use the Flash Object in HTML etc.

With AIR, you can work in a HTML, Flex, or Flash environment. The HTML environment allows you to use JS as it known in browser scripting, and act as if you are working on a browser based application. HTML environment built around Webkit, which is the underlying platform for Safari and Chrome browsers and provides the Ecmascript runtime. The storage is embedded SQLite.

The Flex environment is similar to XUL, as it uses MXML, which is Adobe’s XML markup/standard for UI development. MXML uses Actionscript3.0 which is the latest Ecma specifications. It has some really nice scripting features, classes, libraries, namespaces, E4X etc. and great bindings with the XML UI (MXML).

I don’t have any experience with the Flash environment. Should be the same as FLEX but with the Flash IDE so you code less and do more visual development.

Silverlight I believe provides a similar environment, but I haven’t tried it yet.

Both XUL and AIR are very high level, so you don’t have to worry much about the environment and can focus more on building your App. It is great if you develop by yourself, as you can achieve more faster.

RSS Feeds via cross domain JSON proxy

January22

JavaScript remoting functions are limited to the same domain. For example, XMLHttpRequest can only retrieve URLs on the same domain, and the same applies for the Flash remoting methods.

JavaScript files however, can be hosted on a different domain, and this is the basis of a well known JavaScript remoting method.

To retrieve RSS feeds, you don’t need a proxy on the same domain.
You can actually have a proxy on a different domain, but have that proxy create a JavaScript file of the RSS XML text.

That is, encapsulate the RSS XML text in a JavaScript variable or function.

That way you can include the RSS as a JavaScript file.

For example, the yahoo top stories:
http://rss.news.yahoo.com/rss/topstories

Could be proxied as:

http://json-proxy.appjet.net/?url=http://rss.news.yahoo.com/rss/topstories

And retrieved via a simple <script> tag:

<script src="http://json-proxy.appjet.net/?url=http://rss.news.yahoo.com/rss/topstories"></script>

And recieved in JavaScript as:

function callback(rss) {
// manipular rss here..
}

I’ve written a JavaScript class that will do the heavy lifting, and allow you to retrieve RSS feeds cross-domain from within JavaScript.

You can view the JavaScript source at the project page.

I’m using this JavaScript class to power the images at the top of my blog. They are retrieved from a Flickr RSS feed via JavaScript, with no server side interaction on my domain - just the JSON proxy at Appjet.

Update: 25th Sept, 2009

Appjet has closed down. The JSON/RSS proxy now resides at: http://json-proxy.jgate.de/

So the URL to proxy any webpage would be: http://json-proxy.jgate.de/?url={url}

The JavaScript RSS Proxy library has been updated to reflect this.

Google AJAX Language API with PHP

January20

I had noticed some time ago that Google had released an API for their language translation service. A recent forum discussion made me revisit the API, and since I had a wee bit of time on my hands, I wrote a very rudimentary PHP class implementation of the API.

Google seems to like flaunting “AJAX”, in their APIs at least. So the API is called “Google AJAX Language API” and the main implementation is .. take a guess, AJAX. However, in addition to their pure JavaScript API, they also have a REST interface (of course JavaScript would need such an interface anyway).

The REST interface is just a HTTP endpoint (URL) that returns JSON. You just need to formulate a HTTP GET passing the parameters described in the API documentation, and Google will send you a nicely formated JSON response with the translated text and some other details.

Here is an example request, to translate “Hello World” to Italian.

http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=hello%20world&langpair=en|it

The parameters are q=hello world and langpair=en|it

The JSON response looks like:

{"responseData": {"translatedText":"ciao mondo"}, "responseDetails": null, "responseStatus": 200}

So a simple implementation in PHP would be to use file_get_contents() to download the JSON text from the URL over HTTP and here it is in a PHP class.

http://code.google.com/p/php-language-api/source/browse/trunk/google.translator.php

Example Usage:

// example usage
$text = 'Welcome "to my " website.';
$trans_text = Google_Translate_API::translate($text, '', 'it');
if ($trans_text !== false) {
	echo $trans_text;
}

The class uses file_get_contents() which assumes your PHP has allow_url_fopen directive enabled in the PHP configuration (PHP.ini). This is usually the case, however, it can be disabled for security reasons (since it allows include() to use the URL wrapper and thus include remote files - a favourite exploit for attackers it to inject remote files into include() functions)

The class doesn’t use a JSON parser, I think its a bit of an overhead including the JSON libraries in PHP4. Intead it just uses regular expressions. The PCRE regular expression functions are pretty fast. PHP5 has native support for JSON however, so the class could be modified to use the PHP5 native JSON functions if you use PHP5 specifically.

Something I ran into was that Google returns not only UTF-8, but UTF-8 escape sequences. That is, they have characters outside the basic ASCII range escaped with the UTF-8 escape sequence which is \u followed by the character’s hex value. For example, the & symbol becomes:

\u0026

Cools aye. Sucks, because PHP does not understand this. JavaScript, which is the main method of invoking the Google Language API, understands this natively. PHP doesn’t even understand UTF-8 in PHP4. First I resorted to this ugly function to unescape the UTF-8 escape sequences (convert those UTF-8 sequence to actual UTF-8 byte sequences).

/**
         * Convert UTF-8 Escape sequences in a string to UTF-8 Bytes. Old version.
         * @return UTF-8 String
         * @param $str String
         */
        function __unescapeUTF8EscapeSeq($str) {
                return preg_replace_callback("/\\\u([0-9a-f]{4})/i", create_function('$matches', 'return html_entity_decode(\'\'.$matches[1].\';\', ENT_NOQUOTES, \'UTF-8\');'), $str);
        }

The function is ugly because it uses html_entity_decode() to do the transformation for us. We just convert the UTF-8 escape sequence to a HTML escape sequence (HTML entities), then use html_entity_decode() which PHP handles well. I decided on a compatible function that uses bitwise operations instead. Both are included in the source however for reference.

The code is very early development and will be buggy. You can check out the latest sources via SVN:

svn checkout http://php-language-api.googlecode.com/svn/trunk/ php-language-api-read-only

Feel free to let me know on the Google project page if you find any bugs.

http://code.google.com/p/php-language-api/issues/list

« Older Entries
Tag Cloud