I've just read this pdf from I.A.'s blog about PHP 5.3 performances.
My comments:
Performances
What I really consider good is the performance increasing (5/10%) that include a smarter behaviour with require/inclusion, smaller binary size and better stack performance.
Features
- Namespaces are OK, but not really necessary. A good code can be written also without them.
- I think the best feature is the late static binding. It was the only big lack about the PHP OO.
- Also closures are sometimes useful to write a clearer code. I've tested their performances (*) and it seems there are no decreasing using them, that's cool.
(*) array_map(function ($n){return($n * $n * $n);}, array(1, 2, 3, 4, 5)); #
- "goto": its utility (especially in a OO language!) doesn't make any sense to me. Very bad code readability with it.
- MySQLInd sounds very interesting to have better performances with MySQL. Client side query cache, written for PHP (not C/C++), performance statistics for bottle-neck analysis [read here]... wow ! I'll probably test it soon although (according to some blog posts) the performance increasing seems not very high.
- Hundreds of bug fixing and improvements, including Directory iterator and date functions => well done PHP community !
Monday, November 16, 2009
Tuesday, November 3, 2009
How to optimize PHP applications
Thursday, October 15, 2009
a Javascript function to manage function timeout
Today I worked with form validations and AJAX requests associated with the event keyup in the input fields.
Sometimes the validation requires an AJAX call (for instance to check if the typed text already exists in the DB). In order to avoid a request for each char typed, a good solution might be using set/clearTimeout.
With the aim of doing only one function to manage all the timeouts of the functions called for each field, I've written a general function that does that, supporting dynamically the function name.
It uses an array (member of the function) to store the timeouts of the functions called, and an eval to launch/stop the each function with the relative timeout.
Tested only on Firefox 3.5.2
/*javascript*/
function callTimeout(funcName, timeout, args) {
if (args==undefined) {args = "";}
if (timeout == undefined) { timeout=500; }
if (this.timeOutsArray == undefined) {
this.timeOutsArray = new Array(); }
if (this.timeOutsArray[funcName] == undefined) {
this.timeOutsArray[funcName] = 0;
}
if (this.timeOutsArray[funcName]) {
clearTimeout(this.timeOutsArray[funcName]);
}
eval("this.timeOutsArray['"+funcName+"'] = setTimeout('" + funcName + "("+args+")', "+timeout+");");
}
Example of use with JQuery:
$("#field1").keyup(function(){callTimeout("validatefield1",500);})
...
...
$("#anotherFieldN").keyup(function(){callTimeout("validatefieldN",500);})
Behaviour: If the user types very quickly in the search box (id=field1), the function is not called hundreds time, but just at the end after 500ms.
Monday, October 12, 2009
printing the backtrace in a complex PHP application
the PHP debug_backtrace [man] function is very useful to understand where a function/method is called.
It prints the back trace of the code.
Example: In the framework I'm currently using there are ORM classes to access the DB. So, it takes long to understand where the query is launched when needed. Solution: save in a global variable (or in a field of the class) the list of the queries launched and the code that has launched each query.
Code:
class DB {
# ...
public function query($query) {
if (DEBUG_MODE) { ##
$_dbt = debug_backtrace();
$_fromFile = isset( $_dbt[0]['file']) ? str_replace(_ROOT, "",$_dbt[0]['file']) : "";
$_fromLine = isset( $_dbt[0]['line']) ? $_dbt[0]['line'] : "";
$_launchedFrom = "launched from $_fromFile:$_fromLine";
$this->logQueries[] = "[$query][$_launchedFrom]";
# ..query.
}
}
# ...
}
Example of result:
Array
(
[0] => [set names utf8][launched from C:\wamp\www\dev\index.php:68]
[1] => [SELECT * FROM `Setting` WHERE `id` = 1][launched from \class\Setting.class.php:20]
[2] => [SELECT id FROM `User`WHERE id = 293968 LIMIT 1][launched from \class\User.class.php:598]
[3] => [SELECT * FROM `User` WHERE `id` = 293968][launched from \class\User.class.php:45]
[4] => [SELECT f.store_id FROM `Store_Monthly_Featured` f ORDER BY f.order ASC][launched from \class\Store_Monthly_Featured.class.php:16]
)
Useful function to print only line and number
function debug_backtrace_filelines() {
$ret = array();
$b = debug_backtrace();
foreach ($b as $elements) {
$ret[] = $elements['file'].':'.$elements['line'];
}
}
How to search complex source code using regular expressions
Today I needed to search the source code that ware calling a function with a complex array of parameters
URL::Make( 'site/store.inc.php', array( 'action'=>'view','id'=> [... STH...] ,'idTwo'=> [... STH...] ) );
Considering this requirements:
- there are lots of similar function (to exclude from the search) with an argument less or an argument more
- there may are additional spaces in the line
- some other delopers have probably used double quotes instead of single quotes for array keys
- the value of the keys may be a PHP code
- every function call is in one line
An elegant solution is to search using regular expression. Netbeans (my favorite editor) supports POSIX extended regular expression and is very quickly to search a complex epression in a huge amount of source code.
Solution 1
#using char length and patter ".{0,MAX}" (that is: every sequence, MAX maximum chars). Easy !
URL::Make.{0,10}site/store.{0,40}action.{0,9}view.{0,20}id.{ 0,200}idTwo.{0,30}\).{0,10}\)
Solution 2
#using regexpr for separators. more complex. more accurate in some cases but fails in some cases (eg: comments inside)
URL::Make.*\([ ]*['"]site/store.inc.php['"][ ]*,[ ]*array[ ]*\([ ]*['"]action['"][ ]*=>[ ]*['"]view['"][ ]*,[ ]*['"]id['"][ ]*=>.{0,100}['"]idTwo['"][ ]*=>.{0,100}[ ]*\)[ ]*\)
An idea for DB server load balancing
If the application has a heavy load of reading queries and there are no problems with the application requirements,
a simple possible idea to balance the db server loading is to run the INSERT/UPDATE/ALTER queries in all the servers and run the SELECT queries in just one server (chosen randomly).
Have a look at the 'loadBalancedQuery' function: if the query modifies the db it will be executed in both server, otherwise in just one.
private function queryServer($query, $server = "mainServer" ) {
$db = DB::GetInstance($server);
return $db->query();
}
public function loadBalancedQuery($query) {
if (preg_match("/[]*(insert|INSERT|update|UPDATE|alter|ALTER).*/",
$query)){
$this->queryServer( $query, "mainServer" );
$this->queryServer( $query, "secondaryServer" );
} else {
$this->queryServer( $query, rand(0,1) ? "mainServer": "secondaryServer" );
}
}
}
Sunday, October 11, 2009
UTF8 in LAMP applications: overview and how to solve the common issues
The problem
In a LAMP application the text is frequently saved/retrieved from/to a database and files. We must consider all the different encoding (mapping characher-byte value): latin1 iso8859-1, latin9, UTF-8 (utf8) etc...
Lots of applications use ISO8859 encoding and some PHP functions to convert the characters (htmlentities, htmlspecialchars etc...)
The solution
Converting all the text from an encoding to another using PHP functions is unsafe, difficult and annoying.
Ad example the character "é" (encoded as iso8859) will printed as "é" if it's supposed to be encoded as utf8.
The solution is to use only one charset for files, Content-Type of the pages and the db. UTF-8 [wiki] is the best choice: a variable-lenght char encoding for the standard Unicode. If you use this charset in the HTML file, it won't need to convert the characters to the respective entities.
How To use UTF-8
- set your IDE to save and open source files using utf8 encoding
- set the content-type of your application to utf-8 (better a apache/htaccess rule instead of the meta tag).
- set the database server to use utf8 encoding (also tables must be converted). If the db is utf8 but client encoding is latin1, execute first of all the query "SET NAMES utf8"
- if the application was using latin1 and PHP convert functions, remove all the existing function to encode/decode special characters/entities.
Monday, September 28, 2009
Mass rows copying (duplicating) with filed customization - MySQL
Let's suppose we have a table with the following structure and data:
# `table`
id | a | b | c |
---------------------------
1 | "aaa" | "xxx" | "ccc"
2 | "aab" | "yyy" | "ccc"
3 | "aba" | "yyy" | "ccc"
4 | "abc" | "xxx" | "ccc"
5 | "dcz" | "xxx" | "eee"
Now, we want to copy some records (only some columns) to the same table AND change some of them with a fixed value.
Requirements:
- duplicate all the rows with `b` equal to "xxx"
- for the new rows inserted, the value of `c` has to be changed to "ddd" (fixed value!)
- `id` is the primary key, auto increment
Query (pay attention to the quote characters):
INSERT INTO `table`
(SELECT
# `table`
id | a | b | c |
---------------------------
1 | "aaa" | "xxx" | "ccc"
2 | "aab" | "yyy" | "ccc"
3 | "aba" | "yyy" | "ccc"
4 | "abc" | "xxx" | "ccc"
5 | "dcz" | "xxx" | "eee"
Now, we want to copy some records (only some columns) to the same table AND change some of them with a fixed value.
Requirements:
- duplicate all the rows with `b` equal to "xxx"
- for the new rows inserted, the value of `c` has to be changed to "ddd" (fixed value!)
- `id` is the primary key, auto increment
Query (pay attention to the quote characters):
INSERT INTO `table`
(SELECT
NULL as `id`,
`a`,
`b`,
"ddd" as `c`
FROM `table`
WHERE `b`="xxx")
Result:
id | a | b | c |
---------------------------
1 | "aaa" | "xxx" | "ccc" #row1
2 | "aab" | "yyy" | "ccc"
3 | "aba" | "yyy" | "ccc"
4 | "abc" | "xxx" | "ccc" #row4
5 | "dcz" | "xxx" | "eee" #row5
6 | "aaa" | "xxx" | "ddd" #copied from #row1
7 | "aab" | "xxx" | "ddd" #copied from #row4
8 | "dcz" | "xxx" | "ddd" #copied from #row5
Result:
id | a | b | c |
---------------------------
1 | "aaa" | "xxx" | "ccc" #row1
2 | "aab" | "yyy" | "ccc"
3 | "aba" | "yyy" | "ccc"
4 | "abc" | "xxx" | "ccc" #row4
5 | "dcz" | "xxx" | "eee" #row5
6 | "aaa" | "xxx" | "ddd" #copied from #row1
7 | "aab" | "xxx" | "ddd" #copied from #row4
8 | "dcz" | "xxx" | "ddd" #copied from #row5
Note 1:
The insert operation by using a subquery is not an 'atomic' operation for the recent versions of MySQL. If there are table constraints and a new record is not accepted (e.g: duplicate record for a key defined on columns), only that record won't inserted (not all of them!).
Example: If before there was a record with the same values as in line 9 and there was a unique key on columns (a,b,c), the query would insert only rows on line 6,7 and 8 (9 fails).
Some old version of MySQL stop execution in case of duplicate entries. In this case, add IGNORE to the query to skip duplicates: INSERT IGNORE INTO ...
Example: If before there was a record with the same values as in line 9 and there was a unique key on columns (a,b,c), the query would insert only rows on line 6,7 and 8 (9 fails).
Some old version of MySQL stop execution in case of duplicate entries. In this case, add IGNORE to the query to skip duplicates: INSERT IGNORE INTO ...
Note 2:
If you add a new column to the table, the query will fail !!
Tested on MySQL 5.1.36
Tuesday, September 15, 2009
How to recursively check syntax of PHP files
The executable of PHP supports the '-l' option, that checks the syntax instead of parsing the file.
Using the command 'find', it's possibile to do a interesting operation: syntax checking of all the files recursively, to avoid parse errors in some script !!
find ./ -type f -name \*.php -exec php -l {} \; ";
the result will be a list of files, example:
No syntax errors detected in ./codebase/controller/competition.inc.php
No syntax errors detected in ./codebase/controller/feed_data.inc.php
Errors parsing ./codebase/controller/site/contact.inc.php
No syntax errors detected in ./codebase/controller/compare_prices.inc.php
We can improve the script and print only the file with suntax errors using 'grep'
find ./ -type f -name \*.php -exec php -l {} \; | grep "Errors parsing ";
To launch it from a PHP script
passthru('find ./ -type f -name \*.php -exec php -l {} \; | grep "Errors parsing " ');
Updated: To skip .svn directories add the option :
-not -regex '.*/.svn/*.*'
Using the command 'find', it's possibile to do a interesting operation: syntax checking of all the files recursively, to avoid parse errors in some script !!
find ./ -type f -name \*.php -exec php -l {} \; ";
the result will be a list of files, example:
No syntax errors detected in ./codebase/controller/competition.inc.php
No syntax errors detected in ./codebase/controller/feed_data.inc.php
Errors parsing ./codebase/controller/site/contact.inc.php
No syntax errors detected in ./codebase/controller/compare_prices.inc.php
We can improve the script and print only the file with suntax errors using 'grep'
find ./ -type f -name \*.php -exec php -l {} \; | grep "Errors parsing ";
To launch it from a PHP script
passthru('find ./ -type f -name \*.php -exec php -l {} \; | grep "Errors parsing " ');
Updated: To skip .svn directories add the option :
-not -regex '.*/.svn/*.*'
Sunday, August 30, 2009
Sunday, August 23, 2009
Monitor linux web-servers with monit
"monit is a utility for managing and monitoring processes, files, directories and devices on a Unix system. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations. E.g. monit can start a process if it does not run, restart a process if it does not respond and stop a process if it uses to much resources. You may use monit to monitor files, directories and devices for changes, such as timestamps changes, checksum changes or size changes.
Monit is controlled via an easy to configure control file based on a free-format, token-oriented syntax. Monit logs to syslog or to its own log file and notifies you about error conditions via customizable alert messages. Monit can perform various TCP/IP network checks, protocol checks and can utilize SSL for such checks. Monit provides a http(s) interface and you may use a browser to access the monit program."[ref] .
How to install in ubuntu 8.10
- sudo apt-get install monit
- edit /etc/monit/monitrc and remove comments
#launch system mon every 2min
set daemon 120
#set web interface on port 2812
set httpd port 2812 and
# allow localhost and request from IP 1.2.3.4
allow localhost
allow 1.2.3.4
#request username (admin) and password(monit)
allow admin:monit
#check apache
check process apache with pidfile /usr/local/apache/logs/httpd.pid
#check mysql service (other server)
check host myserver with address 192.168.1.1
if failed icmp type echo count 3 with timeout 3 seconds then alert
if failed port 3306 protocol mysql with timeout 15 seconds then alert
#set alerts via email
set mailserver mail.domain.com
set alert administrator@domain.com - check the syntax of the previous file with
sudo monit -t - edit /etc/default/monit
startup=1 - start deamon !!
sudo /etc/init.d/monit start - call web interface
http://yourserverip:2812/
use admin and password specified before
Saturday, August 22, 2009
Compare Files between two directories recursively - PHP script
In this post I'll present my PHP class to show differences (file missing or different file date) between files in two directories (recursively).
It's useful to compare two websites (Example, test and live versioning the same server) and show the differences, much faster than a FTP synchronization, and customizable (size differences instead of time, PHP copy commands with backup etc...).
Example of output:
How to use:
comparedirs.php
It's useful to compare two websites (Example, test and live versioning the same server) and show the differences, much faster than a FTP synchronization, and customizable (size differences instead of time, PHP copy commands with backup etc...).
Example of output:
How to use:
= new CompareDirs("e:/siti/symfapp/", "e:/siti/symfapp2/");
$c->addSkipPath(".svn".DIRECTORY_SEPARATOR);
$c->addSkipPath(".project".DIRECTORY_SEPARATOR);
$c->addSkipPath(".settings".DIRECTORY_SEPARATOR);
$c->compare();
echo "{$c->totalFiles} files. {$c->skippedFiles} skipped.
{$c->differencesNotFound} missing.{$c->differencesTime} time differences. ".$c->getHTML();
?>
View the codecomparedirs.php
Tuesday, July 21, 2009
benchmark: boolean variables value check
$a = NULL;
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
if ($a) {} }
print microtime(true)-$t."\n"; //2.84 sec
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
if ($a==true) {} }
print microtime(true)-$t."\n"; //3.52sec
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
if ($a===true) {} }
print microtime(true)-$t."\n"; //3.0 sec
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
if (isset($a)) {} }
print microtime(true)-$t; //3.08 sec
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
if ($a) {} }
print microtime(true)-$t."\n"; //2.84 sec
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
if ($a==true) {} }
print microtime(true)-$t."\n"; //3.52sec
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
if ($a===true) {} }
print microtime(true)-$t."\n"; //3.0 sec
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
if (isset($a)) {} }
print microtime(true)-$t; //3.08 sec
benchmark: array_key_exists vs isset
When the aim is to check if the element exists in an array and has a value not null, better to use isset(array[index]), that is much faster than array_key_exists(index, array).
note: array_key_exists(index, array)and isset(array[index]) are a little different:
#PHP 5.3.0 command line - Amd turion 64 1.6 Ghz
#create array
$a = array();
for($i = 0; $i<100000; $i++) {
$a[] = "$i asd a".($i*2)." sd";
}
#isset
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
isset($a[$i]);
}
print microtime(true)-$t; //4.16 sec
#array_key_exists
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
array_key_exists($i, $a);
}
print microtime(true)-$t; //11.7 sec
$a = array();
for($i = 0; $i<100000; $i++) {
$a[] = "$i asd a".($i*2)." sd";
}
#isset
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
isset($a[$i]);
}
print microtime(true)-$t; //4.16 sec
#array_key_exists
$t = microtime(true);
for($i = 0; $i<10000000; $i++) {
array_key_exists($i, $a);
}
print microtime(true)-$t; //11.7 sec
$a = array(0=>0, 1=>NULL);
print isset($a[1])?1:0; //0
print array_key_exists(1, $a)?1:0; //1
print isset($a[1])?1:0; //0
print array_key_exists(1, $a)?1:0; //1
note: eAccelerator ( and probably also other PHP accelerators) don't improve the array access time
note: xDebug (and probably zend optimizer) activated can slow down a lot the performance (by 6/7 times) with huge arrays like these !
Saturday, July 18, 2009
Functional testing for web applications: Selenium IDE for firefox
In a previous post I wrote about PHP Unit testing.
In software science, the levels above Unit testing are Integration, System and System integration testing.
For a standard web application I suggest to use only Selenium [official site], a powerful complete framework , distributed with apache licence 2.0 (a free-software, GPL compatible).
Selenium includes many components. In my opinion the most useful is Selenium IDE [page], a firefox plugin to create action tests, play and check results.
Basically, a test case is a sequence of commands, and a test suite is a cluster of test cases.
To create actions the tool allows to record user actions (clicks, page opening...) or manually insert/edit (mainly to insert verify-actions) and play the test case (or test suite), then check results.
It's possible to export the test cases in many formats (html table, o export (html table format, PHPunit selenium extension, etc...) and to import (only html table).
Every command has only one or two arguments. The command reference is present in selenium IDE. Here are the commands I mainly use and I suggest:
- open dir/page.html
open the url specified - type id=phone 123456
fill in field with id=phone with the text "123456" - clickAndWait id=elementID
click on the link with ID equal to "elementID" . - clickAndWait link=text
click on the link with "text" as internal text (a "text-dependent" approach is plainly not recommended) - verifyTextPresent done
Verify the page contains the text "done". If not, the test will fail at runtime. (be careful: "text-dependent"). - verifyElementPresent id=elementID
Verify the page contains the element with ID equal to "elementID". If not, the test will fail at runtime. - assertConfirmation
assert a javascript confirmation appear
Instead of use the ID to locate an element (example: "id=linkA") it's possibile to use XPath expressions [syntax by w3c school] as locators. Example: "//div//a[2]" is the second link inside the "div" tag. I don't suggest to use XPath: if you (or the designer) change the template (position, or text inside links, or change teh text) the test will not work !
To write solid reliable tests, remember to use a univocal ID for elements to test (message confirmation, links, form fields..), and write the test as soon as the application is working (you will easily and quickly test the correct application behaviour after adding new features).
To test CRUD forms, I suggest to write the html code (template) using a different ID for confirmation messages and error messages
[?php if ($result ): ?][div id="postok"]submit ok[/div][?php else: ?][div id="posterr"][?php echo $messageError ?][/div][?php endif; ?]
To test the result of this form:
verifyElementPresent postok
or
verifyTextNotPresent posterr
Selenium IDE screenshot
Subscribe to:
Posts (Atom)
