Showing posts with label mysql. Show all posts
Showing posts with label mysql. Show all posts

Thursday, November 19, 2009

Profiling MySQL

To analyze the db server usage in a complex PHP application, the first step is to profile the db server.
There are lots of tools to profile, but I think it's very easy to make a customized code to save the data really needed.
The idea is save information about some queries in the production environment (about 1% of the queries is usually enough, depending on the traffic).


MySQL profiling

Hoping there is a class used to manage queries (or at least mysqli class), it doesn't take long to replace a function that manages the queries with something similar to the following code (written in a simple way to show the idea):


class DB extends mysqli {
...
function query ($q) {

$start = microtime(1);
$this->query($q);
$wtime = microtime(1) - $start;
#save 1% of the queries + info
if ( rand(0,100)<1>query("INSERT DELAYED INTO `ProfileData` (q,wtime,created,...) ($q,$wtime, NOW(), ...) ");
}

}
...
}


What other info to save ? some ideas:
  • client IP
  • other $_SERVER info: user-agent, request_method, etc...
  • PHP backtrace (to understand which line launched the query)
  • web server load
  • mysql server load
  • ...
How to analyze results making queries on the `ProfileData` table.
example: queries grouping by query and showing the average time of the queries. In this way, you can find what queries are the slowest ones.



-- select the slowest queries (average time) in the last 24 h
-- exclusion of the queries executed only once to exclude missing sql cache
SELECT `q`,AVG(`wtime) as "medium time", COUNT(`id`) as "occurences"
FROM `ProfileData`
WHERE `created` > DATE_ADD(NOW(), INTERVAL -1 DAY)
GROUP BY `q`
HAVING COUNT(`id`) > 2
ORDER BY AVG(`wtime) DESC

MySQL dump importing

Today I realized that "mysqlimport" is not working as expected on Wamp environment.
A working way to import a sql/dump file is to use the "mysql" executable

#localhost
mysql --u root -p --user=root --force [DBNAME] < [FILE.SQL]

Wednesday, June 10, 2009

MySQL : version differences notes (draft)

Relevant features added in Mysql 4.0
  • FULLTEXT search
  • INNODB tables (and on [delete|update features]
  • UNION statement
  • TRUNCATE TABLE statement
  • multiple table delete and update
Relevant features added in Mysql 4.1
  • Subquery (nested query) supported
  • "CREATE table LIKE table 2" syntax supported
  • added storages:
    - EXAMPLE (for developers)
    - NBCLUSTER (table partitioned over many computers)
    - CSV (comma separated values storage)
    - BLACKHOLE (it doesn't store data)
  • added command "HELP"
  • API improved (multiple statement in a single call)
  • "INSERT ... ON DUPLICATE KEY UPDATE" syntax added
  • GROUP_CONCAT() added
Relevant features added in Mysql 5.0
  • BIT data type
  • Math precision improved
  • SQLSTATE error message
  • TRIGGERS (limited support)
  • VARCHAR max length increased to 2^16 byte (64 Kb)
  • "greedy" optimizer, "merge index" to improve join performance
  • Improvement of "NOT IN"
  • VARCHAR more efficient
  • INNODB less disk space usage (-20%)
  • Added storages ARCHIVE and FEDERATED
Relevant features added in Mysql 5.1
  • Partitioning (tables in different locations)
  • plugin API to load/unload components at runtime
  • event scheduler (like cronotab)
  • customizable MySQL logs (to file or tables)
  • XML and XPath functions
Relevant features added in Mysql 6.0
  • UTF8 (4 bytes), UTF16 and UTF32
  • BACKUP_DATABASE and RESTORE statements (to backup)
  • performance optimization for subqueries and joins
  • transactional locks
  • LOAD_XML statement
 

PHP and tips|PHP