views:

43

answers:

2

Hi,

I turned the slow queries on for a Drupal site and following are the top 10 slow queries. Any idea on how I can find their source or origin

 Reading mysql slow query log from /var/log/mysql/mysql-slow.log
Count: 1  Time=220.72s (220s)  Lock=0.00s (0s)  Rows=14293563.0 (14293563), root[root]@localhost
  SELECT /*!N SQL_NO_CACHE */ * FROM `sessions`

Count: 1  Time=42.62s (42s)  Lock=0.00s (0s)  Rows=4181557.0 (4181557), root[root]@localhost
  SELECT /*!N SQL_NO_CACHE */ * FROM `search_index`

Count: 1  Time=28.26s (28s)  Lock=0.00s (0s)  Rows=1.0 (1), exdigital[exdigital]@[10.200.1.1]
  SELECT * FROM term_data WHERE tid IN (N)

Count: 1  Time=27.87s (27s)  Lock=0.00s (0s)  Rows=7.0 (7), exdigital[exdigital]@[10.200.1.1]
  SELECT * FROM nodewords WHERE type = N AND id = N

Count: 6  Time=19.98s (119s)  Lock=0.08s (0s)  Rows=1.0 (6), exdigital[exdigital]@[10.200.1.1]
  SELECT COUNT(*) FROM (SELECT DISTINCT(node.nid) AS nid,
  node.vid AS node_vid,
  node.title AS node_title,
  node.created AS node_created
  FROM node node
  LEFT JOIN term_node term_node ON node.vid = term_node.vid
  LEFT JOIN term_data term_data ON term_node.tid = term_data.tid
  LEFT JOIN term_node term_node_value_0 ON node.vid = term_node_value_0.vid AND term_node_value_0.tid = N
  LEFT JOIN term_node term_node_value_1 ON node.vid = term_node_value_1.vid AND term_node_value_1.tid = N
  LEFT JOIN term_node term_node_value_2 ON node.vid = term_node_value_2.vid AND term_node_value_2.tid = N
  LEFT JOIN term_node term_node_value_3 ON node.vid = term_node_value_3.vid AND term_node_value_3.tid = N
  LEFT JOIN term_node term_node_value_4 ON node.vid = term_node_value_4.vid AND term_node_value_4.tid = N
  LEFT JOIN term_node term_node_value_5 ON node.vid = term_node_value_5.vid AND term_node_value_5.tid = N
  LEFT JOIN term_node term_node_value_6 ON node.vid = term_node_value_6.vid AND term_node_value_6.tid = N
  LEFT JOIN term_node term_node_value_7 ON node.vid = term_node_value_7.vid AND term_node_value_7.tid = N
  LEFT JOIN term_node term_node_value_8 ON node.vid = term_node_value_8.vid AND term_node_value_8.tid = N
  LEFT JOIN term_node term_node_value_9 ON node.vid = term_node_value_9.vid AND term_node_value_9.tid = N
  LEFT JOIN term_node term_node_value_10 ON node.vid = term_node_value_10.vid AND term_node_value_10.tid = N
  LEFT JOIN term_node term_node_value_11 ON node.vid = term_node_value_11.vid AND term_node_value_11.tid = N
  LEFT JOIN term_node term_node_value_12 ON node.vid = term_node_value_12.vid AND term_node_value_12.tid = N
  LEFT JOIN term_node term_node_value_13 ON node.vid = term_node_value_13.vid AND term_node_value_13.tid = N
  LEFT JOIN term_node term_node_value_14 ON node.vid = term_node_value_14.vid AND term_node_value_14.tid = N
  LEFT JOIN term_node term_node_value_15 ON node.vid = term_node_value_15.vid AND term_node_value_15.tid = N
  LEFT JOIN term_node term_node_value_16 ON node.vid = term_node_value_16.vid AND term_node_value_16.tid = N
  LEFT JOIN term_node term_node_value_17 ON node.vid = term_node_value_17.vid AND term_node_value_17.tid = N
  LEFT JOIN term_node term_node_value_18 ON node.vid = term_node_value_18.vid AND term_node_value_18.tid = N
  LEFT JOIN term_node term_node_value_19 ON node.vid = term_node_value_19.vid AND term_node_value_19.tid = N
  LEFT JOIN term_node term_node_value_20 ON node.vid = term_node_value_20.vid AND term_node_value_20.tid = N
  LEFT JOIN domain_access domain_access ON node.nid = domain_access.nid
  INNER JOIN node_access na ON na.nid = node.nid LEFT JOIN node n ON node.nid = n.nid WHERE (na.grant_view >= N AND ((na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S'))) AND ((n.moderate != N)) AND (  (node.moderate = N) AND (node.status <> N) AND (term_data.vid in ('S', 'S', 'S', 'S', 'S')) AND (node.nid != N) AND (term_node_value_0.tid = N OR term_node_value_1.tid = N OR term_node_value_2.tid = N OR term_node_value_3.tid = N OR term_node_value_4.tid = N OR term_node_value_5.tid = N OR term_node_value_6.tid = N OR term_node_value_7.tid = N OR term_node_value_8.tid = N OR term_node_value_9.tid = N OR term_node_value_10.tid = N OR term_node_value_11.tid = N OR term_node_value_12.tid = N OR term_node_value_13.tid = N OR term_node_value_14.tid = N OR term_node_value_15.tid = N OR term_node_value_16.tid = N OR term_node_value_17.tid = N OR term_node_value_18.tid = N OR term_node_value_19.tid = N OR term_node_value_20.tid = N) AND (domain_access.gid = 'S')
  )GROUP BY nid
  ORDER BY node_created DESC
  LIMIT N) count_alias

Count: 13  Time=19.47s (253s)  Lock=0.03s (0s)  Rows=1.0 (13), exdigital[exdigital]@[10.200.1.1]
  SELECT COUNT(*) FROM (SELECT DISTINCT(node.nid) AS nid,
  node.vid AS node_vid,
  node.title AS node_title,
  node.created AS node_created
  FROM node node
  LEFT JOIN term_node term_node ON node.vid = term_node.vid
  LEFT JOIN term_data term_data ON term_node.tid = term_data.tid
  LEFT JOIN term_node term_node_value_0 ON node.vid = term_node_value_0.vid AND term_node_value_0.tid = N
  LEFT JOIN term_node term_node_value_1 ON node.vid = term_node_value_1.vid AND term_node_value_1.tid = N
  LEFT JOIN term_node term_node_value_2 ON node.vid = term_node_value_2.vid AND term_node_value_2.tid = N
  LEFT JOIN term_node term_node_value_3 ON node.vid = term_node_value_3.vid AND term_node_value_3.tid = N
  LEFT JOIN term_node term_node_value_4 ON node.vid = term_node_value_4.vid AND term_node_value_4.tid = N
  LEFT JOIN term_node term_node_value_5 ON node.vid = term_node_value_5.vid AND term_node_value_5.tid = N
  LEFT JOIN term_node term_node_value_6 ON node.vid = term_node_value_6.vid AND term_node_value_6.tid = N
  LEFT JOIN term_node term_node_value_7 ON node.vid = term_node_value_7.vid AND term_node_value_7.tid = N
  LEFT JOIN term_node term_node_value_8 ON node.vid = term_node_value_8.vid AND term_node_value_8.tid = N
  LEFT JOIN term_node term_node_value_9 ON node.vid = term_node_value_9.vid AND term_node_value_9.tid = N
  LEFT JOIN term_node term_node_value_10 ON node.vid = term_node_value_10.vid AND term_node_value_10.tid = N
  LEFT JOIN term_node term_node_value_11 ON node.vid = term_node_value_11.vid AND term_node_value_11.tid = N
  LEFT JOIN term_node term_node_value_12 ON node.vid = term_node_value_12.vid AND term_node_value_12.tid = N
  LEFT JOIN term_node term_node_value_13 ON node.vid = term_node_value_13.vid AND term_node_value_13.tid = N
  LEFT JOIN term_node term_node_value_14 ON node.vid = term_node_value_14.vid AND term_node_value_14.tid = N
  LEFT JOIN term_node term_node_value_15 ON node.vid = term_node_value_15.vid AND term_node_value_15.tid = N
  LEFT JOIN domain_access domain_access ON node.nid = domain_access.nid
  INNER JOIN node_access na ON na.nid = node.nid LEFT JOIN node n ON node.nid = n.nid WHERE (na.grant_view >= N AND ((na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S'))) AND ((n.moderate != N)) AND (  (node.moderate = N) AND (node.status <> N) AND (term_data.vid in ('S', 'S', 'S', 'S', 'S')) AND (node.nid != N) AND (term_node_value_0.tid = N OR term_node_value_1.tid = N OR term_node_value_2.tid = N OR term_node_value_3.tid = N OR term_node_value_4.tid = N OR term_node_value_5.tid = N OR term_node_value_6.tid = N OR term_node_value_7.tid = N OR term_node_value_8.tid = N OR term_node_value_9.tid = N OR term_node_value_10.tid = N OR term_node_value_11.tid = N OR term_node_value_12.tid = N OR term_node_value_13.tid = N OR term_node_value_14.tid = N OR term_node_value_15.tid = N) AND (domain_access.gid = 'S')
  )GROUP BY nid
  ORDER BY node_created DESC
  LIMIT N) count_alias

Count: 5  Time=17.67s (88s)  Lock=0.00s (0s)  Rows=3.0 (15), exdigital[exdigital]@[10.200.1.1]
  SELECT DISTINCT(node.nid) AS nid,
  node.vid AS node_vid,
  node.title AS node_title,
  node.created AS node_created
  FROM node node
  LEFT JOIN term_node term_node ON node.vid = term_node.vid
  LEFT JOIN term_data term_data ON term_node.tid = term_data.tid
  LEFT JOIN term_node term_node_value_0 ON node.vid = term_node_value_0.vid AND term_node_value_0.tid = N
  LEFT JOIN term_node term_node_value_1 ON node.vid = term_node_value_1.vid AND term_node_value_1.tid = N
  LEFT JOIN term_node term_node_value_2 ON node.vid = term_node_value_2.vid AND term_node_value_2.tid = N
  LEFT JOIN term_node term_node_value_3 ON node.vid = term_node_value_3.vid AND term_node_value_3.tid = N
  LEFT JOIN term_node term_node_value_4 ON node.vid = term_node_value_4.vid AND term_node_value_4.tid = N
  LEFT JOIN term_node term_node_value_5 ON node.vid = term_node_value_5.vid AND term_node_value_5.tid = N
  LEFT JOIN term_node term_node_value_6 ON node.vid = term_node_value_6.vid AND term_node_value_6.tid = N
  LEFT JOIN term_node term_node_value_7 ON node.vid = term_node_value_7.vid AND term_node_value_7.tid = N
  LEFT JOIN term_node term_node_value_8 ON node.vid = term_node_value_8.vid AND term_node_value_8.tid = N
  LEFT JOIN term_node term_node_value_9 ON node.vid = term_node_value_9.vid AND term_node_value_9.tid = N
  LEFT JOIN term_node term_node_value_10 ON node.vid = term_node_value_10.vid AND term_node_value_10.tid = N
  LEFT JOIN term_node term_node_value_11 ON node.vid = term_node_value_11.vid AND term_node_value_11.tid = N
  LEFT JOIN term_node term_node_value_12 ON node.vid = term_node_value_12.vid AND term_node_value_12.tid = N
  LEFT JOIN term_node term_node_value_13 ON node.vid = term_node_value_13.vid AND term_node_value_13.tid = N
  LEFT JOIN term_node term_node_value_14 ON node.vid = term_node_value_14.vid AND term_node_value_14.tid = N
  LEFT JOIN term_node term_node_value_15 ON node.vid = term_node_value_15.vid AND term_node_value_15.tid = N
  LEFT JOIN term_node term_node_value_16 ON node.vid = term_node_value_16.vid AND term_node_value_16.tid = N
  LEFT JOIN term_node term_node_value_17 ON node.vid = term_node_value_17.vid AND term_node_value_17.tid = N
  LEFT JOIN term_node term_node_value_18 ON node.vid = term_node_value_18.vid AND term_node_value_18.tid = N
  LEFT JOIN term_node term_node_value_19 ON node.vid = term_node_value_19.vid AND term_node_value_19.tid = N
  LEFT JOIN term_node term_node_value_20 ON node.vid = term_node_value_20.vid AND term_node_value_20.tid = N
  LEFT JOIN domain_access domain_access ON node.nid = domain_access.nid
  INNER JOIN node_access na ON na.nid = node.nid LEFT JOIN node n ON node.nid = n.nid WHERE (na.grant_view >= N AND ((na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S'))) AND ((n.moderate != N)) AND (  (node.moderate = N) AND (node.status <> N) AND (term_data.vid in ('S', 'S', 'S', 'S', 'S')) AND (node.nid != N) AND (term_node_value_0.tid = N OR term_node_value_1.tid = N OR term_node_value_2.tid = N OR term_node_value_3.tid = N OR term_node_value_4.tid = N OR term_node_value_5.tid = N OR term_node_value_6.tid = N OR term_node_value_7.tid = N OR term_node_value_8.tid = N OR term_node_value_9.tid = N OR term_node_value_10.tid = N OR term_node_value_11.tid = N OR term_node_value_12.tid = N OR term_node_value_13.tid = N OR term_node_value_14.tid = N OR term_node_value_15.tid = N OR term_node_value_16.tid = N OR term_node_value_17.tid = N OR term_node_value_18.tid = N OR term_node_value_19.tid = N OR term_node_value_20.tid = N) AND (domain_access.gid = 'S')
  )GROUP BY nid
  ORDER BY node_created DESC
  LIMIT N, N

Count: 24  Time=13.51s (324s)  Lock=0.00s (0s)  Rows=2.9 (70), exdigital[exdigital]@[10.200.1.1]
  SELECT DISTINCT(node.nid) AS nid,
  node.vid AS node_vid,
  node.title AS node_title,
  node.created AS node_created
  FROM node node
  LEFT JOIN term_node term_node ON node.vid = term_node.vid
  LEFT JOIN term_data term_data ON term_node.tid = term_data.tid
  LEFT JOIN term_node term_node_value_0 ON node.vid = term_node_value_0.vid AND term_node_value_0.tid = N
  LEFT JOIN term_node term_node_value_1 ON node.vid = term_node_value_1.vid AND term_node_value_1.tid = N
  LEFT JOIN term_node term_node_value_2 ON node.vid = term_node_value_2.vid AND term_node_value_2.tid = N
  LEFT JOIN term_node term_node_value_3 ON node.vid = term_node_value_3.vid AND term_node_value_3.tid = N
  LEFT JOIN term_node term_node_value_4 ON node.vid = term_node_value_4.vid AND term_node_value_4.tid = N
  LEFT JOIN term_node term_node_value_5 ON node.vid = term_node_value_5.vid AND term_node_value_5.tid = N
  LEFT JOIN term_node term_node_value_6 ON node.vid = term_node_value_6.vid AND term_node_value_6.tid = N
  LEFT JOIN term_node term_node_value_7 ON node.vid = term_node_value_7.vid AND term_node_value_7.tid = N
  LEFT JOIN term_node term_node_value_8 ON node.vid = term_node_value_8.vid AND term_node_value_8.tid = N
  LEFT JOIN term_node term_node_value_9 ON node.vid = term_node_value_9.vid AND term_node_value_9.tid = N
  LEFT JOIN term_node term_node_value_10 ON node.vid = term_node_value_10.vid AND term_node_value_10.tid = N
  LEFT JOIN term_node term_node_value_11 ON node.vid = term_node_value_11.vid AND term_node_value_11.tid = N
  LEFT JOIN domain_access domain_access ON node.nid = domain_access.nid
  INNER JOIN node_access na ON na.nid = node.nid LEFT JOIN node n ON node.nid = n.nid WHERE (na.grant_view >= N AND ((na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S'))) AND ((n.moderate != N)) AND (  (node.moderate = N) AND (node.status <> N) AND (term_data.vid in ('S', 'S', 'S', 'S', 'S')) AND (node.nid != N) AND (term_node_value_0.tid = N OR term_node_value_1.tid = N OR term_node_value_2.tid = N OR term_node_value_3.tid = N OR term_node_value_4.tid = N OR term_node_value_5.tid = N OR term_node_value_6.tid = N OR term_node_value_7.tid = N OR term_node_value_8.tid = N OR term_node_value_9.tid = N OR term_node_value_10.tid = N OR term_node_value_11.tid = N) AND (domain_access.gid = 'S')
  )GROUP BY nid
  ORDER BY node_created DESC
  LIMIT N, N

Count: 12  Time=11.31s (135s)  Lock=0.71s (8s)  Rows=1.0 (12), exdigital[exdigital]@[10.200.1.1]
  SELECT COUNT(*) FROM (SELECT DISTINCT(node.nid) AS nid,
  node.vid AS node_vid,
  node.title AS node_title,
  node.created AS node_created
  FROM node node
  LEFT JOIN term_node term_node ON node.vid = term_node.vid
  LEFT JOIN term_data term_data ON term_node.tid = term_data.tid
  LEFT JOIN term_node term_node_value_0 ON node.vid = term_node_value_0.vid AND term_node_value_0.tid = N
  LEFT JOIN term_node term_node_value_1 ON node.vid = term_node_value_1.vid AND term_node_value_1.tid = N
  LEFT JOIN term_node term_node_value_2 ON node.vid = term_node_value_2.vid AND term_node_value_2.tid = N
  LEFT JOIN term_node term_node_value_3 ON node.vid = term_node_value_3.vid AND term_node_value_3.tid = N
  LEFT JOIN term_node term_node_value_4 ON node.vid = term_node_value_4.vid AND term_node_value_4.tid = N
  LEFT JOIN term_node term_node_value_5 ON node.vid = term_node_value_5.vid AND term_node_value_5.tid = N
  LEFT JOIN term_node term_node_value_6 ON node.vid = term_node_value_6.vid AND term_node_value_6.tid = N
  LEFT JOIN term_node term_node_value_7 ON node.vid = term_node_value_7.vid AND term_node_value_7.tid = N
  LEFT JOIN term_node term_node_value_8 ON node.vid = term_node_value_8.vid AND term_node_value_8.tid = N
  LEFT JOIN term_node term_node_value_9 ON node.vid = term_node_value_9.vid AND term_node_value_9.tid = N
  LEFT JOIN term_node term_node_value_10 ON node.vid = term_node_value_10.vid AND term_node_value_10.tid = N
  LEFT JOIN term_node term_node_value_11 ON node.vid = term_node_value_11.vid AND term_node_value_11.tid = N
  LEFT JOIN term_node term_node_value_12 ON node.vid = term_node_value_12.vid AND term_node_value_12.tid = N
  LEFT JOIN term_node term_node_value_13 ON node.vid = term_node_value_13.vid AND term_node_value_13.tid = N
  LEFT JOIN domain_access domain_access ON node.nid = domain_access.nid
  INNER JOIN node_access na ON na.nid = node.nid LEFT JOIN node n ON node.nid = n.nid WHERE (na.grant_view >= N AND ((na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S'))) AND ((n.moderate != N)) AND (  (node.moderate = N) AND (node.status <> N) AND (term_data.vid in ('S', 'S', 'S', 'S', 'S')) AND (node.nid != N) AND (term_node_value_0.tid = N OR term_node_value_1.tid = N OR term_node_value_2.tid = N OR term_node_value_3.tid = N OR term_node_value_4.tid = N OR term_node_value_5.tid = N OR term_node_value_6.tid = N OR term_node_value_7.tid = N OR term_node_value_8.tid = N OR term_node_value_9.tid = N OR term_node_value_10.tid = N OR term_node_value_11.tid = N OR term_node_value_12.tid = N OR term_node_value_13.tid = N) AND (domain_access.gid = 'S')
  )GROUP BY nid
  ORDER BY node_created DESC
  LIMIT N) count_alias

Count: 23  Time=10.95s (251s)  Lock=0.40s (9s)  Rows=1.0 (23), exdigital[exdigital]@[10.200.1.1]
  SELECT COUNT(*) FROM (SELECT DISTINCT(node.nid) AS nid,
  node.vid AS node_vid,
  node.title AS node_title,
  node.created AS node_created
  FROM node node
  LEFT JOIN term_node term_node ON node.vid = term_node.vid
  LEFT JOIN term_data term_data ON term_node.tid = term_data.tid
  LEFT JOIN term_node term_node_value_0 ON node.vid = term_node_value_0.vid AND term_node_value_0.tid = N
  LEFT JOIN term_node term_node_value_1 ON node.vid = term_node_value_1.vid AND term_node_value_1.tid = N
  LEFT JOIN term_node term_node_value_2 ON node.vid = term_node_value_2.vid AND term_node_value_2.tid = N
  LEFT JOIN term_node term_node_value_3 ON node.vid = term_node_value_3.vid AND term_node_value_3.tid = N
  LEFT JOIN term_node term_node_value_4 ON node.vid = term_node_value_4.vid AND term_node_value_4.tid = N
  LEFT JOIN term_node term_node_value_5 ON node.vid = term_node_value_5.vid AND term_node_value_5.tid = N
  LEFT JOIN term_node term_node_value_6 ON node.vid = term_node_value_6.vid AND term_node_value_6.tid = N
  LEFT JOIN term_node term_node_value_7 ON node.vid = term_node_value_7.vid AND term_node_value_7.tid = N
  LEFT JOIN term_node term_node_value_8 ON node.vid = term_node_value_8.vid AND term_node_value_8.tid = N
  LEFT JOIN term_node term_node_value_9 ON node.vid = term_node_value_9.vid AND term_node_value_9.tid = N
  LEFT JOIN term_node term_node_value_10 ON node.vid = term_node_value_10.vid AND term_node_value_10.tid = N
  LEFT JOIN term_node term_node_value_11 ON node.vid = term_node_value_11.vid AND term_node_value_11.tid = N
  LEFT JOIN term_node term_node_value_12 ON node.vid = term_node_value_12.vid AND term_node_value_12.tid = N
  LEFT JOIN domain_access domain_access ON node.nid = domain_access.nid
  INNER JOIN node_access na ON na.nid = node.nid LEFT JOIN node n ON node.nid = n.nid WHERE (na.grant_view >= N AND ((na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S') OR (na.gid = N AND na.realm = 'S'))) AND ((n.moderate != N)) AND (  (node.moderate = N) AND (node.status <> N) AND (term_data.vid in ('S', 'S', 'S', 'S', 'S')) AND (node.nid != N) AND (term_node_value_0.tid = N OR term_node_value_1.tid = N OR term_node_value_2.tid = N OR term_node_value_3.tid = N OR term_node_value_4.tid = N OR term_node_value_5.tid = N OR term_node_value_6.tid = N OR term_node_value_7.tid = N OR term_node_value_8.tid = N OR term_node_value_9.tid = N OR term_node_value_10.tid = N OR term_node_value_11.tid = N OR term_node_value_12.tid = N) AND (domain_access.gid = 'S')
  )GROUP BY nid
  ORDER BY node_created DESC
  LIMIT N) count_alias

If you suggest some tool which can backtrack queries to views or modules used, it would be helpful; provided if such a thing exists. If looking at the queries strikes a chord, kindly be helpful to write a comment.

Much Thanks..

+3  A: 

I wouldn't worry too much about the ones that only occur once. Most is just SELECT * FROM big_table which most likely is done through some admin. I'm not sure what would fetch the entire session table, which isn't much usefull.

Anyways, the other queries that occur more than once, is most likely generated by views:

LEFT JOIN term_node term_node_value_0
  ON node.vid = term_node_value_0.vid AND term_node_value_0.tid = N

Doing 20 of these joins will slow down any query, which is also why it looks like views. Views make generic queries without know what you really want to do. So it can make any assumptions which is why it in some cases can make crazy queries like this. The only fix, would be to custom code the view or just the query instead. Views have a hard time making good queries on the terms when things are a bit complex, because terms have a 1 to many relationship (1 term can be associated to make nodes).

So check out the which views you have, you might be able to handle this with cache from within the views UI.

googletorp
A: 

Install and enable the devel module.

Navigate to the module settings page (example.com/admin/settings/devel) and check

  • Collect query info
  • Display query log

This will write every query for a page request to the bottom of the page and tell you the function that wrote the query (the where column). The function name is the moduleName_functionName.

And googletorp is exactly correct.

Ted