FOUR-17365 39399 - Fidelity Bank Performance, queries constantly running by gproly · Pull Request #7327 · ProcessMaker/processmaker
Issue & Reproduction Steps
On the our analysis we found 3 queries that are constantly running in the database
-
For the first 2, we searched the process and also other assets int he database and there is no place in the scripts, screens and saved searches where the criteria for the queries was explicitly set from the UI, so I assume it is internal, however, on a quick review I can tell that tue queries are poorly written:
SELECT * FROM collection_x WHERE ( LOWER ( DATA ) LIKE ? OR id LIKE ? OR created_at LIKE ? OR updated_at LIKE ? ) ORDER BY id ASC LIMIT ? OFFSET ?
SELECT COUNT ( * ) AS AGGREGATE FROM collection_x WHERE ( LOWER ( DATA ) LIKE ? OR id LIKE ? OR created_at LIKE ? OR updated_at LIKE ? ) -
For this 3rd query, we notices this runs for every customer, but it case of Fidelity it shows high peaks in coudwatch
select table_name as name, (data_length + index_length) as size, table_comment as comment, engine as engine, table_collation as collation from information_schema.tables where table_schema = 'test' and table_type = 'BASE TABLE' order by table_name
Solution
This line of code Schema::hasTable('scheduled_tasks') makes a query to
select table_name as name, (data_length + index_length) as size, table_comment as comment, engine as engine, table_collation as collation from information_schema.tables where table_schema = 'test' and table_type = 'BASE TABLE' order by table_name
, so it should be avoided whenever possible.
With this solution, we are going to reduce the use of the following query (please verify this in the monitor)
These two queries are used to perform searches based on a filter string.
SELECT * FROM collection_x WHERE ( LOWER ( DATA ) LIKE ? OR id LIKE ? OR created_at LIKE ? OR updated_at LIKE ? ) ORDER BY id ASC LIMIT ? OFFSET ?
SELECT COUNT ( * ) AS AGGREGATE FROM collection_x WHERE ( LOWER ( DATA ) LIKE ? OR id LIKE ? OR created_at LIKE ? OR updated_at LIKE ? )
The DATA field is a JSON string, so LIKE is used to filter records that match the filter string and then retrieve the total results to display them in a paginated format for the client. This table is frequently used by the client, which is why high spikes are observed.
The best solution to optimize the queries on this table would be to avoid using JSON to store the information, but this would involve significant changes (For now, we are not doing anything).
How to Test
Check in the query execution monitor to ensure that this execution has been reduced.