CQRLOG hanging during many commands due to MySQL & Maria DB differences

11 posts / 0 new
Last post
sm0kbd
CQRLOG hanging during many commands due to MySQL & Maria DB differences

Hi,

I have a very strange problem here, were CQRLOG hangs during many operations like upload/download from LotW / EQSL and also during updates of log-entries in contest mode. (this is not yet confirmed to be the same issue, though, but shows the same locking) The last working import / export I did was on 2025-07-08.

I am using / have tried the following:

Mint 22.2, Lazarus 3.0 (from official Mint distribution repositories), cqrlog 138 (compiled by me) / 138 (compiled by OH1KH) / 139 (Compiled by me)

2.5.2 from the official distribution repositories opens with an access violation. This could of course be the same problem, as this also locks up CQRLOG.

Trying to narrow down were this happens it is when this is called:

Application.ProcessMessages;

As this is a sort of cooperative multitasking I first thought that something locked a thread, but then more people should have seen this... Now it more points towards that there is a bug in a library somewhere, but WHICH???

Which is the recommended Lazarus?

BR

/Thomas, SM0KBD

sm0kbd
After upgrading to Lazarus 3

After upgrading to Lazarus 3.8 I get this error message when running with the debugger. Unfortunately the address is deep down in a library somewhere.

/Thomas, SM0KBD

File: 

---
Thomas
SM0KBD

oh1kh
After upgrading to Lazarus 3

Hej Thomas!

I have used Lazarus 3.8 for some time now and have not accessed that kind of problem. I'm currently running CqrlogAlpha139 from devel branch.
Only problem so far I have seen is that 3.8 modifies parameters in .lfm resulting that older version of Lazarus compiles ok, but running the result causes error.
See: https://www.cqrlog.com/comment/13230#comment-13230

See also this PR https://github.com/ok2cqr/cqrlog/pull/578

--
Saku
OH1KH

sm0kbd
Thanks for your comments!

Thanks for your comments!

I have now also confirmed that I have the same problem on my both computers here (desktop, where the above info is from and on my laptop). The way it is setup up here is that there is a common Mysql server that both connects to, to the same database.

The laptop is running Ubuntu 24.4

A first wireshark dump doesn't reveal any strange with that communication.

/Thomas

---
Thomas
SM0KBD

oh1kh
After upgrading to Lazarus 3

Hej!

Interesting.
I have also a common SQL server in use here, but it is not Mysql but MariaDB. So I am not using the most common way to start a new SQL server to ~/.config/cqrlog/database

You say "Mysql" and that makes me think was there any MariaDB specific commands used somewhere in source. Mostly Mysql and MariaDB are compatible, and on the other hand if you have spotted the problem to change of Lazarus version it cannot be the root cause.

Have you tested to compile GTK2, QT5 and QT6 versions? Do they all have same problem?

Application.ProcessMessages causes some graphics and I/O updates, at least.
I have found out that in some loops that have retry watchdog counter putting a Application.ProcessMessages in loop may be needed, sometimes it will cause errors. Weird!(or too little understanding what the command actually does)

Running debug from inside Lazarus can also cause weird results sometimes. Even Lazarus hangups.
How ever when running same program from console and without debug code compiled inside all works ok.
So be aware if you are trying to find the bug running code in debug mode from inside of Lazarus.

Best way is to add (or release some debug prints without IF) some writeln(s) and run from console without debug code. (Project/Project Options/debugging)
Yes, it is the hard way....and you must first pinpoint the problem to certain unit.

After all it must be something in you systems, otherwise there would have been more reports about it.

--
Saku
OH1KH

sm0kbd
Finally!!!

Finally!!!

This line is the culprit:

CREATE TRIGGER `cqrlog_main_ai` AFTER INSERT ON `cqrlog_main` FOR EACH ROW
insert into log_changes(id_cqrlog_main,cmd,qsodate,time_on,callsign,mode,freq,band) values
(NEW.id_cqrlog_main,'INSERT',NEW.qsodate,NEW.time_on,NEW.callsign,NEW.mode,NEW.freq,NEW.band);

The line is generated in dLogUpload.pas in procedure TdmLogUpload.EnableOnlineLogSupport(RemoveOldChanges : Boolean = True);

t.SQL.Add(dmData.scOnlineLogTriggers.Script.Strings[i]);
if dmData.DebugLevel>=1 then Writeln(t.SQL.Text);
writeln('Execute: ', i);
t.ExecSQL;
writeln('Execute done: ', i);
t.SQL.Text := ''

the t.ExecSQL does never return... (The writeln's are mine debug ones...)

BTW: Seems to be mysql on my desktop (Mint 22.2) and maria on the server (Debian).

QED!

---
Thomas
SM0KBD

oh1kh
Finally!!

Hej!

For me, if I compare your linked Mysql and Mariadb create trigger with the Cqrlog SQL clause, I do not see differences that should effect the result.
Try to remove "if dmData.DebugLevel >=1 then" (easiest way is to add newline between "then" and "Writeln" and after that add "//" before IF. When not needed any more just remove "//" )
When you get the SQL clause printed with your added writelns you see does it happen with all clauses, or does it happen only "sometimes" and if so what kind of SQL clause it is.
When it does not return, does it have something wrong?
It seems that resulting SQL clause is a collection of "something" until the ";" is reached.

If clauses seem to be ok, but there are no returns from exec, then if you look at lines:
t := TSQLQuery.Create(nil);
tr := TSQLTransaction.Create(nil);
try
t.Transaction := tr;
tr.DataBase := dmData.MainCon;
t.DataBase := dmData.MainCon;
if RemoveOldChanges then

You see that query and transaction created are using the MainCon connection.

I have added more connections into dData

MainCon : TSQLConnection;
BandMapCon : TSQLConnection;
RbnMonCon : TSQLConnection;
LogUploadCon : TSQLConnection;
dbDXC : TSQLConnection;
WGMWCon : TSQLConnection;
UpStatCon : TSQLConnection;

Because I have found out that if one connection is stressed too much it may have random "hiccups". Separating different functions to own connections have helped.

You cold try to change dmData.MainCon to dmData.UpStatCon in procedure and see if that makes any difference.

The only use for UpStatCon is when creating a new US State table for wsjt-x use. (see Ctrl+Shift+F "UpStatCon" and Ctrl+Shift+F "trUpStat" with Lazarus ide)
So the connection does not have lot of traffic all the time.

--
Saku
OH1KH

sm0kbd
If I read the Mariadb

If I read the Mariadb documentation correctly the CREATE TRIGGER statement looks completely different for a Maria DB.

https://dev.mysql.com/doc/refman/8.4/en/create-trigger.html

https://mariadb.com/docs/server/server-usage/triggers-events/triggers/tr...

The definition of the trigger statement is in the dData.lfm

I see that there is no version for in SQLdb for Maria DB, at least what I can find during a quick search. Anyhow the problem is that the statement used is fixed and not created with the SQL API. In worst case all the TRIGGER statements need to be rewritten in a more simple form.

https://wiki.freepascal.org/SQLdb_tab

---
Thomas
SM0KBD

sm0kbd
The culprit statement is in a

The culprit statement is in a try / except clause but the except is not triggered...

---
Thomas
SM0KBD

sm0kbd
The way I read the trigger

The way I read the trigger documentation the statements look like this:

MySQL:
CREATE TRIGGER ins_sum BEFORE INSERT ON account
FOR EACH ROW SET @sum = @sum + NEW.amount;

Maria DB:
CREATE TRIGGER increment_animal
AFTER INSERT ON animals
FOR EACH ROW
UPDATE animal_count SET animal_count.animals = animal_count.animals+1;

Note that the "NEW" keyword is not used in Maria DB. The MySQL version is what is found in dData.lfm

Anyhow I will also check your proposals!

/Thomas

---
Thomas
SM0KBD

oh1kh
The way I read the trigger

If you look at:
https://www.mariadbtutorial.com/mariadb-triggers/mariadb-create-trigger

QUOTE:
Inside the trigger body, you can access values of columns that are affected by an insert, update or delete statement. The old and new modifiers allow you to access values of the columns before and after the triggering event.

----------

What I understand the keywords NEW and OLD are used in MariaDB in these cases.
Am I wrong?

If script.strings would be wrong in dData.lfm why would you be the first having problems with this?
I have done LoTW/eQSL up/DNloads without any database problems ever.

There must be some other reason for this. But what?

Have you tried to "save data to local machine" and create a new test log there? Does it work?
What if the problem lays on the common mysql server used. Have you checked the server's error logs?

--
Saku
OH1KH