CQRLOG and MariaDB 10.4 = Problem

25 posts / 0 new
Last post
DF1NIF
CQRLOG and MariaDB 10.4 = Problem

Hello,

ok, it's not realy a but report but maybe a discussion, if CQRLOG has a problem with MariaDB 10.4 and how to fix it. At the moment, there are two topics with problems which are based on MariaDB 10.4

* https://www.cqrlog.com/node/2269
* https://www.cqrlog.com/node/2272

I use CQRLOG on ArchLinux (2.3.0) but not with a local stored database. I have a server in my home network which runs with MariaDB 10.4. So I have not the problem with the "mariadb-upgrade" to bring the database to the new format. But I can see different other problems.

So I am not able to import an adif file to the database (maybe a general problem with my adif files but I tried three different files and got this error and I imported them a year ago...) - CQRLOG reports complete and shows me only a view records of the adif file...:

1
2
3
4
5
insert into cqrlog_main (qsodate,time_on,time_off,callsign,freq,mode,rst_s,rst_r,name,qth,qsl_s,qsl_r,qsl_via,iota,pwr,itu,waz,loc,my_loc,remarks,county,adif,idcall,award,band,state,cont,profile,lotw_qslsdate,lotw_qsls,lotw_qslrdate,lotw_qslr,qsls_date,qslr_date,eqsl_qslsdate,eqsl_qsl_sent,eqsl_qslrdate,eqsl_qsl_rcvd, prop_mode, satellite, rxfreq) values(:qsodate,:time_on,:time_off,:callsign,:freq,:mode,:rst_s,:rst_r,:name,:qth,:qsl_s,:qsl_r,:qsl_via,:iota,:pwr,:itu,:waz,:loc,:my_loc,:remarks,:county,:adif,:idcall,:award,:band,:state,:cont,:profile,:lotw_qslsdate,:lotw_qsls,:lotw_qslrdate,:lotw_qslr,:qsls_date,:qslr_date,:eqsl_qslsdate,:eqsl_qsl_sent,:eqsl_qslrdate,:eqsl_qsl_rcvd, :prop_mode, :satellite, :rxfreq)

Import failed! TMySQL56Connection : Error executing query: Incorrect string value: '\xE1bor' for column `cqrlog002`.`cqrlog_main`.`name` at row 1
It takes about 0 minutes and 0 seconds 69 milliseconds

I can also reproduce this problem with old CQRLOG backups.

(without debuglog and without I can reproduce that)
Yesterday I edit a record (change the name), the log record was lost. That should never happen. Today every change of a record works fine.

Rebuilding the DXCC statistic with only 149 QSOs created an crash of the program.
select band,count(distinct adif) from cqrlog_main where adif <> 0 and qsl_r = 'Q' and (adif not in (2,8,19,23,25,26,28,30,39,42,44,55,57,58,59,67,68,81,85,93,101,102,113,115,119,127,128,134,139,151,154,155,164,178,183,184,186,193,193,194,196,198,200,208,210,218,220,226,228,229,231,243,244,255,258,261,264,267,268,271,307,488,493)) and ((mode<>'CW') and (mode<>'CWR') and (mode<>'SSB') and (mode<>'FM')and (mode<>'AM')) group by band

Section:frmDXCCStat
Saving window size a position (frmDXCCStat) (height|width|top|left):529|711|27|20
SELECT COUNT(*) FROM cqrlog_main

select id_cqrlog_main,qsodate,callsign,adif,qso_dxcc from cqrlog_main

select * from view_cqrlog_main_by_qsodate LIMIT 500

TApplication.HandleException Access violation
Stack trace:
$00000000009216A8
$00000000008B2EF2
$00000000008B31D5
$00000000008B250D
$00000000008AE0FA
$000000000084B210
$00000000008ADF43
$00000000008499B5
$0000000000670B56
$00000000006577B8
$000000000065BCF9
$0000000000670A8C
$0000000000431FAF
$0000000000658D8D
$000000000084CCEA
$00000000008A90DD
$000000000082DB32

That are only a view things a tried. But I cannot trust a program which does not store the data clearly. So what can we do? Back to MariaDB 10.2/.3?

73 Tobias, DF1NIF

DL7OAP
Hi, i am using cqrlog with

Hi, i am using cqrlog with ubuntu 19.10 and mariadb 10.4.6 in a docker container. this works fine for some month now.

maybe there is a problem with mariaDB and archlinux (and not cqrlog in special).

there is still a big hint on https://www.archlinux.de/ for problems with mariadb and archlinux and how to solve them with some manual workarounds.

bye Andreas, DL7OAP

DF1NIF
Good morning Andreas,

Good morning Andreas,

that's not a problem. That's only the information, that the configuration path has changed. But that happened to my server and I hava done this weeks ago. So this is not a problem which should take care here... My MariaDB Server is working well on my dedicated server (there are still other products which running on it).

But there are other things which are much more important in 10.4 then in older releases. The SQL-syntax is much more strict. Than can generate problems - I had a view problems in own programs based on this.

But it is good to know, that it will work without problems on your system. Then I will check it again. Maybe I try to compile it myself...

Regards, Tobias, DF1NIF

DF1NIF
So, some news to my problems.

So, some news to my problems... Maybe that will help someone...

I created complete new Logs. Remember: I use a extra MariaDB Server and not the local db. If I try to import the ADIF files, I got a error. The error you can find above is a encoding error of the file. You can fix that, if you save the ADIV file explicit in UTF8 encoding. But I got another error after that.

[code]Import failed! TMySQL56Connection : Error executing query: Incorrect string value: '\xC3' for column `cqrlog002`.`cqrlog_main`.`name` at row 1[/code]

So I installed HRD in a VM an exported my old logs again. I used ADIF 2 format without the HRD extras. Here you have again to convert the file tu UTF8 encoding! The problem with the characters still exists.

So I checked the file I found the problem. The stopped on every log entry which has special characters. For example a name like "René". So I changed the names, documented this and changed that back after the import.

In my case it is not a big problem because I had to do this only for a view records. But if you import a lot of entries this will not be a good solution.

ArchLinux, MariaDB 10.4.7 on a ded. ArchLinux Server

73 de Tobias, DF1NIF

F6DYA
Hello

Hello

I have a similar problem on UBUNTU 18. I have installed CQRLOG from the UBUNTU repository and it works fine on many aspects. I did not change anything in the database configuration (definitely unskilled to do that)
I have uploaded an adif coming from LOGGER32 (windows). The imported QSO number is correct (no error during import), but I can only see a small bunch of QSO's on the list.
I opened a nex log and import the same adif. Same problem, but not the same bunch of qso's displayed.
I did that several times, creating a nex log for each import, and the process is definitely not deterministic! I even had no qso displayed.
Any suggestion to fix the problem ?

73 Alain

DF1NIF
Hallo Alain,

Hallo Alain,

after the import, did you click the refresh button? The field with the QSO numbers is not always updated correctly. I search problems like this to write a bug report later.

Or do you have the problem with no displayed QSOs? Maybe that's also a problem with the MariaDB/MySQL-Server. Have a look in the debug messages of CQRLog. If that's the problem, there are some hints in the ArchLinux thread.

I fixed that problem by using my MariaDB-Server on my central server in my home network. There I can do a mariadb-upgrade without any problems.

73 Tobias

F6DYA
Hello Tobias

Hello Tobias

Yes, I have refreshed the screen after import: that does not change anything.
I have also checked the number of QSO cards in the ADIF and in the import: it fits, and we can imagine that the import has been correct, all the cards have been imported and understood. (about 9000 QSO's)
Software use to be deterministic, but this one is not 8) as the import of the same adif file does not always show the same QSO's, even if the QSO count is always correct. I have something like 2000 QSO's displayed down to none.
I tried unsuccessfully to find the the debug messages of CQRLog . Can you give me the name of this file to check if we could find a track.

73 Alain

DF1NIF
Just start the program

Just start the program directly from the command line with the debug parameter:

cqrlog debug=1

then you see the messages in the command line. If you want a logfile, you can pipe that to file

cqrlog debug=1 > logfile.txt

73 Tobias

F6DYA
Hello Tobias

Hello Tobias

Interesting!
First, I have that message on the console, not written on the log:

alain@grobuntu:~/Documents$ cqrlog debug=1 > F6DYA_log.txt
Gtk-Message: 21:01:44.619: Failed to load module "canberra-gtk-module"
I don't know what it means and what to do, but obviously not expected.

On the log file, line 33 seems also suspect.

It makes sense for you ?

73 Alain

File: 

oh1kh
gtk-module

Hi!

Yo need to have gtk2 installed. Some Linuxes may not have that as default, or it may be gtk3 that is not suitable.

--
Saku
OH1KH

oh1kh
gtk-module

--
Saku
OH1KH

F6DYA
Well Saku

Well Saku

it was a good track. The problem was coming from libcanberra-gtk-module. A fresh installation and things are better.
The only error remaining in the log is : TMySQL57Connection : Server connect failed.
I have read the comments about mariaDB but it is definitely unclear for me. Any practical suggestion ?
Best 73

oh1kh
gtk-module

Hi !
When you use "save log data to local machine" it is normal to see few of those errors at the begin of debug message file.
Depending of the speed you PC can start mysql_safe to run.
How ever not more than 1-5. If I remember right cqrlog tries 10 times before it quits.

But you should start by reading this message:
https://www.cqrlog.com/comment/7452#comment-7452

And if that does not help there are more messages when you put Google these words as search criteria:

TMySQL57Connection : Server connect failed cqrlog.com

It may well be the apparmor problem that still makes harm. Unfortunately Ubuntu (and relative) versions seems to cause the biggest problems.

--
Saku
OH1KH

F6DYA
Thank you for your feedback(s) Saku and Andreas

So, I have uninstalled everything and followed the procedure described by Andreas for a fresh installation.
No problem with that. But at the end, the problem is still there as you can see line 31 of the log.
The effect is the same: QSO from today down to 2018 (but the log has started in 1976!).
Of course, I have removed the log and import again the adif, with the same results. I did it several times, and it seems now deterministic.
Please note that I am not running UBUNTU 19 but UBUNTU 18 and the installation is not in a virtual box: this machine runs only UBUNTU.
Sorry for the bad news... It's a bit more tricky 8(

File: 

F6DYA
In addition...

I don't know if that can help, but I have just discovered a funny thing: I can only see a bunch of the log and many QSO are not displayed on the QSO page.
However, if I search for an un-displayed QSO, it is then found and displayed.
In the same way, if I type (in the input window) a callsign of a station formerly contacted but not visible, then it appears in the upper lines as formerly contacted.
So in my understanding, the base is correct, but not correctly displayed.

73 de Alain

oh1kh
In addition...

Hi!
Sounds bad.

What if you open qso list/filter/sql console then write to upper part window:
select * from cqrlog_main
After that press arrow icon (leftmost at the top icon line). I tested that with +8000 qsos and it took few seconds.
Can you see all your qsos? Click 1st qso, then you can move down qso by qso with "arrow down"

If that shows all qsos then next question is: do you use "QTH profiles" ?

If you do not use QTH profiles the preferences/QTHprofiles should show unchecked value. And when in QSO list/filter/sql console a line:
select * from profiles
Should give an empty result.

If there is something unclear make a new log. Check that it does not have QTH profiles set and then import all your qsos to that log.

Or with present log you can open QSO list/File/Group edit. To upper box select "QTH profile" and to other one of your profiles (if many) or leave empty and press "Apply".

I assume you have backups from your log. With Group edit you can easily destroy your whole log with one Apply-button click.

--
Saku
OH1KH

F6DYA
Well, the results:

Well, the results:
- I can see all my QSO's in QSL console. No problem.
- I never tried to use QTH profiles. In the preference menu, it is not checked.
- When I select * from profiles in QSL console, I have an empty list
- When I open QSO list/File/Group edit and force an empty profile, I have no effect.

At the end, no change. I have also tried to sort the QSO's by date ascending or descending: I can see both ends of the list and it seems that the display is limited to a certain number of lines. I can't easily count them but roughly it seems to be the same number., let say about 2000.

73 de Alain.

oh1kh
Well, the results:

Hi!
Strange!
Can you make a zip using console:

zip -r /tmp/test.zip ~/.config/cqrlog

If it is less than 2M you may send it via email as attachment. Otherwise can you upload it somewhere (Google drive or similar) and send a link.
mycallsign
at
sral dot fi

Then I could try it here locally.

--
Saku
OH1KH

F6DYA
Shared on google drive. Thank you.

73 de Alain

F6DYA
Hi Saku

Could you find a track ?
On my side, I tried several "wipe all and reinstall", with always the same effect. Well, it is a blind approach as I am not skilled in software development.
Best 73.

on5hb
on5hb's picture
Running Ubuntu...

Hi Alain,

The problem is that MySQL has been replaced by MariaDB for some time now.
Regardless of the Ubuntu or Debian (or whatever Linux), you need to replace MySQL with MariaDB.

This is in fact quite simple:

>> sudo apt update <<
>> sudo apt install mariadb-server <<

Mostly that is enough, do NOT remove MySQL first, the command will replace all libraries and everything to MariaDB.

If that isn't working, then try this:

>> sudo apt install cqrlog libmariadb3 libmariadb-dev mariadb-common mariadb-server <<

Greetings, Bas. ON5HB

oh1kh
Hi Saku

HI!

I answered to you via email and suggested some things.
Have'nt you received it?

I have now resent it to your gmail.

--
Saku
OH1KH

so5wd
Ubuntu apparmor

In Ubuntu apparmor is blocking mysql. Removing apparmor solving this issue (I do not know consequences).

73, Mirek so5wd

so5wd
Ubuntu apparmor

In my opinion, on fresh install Kubuntu 19.10 cqrlog will do not work at all.
Apparmor is default application.
From Kubuntu 19.10 system log:
"13.11.19. 18.54 kernel audit: type=1400 audit(1573667641.449:34): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/home/mk/.config/cqrlog/database/mysql.cnf" pid=1603 comm="mysqld" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000"
In a very close future apparmor will be in every Linux distro.
There is some challange for cqrlog team to open mysql port in apparmor by cqrlog.

73, Mirek so5wd

on5hb
on5hb's picture
You look too far....

No it's not....it simply because MySQL is replaced by MariaDB....however not all distro's install MariaDB by default or replace it by an upgrade.

Try this:

>> sudo apt update <<
>> sudo apt install mariadb-server <<

Mostly that is enough, do NOT remove MySQL first, the command will replace all libraries and everything to MariaDB.

If that isn't working, then try this:

>> sudo apt install cqrlog libmariadb3 libmariadb-dev mariadb-common mariadb-server <<

Greetings, Bas. ON5HB