Invisible QSOs - Old Bug - Fixed

13 posts / 0 new
Last post
KW2P
Invisible QSOs - Old Bug - Fixed

After all the database and version problems I had in December, everything seems to be working fine. However, tonight I scrolled to the end of the QSO list and there are no QSOs shown prior to 2020-12-15. This was alarming!

I made sure there were no filters. I tried re-sorting the list. Same problem.

Then, in desperation I checked my ADIF backup files and all QSOs are there. Whew!

Then, in CQRlog I set up a filter for QSOs from 2016-01-01 to 2019-12-31 and the missing QSOs appeared in the list. I cleared the filter and they disappeared again.

The combination that I found that has been working is CQRlog 2.4.0 with a fresh install of Linux Mint and MySQL.

Any suggestions? I'd really rather avoid repeating all the experimentation with operating system installs, different databases, and app versions, if possible. Lol.

Thanks in advance!

oh1kh
Invisible QSOs

Hi!

You say all qsos are in adif backup.
What if you create a new log and import backup to that. Does they then show up?

Before doing that open cqrlog up to "database connection" window. Select your current log and the press button "utils" and select configuration/export. Give suitable name for your log configurations.

Then make new log from button "new log".
Open your new log and import your backup having all your qsos. Check do you now see all qsos? Close cqrlog.
Open again to "database connection" , select your just created new log (if not already selected), but do not open it yet.
Instead press "Utils" and select configuration/import and find your just created configuration export file and load it.
Then open the log. Do you still/now see all qsos. This is to test does your log settings effect to what you see.

If you see them start using new log. If not we have to try something else.

--
Saku
OH1KH

KW2P
Haha. I see we were both

Haha. I see we were both writing at the same time. Please see my other post.

Thank you, Saku!

KW2P
Aha! I have my QSOs back.

Aha! I have my QSOs back.

First, something weird: I spent the past hour editing my 50 most recent QSOs, adding information, etc. I scrolled down to the bottom of the QSO list and two of the missing qsos were now visible. (?)

Then, I decided to use the filter to make the missing QSOs visible (as described in my first post) and take a look at them to see if something was odd about them. On the filter dialog, I noticed a "Clear filter" button. So I hit that and engaged the filter, and everything reappeared as it should. Maybe some kind of garbage was still in the filter settings but not visible in the filter dialog.

So the Filter / Cancel Filter function on the QSO list didn't actually cancel the filter. Some kind of filter was still engaged. Only when I went to create a new filter (which was empty of course) and hit the Clear Filter button, did the filtering actually go away.

So the problem is fixed and hopefully this provides a clue.

73 de KW2P

KW2P
More interesting behavior:

More interesting behavior:

With the filter cleared and all QSOs visible, if I shut down CQRlog, then restart it, the missing QSOs disappear again. If I then choose Create Filter to open the filter dialog, Clear the filter and apply it, the QSOs reappear.

Then, while all the QSOs are showing on the QSO list, if I hit Cancel Filter, the QSOs before Dec 15 disappear again.

Is there maybe a default filter settings file that might be corrupted? I see that I can save filters as .fil files. I've never done that but it implies to me that there may be a default file.

/phil

oh1kh
Invisible QSOs

HI Phil!

Clear filter clears the filter window. It has no other effect. It just makes a clean start for new filter design.
When cancel filter (Shift+F12) is selected it should disable filtering completely.

My guess is QTH profiles.
Do you use them?
I guess not.

You could try:
First take backup. Full adif or/and copy of the ~/.config/cqrlog
Make your tricks that shows all qsos.
Then open QSO list and select File/group edit.
It should say effect is for whole log.
Select field "QTH profile"
If you do not use qth profiles then only value that can be selected is "empty"
Press button "apply"

Then look can you see all qsos. Close and open cqrlog and again check can you still see all qsos.

--
Saku
OH1KH

KW2P
Hello Saku,

Hello Saku,

I was hoping, but that didn't do it. I do use QTH profiles. I think it's a great feature. Fortunately, all the QSOs in the problem log should have the same QTH profile, for now. So changing them all didn't cause problems. When the weather improves and I start doing activations then there will be other profiles in that log.

The procedure went smoothly but didn't fix the problem. Upon shutdown and restart, or if I hit Filter/Cancel filter, the QSO's before Dec 10 disappear.

I wonder if some other field is messed up. That date, around December 10-15 is when I finally got the version and database problems fixed. So, there may be some kind of damage to the records before that date. Now that I have some time, I'm going to closely examine QSO records before and after that date and see if I can see something broken.

Thank you again. Maybe I'll find something.

73 de KW2P

KW2P
Hello Saku,

Hello Saku,

I saw some bits of erroneous data in the hidden qsos, like a signal report of -79. Haha. But that's still valid data and shouldn't affect anything. Correcting it did not make a difference.

Since I have an ADIF export that I've visually inspected to make sure the invisible QSOs are there, I'm thinking of taking your advice and either building a new database or deleting the invisible QSOs and importing from the ADIF.

I believe the only thing I'll lose are the "comments to callsigns" since I don't see that in the export.

I think that's the only reason to delete the bad Q's and only import only what's missing -- to keep all my settings and preferences, and comments to callsigns.

Can you think of anything else I'll lose or be should beware of?

Thank you,
Phil

KW2P
New Information

Hello Saku,

I have some new information. First, you were quite right about the Clear button on the filter dialog. To make the missing QSOs appear, all I have to do is hit Filter/Create and OK. No need to hit Clear.

Since there's no harm in experimenting, I used the Utils function to export the preferences from the old database, created a new database, and imported the settings. Then I manually added my one QTH profile, exactly as it is in the old database. Then I imported the ADIF. All 659 QSOs were imported. And, the same thing happened. Scroll to the end of the QSO list and nothing before Dec 10, 2020 appears. Hit Filter/Create, OK, and they appear.

Thinking maybe there's something broken in the preferences, I created another experimental database, but didn't touch a single thing in the preferences. I simply imported the ADIF straight off. All 659 QSOs were imported, and the behavior is the same. Nothing earlier than some time on Dec 10, 2020 is visible until I engage an empty filter.

Something in the data itself that gets exported to the ADIF is triggering this behavior, but I can't see anything wrong. The ADIF import code is obviously happy or it would complain about a broken ADIF file.

If it would help, I can send you the ADIF file. I can use a text editor to add a line of dollar signs or something that marks the point beyond which things don't display.

This problem isn't holding up my operations here. I continue to add QSOs, upload to LotW, etc. It's working fine except for what appears to be a filtering or query problem, but is likely something in my data itself that's upsetting things. Whatever is wrong, it gets exported to the ADIF.

Thank you,
Phil

PS. I wonder if some clever ham has embedded SQL code his QSL comments on QRZ.com, just to mess with logging programs. Hahaha.

oh1kh
Invisible QSOs

Quite interesting.

I think your comments to callsign should go to adif tag <NOTES Notes are kept in different database table but they are extracted by callsign at adif export phase.

Your profiles should also follow in export/imports with tag <APP_CQRLOG_PROFILE

QSO list loads qsos to view in blocks of 500 qsos. Could it be that those ones missing belong to other block. Have you tried PgUP PgDN to scroll qsos when you are near of the list end?
In earlier versions of cqrlog there has been problems with this block loading. But they should not exist any more in 2.5.2 version.

You can send your export via email. Look for qrz or hamqth for address.

I think it is quite hard to get comment sql executed at import :-)

--
Saku
OH1KH

KW2P
Yes, quite interesting, and I

Yes, quite interesting, and I think you've identified the problem.

I've been busy recently and not able to play with it much. But I imported another block of 65 new QSOs and noticed that the ending date for the QSO list changed from mid-December to December 29th. Aha! That implies the list is only showing the X most recent QSOs where X is around 500. Then I remembered your comment and tried the PgUp/Dn trick and that works.

I'm running 2.4.0, which is the most recent version I could find to run on the latest Linux Mint. This appears to be an old bug. Why the empty filter trick works is a bit of a mystery, but that doesn't matter.

The important thing is it sounds like the database is fine and it's just a display problem. And I need to figure out how to run a newer version on this machine. Hmm.

oh1kh
Invisible QSOs

Ah, 2.4.0!
No wonder any more...

You could try to install new repository of Petr from download section, but I do not know does it work with Mint.
Or if you have x86-64 machine I have a binary that is compiled from latest official source with Mint20.

My alpha test binary page is https://github.com/OH1KH/cqrlog/tree/loc_testing/compiled
There are binary files that can be used to update cqrlog without installing from package.
Some of files have additions that do not exist in official version, but there is also plain official version for x86_64 machines.

Please read first the README.md that is on the page.

Quick quide for update to official:

Click "newupdate.zip", a new page opens.
At right side is button "Download" press that.
Your browser downloads newupdate.zip
Find downloaded newupdate.zip and extract it. You get newupdate.sh script file.
Start newupdate.sh in command console

./newupdate.sh

It quides you through update and downloads needed files.
Selection 1) is official version for x86_64 compiled with Linux Mint 20

--
Saku
OH1KH

KW2P
Sorry, yes, I mentioned the

Sorry, yes, I mentioned the version in the first post. I know it's not the latest but it's far more along than the 2.0.0 I started out with. Hah.

I'll see what I can do with the above on Sunday.

Thank you, Saku