Perl Tk app sometimes crashes after exceeding 4GB ram usage

Question

I have a Perl Tk GUI application that sometimes crashes after it exceeds 4GB of RAM usage. I can exceed 4GB of RAM usage in some cases using Perl Tk, and I have no issues exceeding 4GB when running tests in a console application.

• Operating system: Microsoft Windows [Version 10.0.19044.2006]
• Perl version: v5.30.3
• Tk version: 804.036 (latest available on CPAN)

Perl spits out this error almost every time it crashes, but sometimes it crashes without an error:

Free to wrong pool 678ea0 not e228dd0 at .\common\GUI_TESTS\test_memory_hog_gui.pl line 41.

When searching for this error, everything I could find was multi-threading related, and our application does not use multi-threading.

I thought it may be because we have something configured as 32-bit instead of 64-bit, so I followed the instructions in this question and found that everything is configured as 64-bit.

perl -V:ivsize          # ivsize='8';
perl -V:ptrsize         # ptrsize='8';
perl -V:archname        # archname='MSWin32-x64-multi-thread';

Below is an example GUI application that crashes after the memory exceeds 4GB. I have boiled this down from our application and the crashing behavior is the same. The data structure that we use is obviously much larger, so I am cloning a simplified version of ours many times to pass the 4GB threshold.

use strict;
use warnings;

use Tk;
use Tk::LabFrame;

use Clone;

my $MAIN_WINDOW = MainWindow->new;

$MAIN_WINDOW->minsize(400, 400);

my @dataStructureClones = ();
my $textBox;
my $button_frame = $MAIN_WINDOW->LabFrame(-label => "Test", -relief => 'groove', -borderwidth => 2)->pack();

$button_frame->Button(
    -text    => 'Run Crashing Operation',
    -command => sub {

        my $dataStructureThatCrashes = {
            NETLIST_INFO => {
                EXTRA_PROPERTIES => {
                    C_SIGNAL => {},
                    NET      => {},
                },
                NET_LIST => [
                    # omitting this call will allow the program to exceed 4GB until after it finishes the loop
                    { NL_INDEX => 0, }
                ]
            },
        };

        my $lastUpdate = time();

        push @dataStructureClones, $dataStructureThatCrashes;
        for (1 .. 5000000) {
            if (time() - $lastUpdate > 1) {
                # omitting this call will allow the program to exceed 4GB
                $textBox->insert("end", "Cloning hash ($_)...\n");
                $MAIN_WINDOW->update();
                $lastUpdate = time();
            }
            push @dataStructureClones, Clone::clone($dataStructureThatCrashes);
        }
    }
)->grid(-row => 0, -column => 0);

$textBox = $MAIN_WINDOW->Scrolled(
    'Text',
    -relief     => 'groove',
    -background => 'light grey',
    -foreground => 'black',
    -wrap       => 'char',
    -scrollbars => 'osoe',
    -width      => 110,
    -height     => 24,
)->pack(-side => 'top', -fill => 'both', -expand => 1);

MainLoop;

Things to note:

1. Commenting out line 28 makes the program run properly.
2. Commenting out line 38 makes the cloning loop finish executing, but then it crashes with a similar error about 15 seconds after the cloning has finished.

The similar error:

Free to wrong pool 1008ea0 not fcedf7a8 at C:/Strawberry/perl/site/lib/Tk.pm line 424.

I tried this on a Linux VM that we have (CentOS 7) and the issue does not happen.

Blevins

I have no idea how to write it in perl. But instead of a for loop that uses update, you should consider to use tkinters after method. If this alone does not help you need to consider to show just data that is needed instead of showing it all at once. – Loafer

The loop is building a large data structure, it is showing the problem, but it isn't the problem. The problem is that when the program exceeds 4GB of ram usage, which our application does, it crashes. – Blevins

I see. Did you take this into account as well? – Loafer

@Loafer Yes. I have 32GB of RAM on the machine that is running this. As I said in the first sentence, I have no issues exceeding 4GB when running tests in a console application. It is only when I'm using Tk that this happens, and only when I'm cloning this particular type of data structure. As I said in the example, using a simpler data structure does not cause the program to crash. – Blevins

Before I start a bounty on your question, could you tell me first if your OS is installed as 64bit as well? – Loafer

Yes, it is a 64-bit Windows 10 operating system. – Blevins

I tested the sample program on Windows 11, Strawberry perl version 5.32.1. I have 8GB ram on my laptop. I opened task manager, and could verify that the program's memory print increased steadily until it reached 4GB, then the program crashed without any error messages printed. – Than

Cannot reproduce the bug on Ubuntu 22.10 with the default perl+perltk. So the problem might be with window's perl/perltk. I have no idea how perl works but is it possible that perltk is 32 bit even if perl itself is 64 bit? I think that might cause the same issue. – Chivy

I think you should try to profile it with sysinternals tools like VMMap. This for 99% caused by 32bit limit of some kind. You use the Process Explorer also from sysinternals to see what is actually running. – Stringer

@Stringer what makes me wonder is --I have no issues exceeding 4GB when running tests in a console application. --, so I don't think it is necessarily a 32bit limitation. I could imagine something like this or that perl start a 32bit process regardless if it is installed 64bit or not. Another thing I had found on the web is, that a driver is 32bit, but I'm not much of an expert on those matters. – Loafer

Also it is might be worth to check this link where they explain "Utilising More Than 4GB of Memory in 32-bit Windows Process". – Loafer

@Loafer did you try to profile it or check it with Process explorer? You need to profile it to see details what is actually happening. – Stringer

While I don't use perl and I don't have a related goal with your question I will leave you two more links that may or may not help you and wish you good luck with your question. As last resort I have linked your question on Meta Stack Overflow but don't know if it will lead to something. Perl related issue and a somewhat informative VS thread. – Loafer

...and our application does not use multi-threading... Doesn't Tk use multiple threads on its own? – Ridiculous

@PresidentJamesK.Polk AFAIK it does not, each application (instances of Tk) comes with a interpreter and each interpreter runs in it's own thread and this model is also applied in the C-code which tcl is written in. That is what I have read at least. – Loafer

compare --"At the C programming level, Tcl’s threading model requires that a Tcl interpreter be managed by only one thread. However, each thread can create as many Tcl interpreters as needed running under its control. As is the case in even a single-threaded application, each Tcl interpreter has its own set of variables and procedures."-- in Multi-Threaded Tcl Scripts – Loafer

@Loafer Perl/Tk does not use tcl. However it has a similar requirement that all Tk operations are done in one thread. In any case this program is not multithreaded. – Hydrophobic

Looks like the code for Clone uses I32 for size of some objects. See metacpan.org/release/GARU/Clone-0.46/source/Clone.xs#L68 You might try modifying the code and file a bug if this is the problem. – Hydrophobic

Good catch. I tried several (Data::Clone, Storable, Sereal::Dclone) other cloning modules, including using JSON serialization/deserialization, and netted the same result. – Blevins

@stark, also note that when using this same code outside of a TK GUI application, there is no crash. It only crashes inside of a TK update call. – Blevins

@ErikAngerstig if that's the case, it would be interesting if the program still crashes if you insert the data in chunks. – Loafer

@Loafer when I populate the array without doing any TK calls (e.g. not calling $MW->update), it crashes after control is passed back to Tk when the button-click callback subroutine is complete. – Blevins

You shouldn't call update anyway. Does Perl offer threads to outsource the long blocking code or coroutines ? – Loafer

Answer 1

The first problem is that the underlying Tk library has internal memory allocators that can only handle memory allocations up to around 2GB (due to an ABI botch long ago that forced the memory object size parameter to be an int; fixing that is messy and breaks many things). The internal data structures mean that you don't necessarily blow things up as soon as you go over 2GB, but you're playing with fire, or maybe with nitroglycerin would be a better analogy.

The second problem, that you've not yet hit per se but will soon, is that 4GB of text data is a lot to actually work with in a GUI! It simply overwhelms users to have all the detail at once.

The fix, which I can only outline here, is to use the fact that that much data will never show all on the screen at once. Instead, you will need to be scrolling through the data... but you can connect up the scrolling mechanisms to scroll a window through the data itself instead of moving the widget's view into the data. When a scrolling event happens, you delete the contents of the window and replace it from the underlying data source. (This is only an outline because I don't know how to do this in Perl/Tk... and would have to look up the details for even Tcl/Tk despite knowing that much better.) I've read about people doing this to visualise very large result sets from DB queries, and suspect something similar would be useful for you.

Of course, if you go with putting all that data in a DB (for their winnowing support) then you might find better ways of querying and updating things that doesn't involve looking at huge amounts of data at one time...

Termor

This is a Community Wiki answer because I think it is only a partial answer, perhaps more of a pointer to a full answer than anything else. Feel free to edit to add in actual code and/or links! – Termor

I appreciate your response. The first part seems to unfortunately be spot on. The second part, though I appreciate the effort in your post and it is valuable information, is not really how our application works. We do a lot of background processing before building reports (excel/html) that open externally. The perl GUI is really just to allow the user to configure settings and start the operation. – Blevins

@ErikAngerstig As I'm not writing in perl I'm not fully sure, but I think this line $textBox->insert("end", "Cloning hash ($_)...\n"); will insert your data into the Text widget. Doesn't it ? If you don't need to show all of this data the question is quite misleading and there seems to be no more reason to do this background processing in your GUI-Thread or even process. – Loafer

I've done something similar with python and tkinter what is proposed by @DonalFellows here. --The fix, which I can only outline here, is to use the fact that that much data will never show all on the screen at once. Instead, you will need to be scrolling through the data... but you can connect up the scrolling mechanisms to scroll a window through the data itself instead of moving the widget's view into the data.-- In one case I used generator and in a different approach I was using a memory-mapped file object. – Loafer

@Loafer Yes, that line inserts data into the Text widget. If you were to run this program, you would see that the textbox receives 5-15 (depending on how fast your machine is) lines printed to it before it crashes. The lines are just progress indicators for the hash cloning and serve to illustrate that when there are GUI events the program crashes after exceeding the memory threshold. I don't think I need any more help with this, it is clear that the solution is outside the scope of possibility. Thank you for all your help. – Blevins