Strategy or tools to find "non-leak" memory usage problems in Delphi?

Question

One old application started to consume memory a lot after server update. Memory usage seems to rise with out limit until program hangs.

According to FastMM4 and EurekaLog, there's no memory leak (except 28 bytes), so I assume all memory is freed when application is shutdown.

Are there any tools or strategies suitable for tracking this kind of memory problem?

Skein

Answer 1

1. The growing memory consumption is an application issue. It is not a bug, which can discover FastMM4 or EurekaLog. As from they point of view - application just correctly uses the memory.
2. Using AQTime, MemProof (hard to find, D7 is last supported version (?)), SleuthQA (similar to MemProof) or similar memory profilers, you can track the memory usage outside of application in real-time.
3. Using FastMM4, GetMemoryManagerState / GetMemoryManagerUsageSummary you can track memory usage from application. Output this information into trace file and analyze it after run. Or make simple wrapping function for one of the above procedures, which will return curent memory usage. And call it from IDE Debugger Evalute / Modify, add to Watches or call OutputDebugString, and see the current memory usage.

Note, if memory is eated by some DLL then you may not see her memory usage using (3). Use (2).

Analyzing the memory usage and the tasks performed by the application, you may discover what leads to raised memory usage.

Pasahow

MemProof (v.0.948, not sure was this the last one) is still on Torry's: torry.net/authorsmore.php?id=1229 – Manolo

The problem is that this is server type application, so pinpointing single action is very hard. – Skein

@Skein The GetMemoryManagerUsageSummary solution is what you need. I used this in a mult-threaded service and it at least points to the size of the memory items that are going wild. – Hortatory

For a multi-threaded application the tracing capability is the must. – Pasahow

@mj2008: Could the explain shortly how you used GetMemoryManagerUsageSummary? – Skein

@Skein The details I don't remember, but I modified the sample that uses the window output to save to a log file periodically. That gave me details over time. I also recall using some code in madExcept to give me per-thread memory use, but that may be CPU use - not sure! It was a while ago unfortunately. You mention this is a server app, but if it is not a service then AQTime is probably the best option as you can see full stack traces. If it is a service, make an app that creates the service module and use that. – Hortatory

Answer 2

Since September 2012, there is a very simple and comfortable way to find this type of "run-time only" memory leaks.

FastMM4991 introduced a new method, LogMemoryManagerStateToFile:

Added the LogMemoryManagerStateToFile call. This call logs a summary of the memory manager state to file: The total allocated memory, overhead, efficiency, and a breakdown of allocated memory by class and string type. This call may be useful to catch objects that do not necessarily leak, but do linger longer than they should.

To discover the leak at run time, you only need these steps

1. add a call to LogMemoryManagerStateToFile('memory.log', '') in a place where it will be called in intervals
2. run the application
3. open the log file with a "tail" program (for example Notepad++ with tail mode switched on), which will auto-refresh when the file content changes
4. watch the first lines of the file, they will contain the memory allocations which occupy the highest amount of memory
5. if you see a class or memory type constantly has a growing number of instances, this can be the reason of your leak

Libation

Could you help to comment what should be done if pagefile keeps growing after each execution but memory.log shows nothing constantly growing ? – Corselet

Answer 3

AQTime (a commercial tool which is quite expensive) can report your memory usage, down to the line of source code that allocated each object. In the case of very large memory usage scenarios, you might want the AQTime functionality that can show the number of objects and the size (total plus individual instance size) for each object. AQTime worked great for me, starting with Delphi 7, and all later versions, including your version (2006) and the latest versions (XE and XE2).

As the program memory usage grows, AQTime can be used to grab "snapshots" of the runtime heap, you can use to understand memory usage of your application; What is being created, and how many of each object exists. Even when no leaks exist, understanding the runtime behaviour of your application in terms of the objects it creates and manages, is very important, and AQTime is the most powerful tool I know of for Delphi users.

If you are willing to upgrade to Delphi XE/XE2, you might have an included light version of AQTime already, if so, check it out. If not, I recommend you try their demo. I am unaware of any free or open source alternatives that can provide the same functionality.

Lesser functionality could be cobbled together manually by writing lots of trace messages, or using the FastMM full-debug-mode. If you could write a complete dump of your memory usage into a very large file, you might be able to write some tools to parse, and create a summary. The problem I have with FastMM in this case, is that you will be drowned in detail information, without the ability to extract exactly the summary information that helps you understand your situation. So, you can try to write your own tool to summarize the memory usage. In one application I had that used a series of components that I knew would use a lot of memory, I wrote a dialog box into my application that showed current memory usage by these large memory-blob-of-data objects.

Pentomic

Just note: Denomo doesn't compile under D2006, and when fixed, it corrupts something and application is unusable. – Skein

Answer 4

Have you ever think about the Leak that is causing the IDE... it is so huge!!!

In my case (2GB of RAM) i do the next... 1. Open the IDE 2. Leave it minimized for near six hours 3. See how Physical memory is getting used

The result: While IDE is oppened (remember i also do the test having it minimized) it is getting more and more RAM... till no more ram free. It gets all 2GB RAM + all Pagefile hard disk space (i have it configured to a mas of 4GB) In less that six hours (doing nothing on IDE) it tries to use more than 6GB.

That is called a Memory Leak casused by the IDE... i do not type any letter on IDE, do not compile anything, do not even open any project... just open IDE and minimize it... leave the computer without doing anything on it for about six hours and IDE is consuming 6GB of memory.

Of course, after that, the IDE start with annoying messages of SystemOutOfMemory... and i must kill it... then all that 6GB are freed!!!

When on the hell will this get fixed?

Please note i have all patches applied, i also tested without applying each patch/hotfix, etc...

The best i got was dissabling some options on Tools, like the one that underlines bad code, etc... so why on the hell that option has any influence... i am not typing anything on the IDE (on the tests)... and if i have it dissabled the memory leak gets reduced a lot...

Of course, if i use the IDE (write code on an opened project) without even compiling / running it... the thing goes much more worst... memory leak upto 6GB can got reached on less than an hour, sometimes occurs after 15 minutes of Copy/Paste source code.

Seems there will not be a solution in a short time!!!

So i got the next solution that works perfect: -Close the IDE an reopen it each 15 minutes or less

Ugly solution, i know... but works!!!

Aquitaine