Windows Heap Chunk Header Parsing and Size Calculation

Asked 12/2, 2015 at 17:4 Answered 13/2, 2015 at 14:15

Solved debugging heap-memory reverse-engineering windbg

How can I calculate heap chunk size from raw bytes read from memory. I tried below thing.

0:001> !heap
Index   Address  Name      Debugging options enabled
  1:   00500000                
  2:   00280000                
  3:   008f0000                
  4:   00ab0000                
  5:   00cc0000                

0:001> !heap -a 00500000
    ..
    ..
Heap entries for Segment00 in Heap 00500000    
     address: psize . size  flags   state (requested size)    
    00500000: 00000 . 00588 [101] - busy (587)    
    00500588: 00588 . 00240 [101] - busy (23f)    
    005007c8: 00240 . 00020 [101] - busy (18)    
    005007e8: 00020 . 00ca0 [101] - busy (c94)    
    ..
    ..
!heap -a 00500000 shows that size of first chunk is 588 bytes.

If we dump the chunk header using dt _HEAP_ENTRY, it somehow shows size is 0x3822

0:001> dt _HEAP_ENTRY 00500000
ntdll!_HEAP_ENTRY
   +0x000 Size             : 0x3822
   +0x002 Flags            : 0xfc ''
   +0x003 SmallTagIndex    : 0xbb ''
   +0x000 SubSegmentCode   : 0xbbfc3822 Void
   +0x004 PreviousSize     : 0x1849
   +0x006 SegmentOffset    : 0 ''
   +0x006 LFHFlags         : 0 ''
   +0x007 UnusedBytes      : 0x1 ''
   +0x000 FunctionIndex    : 0x3822
   +0x002 ContextValue     : 0xbbfc
   +0x000 InterceptorValue : 0xbbfc3822
   +0x004 UnusedBytesLength : 0x1849
   +0x006 EntryOffset      : 0 ''
   +0x007 ExtendedBlockSignature : 0x1 ''
   +0x000 Code1            : 0xbbfc3822
   +0x004 Code2            : 0x1849
   +0x006 Code3            : 0 ''
   +0x007 Code4            : 0x1 ''
   +0x000 AgregateCode     : 0x01001849`bbfc3822

When I dump the address 0x00500000 I find first two bytes are 22 and 38.

00500000 22 38 fc bb 49 18 00 01 ee ff ee ff 00 00 00 00 a8 00  "8..I.............
00500012 50 00 a8 00 50 00 00 00 50 00 00 00 50 00 00 01 00 00  P...P...P...P.....
00500024 88 05 50 00 00 00 60 00 cf 00 00 00 01 00 00 00 00 00  ..P...`...........
00500036 00 00 f0 0f 53 00 f0 0f 53 00 02 00 00 00 00 00 00 00  ....S...S.........
00500048 00 00 00 00 00 00 10 00 93 38 fd 0b 49 18 00 00 17 ff  .........8..I.....
0050005a bb 44 00 00 00 00 00 fe 00 00 ff ee ff ee 00 00 10 00  .D................
0050006c 00 20 00 00 00 08 00 00 00 20 00 00 2e 04 00 00 ff ef  . ....... ........
0050007e fd 7f 01 00 38 01 00 00 00 00 00 00 00 00 00 00 00 00  ....8.............
00500090 e8 0f 53 00 e8 0f 53 00 0f 00 00 00 f8 ff ff ff a0 00  ..S...S...........
005000a2 50 00 a0 00 50 00 10 00 50 00 10 00 50 00 00 00 00 00  P...P...P...P.....

My question is how does 22 and 38 (or 0x3822) becomes 0x588

Humiliate answered 12/2, 2015 at 17:4 Comment(2)

The column with "588" in it seems to be in hex ("00ca0" is in the same column). I have zero experience with OS memory debugging and heaps though. – Weigel 12/2, 2015 at 17:37

Yes! because 0x00500000 + 0x588 will become 0x00500588. Which is the start of next heap chunk. – Humiliate 12/2, 2015 at 17:39

Summary: heap entries are now encoded, the key is in the heap itself.

Let's say I have a heap at 0x00d60000:

0:000> !heap -a 00d60000 
Index   Address  Name      Debugging options enabled
  2:   00d60000 
    Segment at 00d60000 to 00d70000 (00001000 bytes committed)
    Flags:                40000061
    ForceFlags:           40000061
    Granularity:          8 bytes
    Segment Reserve:      00100000
    Segment Commit:       00002000
    DeCommit Block Thres: 00000200
    DeCommit Total Thres: 00002000
    Total Free Size:      00000149
    Max. Allocation Size: 7ffdefff
    Lock Variable at:     00000000
    Next TagIndex:        0000
    Maximum TagIndex:     0000
    Tag Entries:          00000000
    PsuedoTag Entries:    00000000
    Virtual Alloc List:   00d6009c
    Uncommitted ranges:   00d6008c
            00d61000: 0000f000  (61440 bytes)
    FreeList[ 00 ] at 00d600c0: 00d605a0 . 00d605a0  
        00d60598: 00118 . 00a48 [104] - free

    Segment00 at 00d60000:
        Flags:           00000000
        Base:            00d60000
        First Entry:     00d60480
        Last Entry:      00d70000
        Total Pages:     00000010
        Total UnCommit:  0000000f
        Largest UnCommit:00000000
        UnCommitted Ranges: (1)

    Heap entries for Segment00 in Heap 00d60000
         address: psize . size  flags   state (requested size)
        00d60000: 00000 . 00480 [101] - busy (47f)
        00d60480: 00480 . 00118 [107] - busy (100), tail fill
        00d60598: 00118 . 00a48 [104] free fill
        00d60fe0: 00a48 . 00020 [111] - busy (1d)
        00d61000:      0000f000      - uncommitted bytes.

There's a busy block at 0x00d60480: its allocated size is 0x118 (the size of the previous block is 0x480).

If we dump this block we can see it's encoded:

0:000> dt _heap_entry 00d60480
ntdll!_HEAP_ENTRY
   +0x000 Size             : 0x7387
   +0x002 Flags            : 0xf5 ''
   +0x003 SmallTagIndex    : 0x64 'd'
   +0x000 SubSegmentCode   : 0x64f57387
   +0x004 PreviousSize     : 0xb95d
   +0x006 SegmentOffset    : 0 ''
   +0x006 LFHFlags         : 0 ''
   +0x007 UnusedBytes      : 0x18 ''
   +0x000 FunctionIndex    : 0x7387
   +0x002 ContextValue     : 0x64f5
   +0x000 InterceptorValue : 0x64f57387
   +0x004 UnusedBytesLength : 0xb95d
   +0x006 EntryOffset      : 0 ''
   +0x007 ExtendedBlockSignature : 0x18 ''
   +0x000 Code1            : 0x64f57387
   +0x004 Code2            : 0xb95d
   +0x006 Code3            : 0 ''
   +0x007 Code4            : 0x18 ''
   +0x004 Code234          : 0x1800b95d
   +0x000 AgregateCode     : 0x1800b95d`64f57387

Back to the heap, pay a particular attention to the field named "Encoding " (at offset 0x50):

0:000> dt _heap encoding
ntdll!_HEAP
   +0x050 Encoding : _HEAP_ENTRY

Dumping the whole _HEAP structure:

0:000> dt _heap 00d60000
ntdll!_HEAP
   +0x000 Entry            : _HEAP_ENTRY
   +0x008 SegmentSignature : 0xffeeffee
   +0x00c SegmentFlags     : 0
   +0x010 SegmentListEntry : _LIST_ENTRY [ 0xd600a4 - 0xd600a4 ]
   +0x018 Heap             : 0x00d60000 _HEAP
   +0x01c BaseAddress      : 0x00d60000 Void
   +0x020 NumberOfPages    : 0x10
   +0x024 FirstEntry       : 0x00d60480 _HEAP_ENTRY
   +0x028 LastValidEntry   : 0x00d70000 _HEAP_ENTRY
   +0x02c NumberOfUnCommittedPages : 0xf
   +0x030 NumberOfUnCommittedRanges : 1
   +0x034 SegmentAllocatorBackTraceIndex : 0
   +0x036 Reserved         : 0
   +0x038 UCRSegmentList   : _LIST_ENTRY [ 0xd60ff0 - 0xd60ff0 ]
   +0x040 Flags            : 0x40000061
   +0x044 ForceFlags       : 0x40000061
   +0x048 CompatibilityFlags : 0
   +0x04c EncodeFlagMask   : 0x100000
   +0x050 Encoding         : _HEAP_ENTRY
   +0x058 Interceptor      : 0
   +0x05c VirtualMemoryThreshold : 0xfe00
   +0x060 Signature        : 0xeeffeeff
   +0x064 SegmentReserve   : 0x100000
   +0x068 SegmentCommit    : 0x2000
   +0x06c DeCommitFreeBlockThreshold : 0x200
   +0x070 DeCommitTotalFreeThreshold : 0x2000
   +0x074 TotalFreeSize    : 0x149
   +0x078 MaximumAllocationSize : 0x7ffdefff
   +0x07c ProcessHeapsListIndex : 2
   +0x07e HeaderValidateLength : 0x248
   +0x080 HeaderValidateCopy : (null) 
   +0x084 NextAvailableTagIndex : 0
   +0x086 MaximumTagIndex  : 0
   +0x088 TagEntries       : (null) 
   +0x08c UCRList          : _LIST_ENTRY [ 0xd60fe8 - 0xd60fe8 ]
   +0x094 AlignRound       : 0x17
   +0x098 AlignMask        : 0xfffffff8
   +0x09c VirtualAllocdBlocks : _LIST_ENTRY [ 0xd6009c - 0xd6009c ]
   +0x0a4 SegmentList      : _LIST_ENTRY [ 0xd60010 - 0xd60010 ]
   +0x0ac AllocatorBackTraceIndex : 0
   +0x0b0 NonDedicatedListLength : 0
   +0x0b4 BlocksIndex      : 0x00d60248 Void
   +0x0b8 UCRIndex         : (null) 
   +0x0bc PseudoTagEntries : (null) 
   +0x0c0 FreeLists        : _LIST_ENTRY [ 0xd605a0 - 0xd605a0 ]
   +0x0c8 LockVariable     : (null) 
   +0x0cc CommitRoutine    : 0x7944d754     long  +7944d754
   +0x0d0 FrontEndHeap     : (null) 
   +0x0d4 FrontHeapLockCount : 0
   +0x0d6 FrontEndHeapType : 0 ''
   +0x0d7 RequestedFrontEndHeapType : 0 ''
   +0x0d8 FrontEndHeapUsageData : (null) 
   +0x0dc FrontEndHeapMaximumIndex : 0
   +0x0de FrontEndHeapStatusBitmap : [257]  ""
   +0x1e0 Counters         : _HEAP_COUNTERS
   +0x23c TuningParameters : _HEAP_TUNING_PARAMETERS

Dumping the encoding field as two DWORDs:

0:000> dd 00d60000 + 0x50 L2
00d60050  40f273a4 0000b9cd

Now dumping the heap entry as two DWORDs:

0:000> dd 00d60480 L2
00d60480  64f57387 1800b95d

Let's XOR them:

0:000> ? 40f273a4 ^ 64f57387 
Evaluate expression: 604438563 = 24070023

0:000> ? 0000b9cd ^ 1800b95d 
Evaluate expression: 402653328 = 18000090

Now just writing a fake _HEAP_ENTRY so we can 'dt' it:

0:000> ed 00d604b0
00d604b0 00000000 24070023
24070023
00d604b4 00000000 18000090
18000090
00d604b8 00000000 

0:000> dt _HEAP_ENTRY 00d604b0
ntdll!_HEAP_ENTRY
   +0x000 Size             : 0x23
   +0x002 Flags            : 0x7 ''
   +0x003 SmallTagIndex    : 0x24 '$'
   +0x000 SubSegmentCode   : 0x24070023
   +0x004 PreviousSize     : 0x90
   +0x006 SegmentOffset    : 0 ''
   +0x006 LFHFlags         : 0 ''
   +0x007 UnusedBytes      : 0x18 ''
   +0x000 FunctionIndex    : 0x23
   +0x002 ContextValue     : 0x2407
   +0x000 InterceptorValue : 0x24070023
   +0x004 UnusedBytesLength : 0x90
   +0x006 EntryOffset      : 0 ''
   +0x007 ExtendedBlockSignature : 0x18 ''
   +0x000 Code1            : 0x24070023
   +0x004 Code2            : 0x90
   +0x006 Code3            : 0 ''
   +0x007 Code4            : 0x18 ''
   +0x004 Code234          : 0x18000090
   +0x000 AgregateCode     : 0x18000090`24070023

Size field is 0x23, granularity is 8 bytes (as reported by the !heap -a command output). The real size of the block is the Size field value multiplied by the granularity, so:

0:000> ? 23 * 8
Evaluate expression: 280 = 00000118

It also works for the size of the previous block (reported to be 0x480):

0:000> ? 0x90 * 8
Evaluate expression: 11552 = 00000480

We found the same sizes.

Granularity

Granularity (as given by the !heap -a command output) is not indicated by a specific field, it's just the size of a HEAP_ENTRY structure:

8 bytes on x86 systems (or WOW64):

0:000> ?? sizeof(_HEAP_ENTRY)
unsigned int64 8

16 bytes on x64 systems:

0:000> ?? sizeof(_HEAP_ENTRY)
unsigned int64 0x10

Paver answered 13/2, 2015 at 14:15 Comment(8)

Excellent, I admit that my statement “hard task” is overstated :-) – Municipal 13/2, 2015 at 16:12

I should have read Chris Valasek's LFH paper more carefully :) – Humiliate 13/2, 2015 at 17:41

And what about "requested size" and "state"? How does windbg find that ? – Humiliate 13/2, 2015 at 20:18

@Deb.K.: I think your question is answered. Please open a new question if you really need that info. – Archduchy 13/2, 2015 at 21:45

@Deb.K. "requested size" is: (_HEAP_ENTRY.Size * granularity) - _HEAP_ENTRY.UnusedBytes ; "state" is grabbed from _HEAP_ENTRY.Flags – Paver 14/2, 2015 at 1:5

@Paver Can you please take a look at this one #28525119 – Humiliate 15/2, 2015 at 11:39

How do you know what the Granularity is? Where is the information contained? Thanks a lot for your post it's been enormous help for me – Venosity 13/7, 2018 at 10:20

@MarcinK. I added an edit on the above post; basically granularity is not given by a field, it's just the size of the HEAP_ENTRY structure. – Paver 14/7, 2018 at 8:17

Form Vista and later, the heap entries are scrambled so it’s a hard task to do any calculations. Check this link read about randomization.

The DT command are therefore unable do display any sensible information at all. Take a look at the offsets:

0:001> dt _HEAP_ENTRY
+0x000 Size                    
+0x000 FunctionIndex    
+0x000 InterceptorValue 
+0x000 AgregateCode

A lot of elements with same offset, hence same memory.

Also observe your

+0x004 PreviousSize     : 0x1849

Does not correspond with the psize of 0000 from !heap –a.

On Win XP and earlier your technique was possible, but here the

_HEAP_ENTRY-> Size

was number of heap blocks, usually of 8 bytes.

Edit: I’m not aware of any manual method to decode the heap entry, but I guess it’s possible. I have used the !heap –i command to do it for me. First:

!heap –i <heap>   , in your case !heap –i 00500000

Then

!heap –I <heap entry> , in your case !heap –I 00500588 (for second entry)

Sample:

address: psize . size  flags   state (requested size)
00240000: 00000 . 00588 [101] - busy (587)
00240588: 00588 . 00240 [101] - busy (23f)
....

0:000> !heap -i 00240000                
Heap context set to the heap 0x00240000
0:000> !heap -i 00240588
Detailed information for block entry 00240588
Assumed heap       : 0x00240000 (Use !heap -i NewHeapHandle to change)
Header content     : 0x32343AD9 0x0100B0F1 (decoded : 0x49010048 0x010000B1)
Owning segment     : 0x00240000 (offset 0)
Block flags        : 0x1 (busy )
Total block size   : 0x48 units (0x240 bytes)
Requested size     : 0x23f bytes (unused 0x1 bytes)
Previous block size: 0xb1 units (0x588 bytes)
Block CRC          : OK - 0x49  
Previous block     : 0x00240000
Next block         : 0x002407c8

Recommended topics

Hot tags