VBA: Why would the Not operator stop working? [duplicate]

Question

This has me utterly baffled.

Sub testChangeBoolean()
  Dim X As Boolean       ' default value is False
  X = True               ' X is now True
  X = Not X              ' X is back to False
End Sub

But I'm trying to toggle the .FitText property in a table cell (in Word). .FitText starts as True.

If I assign it to X:

Sub testChangeBoolean()
  Dim X As Boolean                  ' again, default is False
  X = Selection.Cells(1).FitText    ' does set X to True
  X = Not X                         ' X is still True!
End Sub

I just don't understand what I'm doing wrong.

Inhabited

Replying to the closing as duplicate by a gold badge: While the other question discusses when Not does not work as expected, it does not handle the why behind the behavior here: The Word object model is not behaving according to the rules of VBA. A discussion of that aspect is completely missing in the question linked as a duplicate. – Parget

@Cindy The "why" is exactly the same in both cases. This is a classic problem with Visual Basic as well as VBA. False is 0 (all bits clear), and True is defined as Not False, which means -1 (all bits set).Causes WinAPI interop issues, too. Any non-zero value is treated as True, so it's only a problem if you start manipulating bits without understanding what;s happening, compounded by the lack of a distinction between bitwise/logical operators. I don't see the merit in re-opening this when the root cause and the solution are fundamentally identical. – Kilowatt

@CindyMeister If it had not already been explained here that it's the Word object model that generates a malformed Boolean, I would have added that as a comment in addition to voting to close. But because there already was a great answer explaining it, I did not have to. The underlying reason is identical, like Cody Gray mentioned, and in addition to that, I see closing as duplicate as a way to link helpful questions together rather than to punish the later asker. – Flivver

Answer 1

I believe the explanation has to do with how older programming languages (WordBasic and early VBA) stored the integer values of True and False. In those days, True = -1 and False = 0.

Newer programming languages still use 0 for False, but 1 for True.

The majority of Word's Boolean type properties continue to use -1 for True (Font.Bold, for example), which has been cause for confusion and frustration for programmers working with the Interop in newer languages. So, at some point, some developers at Microsoft decided to use the new way and assigned the integer value of 1 to True for some new functionality. Such as FitText.

Considering the following code sample, where X is of type Boolean and y of type Integer:

• If FitText is True, the integer value is 1
• If reversing the values, using Not shows that the Boolean remains "True" because its integer value is not 0, it's -2
• Setting the integer value directly to True gives -1

This is confusing, indeed, but does explain why Not is not giving the expected result.

Sub testChangeBoolean()
  Dim X As Boolean                  ' again, default is False
  Dim Y As Integer
  X = Selection.Cells(1).FitText    ' does set X to True
  Y = Selection.Cells(1).FitText
  Debug.Print X, Y                  ' result: True    1
  X = Not X                         ' X is still True!
  Y = Not Y
  Debug.Print X, Y                  ' result: True   -2
  X = False
  Y = True
  Debug.Print X, Y                  ' result: False  -1
End Sub

Parget

Yeah this is amazing indeed. Great explanation. Makes me sooo happy I don't program in Word. – Nerva

@Nerva the part that's baffling, is the part where the integer value is somehow still present under the hood of the Boolean variable. This basically means Boolean is not type-safe in VBA. – Botha

@MathieuGuindon Considering the age of the Office programming languages, this should not be surprising. In the really old days, "Boolean" was a "nice way to think" about off/on, which was simply stored as 0... and something else, by convention -1. Backwards compatibility, even for WordBasic, has been a priority, so no surprise that there's been no modernization. And also keep in mind that the core of Word - the stuff they cannot change, even to fix bugs - is C. Some things cannot be changed without starting from 0, which would be too expensive. – Parget

Yeah... yet if I do Dim a As Boolean, then a = 2, and then Debug.Print CInt(a), a, Not a, I get -1 / True / False for output. That 2 -> -1 conversion seems to not be happening in Word's internals. – Botha

@MathieuGuindon That is interesting. So the "boolean" status being assigned by the property is apparently somehow being stored at another level so that it "sticks". I found the -2 result of Not very odd - wonder what that equates to...? Some special kind of false that VBA isn't recognizing? – Parget

The -2 is because logical operators do bitwise operations - tricky with the sign bit, but basically flipping all the bits of 1 results in the sign bit turning on and the 01 becoming 10, which is 2 - with the sign bit that reads -2. – Botha

Ah, so I'm not insane, for once. Good to know. Thank you so much! – Inhabited

Some things cannot be changed without starting from 0 -- keeping in mind that in VBA you should start from UBound. – Sennet

@ZevSpitz Not sure what that has to do with this particular discussion, which is about Boolean and not arrays... – Parget

@CindyMeister I took as a sort of pedantic joke maybe?, Idk, I was quite confused by Zev's comment too. – Nerva

@ZevSpitz shouldn't you start from LBound()? – Jannelle

@Nerva Yes, it was meant to be a pedantic joke; I apologize for any confusion. And yes, I meant LBound. Should I delete the comment? – Sennet

@ZevSpitz No need from my POV, now that we know :-) – Parget

Answer 2

To add on to Cindy's excellent answer, I want to point out that while VBA normally has safeguards to coerce the values when assigning to a Boolean data type, this can be circumvented. Basically, if you write a random value to a memory address that's not yours, then you should expect undefined behavior.

To help demonstrate this, we'll (ab)use LSet which essentially allow us to copy the value without actually assigning.

Private Type t1
  b As Boolean
End Type

Private Type t2
  i As Integer
End Type

Private Sub Demo()
  Dim i1 As t2
  Dim b1 As t1
  Dim b As Boolean

  i1.i = 1
  
  LSet b1 = i1

  b = b1.b

  Debug.Print b, b1.b, i1.i
  Debug.Print CInt(b), CInt(b1.b), i1.i

End Sub

Note the line b = b1.b is basically equivalent to what we did in the OP code

X = Selection.Cells(1).FitText

That is, assigning a Boolean to another Boolean. However, because I wrote to the b1.b using LSet, bypassing VBA runtime checks, it doesn't get coerced. When reading the Boolean, VBA does implicitly coerce it into either True or False, which seems misleading but is correct because any falsy results is one that equals 0 (aka False), and any truthy results is one that doesn't. Note that the negative for truthy means that both 1 and -1 are truthy.

Had I assigned the 1 to a Boolean variable directly, VBA would have had coerced it into -1/True and thus there'd be no problem. But evidently with FitText or LSet, we are basically writing to the memory address in an uncontrolled fashion, so that VBA start to behave strangely with this particular variable since it expects the Boolean variable to already had its contents coerced but wasn't.

Aerugo

So... the bug is in the Word library (fixable) or in VBA itself (fix unlikely)? – Botha

Wonderful, what a seemingly simple, inocuous question can result in... Thank you for this, which I'm going to have to look at in more detail, tomorrow, when I'm awake :-) – Parget

Answer 3

It's because of the internal Long value coming from this property, as explained by Cindy Meister. We should always use CInt to avoid this.

Sub testChangeBoolean2()
  Dim X As Boolean                     ' again, default is False
  X = CInt(Selection.Cells(1).FitText) ' [Fixed] does set X to True
  X = Not X                            ' X is False!
End Sub

Gumwood

Upvoted for including an easy fix to get the desired behavior. – Jotun