Implementation change to .NET's Random()

Question

I am migrating a method that is used for decoding from .NET Framework 1.1 to .NET Framework 4. I noticed that implementation of Random changed. So given the same seed, Random.NextBytes returns different result.

So if I run the following code.

byte[] bytes = new byte[4];
System.Random random = new System.Random(50);
random.NextBytes(bytes);

for(int i=0; i< bytes.Length; i++)
{
  Console.WriteLine("bytes[" + i + "] = " + bytes[i]);
}

Under .NET Framework 1.1 it returns:

bytes[0] = 216
bytes[1] = 124
bytes[2] = 183
bytes[3] =  58

Under .NET framework 4 it returns:

bytes[0] = 154
bytes[1] =  49
bytes[2] = 183
bytes[3] =  48

What is the best way to resolve this problem?

Mosira

What exactly is the problem? Does your program depend on the particular random number generation? – Ambo

I'm with Stilgar. Random is meant to produce pseudo-random results. If you need specific bytes, then why not just place those bytes in your code? @Moo-juice, but if you're upgrading to a later framework, then they'll all be using the same implementation after the upgrade...? – Indo

@Moo-Juice sure. Just make sure that the versions are compatible before allowing clients to connect. Games have been doing this kind of versioning for decades. – Ambo

@Ambo This is a part of method that is used for decoding. So it is required that it generates the same numbers for a given seed. – Mosira

The algorithm by which Random generates random numbers is not documented and, therefore, may change between versions. Using Random for version-independent number sequences is a bug. Period. – Catholicon

@Mosira if you are decoding something that is stored somewhere (like a database) the architecture is simply wrong and you are in trouble. If it is something generated on the fly all you need to do is update both parts at the same time or apply some kind of versioning. I know this is not really helpful :( – Ambo

@GregD I completely disagree. The Random class explicitly provides "repeatable sequences of psuedo-random numbers". If that's what your program needs, then having the .NET framework change the sequence across versions is a bug, regardless of the reason why you need it. – Monophysite

This was changed in .NET 3.5 (at least) documentation, when the class specifically highlights the fact that the implementation may not be the same across major .NET versions. The .NET 1.1 documentation says the algorithm follows a "definite mathematical algorithm", so I'd say this design change is not backwards compatible. – Aluminous

There's an oxymoron for you - a "predictable random number sequence". I agree that this kind of usage is a bug. – Algesia

@Algesia the documentation of Random states that setting the same seed will result in identical sequence of random numbers. So using it this way is not realy a bug. Also Random is a pseudo random number generator and its "randomness" is based only on internal state and the used algorithm. – Sansbury

@Sansbury - The documentation says that so as to provide a warning that the sequence is not truly random. I don't think it is trying to suggest that it is a stable behaviour throughout versions of the .NET framework. – Algesia

@Algesia there would be no reason to provide a seed if the results where completely random, so seeing it as a warning seems redundant. For me it describes a feature useful for debugging/replaying a series of random events. The error here was assuming that it was stable across different versions without the documentation stating this (or the documentation was erring by not mentioning possible differences between versions). – Sansbury

@Sansbury - I wholeheartedly agree. :-) – Algesia

Answer 1

You can just use Reflector to copy the Random class from the 1.1 mscorlib.

public class Random1_1
{
    // Fields
    private int inext;
    private int inextp;
    private const int MBIG = 0x7fffffff;
    private const int MSEED = 0x9a4ec86;
    private const int MZ = 0x0;
    private int[] SeedArray;

    // Methods
    public Random1_1()
        : this(Environment.TickCount)
    {
    }

    public Random1_1(int Seed)
    {
        this.SeedArray = new int[0x38];
        int num2 = 0x9a4ec86 - Math.Abs(Seed);
        this.SeedArray[0x37] = num2;
        int num3 = 0x1;
        for (int i = 0x1; i < 0x37; i++)
        {
            int index = (0x15 * i) % 0x37;
            this.SeedArray[index] = num3;
            num3 = num2 - num3;
            if (num3 < 0x0)
            {
                num3 += 0x7fffffff;
            }
            num2 = this.SeedArray[index];
        }
        for (int j = 0x1; j < 0x5; j++)
        {
            for (int k = 0x1; k < 0x38; k++)
            {
                this.SeedArray[k] -= this.SeedArray[0x1 + ((k + 0x1e) % 0x37)];
                if (this.SeedArray[k] < 0x0)
                {
                    this.SeedArray[k] += 0x7fffffff;
                }
            }
        }
        this.inext = 0x0;
        this.inextp = 0x15;
        Seed = 0x1;
    }

    public virtual int Next()
    {
        return (int)(this.Sample() * 2147483647.0);
    }

    public virtual int Next(int maxValue)
    {
        if (maxValue < 0x0)
        {
            throw new ArgumentOutOfRangeException("maxValue");
        }
        return (int)(this.Sample() * maxValue);
    }

    public virtual int Next(int minValue, int maxValue)
    {
        if (minValue > maxValue)
        {
            throw new ArgumentOutOfRangeException("minValue");
        }
        int num = maxValue - minValue;
        if (num < 0x0)
        {
            long num2 = maxValue - minValue;
            return (((int)((long)(this.Sample() * num2))) + minValue);
        }
        return (((int)(this.Sample() * num)) + minValue);
    }

    public virtual void NextBytes(byte[] buffer)
    {
        if (buffer == null)
        {
            throw new ArgumentNullException("buffer");
        }
        for (int i = 0x0; i < buffer.Length; i++)
        {
            buffer[i] = (byte)(this.Sample() * 256.0);
        }
    }

    public virtual double NextDouble()
    {
        return this.Sample();
    }

    protected virtual double Sample()
    {
        int inext = this.inext;
        int inextp = this.inextp;
        if (++inext >= 0x38)
        {
            inext = 0x1;
        }
        if (++inextp >= 0x38)
        {
            inextp = 0x1;
        }
        int num = this.SeedArray[inext] - this.SeedArray[inextp];
        if (num < 0x0)
        {
            num += 0x7fffffff;
        }
        this.SeedArray[inext] = num;
        this.inext = inext;
        this.inextp = inextp;
        return (num * 4.6566128752457969E-10);
    }
}

Tested and it gives the desired output.

Saintjust

Incidentally, what is the difference between the two, if you reflect them? – Syrinx

The main thing seems to be that they added support for larger samples. Here is the random class from .net 4.0. pastebin.com/stFuTzCk. – Saintjust

Legally dubious. You can't just decompile and use other people's code. – Slung

I'm sure he didn't decompile it. He just wrote test cases until his implementation happened to match the 1.1 version. Right? :-) – Monophysite

Answer 2

This is not a problem with Random, it satisfies its documented interface perfectly fine. This is a problem with your software relying on an implementation detail. Learn from this mistake and don't do it again.

As far as fixing the problem, you can implement your own version of 1.1's pseudorandom number generation for decoding and then implement a new encoding/decoding algorithm that doesn't rely on unstable behavior (such as the implementation of Random or GetHashCode) for your new version of the software.

Catholicon

That's very unfair; Storing the Seed used has been a reliable way of using Random since I can remember. – Telemann

It's not at all unfair. The algorithm isn't documented and thus is an implementation detail. This is a fundamental concept around using practically any proper API: Do not rely on implementation details or you may get burnt. What's unfair is using an api that's provided and then forcing that API to never change its implementation to something superior because you relied on silly implementation details instead of the published contract. – Catholicon

@RussC really? From the documentation: "The implementation of the random number generator in the Random class is not guaranteed to remain the same across major versions of the .NET Framework. As a result, your application code should not assume that the same seed will result in the same pseudo-random sequence in different versions of the .NET Framework." – Ambo

No, "Learn from this mistake and don't do it again." is blunt and unfair. He has documented a perfectly reasonable reason for maintaining the seed. The algorithm may not be documented, but passing a Seed is, and the behaviour should be expected. That it changes between frameworks is valid and he will need to account for that, but how often do you change frameworks ? – Telemann

@RussC: No, read Stilgar's comment. This is such a ridiculously common bit of incompetence that the documentation explicitly states the implicit implementation detail. If he wants a version-independent algorithm that'll result in the same sequence from the same seed, Random is the wrong tool. – Catholicon

To clarify, I'm saying it's perfectly fine to do this when you don't expect to change Frameworks. That he's held out from .Net 1 to 4 is impressive in itself! – Telemann

The only reasonable way to NOT expect change of frameworks is versioning in some client/server scenario or scenarios when the value is used in a single application execution. If the value is persisted then the implementation is simply wrong. – Ambo

In my mind, the problem is the seed. From the MSDN: "Providing an identical seed value to different Random objects causes each instance to produce identical sequences of random numbers." This implies the same sequence will occur regardless of the implementation. Even with the disclaimer, it's hard to justify the change across versions when the functionality is completely characterized by "a repeatable sequence". – Monophysite

Please note that in the .NET 1.1 documentation, the random class is documented to follow a "definite mathematical algorithm". While the specific algorithm and its details is not documented, the documentation could be construed as specifying that this "definiteness" is documented. Only in .NET 3.5 and forward was it documented that the specific implementation could change. – Aluminous

Answer 3

If you're absolutely reliant on the .NET 1.1 version of Random then the only thing I can think of is to create a new assembly that targets 1.1 and call that from your upgraded .NET 4 application.

However, can you detail why it is so essential for you to maintain this seed? There might be a better way.

Telemann

I think High's answer is possibly the way to go here. – Telemann

That 1.1 assembly would still run in the higher CLI, no? – Backbite

Answer 4

No answer here but contrary to many people here I don't think that documenting ridiculous behaviour is enough to justify it.

Because why would you provide a seeding mechanism in the first place? Well I'll tell you: so that you can always reproduce a random sequence from a single seed rather than having to persist perhaps millions of random numbers. Note that I said 'always', and not 'until you upgrade to the next version of .NET'. By not being consistent across versions the current .NET random number generators do not provide this functionality. Microsoft should have done a better job of implementing this (or not have implemented it at all) instead of just documenting the defective behaviour.

And by the way, although the algorithm is indeed an implementation detail, how on earth can you call the result of a method call an implementation detail? Should I really have to check the documentation of every method in the .NET framework to be sure that in the next version I do not risk getting a different result from concatenating two strings or calculating a square root?

So in my opinion what we have here is simply a badly implemented random number generator. And of course the entire problem could have been easily avoided by giving the new functionality (based on the new implementation) a different name.

Mongol

"why would you provide a seeding mechanism in the first place?" Because that's how pseudorandom number generators work! They are not actually random, but generate a sequence of numbers based on... a seed. No seed, no pseudorandom numbers. That's why. – Empennage

Answer 5

Alternatively, might I suggest using System.Security.Cryptography.RandomNumberGenerator class to generate cryptographically strong random byte arrays?

RandomNumberGenerator rng = RandomNumberGenerator.Create();
byte[] bytes = new byte[128];
rng.GetBytes(bytes);

I will join the rest of the comments, and mention that relying on an undocumented implementation is bad. More so, if you're actually relying on a predictable "randomness" - if you're using this for anything that should be "secure" - it is totally wrong.

Moline

"relying on an undocumented implementation is bad". Are you crazy? Every single method and property in all the standard .NET classes have undocumented implementations. What's important is the result - which should not change between versions. – Monophysite