Generate random UUID non blocking
Asked Answered
N

3

15

By using Blockhound io.projectreactor.tools blockhound-junit-platform i found out that UUID.randomUUID is a blocking call which is a problem for us since we are using Spring boot Webflux version 2.2.2.RELEASE

Is there any other way to get a random uuid in a non blocking way or is there any other java library recommended for non blocking generation of randomized strings.

Stack trace from blockhound:

java.lang.Error: Blocking call! java.io.FileInputStream#readBytes
at reactor.blockhound.BlockHound$Builder.lambda$new$0(BlockHound.java:196) ~[blockhound-1.0.1.RELEASE.jar:na]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
|_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain][ExceptionHandlingWebHandler]
Stack trace:
    at reactor.blockhound.BlockHound$Builder.lambda$new$0(BlockHound.java:196) ~[blockhound-1.0.1.RELEASE.jar:na]
    at reactor.blockhound.BlockHound$Builder.lambda$install$6(BlockHound.java:318) ~[blockhound-1.0.1.RELEASE.jar:na]
    at reactor.blockhound.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:46) ~[na:na]
    at java.base/java.io.FileInputStream.readBytes(FileInputStream.java) ~[na:na]
    at java.base/java.io.FileInputStream.read(FileInputStream.java:279) ~[na:na]
    at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133) ~[na:na]
    at java.base/sun.security.provider.NativePRNG$RandomIO.readFully(NativePRNG.java:424) ~[na:na]
    at java.base/sun.security.provider.NativePRNG$RandomIO.ensureBufferValid(NativePRNG.java:526) ~[na:na]
    at java.base/sun.security.provider.NativePRNG$RandomIO.implNextBytes(NativePRNG.java:545) ~[na:na]
    at java.base/sun.security.provider.NativePRNG.engineNextBytes(NativePRNG.java:220) ~[na:na]
    at java.base/java.security.SecureRandom.nextBytes(SecureRandom.java:741) ~[na:na]
    at java.base/java.util.UUID.randomUUID(UUID.java:150) ~[na:na]
Newcomer answered 6/2, 2020 at 13:46 Comment(10)
Is this really a problem? Getting enough random bytes from the OS random number source used to be a problem on older Linux versions, but it should be pretty fast now. And maybe that this only happens the first time you generate a UUID?Hoyos
See this thread for a discussion about why the default entropy source can be slow and how to select a different one: #137712Hoyos
If you do run into slowness and don't want to block your "reactive" thread, you could generate all UUID on a dedicated worker pool, same as you would do with other blocking things. But it does seem overkill.Hoyos
Thanks for answers. Blocking calls should not be used in reactive program even though they are fast. I agree creating a separate thread pool for generating the uuid seems overkill. No other library available for generating random id's that is recommended to use for reactive applications?Newcomer
There is no problem with this (unless seeding the SecureRandom is really slow). Class-loading is blocking as well. Don't fall into the premature "optimisation" trap and focus on demonstrable performance issues.Hoyos
my question is, is it a problem that the uuid random is blocking? have you measured it to be a problem?Farleigh
Is it possible to provide a workaround on this blocking call, regardless of how small is the impact?Disrate
If there's no way to say to blockhound that this is not an error I think the error is that one should disable the entire blockhound checks to use the UUID, was any solution found for this?Ochone
You should check on the first comment about why the call is taking so much time. But in general to wrap a synchronous blocking call you go Mono.fromCallable(UUID::randomUUID).subscribeOn(Schedulers.boundedElastic())Iand
+1 to suggestion by @Iand - this is the right fix for an operation like this that does have IO and is blocking even if expected to be quick.Lishalishe
H
3

This is a list of three examples on how to use uuid-creator to reduce thread contention on Linux.

An additional example uses ulid-creator to generate Monotonic ULIDs and then turn them into RFC-4122 UUID (standard). Monotonic ULID generation is very fast.

The benchmark code is available on a GitHub Gist.

Example 1: using UuidCreator and SHA1PRNG

How to setup the SHA1PRNG algorithm to be used by UuidCreator:

# Append to /etc/environment or ~/.profile
# Use the the algorithm SHA1PRNG for SecureRandom
export UUIDCREATOR_SECURERANDOM="SHA1PRNG"
// generate a random-based UUID
UUID uuid = UuidCreator.getRandomBased();

UuidCreator uses a fixed pool of SecureRandom. The variable UUIDCREATOR_SECURERANDOM tells UuidCreator to use another SecureRandom algorithm, instead of NativePRNG on Linux. UuidCreator along with SHA1PRNG algorithm generates UUIDs with less thread contention (less-blocking) than UUID.randomUUID().

See the benchmark using 4 threads:

----------------------------------------------------------------------
Benchmark                      Mode  Cnt      Score     Error   Units
----------------------------------------------------------------------
UUID.randomUUID()             thrpt    5   1423,060 ±  30,125  ops/ms
UuidCreator.getRandomBased()  thrpt    5  10616,016 ± 281,486  ops/ms
----------------------------------------------------------------------

Example 2: using ThreadLocalRandom

How to implement a UUID generator using ThreadLocalRandom:

public class UuidGenerator {

    private static final RandomBasedFactory UUID_FACTORY;

    static {
        UUID_FACTORY = new RandomBasedFactory((int length) -> {
            final byte[] bytes = new byte[length];
            ThreadLocalRandom.current().nextBytes(bytes);
            return bytes;
        });
    }

    public static UUID generate() {
        return UUID_FACTORY.create();
    }
}

ThreadLocalRandom is a fast (and unsafe) random generator without thread contention (non-blocking).

See the benchmark using 4 threads:

----------------------------------------------------------------------
Benchmark                     Mode  Cnt      Score      Error   Units
----------------------------------------------------------------------
UUID.randomUUID()            thrpt    5   1423,060 ±   30,125  ops/ms
UuidGenerator.generate()     thrpt    5  85390,979 ± 1564,589  ops/ms
----------------------------------------------------------------------

Example 3: using RandomBasedFactory[] and SHA1PRNG

How to implement a UUID generator using ana array of RandomBasedFactory and SHA1PRNG algorithm:

    public static class UuidGenerator {

        private static final int SIZE = 8; // you choose
        private static final RandomBasedFactory[] FACTORIES;

        static {
            FACTORIES = new RandomBasedFactory[SIZE];
            try {
                for (int i = 0; i < FACTORIES.length; i++) {
                    // SHA1PRNG or DRBG can be used to reduce thread contention.
                    SecureRandom argument = SecureRandom.getInstance("SHA1PRNG");
                    FACTORIES[i] = new RandomBasedFactory(argument);
                }
            } catch (NoSuchAlgorithmException e) {
                // oops!
            }
        }

        public static UUID generate() {
            // calculate the factory index given the current thread ID
            final int index = (int) Thread.currentThread().getId() % SIZE;
            return FACTORIES[index].create();
        }
    }

An array of RandomBasedFactory with and SHA1PRNG algorithm can generate UUIDs with less thread contention (less-blocking).

See the benchmark using 4 threads:

----------------------------------------------------------------------
Benchmark                     Mode  Cnt      Score      Error   Units
----------------------------------------------------------------------
UUID.randomUUID()            thrpt    5   1423,060 ±   30,125  ops/ms
UuidGenerator.generate()     thrpt    5  10048,747 ±  195,209  ops/ms
----------------------------------------------------------------------

Example 4: using UlidCreator and Monotonic ULIDs

How to use ulid-creator to generate RFC-4122 UUIDs from Monotonic ULIDs:

// create a Monotonic ULID and convert it to RFC-4122 UUID v4
UUID uuid = UlidCreator.getMonotonicUlid().toRfc4122().toUuid();

See the benchmark using 4 threads:

-----------------------------------------------------------------------
Benchmark                       Mode  Cnt     Score      Error   Units
-----------------------------------------------------------------------
UUID.randomUUID()              thrpt    5  1423,060 ±   30,125  ops/ms
UlidCreator.getMonotonicUlid() thrpt    5  7391,917 ±  871,799  ops/ms
-----------------------------------------------------------------------

EDIT: added the 4th example that uses Monotonic ULID.


Disclaimer: I'm the main contributor of uuid-creator.

Haymes answered 9/12, 2021 at 7:57 Comment(0)
E
3

There is a way to tell BlockHound to ignore UUID.randomUUID().

And there is a good reason to do so. Blockhound blames UUID.randomUUID() to use a file read operation, because file operations contain the risk to transfer some data to/from a disk.

/dev/random and /dev/urandom are not of that kind. The data is coming from the memory directly.

Since Java 8, Java uses /dev/urandom, which guarantees on top of /dev/random, that it won't block even if the entropy pool runs low. Which means, that in the whole stack down to the kernel, the CPU has always work to do.

Not using /dev/urandom makes solutions cryptographically weak. You could directly fallback to time-based UUIDs. Pushing it to other Threads as suggested by @downvoteit just adds overhead. The work needs to be done anyway to handle the request.

The performance-measurements by @fabiolimace indicate, that even using a cryptographically questionable solution and only generating UUIDs, you gain the factor 10 at its best. In reality, when UUID generation is not your bottleneck, you probably won't even notice the difference in terms of throughput.

Erroneous answered 9/3, 2023 at 17:11 Comment(0)
I
1

The original solution has been suggested by JEY (https://stackoverflow.com/users/1976843/jey) in the comments.

I am just sharing my BlockHound tested code for UUID and SecureRandom.

For UUID:

Mono.fromCallable(() -> UUID.randomUUID().toString())
  .subscribeOn(Schedulers.boundedElastic());

And another commonly used apparently blocking, SecureRandom:

private final SecureRandom random = new SecureRandom();

Mono.fromCallable(() -> random.nextInt(1_000))
  .subscribeOn(Schedulers.boundedElastic());

Tested on:

  • Java 17
  • Spring Boot 2.7.6
  • BlockHound 1.0.7.RELEASE (no modifications)
Instantly answered 25/2, 2023 at 22:39 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.