Using 64 bits integers with Perl XS

Question

I want to port a C function using 64 bits integers to Perl code. To do that, I use Perl XS.

My problem is there are no 64 bits integers in Perl XS types (only U32, U16 and U8).

So, what is the best way to use 64 bit integers in Perl XS code?

Here is a code example of what I want to do:

uint64_t
foo(integer)
   uint64_t integer

   CODE:
      RETVAL = foo(integer);

   OUTPUT:
      RETVAL

foo() has the C prototype:

uint64_t foo(uint64_t);

I didn't find anything useful in the perlxs documentation and in stackoverflow.com.

Quathlamba

I don't know Perl (XS), but if tthe language does not provide integers larger than 32 bit, you cannot "use 64 bit integers". It's the same as if you have no hammer, you cannot use a hammer. Only way might be to use two 32 bit variables and program the required operations yourself, but that is not trivial. (Sidenote: U32 etc sounds like unsigned, while the term integer normally includes the negative range) – Ingroup

NVs can represent precisely integers up to 2**53. In practice that is enough in quite a few cases, otherwise use Math::Int64 as advised bellow. – Epode

Answer 1

Perl's internal type for unsigned integers is UV. If your platform has 64-bit UVs, you should be fine.

With 32-bit UVs (32-bit OS and Perl was compiled without use64bitint), you can convert large integers to and from floating point numbers (NV which is typically a double). But since IEEE doubles only have 53 bits of mantissa, this will cause a loss of precision for large numbers.

uint64_t integer;

// Convert uint64_t to SV.

if (integer <= UV_MAX) {
    RETVAL = newSVuv((UV)integer);
}
else {
    RETVAL = newSVnv((NV)integer);
}

// Convert SV to uint64_t with range checks.

if (SvUOK(sv)) {
    integer = (uint64_t)SvUV(sv);
}
else if (SvIOK(sv)) {
    IV iv = SvIV(sv);
    if (iv < 0) croak(...);
    integer = (uint64_t)iv;
}
else if (SvNOK(sv)) {
    NV nv = SvNV(sv);
    if (nv < 0.0 || nv >= 18446744073709551616.0) croak(...);
    integer = (uint64_t)nv;
}
else {
    // Parse a uint64_t from the string value, or whatever.
}

If you can't live with loss of precision and want to support 32-bit UVs, you can use the C API of Math::Int64 (also see Module::CAPIMaker):

#define MATH_INT64_NATIVE_IF_AVAILABLE
#include "perl_math_int64.h"

MODULE = ...

BOOT:
    PERL_MATH_INT64_LOAD_OR_CROAK;

SV*
foo(args...)
CODE:
    uint64_t integer = ...
    RETVAL = newSVu64(integer);
OUTPUT:
    RETVAL

With a typemap like

TYPEMAP
uint64_t T_UINT64

INPUT
T_UINT64
    $var = SvU64($arg)

OUTPUT
T_UINT64
    $arg = newSVu64($var);

you can use uint64_t directly in your XS:

uint64_t
foo(arg)
    uint64_t arg

Brookweed

Answer 2

The uint64_t type is not defined in default typemap, that comes with Perl distribution. According to perlxstypemap:

In more practical terms, the typemap is a collection of code fragments which are used by the xsubpp compiler to map C function parameters and values to Perl values.

You can define your own typemap (in the same directory as .xs file). It might look like:

TYPEMAP
unsigned long long T_U_LONG_LONG
uint64_t T_U_LONG_LONG              # equivalent to typedef unsigned long long uint64_t;

INPUT
T_U_LONG_LONG
    $var = (unsigned long long)SvUV($arg)

OUTPUT
T_U_LONG_LONG
    sv_setuv($arg, (UV)$var);

I am not sure about these mappings, especially SvUV and UV part, so you need to test it carefully in any real code. I suspect that they are just "plain" integers, thus uint64_t is fully functional only for internal usage, that is within function's definition.

Note that unsigned long long type is at least 64-bits wide according to C Standard (since C99), but it's 64-bit on every implementation I am aware of.

With test.xs as:

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "ppport.h"

MODULE = test       PACKAGE = test

unsigned int
u64_mul_high(a, b)
    unsigned int a
    unsigned int b
CODE:
    RETVAL = ((uint64_t) a * b) >> 32;
OUTPUT:
    RETVAL

and test.pl defined as:

#!/usr/bin/perl
use ExtUtils::testlib;
use test;

print test::u64_mul_high(2147483647, 1000), "\n";

you get an result of:

499 (0001 1111 0011)

that is higher 32-bit limb of an 32-bit x 32-bit multiplication that results into 64-bit integer.

I checked this on 32-bit GNU/Linux with sizeof(long) = 4 and Perl 5.10.

Subordinate

Answer 3

$ perl -MConfig -le 'print $Config{use64bitint}'

will show if your perl was compiled to use 64 bit integers. If so, IV is 64-bit.

See also perldoc perlguts:

What is an "IV"?

Perl uses a special typedef IV which is a simple signed integer type that is guaranteed to be large enough to hold a pointer (as well as an integer). Additionally, there is the UV, which is simply an unsigned IV.

Perl also uses two special typedefs, I32 and I16, which will always be at least 32-bits and 16-bits long, respectively. (Again, there are U32 and U16, as well.) They will usually be exactly 32 and 16 bits long, but on Crays they will both be 64 bits.

See also Math::Int64:

This module adds support for 64 bit integers, signed and unsigned, to Perl.

...

Fallback to native 64bit support if available

If the lexical pragma Math::Int64::native_if_available is used in your program and the version of perl in use has native support for 64bit integers, the functions imported from the module that create 64bit integers (i.e. uint64, int64, string_to_int64, native_to_int64, etc.) will return regular perl scalars.

Heteropolar