When should you use XS?

Question

I am writing up a talk on XS and I need to know when the community thinks it is proper to reach for XS.

Astrology

Is your talk/presentation available? – Hesper

Answer 1

I can think of at least three reasons to use XS:

1. You have a C library you want to access in Perl 5
2. You have a block of code that is provably slowing down your program and it would be faster if written in C
3. You need access to something only available in XS

Reason 1 is obvious and should need no explaination.

When you really need reason 2 is less obvious. Often you are better off looking at how the code is structured. You should only invoke reason 2 if you have profiled your code and have a benchmark and test suite to prove that the XS code is faster and correct.

Reason 3 is a dangerous reason. It is rare that you actually need to look into Perl's guts to do something, but there is at least one valid case.

Astrology

It would be great if you could also cover the advantages/disadvantages of using Inline::C vs. pure XS. – Lavalley

Yes, a comparison would be great. One more entry to the wishlist: what sort of things can be done only with XS, and what can be done only with Inline::C. – Washko

Inline::C is just a wrapper around XS. It makes it very easy to call out to C, but has the disadvantage that you need the same C compiler used to build your perl. In general, you should only use Inline::C to build the XS code for you, then take that code and strip out Inline::C. That's the best of both worlds. – Tablet

@Washko As far as I know, there is nothing that XS does with Perl's guts that Inline::C can't. The major difference is when the code gets compiled (at build time vs run time) which has a major effect on how easy it is to deploy code written with Inline::C. Ah, the war stories I could, and will, tell about Inline::C deployments. – Astrology

@frezik, the compiler requirements are the same for XS and Inline::C. – Newlin

@Chas. Owens, There's also permission issues, since Inline::C actually needs to create files. Inline::C is easier to work with, so it can be useful as a prototyping tool. – Newlin

@Chas. Owens, (3) is not rare at all. Examples: Pretty much anything in Scalar::Util, modifying Perl's syntax (without using a source filter), adding magic other than tie to variables, creating a PerlIO layer, etc, etc – Newlin

@Newlin Good point, but when you compare the number of uses for 1 and 2, the number of sane cases of 3 are very small. – Astrology

Answer 2

In a few cases, better memory management is another reason for using XS. For example, if you have a very large block of objects of some similar type, this can be managed more efficiently through XS. KinoSearch uses this for tokens, for example, where start and end offsets in a large string can be managed more effectively through XS than as a huge pool of scalars. PDL also has a memory management aspect to it, as well as speed.

There are proposals to integrate some of this approach into core Perl in the long term, initially because it offers a chance to make sharing data in threading better: see: http://openparallel.com/2011/07/05/a-new-hope-for-efficient-safe-data-sharing-between-threads-in-perl/.

Eustoliaeutectic

This is a good point, I think it falls under reason 2, but my reason 2 was too narrowly defined. I think reason 2 should probably be something like "To make more efficient use of the computer's resources". – Astrology

@Chas. Owens, agreed. You'd typically use different tools to assess it, and on 32-bit systems might be forced sooner to adopt this strategy because memory has a hard limit, but is another resource issue – Eustoliaeutectic