Getting locale functions to work in glibc

Question

I need to make some modifications to the the C standard library (glibc) in order to get some performance improvements. Specifically, I am going to be writing some specialized versions of some of the locale-dependent functions (which in glibc perform poorly), such as strcoll.

I have checked out the glibc source from here, into ~/Desktop/glibc. I then ran the following sequence of commands without error.

$ cd ~/Desktop
$ mkdir bglibc
$ cd bglibc
$ ./../glibc/configure --prefix=~/Desktop/bglibc
$ make
$ make install

At this point, I have successfully compiled and installed glibc into ~/Desktop/bglibc. I then created the following test program (ct.c) in the bglibc directory:

#include <stdio.h>
#include <locale.h>

int main ()
{
  char *locale = NULL;
  locale = "en_US.utf8";

  char *result = setlocale(LC_COLLATE, locale);
  if (result == NULL) {
    printf("locale not set\n");
  }   

  printf("strcoll: %d\n", strcoll("some", "string"));

  return 0;
}

I then build it with this script:

iSYSROOT=~/Desktop/bglibc

gcc -o ct ct.c \
  --sysroot=${SYSROOT} \
  -Wl,-rpath=${SYSROOT}/lib \
  -Wl,--dynamic-linker=${SYSROOT}/lib/ld.so.1

Which builds it properly. I then run it with this script:

#!/bin/sh
builddir=`dirname "$0"`
GCONV_PATH="${builddir}/iconvdata" \
exec    "${builddir}"/elf/ld-linux-x86-64.so.2 --library-path "${builddir}":"${builddir}"/*:"${builddir}"/*/*:"${builddir}"/*/*/* ${1+"$@"}

Which is names testrun.sh. To run it on the program I previously compiled (ct), I run ./testrun.sh ./ct.

This successfully runs the program, however the program prints out locale not set, meaning that it was unable to set the locale to "en_US.utf8". Thus, the locale keeps the default ("C"), in which case strcoll simply returns the result of strcmp. However, I need this call to run the strcoll code in order to run tests on its performance, and then tune it to run faster for specific locales.

I know that "en_US.utf8" is a valid locale for my system (Ubuntu 12.04 lts), because I see this:

$ locale -a | grep US
en_US.utf8

I have also tried running this program but setting the locale variable to other strings such as "en_US.UTF-16", "", "en_US.UTF-8", etc. all with no luck.

I imagine this isn't the first issue I will run into when trying to get locale stuff to work with my modified version of glibc, but its the first.

Any ideas On what I can do to get the locale functions (specifically setlocale) to work right?

Newsom

have you tried setLocale(LC_ALL, locale)? – Monkhood

Perhaps this link will help: gnu.org/software/libc/manual/html_node/Setting-the-Locale.html – Monkhood

@Clocks, yes, I have tried using LC_ALL instead and it didn't work. As far as the link goes, that is mostly just explaining "how locale and setlocale works". I understand how the setlocale function and the system locale works fairly well, the issue at hand is a bit more complex. Thanks though. – Newsom

For me it the answer from user2898218 worked. Consider to comment it or to accept his answer. – Connors

Answer 1

My guess: You forgot to "make" some locales. Try:

$ make
$ make install
$ make localedata/install-locales

see also GNU libc make manual

After installation you might want to configure the timezone and locale installation …

Gigantean

make localedata/install-locales works ! – Fauces

Answer 2

Try to use the strace to find out where the glibc tries to read the locales from. I suspect that since you set the prefix glibc tries to find them in ~/Desktop/bglibc/share/locale/ or something similar. And certainly UTF-16 will not work with 8 bit string types...

Frae

Thanks for the suggestion. I looked as some of the system calls using strace, and there are a few locale-related ones that like this: open("/home/username/Desktop/bglibc/lib/locale/en_US.utf8/LC_IDENTIFICATION", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory). So its looking for the locale files in a place where they don't exist (bglib/lib doesn't even exist). I looked around and it seems like they actually reside in this dir: bglibc/share/i18n/locales. Why would it be looking in the wrong location? It seems like it should know the correct place to look. – Newsom

That I do not know, anyway, a simple symlink would suffice now to just test if the locales work when pointed to right directory – Lives

I have added a symlink to the correct file, but the issue persists. I don't know if you have any other suggestions, but either way thanks for the help. – Newsom