Flatpak Meson Not Finding Vala Libraries From Gnome Builder

Question

From Linux, I'm using Meson (0.44.0) within Gnome Builder (3.26.4) for a console program that will use Gee and GXml. My intent is to write this in Genie.

When I use Meson within Gnome Builder it fails but the same succeeds when invoked from the command line using valac (0.38.8) as follows:

valac --pkg=gtk+-3.0 --pkg=gee-0.8 --pkg=gxml-0.16 main.gs

There is no error from the above. I've tried setting up meson.build with gee and gxml as dependency and alternatively as vala_args. Same error.

Checking pkg-config, I get the following:

$ pkg-config --libs gxml-0.16
-L/usr/local/lib64 -lgxml-0.16 -lgio-2.0 -lxml2 -lgee-0.8 -lgobject-2.0 -lglib-2.0
$ pkg-config --libs gee-0.8
-lgee-0.8 -lgobject-2.0 -lglib-2.0
$ pkg-config --libs gee-1.0
-lgee -lgobject-2.0 -lglib-2.0

Perhaps I'm doing something wrong. Here is the local meson.build file followed by the top level meson.build and the error:

example_sources = [
  'main.gs'
]

example_deps = [
  dependency('gio-2.0', version: '>= 2.50'),
  dependency('gtk+-3.0', version: '>= 3.22'),
  dependency('glib-2.0', version: '>= 2.50')
]

gnome = import('gnome')

example_sources += gnome.compile_resources(
  'example-resources',
  'example.gresource.xml',
  c_name: 'example'
)

executable(
  'example',
  example_sources,
  vala_args: '--target-glib=2.50 --pkg=gee-0.8 --pkg=gxml-0.16',
  dependencies: example_deps,
  install: true
)

with top-level meson.build:

project(
  'example',
  ['c', 'vala'],
  version: '0.1.0',
  meson_version: '>= 0.40.0',
)

subdir('src')

And the error is:

uses Gee

error: The namespace name 'Gee' could not be found

I'm invoking the build from within Gnome-Builder. Can someone help me understand what is happening? I've tried to find why valac succeeds and meson fails in the documentation but cannot find a solution.

Dandify

Answer 1

Gee and GXml should be dependencies, just like GIO, GLib and GTK+. So you should try:

example_deps = [
  dependency('gio-2.0', version: '>= 2.50'),
  dependency('gtk+-3.0', version: '>= 3.22'),
  dependency('glib-2.0', version: '>= 2.50'),
  dependency('gobject-2.0'),
  dependency('gee-0.8'),
  dependency('gxml-0.16'),
  ]

Usually you won't need to go beyond that. This makes the --pkg options in the vala_flags unnecessary. Meson does that for you. The way Meson works is it uses valac to produce C code then in a separate stage uses a C compiler to produce the binary. By using --pkg you are only telling valac which VAPI file to use, but not notifying the C compiler which pkg-config package to use for the C library.

Also notice I've added gobject-2.0 as a dependency. If I remember correctly GNOME Builder misses that and it does affect the build.

The error message, error: The namespace name 'Gee' could not be found, is troubling. This is an error from the Vala compiler and I would have thought that the compiler would be able to find the VAPI file using the vala_args method you've tried. Maybe you have Gee built from source and not installed system wide?

Meson does allow another VAPI search directory to be added:

add_project_arguments(['--vapidir',
                       join_paths(meson.current_source_dir(), 'vapi')
                      ],
                      language: 'vala'
                     )

There are more details on the Vala page of the Meson Build documentation.

Genie support was added to Meson with version 0.42. So meson_version: should be >= 0.42.0.

If there are still problems then here is an MCVE using Genie, Gee and Meson. This should be compiled from the command line. Save the following Genie program as genie-gee.gs:

[indent=2]
uses Gee

init
  var my_list = new ArrayList of string()
  my_list.add( "one" )
  my_list.add( "two" )
  for item in my_list
    print( item )

Then save the following Meson file as meson.build:

project('minimal-genie-gee-example',
        'vala', 'c'
        )

genie_gee_deps = [
                dependency('glib-2.0'),
                dependency('gobject-2.0'),
                dependency('gee-0.8'),
                ]

executable('genie-gee',
           'genie-gee.gs',
           dependencies: genie_gee_deps
           )

From the command line use Meson to set up the build directory:

meson setup builddir

This should show the dependencies have been found, for example:

Native dependency gee-0.8 found: YES 0.18.0

Then use Ninja build to build the project:

ninja -C builddir

For anyone using Fedora ninja is ninja-build.

Any problems with Meson setting up the build directory are logged to builddir/meson-logs/meson-log.txt.

If this works, but it fails in GNOME Builder, then my only other thought is that GNOME Builder has been installed using Flatpak. The sandboxed environment of Flatpak may be affecting the access to dependencies.

Update: Following the discussion in the comments it appears the runtime used by GNOME Builder was the problem. Builder has a great feature of being able to select the Flatpak runtime used to build your software. If you are following the 'traditional' way of developing by installing libraries and header files on your workstation then make sure Host Operating System is selected instead of a Flatpak runtime. It would appear the GNOME Flatpak runtime does not include libgee.

Update2: When writing a Flatpak builder manifest and a dependency is not in the Flatpak runtime/SDK then add the dependency as another module in the Flatpak builder manifest. This allows GNOME Builder to use Flatpak to build the software with the Flatpak runtime. An example manifest is given in AsymLabs answer.

Antenatal

I'll have a look for alternative vapi locations, I tried also meson.get_compiler('vala').find_library('gee-0.8') within example_deps but this has not worked either. Again, it is found as valac --pkg=gee-0.8 however. – Dandify

Hmm, my understanding was Flatpak was a sandbox environment using OSTree. For system wide installs the files are under /var/lib/flatpak and acessed through a filesystem namespace where, for example, /usr in the namespace is bind mounted. GNOME Builder 3.26.4 seems to be packaged for many distributions now. It looks like Ubuntu has that version in Bionic Beaver. Could be worth trying to install using your distro rather than Flatpak. – Antenatal

Tried a number of different solutions as you suggested assuming flatpak has sandboxed Gnome-Builder and I also installed Gnome-Builder from the repo. The problem appears to be with the Build Preferences of Gnome-Builder. When changing from the project preferences org.gnome.example.json to the default it builds correctly every time, and this is true for both the repo and flatpak versions of Gnome-Builder. The default settings contain Configure Options --libdir=lib and Runtime Host Operating System. My json file may be the culprit. – Dandify

I've raised this as a usability issue with GNOME Builder: Issue 415 For a better understanding of runtimes the plugin documentation is useful: Runtimes and Subprocesses – Antenatal

The Flatpak (and Gnome Builder) developers perhaps should identify the Flatpack Manifest with a different name or file extension so that Gnome Builder users can more easily distinguish it's purpose and origin. Calling it a JSON file is too generic. For example, meson.build is clear, as well as .vala and .gs. But .json does not identify any context. Of course any user can do this but it would be better if the developers established a convention. – Dandify

Answer 2

Well after some exploration and AlThomas' advice above, here is what I discovered. OpenSUSE Tumbleweed provides four (or more) ways to install Gnome-Builder. These are:

1) Via Gnome Software Center. This installs org.gnome.Builder/stable in a sand boxed environment using Flatpak.

2) Via Flathub.org using Flatpak from the command line. This installs org.gnome.Builder/master (nightly) in a sand-boxed environment.

3) Via the package manager zypper and the command line. This installs a stable Gnome-Builder and related libraries system-wide.

4) Via Yast2. This provides the same as Zypper.

All three installations (same version 3.26.4 - different branches/tags - stable, master, nightly - two sand-boxed and one system wide) can be installed side by side and used as needed. During initial setup and testing, all variants yielded the same outcome - when using Gee and GXml only a Default build would work (the Flatpak Manifest would not build) but this has been resolved (it now appears that this is purely a Flatpak issue was a conflict between Flatpak and Fuse).

The Default build enables the Host runtime system. To set the Default build environment, upon opening a project within Gnome-Builder, choose Build Preferences from the upper left popover menu and select Default.

The drawback to a Default configuration is that it is not possible to Export Bundle, but local builds can utilize system-wide features.

So what is a Flatpak Manifest and why is it so important? It is the top level JSON file that contains project information. The Flatpak Manifest, in this case org.gnome.Example.json, pulls together all the features of the project so that it may be packaged for distribution. This includes the runtime, sdk, system connectivity to X11, IPC, Wayland, DBus, etc, the build system (Meson by default), cleanup directives, configuration and build options, submodule details (dependencies) and many other features. One Flatpak package can be installed in just about any Linux distribution, whether Debian, Ubuntu, Red Hat, OpenSuse or their derivatives, for example, and is sand-boxed for security and portability purposes. It will be, in future, fully cross-platform.

For instruction and testing, there are Flatpak Manifest examples to illustrate how they work. There are ways to alter the sand-box permissions using build finish directives. Flatpak documentation is excellent.

Within Gnome Builder when you first create a project, choose Vala + Gnome Application and a valid Flatpak Manifest will be installed. By default this is intended for a GUI rather than command line application; nonetheless it generates a default Flatpak Manifest that can be used as a template (Gnome Builder will allow multiple manifests - just select the build required). The following is the resulting improved Flatpak Manifest that will build submodules for both Gee and GXml (this has been tested within Gnome Builder and works):

{
  "app-id": "org.gnome.Example",
  "runtime": "org.gnome.Platform",
  "runtime-version": "master",
  "sdk": "org.gnome.Sdk",
  "command": "example",
  "finish-args": [
    "--share=network",
    "--share=ipc",
    "--socket=x11",
    "--socket=wayland",
    "--filesystem=xdg-run/dconf",
    "--filesystem=~/.config/dconf:ro",
    "--talk-name=ca.desrt.dconf",
    "--env=DCONF_USER_CONFIG_DIR=.config/dconf"
  ],
  "build-options": {
    "cflags": "-O2 -g",
    "cxxflags": "-O2 -g",
    "env": {
      "V": "1"
    }
  },
  "cleanup": [
    "/bin",
    "/include",
    "/lib",
    "/lib/pkgconfig",
    "/share",
    "/share/vala",
    "*.la",
    "*.a"
  ],
  "modules": [
    {
      "name": "libgee",
      "buildsystem": "meson",
      "config-opts": [
        "--libdir=lib"
      ],
      "builddir": true,
      "sources": [
        {
          "type": "git",
          "tag": "meson",
          "url": "https://github.com/GNOME/libgee.git"
        }
      ]
    },
    {
      "name": "libgxml",
      "buildsystem": "meson",
      "config-opts": [
        "--libdir=lib"
      ],
      "builddir": true,
      "sources": [
        {
          "type": "git",
          "branch": "master",
          "url": "https://gitlab.gnome.org/GNOME/gxml.git"
        }
      ]
    },
    {
      "name": "example",
      "buildsystem": "meson",
      "config-opts": [
        "--libdir=lib"
      ],
      "builddir": true,
      "sources": [
        {
          "type": "git",
          "url": "file:///home/<user>/Projects/example"
        }
      ]
    }
  ]
}

Hat's off to the folks who are developing this package. Combining Flatpak, Meson, Gtk3/4/5/.., Vala, Genie (and soon the Vulkan 3D graphics engine) and beautifully minimalistic UI guidlines/standards in one lightweight development platform is something magical, akin to a modern day alchemy.

As an aside, I tried using Gtk3 with a number of languages, including C/C++, D, Haskell and Python but none of these alternatives could produce stand-alone binaries that were as compact, efficient and fun to write as Vala and Genie. These are greatly underrated languages.

Concluding, anyone who needs a good starting point when trying to understand these technologies and how Gnome-Builder is bringing them together can read AlThomas' post above and this one, along with the comments. It may save a lot of time.

Dandify

My guess would be that you want libgee to be a module in the Flatpak manifest. Then it is built as part of the build with flatpak-builder – Antenatal

It is! I hope ;) The git repo is gitlab.gnome.org/GNOME/gxml.git and then you choose a tag like 0.16.2, but that is probably another StackOverflow question. – Antenatal

I would suggest opening another question on StackOverflow along the lines of "Unable to build dependencies for a Flatpak in GNOME Builder" with a simplified version of your app and its Flatpak manifest. Maybe someone will have the answer. For this SO question I think the answer is use the host runtime. From issue #316 it looks like "runtime" : "host" could be valid. – Antenatal