Swift package manager unable to compile ncurses installed through Homebrew

Question

I'm trying to use ncurses in a library using Swift Package Manager and I'd like to use a specific version of ncurses, not the one included in OS X. To do so I installed a more recent version (6.1) using Homebrew. This is how my Package.swift looks like:

// swift-tools-version:5.0
import PackageDescription

let package = Package(
    name: "NcursesExample",
    products: [
        .executable(name: "NcursesExample", targets: ["NcursesExample"]),
    ],
    dependencies: [
    ],
    targets: [
        .systemLibrary(name: "Cncurses"),
        .target(name: "NcursesExample", dependencies: ["Cncurses"]),
    ]
)

Under the Sources directory I have a subdirectory for Cncurses containing a module.modulemap and shim.h files:

module.modulemap

module Cncurses {
  header "shim.h"
  link "ncurses"
  export *
}

shim.h

#include "/usr/local/Cellar/ncurses/6.1/include/ncurses.h"

However, when compiling I get several errors complaining about conflicting types, apparently because ncurses is also provided by the macOS SDK:

shim.h:1:10: note: in file included from shim.h:1:
#include "/usr/local/Cellar/ncurses/6.1/include/ncurses.h"
         ^
/usr/local/Cellar/ncurses/6.1/include/ncurses.h:60:10: error: 'ncursesw/ncurses_dll.h' file not found with <angled> include; use "quotes" instead
#include <ncursesw/ncurses_dll.h>
         ^
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "shim.h"
        ^
shim.h:1:10: note: in file included from shim.h:1:
#include "/usr/local/Cellar/ncurses/6.1/include/ncurses.h"
         ^
/usr/local/Cellar/ncurses/6.1/include/ncurses.h:674:45: error: conflicting types for 'keyname'
extern NCURSES_EXPORT(NCURSES_CONST char *) keyname (int);              /* implemented */
                                            ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/curses.h:598:45: note: previous declaration is here
extern NCURSES_EXPORT(NCURSES_CONST char *) keyname (int);              /* implemented */

...

I'm trying to compile the package using:

swift build -Xcc -I/usr/local/Cellar/ncurses/6.1/include/ -Xlinker -L/usr/local/Cellar/ncurses/6.1/lib

I also went down the route of specifying the pkgConfig on the package definition, with the same result. Can someone help?

FYI It's worth mentioning that upon ncurses installation through Homebrew I get the following warning as ncurses is already provided by OS X:

ncurses is keg-only, which means it was not symlinked into /usr/local,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have ncurses first in your PATH run:
  echo 'export PATH="/usr/local/opt/ncurses/bin:$PATH"' >> ~/.zshrc

For compilers to find ncurses you may need to set:
  export LDFLAGS="-L/usr/local/opt/ncurses/lib"
  export CPPFLAGS="-I/usr/local/opt/ncurses/include"

For pkg-config to find ncurses you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/ncurses/lib/pkgconfig"

Pkgconfig for ncurses looks like this

# pkg-config file generated by gen-pkgconfig
# vile:makemode

prefix=/usr/local/Cellar/ncurses/6.1
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include/ncursesw
abi_version=6
major_version=6
version=6.1.20180127

Name: ncursesw
Description: ncurses 6.1 library
Version: ${version}
URL: https://invisible-island.net/ncurses
Requires.private:
Libs:  -L${libdir} -lncursesw
Libs.private:
Cflags:  -D_DARWIN_C_SOURCE -I/usr/local/Cellar/ncurses/6.1/include -I${includedir}

Businesswoman

What is in your /usr/local/opt/ncurses/lib/pkgconfig file? You likely need other compiler flags to be passed in as well. – Conservatism

Edited my question to include the contents of the pkgconfig. – Businesswoman

I'm curious, could this be related to the use of ncurses vs ncursesw? – Businesswoman

That is likely the issue. Does it work if you add the Cflags and Libs flags to your swift build or modulemap's link command? – Conservatism

No, doesn't work even with the Cflags/Libs specified. – Businesswoman

Answer 1

Problem

The main problem are conflicts of the header files, since ncurses is also supplied in /Applications/Xcode.app/.../MacOSX10.14.sdk/usr/include.

A common pure C solution in such situations is to just specify the custom include and lib directories with -I repective -L and it would work, see my answer regarding C ncurses here: https://mcmap.net/q/1319136/-clang-on-macos-fails-linking-lmenu-from-ncurses

This approach does not seem to work with the Swift package manager. But that doesn't mean it's not possible with a little effort.

Possible Solution

We need to make sure that the ncurses header files provided by the macOS SDK are ignored. We can do this by specifying the -Xcc -D__NCURSES_H parameter for the swift build command.

This works because in the header file, there is this typical:

#ifndef __NCURSES_H
#define __NCURSES_H
...
#endif

The problem, of course, is that our custom installation of ncurses using Brew is also affected. But we can work around it:

• copy the new ncurses header files into our Sources/Cncurses directory
• replace __NCURSES_H through something different, e.g. __CNCURSES_H (note leading 'C')
• then make sure that all further nested includes are first searched in our local include directory by replace the angle brackets of the includes with quotes, so e.g. instead of #include <ncursesw/unctrl.h> the form '#include "ncursesw/unctrl.h"' is used

This can actually be done with the following command line commands:

cd Sources/Cncurses
cp -r /usr/local/Cellar/ncurses/6.1/include include
find . -name '*.h' -exec sed -i ''  's/__NCURSES_H/__CNCURSES_H/g' {} \;
find . -name '*.h' -exec sed -i '' -E -e "s/<(.*(`find . -name '*.h' -exec basename {} \; |  paste -sd "|" -`))>/\"\1\"/g"  {} \;

The last statement may require some explanation. With the help of an echo command, you can look at the generated sed expression, i.e. if you execute

echo "s/<(.*(`find . -name '*.h' -exec basename {} \; |  paste -sd "|" -`))>/\"\1\"/g" 

you get the following output:

s/<(.*(termcap.h|form.h|term.h|panel.h|ncurses.h|termcap.h|cursesp.h|cursesf.h|etip.h|form.h|cursesw.h|nc_tparm.h|unctrl.h|cursesapp.h|term.h|cursslk.h|panel.h|ncurses.h|tic.h|eti.h|ncurses_dll.h|term_entry.h|menu.h|cursesm.h|curses.h|curses.h|cncurses.h))>/"\1"/g

As you can see, it searches and replaces only local available include files.

Test

For a test we need a simple ncurses example program. It should be built and we should make sure that the correct version of the library is used.

module.modulemap

My header file is called cncurses.h. The module.modulemap looks like this:

module cncurses [system] 
{
    umbrella header "cncurses.h"
    link "ncurses"
    export *
}

cncurses.h

cncurses.h is a one-liner, it imports our copied and customized ncurses.h file from our local include folder:

#include "include/ncurses.h"

main.swift

In the NcursesExample folder we have main.swift where we have a simple cncurses swift app:

import cncurses

initscr()
curs_set(0) 
move(5, 10)
addstr("NCURSES")
move(10, 10)
addstr("Hello World!")
refresh() 

select(0, nil, nil, nil, nil)

Package.swift

Please note here the pkgConfig: "ncurses" in the systemLibrary targets:

// swift-tools-version:5.0
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "NcursesExample",
    dependencies: [
    ],
    targets: [
        .systemLibrary(name: "cncurses", pkgConfig: "ncurses"),
        .target(name: "NcursesExample", dependencies: ["cncurses"]),
        .testTarget(
            name: "NcursesExampleTests",
            dependencies: ["NcursesExample"]),

    ]
)

Build

For pkg-config to do its job properly, we must first call the following:

export PKG_CONFIG_PATH="/usr/local/opt/ncurses/lib/pkgconfig"

Finally we initiate the build with:

swift build -Xcc -D__NCURSES_H 

So first we should test if the correct ncurses lib was used. We can do that with:

otool -L .build/x86_64-apple-macosx/debug/NcursesExample

Among other lines, the output contains this:

/usr/local/opt/ncurses/lib/libncursesw.6.dylib (compatibility version 6.0.0, current version 6.0.0)

which looks promising.

Finally calling the binary:

Xcode Project

If you want to generate a Xcode project, use the following command:

swift package generate-xcodeproj

Then load the project in Xcode and

• select the project node
• in Build settings enter Preprocessor in the search field in the upper right
• under Apple Clang - Preprocessing / Preprocess Macros add __NCURSES_H=1 for Debug and Release

Whitsuntide

Awesome this is exactly what I was looking for, thanks! Out of curiosity why isn't the -I flag working when Swift Package Manager allows passing those flags to the C compiler (is it just a bug)? Also can you advise on how to set up Xcode in order to compile the package, instead of having to go through the command line? – Businesswoman

I've added Xcode project generation to the answer. Regarding -I flag I'm note sure, it seems it's not supported, perhaps because some default packages are loaded? Anyway if you want to dig deeper: with swift build --verbose ... one can see the swiftc command line... – Whitsuntide

Wow, this seems somewhat overkill (in some aspects), but I'll try using this approach with my PoC project (from this question: #56567410 ). I'm not 100% sure about how to apply some specifics to my project, but in case I have some doubts I'll ask @StephanSchlecht who seems to know things :) So eager to try it!!. I want my blink effect back, my 16 colors and all my expected standard fancy effects now !! – Electrophoresis

I historically dislike Brew and am a fan of MacPorts, so I'm trying to adapt your solution to my MacPorts installation... I managed to copy my ncurses.h and ncurses_dll.h (/opt/local/include/ncurses.h /opt/local/include/ncurses_dll.h) into some Source/cncurses folder I created by hand, but from there I get somewhat lost. with some unknown files to me: I have no idea about the required path to deploy them... module.modulemap? cncurses.h ? Never had to deal with this way of using swift compiler, so I'm 100% newbie and I can't really follow you... Still trying to figure out the rest of the stuff – Electrophoresis

@StephanSchlecht I can get it to compile on the command line, but not in Xcode with the instructions you provided. – Businesswoman

Conflicting nurses types, feels like the __NCURSES_H=1 flag ins't being applied properly maybe? Also, do I need to run script phase to set the PKG_CONFIG_PATH? Thanks. – Businesswoman

on the command line you made the call export PKG_CONFIG_PATH="/usr/local/opt/ncurses/lib/pkgconfig"and afterwards swift package generate-xcodeproj? And then you selected in Xcode on the left 'Project' and on the right Build Settings? Added __NCURSES_H=1 to preprocess Macros? Then it should work. Perhaps clean project and remove /Users/<username>/Library/Developer/Xcode/DerivedData and try again. – Whitsuntide

Working - great! Now I just need to understand why functions such as init_pair aren't imported to Swift, although that's a different fight... Thanks for your help! @StephanSchlecht – Businesswoman

Answer 2

I'm not too well-versed in this subject, but it seems like you may want to look into adding a build script for this to your project. In Xcode, under the selected project, go to Build Phases > New Run Script Phase

In that script add your flags:

export LDFLAGS="-L/usr/local/opt/ncurses/lib"
export CPPFLAGS="-I/usr/local/opt/ncurses/include"

The build script should run on install. I'm not sure if you need the pkg-config flag here. Hope this helps point you somewhere in the right direction.

Sainted

This didn't work. Also, I should be able to build on the command line. I had tried swift build -Xcc -I/usr/local/Cellar/ncurses/6.1/include/ -Xlinker -L/usr/local/Cellar/ncurses/6.1/lib before but without results. – Businesswoman

Answer 3

I would change

#include "/usr/local/Cellar/ncurses/6.1/include/ncurses.h"

to

#include <ncurses.h>

because you already have

-I/usr/local/Cellar/ncurses/6.1/include/

Reading the formula, you should have

/usr/local/Cellar/ncurses/6.1/include/ncursesw/ncurses_dll.h

(and some other stuff), but using a quoted-include can interfere with the search path which the -I option sets up (see this for example).

Longterm