Gradle can't find Antlr token file

Question

I created a file MyLexer.g4 inside myproject/src/main/antlr/com/mypackage like:

lexer grammar MyLexer;

DIGIT : '0' .. '9' ;

...

WS  : [ \t\r\n]+ -> skip ;

and then trying to write parser in MyParser.g4 in the same directory:

grammar MyParser;

options
   { tokenVocab = MyLexer; }

SHORT_YEAR: DIGIT DIGIT;

unfortunately, when I run gradle task of generateGrammarSource, the following error occurs:

error(160): com\mypackage\MyParser.g4:4:18: cannot find tokens file MYPROJECT\build\generated-src\antlr\main\MyLexer.tokens

I.e. file is sought in incorrect place.

Actual file is created inside MYPROJECT\build\generated-src\antlr\main\com\mypackage\MyLexer.tokens

Vaporetto

Did you find a solution to this problem? – Schnozzle

Answer 1

As Steven Spungin said, you need to put your ANTLR source files in the directory src/main/antlr/ and not in a subdirectory of that directory.

You do not need to add the @header to your ANTLR source files. Use the following instead.

In your build.gradle you should have (modified for your version of ANTLR):

apply plugin: 'antlr'

dependencies {
    antlr "org.antlr:antlr4:4.7.1"
}

generateGrammarSource {
    arguments += ['-package', 'com.mypackage']
    outputDirectory = new File(buildDir.toString() + "/generated-src/antlr/main/com/mypackage/")
}

Afoot

oh god, you save my life,I have been searching for this all-day – Vaporization

Answer 2

When generating your parser in a package by using:

@header {package org.acme.my.package;}

and declaring tokenVocab in your parser

options {tokenVocab = MyLanguage;}

The MyLanguageLexer.g4 and MyLanguageParser.g4 files must NOT BE in a package directory. due to a bug of sorts.

So this means /src/main/antlr/MyLanguageParser.g4 and not /src/main/antlr/com/acme/my/package/MyLanguageParser.g4.

The java files end up in the wrong directory in build/generated-src/antlr, but somehow make it to the correct directory in build/classes/java/main. And the .tokens file ends up where antlr expects it.

---

Keep in mind this will confuse your IDE; I had to add the compiled classes back to my compileClasspath to avoid visual class-not-found issues.

dependencies {
    testCompile fileTree('build/classes/java/main')
}

Thunderstone

Answer 3

As what suggested above, grammar file with package definition should be put under src/main/antlr folder. Once you compile it, it will generate files in the output folder. Then you can do the file move as what's desired based on the package path. Here is an relevant link how to move files based on your package

Generating ANTLR4 grammar files with package declaration in gradle

In the doLast method, it calls moveAntlrGeneratedFilesToTheirPackages to move the files to right folder to align with the package path.

Streetman