Convert my .g4 to .flex and .bnf for IDEA syntax highlighter
Asked Answered
T

0

7

I have never used JFlex before, and I have no idea how it works. Basically, I've built a runtime for a scheme-esque language in Java, and the parser I have for it was generated using Antlr 4, so I have a .g4 file, which looks like this:

program: expression EOF;
expression: lambdaAbstraction | application| literal;

lambdaAbstraction: LP WS? LAMBDA (WS IDENTIFIER)* WS expression WS? RP;
application: LP WS? expression (WS expression)* WS? RP;
literal: FLOAT_LITERAL | INTEGER_LITERAL | IDENTIFIER | STRING_LITERAL;

STRING_LITERAL: '"' ~["]* '"' | '\'' ~[']* '\'' ;
FLOAT_LITERAL: [0-9]+ '.' [0-9]+;
INTEGER_LITERAL: [0-9]+;
IDENTIFIER: [a-zA-Z+/*-=?@]+;
LAMBDA: '\\';
LP: '(';
RP: ')';
WS: [ \n\r\t]+;

Some examples (so you can get a feel for the grammar).

  1. (+ 1 2) = 3

  2. ((\ a b (+ a b)) 1 2) = 3

  3. (+ "Hello, " "World") = "Hello, World"

What I'm trying to do is build a syntax-highlighter plugin for IntelliJ-IDEA for this language using this guide: http://www.jetbrains.org/intellij/sdk/docs/tutorials/custom_language_support_tutorial.html.

Here is my .flex file

package com.michaelsnowden.yalcil_plugin;

import com.intellij.lexer.FlexLexer;
import com.intellij.psi.tree.IElementType;
import com.michaelsnowden.yalcil_plugin.psi.YalcilTypes;
import com.intellij.psi.TokenType;

%%

%class YalcilLexer
%implements FlexLexer
%unicode
%function advance
%type IElementType
%eof{  return;
%eof}

STRING_LITERAL= "\"" [^\"&]* "\"";
FLOAT_LITERAL=[0-9]+ '.' [0-9]+;
INTEGER_LITERAL=[0-9]+;
IDENTIFIER=[a-zA-Z+/*-=?@]+;
LAMBDA="\\";
LP="(";
RP=")";
WS=[ \n\r\t]+;
%%

<YYINITIAL> {
    {LAMBDA} { yybegin(YYINITIAL); return YalcilTypes.LAMBDA; }
    {FLOAT_LITERAL} { yybegin(YYINITIAL); return YalcilTypes.LAMBDA; }
    {INTEGER_LITERAL} { yybegin(YYINITIAL); return YalcilTypes.INTEGER_LITERAL; }
    {IDENTIFIER} { yybegin(YYINITIAL); return YalcilTypes.IDENTIFIER; }
    {LP} { yybegin(YYINITIAL); return YalcilTypes.LP; }
    {RP} { yybegin(YYINITIAL); return YalcilTypes.RP; }
    {WS} { yybegin(YYINITIAL); return YalcilTypes.WS; }
    {STRING_LITERAL} { yybegin(YYINITIAL); return YalcilTypes.STRING_LITERAL; }
}

. { return TokenType.BAD_CHARACTER; }

And here is my .bnf file

{
  parserClass="com.michaelsnowden.yalcil_plugin.YalcilParser"

  extends="com.intellij.extapi.psi.ASTWrapperPsiElement"

  psiClassPrefix="Yalcil"
  psiImplClassSuffix="Impl"
  psiPackage="com.michaelsnowden.yalcil_plugin.psi"
  psiImplPackage="com.michaelsnowden.yalcil_plugin.psi.impl"

  elementTypeHolderClass="com.michaelsnowden.yalcil_plugin.psi.YalcilTypes"
  elementTypeClass="com.michaelsnowden.yalcil_plugin.psi.YalcilElementType"
  tokenTypeClass="com.michaelsnowden.yalcil_plugin.psi.YalcilTokenType"
}

program ::= expression;
expression ::= lambdaAbstraction | application| literal;
lambdaAbstraction ::= LP WS? LAMBDA (WS IDENTIFIER)* WS expression WS? RP;
application ::= LP WS? expression (WS expression)* WS? RP;
literal ::= FLOAT_LITERAL | INTEGER_LITERAL | IDENTIFIER | STRING_LITERAL;

I get basically nothing but errors when I run this.

enter image description here

For some reason, only strings are highlighted.

enter image description here

The PsiViewer looks like this:

enter image description here

Does anyone see any explanation for why this isn't working?

Tootsie answered 30/4, 2016 at 4:44 Comment(3)
Since you're using ANTLR4, you may want to replace JFlex and Grammar-Kit with this: github.com/antlr/jetbrainsLello
@BastienJansen Wow, that looks perfect, thanks!Tootsie
@michaelsnowden, thanks for your question. I am having a similar question. However, I do not see how ANTLR4 can help here. You still need a Flex file to build the lexers right? Or if you are using ANTLR4, how can that be used in building the Intellij Plugin?Formulaic

© 2022 - 2024 — McMap. All rights reserved.