Using FsLex/Yacc in Vs2013

Question

I'm trying to resurrect an old f# parser project I had working in vs 2008 to work with vs 2013. It uses FsLexYacc.

I got it building ok by using a prebuild step as thus:

fslex --unicode "$(ProjectDir)XpathLexer.fsl"
fsyacc --module XpathParser "$(ProjectDir)XpathParser.fsy"

But this is less than ideal, as it always executes whether or not the inputs have changed.

I then tried just using the old MsBuild actions:

<FsYacc Include="XpathParser.fsy">
<FsLex Include="XpathLexer.fsl">

but these appeared to be completely ignored during the build process. Is that right? Have these build tasks been removed somehow?

I then found some stuff documented under vs C++ that I thought might work:

<CustomBuild Include="XpathParser.fsy">
  <Message>Calling FsYacc</Message>
  <Command>fsyacc --module XpathParser "$(ProjectDir)XpathParser.fsy"</Command>
  <Outputs>$(ProjectDir)XpathParser.fs</Outputs>
</CustomBuild>

and

<PropertyGroup>
    <CustomBuildBeforeTargets>CoreCompile</CustomBuildBeforeTargets>
</PropertyGroup>

(I inspected the Microsoft.Fsharp.Targets file to come up with the "CoreCompile" target.)

Alas, still no cigar.

Is anyone able to shine a light on whether it is indeed possible to properly integrate fslex/yacc into a vs 2013 solution, and if so, how?

Aric

Answer 1

I don't think the those tools are included by default with the F# compiler that is installed with Visual Studio and so the tasks don't exist. I did the following with a Visual Studio 2012 project, but I expect it would be similar in VS 2013. Here were the steps I had to follow:

1. Install FSharp.Powerpack from nuget. This has the fslex and fsyacc tools as well as build tasks and targets.
2. Unload the project and edit the .fsproj file.

3. Add an import statement for the FSharp.Powerpack.target file. This will add the CallFsLex and CallFsYacc build targets. I added this after the import for Microsoft.FSharp.targets:
   <Import Project="$(ProjectDir)\..\packages\FSPowerPack.Community.3.0.0.0\Tools\FSharp.PowerPack.targets" />

4. Add these three properties to main PropertyGroup at the top of the file: <FsYaccToolPath>..\packages\FSPowerPack.Community.3.0.0.0\Tools</FsYaccToolPath> <FsLexToolPath>..\packages\FSPowerPack.Community.3.0.0.0\Tools</FsLexToolPath> <FsLexUnicode>true</FsLexUnicode> This tells the build tasks where to find the necessary tools and sets the unicode option for fslex.

5. To use the targets we've imported, you need to define the FsLex and FsYacc item groups with the input files to use. You also need to add Compile items for the output .fs files. You end up with something like this in an ItemGroup section:
   <Compile Include="Sql.fs" />
<FsYacc Include="SqlParser.fsp">
<Module>SqlParser</Module>
</FsYacc>
<Compile Include="SqlParser.fsi" />
<Compile Include="SqlParser.fs" />
<FsLex Include="SqlLexer.fsl" />
<Compile Include="SqlLexer.fs" />

You might be able to use the FsLex and FsYacc build tasks directly by referencing the FSharp.Powerpack.Build.Tasks.dll, but for me this was easier to get going.

Tenrec

Thanks for that mike. Alas, I still couldn't get it to work. The build just completely ignores the lex/yacc inputs. I tried referencing the FSharp.Powerpack.Build.Tasks.dll directly, but still no cigar. – Aric

that seems in line with my experience – Charqui

@Mike z - This worked for me on VS2013 with 1 extra addition - I had to copy FSharp.Core (4.3.0.0) from C:\Program Files (x86)\Reference Assemblies\Microsoft\FSharp\.NETFramework\v4.0\4.3.0.0 local to fslex. I'm sure theres a better way using redirects and the fslex.exe.config, but this worked for me and I think I'll not question it for now! – Cavalierly

Answer 2

This is what works for me (Windows 7 x64, Visual Studio 2013 Ultimate RTM):

1. Get and install "PowerPack for FSharp 3.0 + .NET 4.x + VS2012" from CodePlex (https://fsharppowerpack.codeplex.com/downloads/get/625449)

2. Create the following Registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\AssemblyFolders\FSharp.PowerPack-1.9.9.9 (for x64 versions of Windows, omit the Wow6432Node for 32bit versions) and set its (Default) value to the installation directory of the F# PowerPack (e.g. "C:\Program Files (x86)\FSharpPowerPack-4.0.0.0\bin"). [This is related to a long standing/regression bug in src/FSharp.PowerPack/CompilerLocationUtils.fs which basically breaks tool discovery.]

3. Import the PowerPack targets (AFTER importing the F# targets) in your *.fsproj file: <Import Project="$(MSBuildExtensionsPath32)\FSharp\1.0\FSharp.PowerPack.targets" />

4. Update your ItemGroup node to something like this (use FsYacc accordingly):

   <ItemGroup>
     <None Include="App.config" />
     <FsLex Include="Lexer.fsl" />
     <Compile Include="Lexer.fs">
       <Visible>False</Visible>
     </Compile>
     <Compile Include="Program.fs" />
   </ItemGroup>
   

5. Include a reference to FSharp.PowerPack.dll and build.

You should end up with a *.fsproj file similar to this:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <SchemaVersion>2.0</SchemaVersion>
    <ProjectGuid>8c565f99-d6bc-43a9-ace9-eadfe429c0f7</ProjectGuid>
    <OutputType>Exe</OutputType>
    <RootNamespace>FsYaccTest</RootNamespace>
    <AssemblyName>FsYaccTest</AssemblyName>
    <TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
    <TargetFSharpCoreVersion>4.3.1.0</TargetFSharpCoreVersion>
    <Name>FsYaccTest</Name>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  <!-- Snip -->
  </PropertyGroup>
  <ItemGroup>
    <Reference Include="FSharp.PowerPack">
      <HintPath>C:\Program Files (x86)\FSharpPowerPack-4.0.0.0\bin\FSharp.PowerPack.dll</HintPath>
    </Reference>
    <Reference Include="mscorlib" />
    <Reference Include="FSharp.Core, Version=$(TargetFSharpCoreVersion), Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
      <Private>True</Private>
    </Reference>
    <Reference Include="System" />
    <Reference Include="System.Core" />
    <Reference Include="System.Numerics" />
  </ItemGroup>
  <PropertyGroup>
    <MinimumVisualStudioVersion Condition="'$(MinimumVisualStudioVersion)' == ''">11</MinimumVisualStudioVersion>
  </PropertyGroup>
  <Choose>
    <When Condition="'$(VisualStudioVersion)' == '11.0'">
      <PropertyGroup Condition="Exists('$(MSBuildExtensionsPath32)\..\Microsoft SDKs\F#\3.0\Framework\v4.0\Microsoft.FSharp.Targets')">
        <FSharpTargetsPath>$(MSBuildExtensionsPath32)\..\Microsoft SDKs\F#\3.0\Framework\v4.0\Microsoft.FSharp.Targets</FSharpTargetsPath>
      </PropertyGroup>
    </When>
    <Otherwise>
      <PropertyGroup Condition="Exists('$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\FSharp\Microsoft.FSharp.Targets')">
        <FSharpTargetsPath>$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\FSharp\Microsoft.FSharp.Targets</FSharpTargetsPath>
      </PropertyGroup>
    </Otherwise>
  </Choose>
  <Import Project="$(FSharpTargetsPath)" />  
  <Import Project="$(MSBuildExtensionsPath32)\FSharp\1.0\FSharp.PowerPack.targets" />  
  <PropertyGroup>
    <FsLexUnicode>true</FsLexUnicode>
  </PropertyGroup>
  <ItemGroup>
    <None Include="App.config" />
    <FsLex Include="Lexer.fsl" />
    <Compile Include="Lexer.fs">
      <Visible>False</Visible>
    </Compile>
    <Compile Include="Program.fs" />
  </ItemGroup>
</Project>

Note: You can probably omit creating the Registry key if you provide a proper FsYaccToolPath as described in mike z's answer.

Degeneracy

Thanks for that. Just skimming your answer makes one realize why the Fsharp.Powerpack is being abandoned - yes? Truly baroque, bizarre even. I find it sad that F# continues to languish due to silly issues such as this. Evidently, M$ is simply not serious about F#. – Aric

The F# PowerPack has moved from CodePlex to GitHub (github.com/fsharp/powerpack), but even there isn't much activity. Most functionality is now developed in different projects, while FsYacc and FsLex seem to be abandoned (though they are probably still used in the F# compiler itself which obviously still gets updates). While I haven't found an official statement anywhere, Microsoft currently seems to put more effort into C++/Win8 than into .Net in general. – Degeneracy

Answer 3

This looks like it works - at least, in my experience, if you use the separate FsLexYacc nuget package as detailed here, and then put the following in your fsproj file (extracted from the github example):

Next to all the other imports:

<Import Project="..\packages\FsLexYacc.6.0.4\bin\FsLexYacc.targets" />

etc, etc

and then for the source files:

<FsYacc Include="Parser.fsp">
  <OtherFlags>--module SqlParser</OtherFlags>
</FsYacc>
<FsLex Include="Lexer.fsl">
  <OtherFlags>--unicode</OtherFlags>
</FsLex>

No need to do anything apart from edit the fsproj file, and install the nuget packages.

Stowers