Packaging multiple Typescript projects depending on the same common local module

Question

I'm working on a set of VSTS extensions. Each extension is its own little Node project with its own package.json and its own node_modules folder. The folder structure is as follows:

 - MyExtension
   - package.json // containing all dev-dependencies
   - tslint.json
   - Tasks
     - tsconfig.json
     - Common
       - common.ts    // containing functioanlity shared across tasks
       - package.json // containing all runtime dependencies for all projects
     - My1stTask
       - package.json // containing all prod-dependencies
       - task.ts      // containing task implementation
     - ...
     - ...
     - My6thTask
       - package.json // containing all prod-dependencies
       - task.ts      // containing task implementation

The way VSTS build tasks work, is that they should be fully self-contained. I've fixed this so far by copying the contents of the Common project into each task and then run tsc to convert them all to JavaScript.

This isn't bad but requires constant copying of the contents of Common to get anything tested.

I tried using local file references, added a dependency in each task's package.json to file:../common, which works at development time, but this doesn't result in the common module being part of the task after generating the extension.

My background isn't in Node develpment, but in C#. I've searched all over and haven't found a solution that works well with vsts-extensions.

• npm pack doesn't seem to work, as the extension expects all files to be there.
• package.json/bundleDependencies looks promising but doesn't bundle the local file reference.
• ///<reference path="../common/common.ts"/> works lovely for editing but still can't run after building the extension.
• project reference with prepend doesn't work, build tasks require the commonjs module resolver. System and AMD aren't able to load the modules. Prepend only works with the latter.

Is there a way I can make this work "seamlessly" without having to take on bower or grunt and simply get each MyXthTask to have a copy of the local common module in their node_modules folder?

Tropopause

Answer 1

I tried @matt-mccutchen's approach, but unfortunately, I couldn't get that to work with the VSTS build tasks due to the fact that these tasks require commonjs:

"compilerOptions": {
  "module": "commonjs",
  "target": "es6", 

But I did find a solution that works for me.

In the Tasks folder I've added a tsconfig.json which defines my default settings and includes the files from the Common library:

{
  "compileOnSave": true,
  "compilerOptions": {
    "module": "commonjs",
    "target": "es6",
    "sourceMap": true,
    "strict": false,
    "strictNullChecks": false,
    "removeComments": true
  },
  "files": [
    "./Common/uuidv5.d.ts",
    "./Common/Common.ts"
  ]
}

Then in each task I created a tsconfig.json which sets the output folder to the current folder for that project and which inherits from the tsconfig.json in the Tasks folder:

{
  "extends": "../tsconfig.json",
  "compilerOptions": {
      "outDir": "./", 
      "sourceRoot": "./"
  },
  "files": [
      "InstallExtension.ts"
  ]
}

This results in:

 - MyExtension
   - package.json // containing all dev-dependencies
   - tslint.json
   - Tasks
     - tsconfig.json   // Including Common by default
     - Common
       - common.ts     // containing functionality shared across tasks
       - package.json  // containing all runtime dependencies for Common
       - tsconfig.json // containing build configuration for just the common files, inherits from ..\Task\tsconfig.json
     - My1stTask
       - package.json  // containing all prod-dependencies for the task
       - task.ts       // containing task implementation
       - tsconfig.json // containing build configuration for the task, inherits from ..\Task\tsconfig.json
     - ...
     - ...
     - My6thTask
       - package.json // containing all prod-dependencies
       - task.ts      // containing task implementation
       - tsconfig.json // containing build configuration for the task, inherits from ..\Task\tsconfig.json

When compiling a task the following

     - My6thTask
       - Common
         - Common.js   // Compiled common
       - My6thTask
         - task.js     // Compiled task
       - package.json  // containing all prod-dependencies
       - task.ts       // containing task implementation
       - task.json     // defining the task UI
       - tsconfig.json // containing build configuration for the task

The only thing I had to add to task.ts is the following:

///<reference path="../Common/Common.ts"/>
import * as common from "../Common/Common";

And to change the execution handler in the task.json to point to the new location:

  "execution": {
    "Node": {
      "target": "InstallExtension/InstallExtension.js", // was: InstallExtension.js
      "argumentFormat": ""
    }
  }

And all seems fine :D. Combined with using glob-exec I've been able to reduce the build time to less than a minute when building clean:

"initdev:npm": "npm install & glob-exec --parallel --foreach \"Tasks/*/tsconfig.json\" -- \"cd {{file.dir}} && npm install\"",
"compile:tasks": "glob-exec \"Tasks/*/tsconfig.json\" -- \"tsc -b {{files.join(' ')}}\"",
"lint:tasks": "glob-exec --parallel --foreach \"Tasks/*/tsconfig.json\" -- \"tslint -p {{file}}\"",

Tropopause

Answer 2

You may be able to use project references with the prepend option if you are satisfied to generate a single file as output for each task (and use a compatible module loader) instead of generating individual modules.

If you need multi-file output, see this question for a TypeScript suggestion and a potential workaround.

Starling

That looks like an option... It would require me to have a tsconfig in each task folder, wouldn't it? I suspect I could fix that by using extends typescriptlang.org/docs/handbook/tsconfig-json.html I'll have to give this a try. – Tropopause

Hmmm, looks like I can't combine outFile + Composite with "compilerOptions": { "module": "commonjs" } – Tropopause

Let's try a different approach. What tool do you use to pack up the extension? When you depend on ../common in package.json, you should get a symlink from node_modules/common to the common dir. Given that the tool is packing up the rest of your node_modules, can you get it to follow that symlink? – Starling

I may have found a solution using the tsconfig.json and output path. In essence building the common module as many times as I have tasks and then outputting the results to each task folder. – Tropopause

Structure can be found here: github.com/Microsoft/vsts-extension-build-release-tasks/tree/… – Tropopause

Not a fan of symlinks. They cause unexpected behaviour for people and are an optional feature for Git. What do you think of my approach? – Tropopause

Your approach of using tsconfig.json inheritance to add the files to the compilation is clever, but ideally in a general solution, one wouldn't have to compile the same files many times. I'd be curious if you have any specific complaints about symlinks beyond the poor support on Windows, since that otherwise seems like the best general solution to me. – Starling

Well, my personal development platform is Windows, the build servers I use are windows and I don't have control over the git symlink support on them. And so far my experience is, that the fs libraries in node tend to not understand many filesystem features on windows. And I'm sick of all the copy-around-scripts I had before. – Tropopause

Answer 3

Project references does not work good in my case so I used webpack with ts-loader to build my projects.

Place all ts code under single root with single tsconfig so it will be like this.

ts
 -core
 -plugin1
 -plugin2

Since 3.10 webpack allows multiple output configurations. So we could use single config file like this.

const path = require("path");

const commonTsRule = {
  test: /\.tsx?$/,
  use: "ts-loader",
  exclude: /node_modules/,
};

const commonConfig = {
  devtool: "inline-source-map",
  mode: "development",
  resolve: {
    extensions: [".tsx", ".ts", ".js"],
  },
};

module.exports = [
  {
    ...commonConfig,
    entry: `${path.resolve(__dirname, "ts")}/plugin1/index.ts`,
    output: {
      filename: "bundle.js",
      path: path.resolve(__dirname, "build/plugin1/js"),
    },
    module: {
      rules: [
        {
          ...commonTsRule,
          // here you can customize rule if required
        },
      ],
    },
  },
  {
    ...commonConfig,
    entry: `${path.resolve(__dirname, "ts")}/plugin2/index.ts`,
    output: {
      filename: "bundle.js",
      path: path.resolve(__dirname, "build/plugin2/js"),
    },
    module: {
      rules: [
        {
          ...commonTsRule
          // here you can customize rule if required
        },
      ],
    },
  },
];

So as final result you will have each project built from its own entry point and to its own output destination.

Putrescent