Using Vega Charts in an Ionic App causes runtime errors in launching on some devices
Asked Answered
L

1

7

Much to my chagrin, I've discovered that an Ionic 4 app that I've developed and tested successfully on my Android (8.0) phone, as well as on an iPhone, freezes on the splash screen on an Android (8.1) tablet and crashes during launch on an iPad. Using adb logcat diagnostic techniques, I observed that on the errant Android tablet, a Syntax Error was being reported in vendor-es5.js, which when I dug into the www folder of my project and went to the referenced line of the error, which said SyntaxError: Unexpected token *, I landed in code that clearly came from node_modules/d3-delaunay/src/delaunay.js and that used the es6 exponentiation operator **, specifically:

r = 1e-8 * Math.sqrt((bounds[3] - bounds[1])**2 + (bounds[2] - bounds[0])**2);

I don't know why this code is problematic on some devices, nor do I know what is causing this code, which is not es5 (?) to end up in the vendor-es5.js file without being transpiled appropriately. To take it a step further, I manually hacked that delaunay.js file to replace all the instances of exponentiation with their equivalent uses of Math.pow() and sure enough, the runtime got further, but eventually ran aground again in a function that came from node_modules/vega-dataflow/src/dataflow/load.js and complained that SyntaxError: Unexpected token function, specifically on this line:

export async function request(url, format) {

Again, obviously async/await is not an es5 construct, so why is it ending up in vendor-es5.js. At this point, I feel like something is systematically wrong here, and I'm not equipped to understand how to overcome it short of maybe switching graphing libraries? I'd like to avoid that if possible, so my questions are:

  1. Why is this happening?
  2. Why does it only have an impact on some, and not all, devices?
  3. Is there a way that I can work around the problem without switching to a different graphing library?

Update #1

Since it's an Ionic4 project, that means it's an Angular 8 project, and that means it's a Webpack project (as in the defaults for the platform). So here's my angular.json file:

{
  "$schema": "./node_modules/@angular/cli/lib/config/schema.json",
  "version": 1,
  "defaultProject": "app",
  "newProjectRoot": "projects",
  "projects": {
    "app": {
      "root": "",
      "sourceRoot": "src",
      "projectType": "application",
      "prefix": "app",
      "schematics": {},
      "architect": {
        "build": {
          "builder": "@angular-devkit/build-angular:browser",
          "options": {
            "outputPath": "www",
            "index": "src/index.html",
            "main": "src/main.ts",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "tsconfig.app.json",
            "assets": [
              {
                "glob": "**/*",
                "input": "src/assets",
                "output": "assets"
              },
              {
                "glob": "**/*.svg",
                "input": "node_modules/ionicons/dist/ionicons/svg",
                "output": "./svg"
              }
            ],
            "styles": [
              {
                "input": "src/theme/variables.scss"
              },
              {
                "input": "src/global.scss"
              }
            ],
            "scripts": []
          },
          "configurations": {
            "production": {
              "fileReplacements": [
                {
                  "replace": "src/environments/environment.ts",
                  "with": "src/environments/environment.prod.ts"
                }
              ],
              "optimization": true,
              "outputHashing": "all",
              "sourceMap": false,
              "extractCss": true,
              "namedChunks": false,
              "aot": true,
              "extractLicenses": true,
              "vendorChunk": false,
              "buildOptimizer": true,
              "budgets": [
                {
                  "type": "initial",
                  "maximumWarning": "2mb",
                  "maximumError": "5mb"
                }
              ]
            },
            "ci": {
              "progress": false
            }
          }
        },
        "serve": {
          "builder": "@angular-devkit/build-angular:dev-server",
          "options": {
            "browserTarget": "app:build"
          },
          "configurations": {
            "production": {
              "browserTarget": "app:build:production"
            },
            "ci": {
              "progress": false
            }
          }
        },
        "extract-i18n": {
          "builder": "@angular-devkit/build-angular:extract-i18n",
          "options": {
            "browserTarget": "app:build"
          }
        },
        "test": {
          "builder": "@angular-devkit/build-angular:karma",
          "options": {
            "main": "src/test.ts",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "tsconfig.spec.json",
            "karmaConfig": "karma.conf.js",
            "styles": [],
            "scripts": [],
            "assets": [
              {
                "glob": "favicon.ico",
                "input": "src/",
                "output": "/"
              },
              {
                "glob": "**/*",
                "input": "src/assets",
                "output": "/assets"
              }
            ]
          },
          "configurations": {
            "ci": {
              "progress": false,
              "watch": false
            }
          }
        },
        "lint": {
          "builder": "@angular-devkit/build-angular:tslint",
          "options": {
            "tsConfig": [
              "tsconfig.app.json",
              "tsconfig.spec.json",
              "e2e/tsconfig.json"
            ],
            "exclude": ["**/node_modules/**"]
          }
        },
        "e2e": {
          "builder": "@angular-devkit/build-angular:protractor",
          "options": {
            "protractorConfig": "e2e/protractor.conf.js",
            "devServerTarget": "app:serve"
          },
          "configurations": {
            "production": {
              "devServerTarget": "app:serve:production"
            },
            "ci": {
              "devServerTarget": "app:serve:ci"
            }
          }
        },
        "ionic-cordova-build": {
          "builder": "@ionic/angular-toolkit:cordova-build",
          "options": {
            "browserTarget": "app:build"
          },
          "configurations": {
            "production": {
              "browserTarget": "app:build:production"
            }
          }
        },
        "ionic-cordova-serve": {
          "builder": "@ionic/angular-toolkit:cordova-serve",
          "options": {
            "cordovaBuildTarget": "app:ionic-cordova-build",
            "devServerTarget": "app:serve"
          },
          "configurations": {
            "production": {
              "cordovaBuildTarget": "app:ionic-cordova-build:production",
              "devServerTarget": "app:serve:production"
            }
          }
        }
      }
    }
  },
  "cli": {
    "defaultCollection": "@ionic/angular-toolkit"
  },
  "schematics": {
    "@ionic/angular-toolkit:component": {
      "styleext": "scss"
    },
    "@ionic/angular-toolkit:page": {
      "styleext": "scss"
    }
  }
}

... and here is the relevant subset of my package.json file for the project:

{
  "dependencies": {
    "@angular/common": "~8.1.2",
    "@angular/core": "~8.1.2",
    "@angular/forms": "~8.1.2",
    "@angular/http": "^7.2.15",
    "@angular/platform-browser": "~8.1.2",
    "@angular/platform-browser-dynamic": "~8.1.2",
    "@angular/router": "~8.1.2",
    "@ionic-native/core": "^5.15.1",
    "@ionic/angular": "^4.7.1",
    "vega": "~5.6.0",
    "vega-lite": "^3.4.0",
    "vega-themes": "^2.4.0",
    "zone.js": "~0.9.1"
  },
  "devDependencies": {
    "@angular-devkit/architect": "~0.801.2",
    "@angular-devkit/build-angular": "~0.801.2",
    "@angular-devkit/core": "~8.1.2",
    "@angular-devkit/schematics": "~8.1.2",
    "@angular/cli": "~8.1.2",
    "@angular/compiler": "~8.1.2",
    "@angular/compiler-cli": "~8.1.2",
    "@angular/language-service": "~8.1.2",
    "@ionic/angular-toolkit": "~2.0.0",
    "@types/jasmine": "~3.3.8",
    "@types/jasminewd2": "~2.0.3",
    "@types/node": "~8.9.4",
    "codelyzer": "^5.0.0",
    "jasmine-core": "~3.4.0",
    "jasmine-spec-reporter": "~4.2.1",
    "karma": "~4.1.0",
    "karma-chrome-launcher": "~2.2.0",
    "karma-coverage-istanbul-reporter": "~2.0.1",
    "karma-jasmine": "~2.0.1",
    "karma-jasmine-html-reporter": "^1.4.0",
    "protractor": "~5.4.0",
    "ts-node": "~7.0.0",
    "tslint": "~5.15.0",
    "typescript": "~3.4.3"
  }
}

Update #2

Continuing to try and work through this, I have made the following set of updates to the package.json:

  "dependences": 
    "tslib": added => "^1.10.0" 
    "vega": "~5.6.0" => "^5.9.0"
    "vega-lite": "^3.4.0" => "^4.0.2"

  "devDependencies": 
    "@angular/compiler": "~8.1.2" => "~8.2.9"
    "@angular/compiler-cli": "~8.1.2" => "~8.2.9"
    "typescript": "~3.4.3" => "~3.5.3"

... with those changes, I think I'm getting apparent es5 compiled output in the www/vendor-es5.js file and my adb logcat results don't appear to be indicating Syntax Errors. Unfortunately, the app still fails to get past the Splash screen (again this is only the case on some devices).

Here is my tsconfig.json file from the project:

{
  "compileOnSave": false,
  "compilerOptions": {
    "baseUrl": "./",
    "outDir": "./dist/out-tsc",
    "sourceMap": true,
    "declaration": false,
    "module": "esnext",
    "moduleResolution": "node",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "importHelpers": true,
    "target": "es2015",
    "typeRoots": [
      "node_modules/@types"
    ],
    "lib": [
      "es2018",
      "dom"
    ]
  },
  "angularCompilerOptions": {
    "fullTemplateTypeCheck": true,
    "strictInjectionParameters": true
  }
}

... and as far as usage of vega the crux of it is:

    const theme = vega.fivethirtyeight;
    this._view = new vega.View(vega.parse(vegaSpec, theme), {})
      .initialize(this.container.nativeElement)
      .logLevel(vega.Warn)
      .renderer('svg');

... on a problematic device if I filter the adb logcat output to E (error) lines, I see this:

01-10 09:17:27.650  6413  6413 E ApkAssets: Error while loading asset assets/natives_blob_64.bin: java.io.FileNotFoundException: assets/natives_blob_64.bin
01-10 09:17:27.651  6413  6413 E ApkAssets: Error while loading asset assets/snapshot_blob_64.bin: java.io.FileNotFoundException: assets/snapshot_blob_64.bin
01-10 09:17:27.680  6413  6413 E         : appName=xxxxxx, acAppName=/system/bin/surfaceflinger
01-10 09:17:27.680  6413  6413 E         : 0
01-10 09:17:27.683  6413  6413 E         : appName=xxxxxx, acAppName=vStudio.Android.Camera360
01-10 09:17:27.683  6413  6413 E         : 0
01-10 09:17:27.781  6413  6413 E MPlugin : Unsupported class: com.mediatek.common.telephony.IOnlyOwnerSimSupport
01-10 09:17:28.153  6413  6464 E libEGL  : validate_display:99 error 3008 (EGL_BAD_DISPLAY)
01-10 09:17:28.432  6413  6464 E         : appName=xxxxxx, acAppName=vStudio.Android.Camera360
01-10 09:17:28.433  6413  6464 E         : 0
01-10 09:17:28.436  6413  6464 E         : appName=xxxxxx, acAppName=vStudio.Android.Camera360
01-10 09:17:28.436  6413  6464 E         : 0
01-10 09:17:28.437  6413  6464 E         : appName=xxxxxx, acAppName=vStudio.Android.Camera360
01-10 09:17:28.437  6413  6464 E         : 0
01-10 09:17:30.514  6413  6455 E         : appName=xxxxxx, acAppName=vStudio.Android.Camera360
01-10 09:17:30.514  6413  6455 E         : 0
01-10 09:17:30.515  6413  6455 E         : app

... and for good measure here are the W (warning) lines:

01-10 09:17:27.835  6413  6413 W chromium: [WARNING:password_handler.cc(33)] create-->contents = 0x9c66ec00, delegate = 0xa4b7edd0
01-10 09:17:27.835  6413  6413 W chromium: [WARNING:password_handler.cc(41)] attaching to web_contents 
01-10 09:17:27.837  6413  6413 W cr_AwContents: onDetachedFromWindow called when already detached. Ignoring
01-10 09:17:28.185  6413  6455 W libEGL  : [ANDROID_RECORDABLE] format: 1
01-10 09:17:28.209  6413  6464 W VideoCapabilities: Unrecognized profile/level 1/32 for video/mp4v-es
01-10 09:17:28.209  6413  6464 W VideoCapabilities: Unrecognized profile/level 32768/2 for video/mp4v-es
01-10 09:17:28.209  6413  6464 W VideoCapabilities: Unrecognized profile/level 32768/64 for video/mp4v-es
01-10 09:17:28.244  6413  6455 W libEGL  : [ANDROID_RECORDABLE] format: 1
01-10 09:17:28.248  6413  6464 W VideoCapabilities: Unsupported mime video/x-ms-wmv
01-10 09:17:28.253  6413  6464 W VideoCapabilities: Unsupported mime video/divx
01-10 09:17:28.262  6413  6464 W VideoCapabilities: Unsupported mime video/xvid
01-10 09:17:28.268  6413  6464 W VideoCapabilities: Unsupported mime video/flv1
01-10 09:17:28.274  6413  6464 W VideoCapabilities: Unrecognized profile/level 1/32 for video/mp4v-es
01-10 09:17:28.485  6413  6413 W cr_BindingManager: Cannot call determinedVisibility() - never saw a connection for the pid: 6413
01-10 09:17:28.568  6413  6413 W cr_BindingManager: Cannot call determinedVisibility() - never saw a connection for the pid: 6413
Lissie answered 6/1, 2020 at 3:47 Comment(4)
Please show your tsconfig.json and some code with vega usage also. I think i can help here. Looks like you just need to force angular to transpile external library.Provincetown
Also i found that vega dropped es5 support github.com/vega/vega/issues/1470#issuecomment-444951182Provincetown
@Provincetown I've continued to struggle through this while waiting help from the S/O community... I'll post an update shortly which includes tsconfig.json and add some detail about vega usageLissie
@Provincetown I've updated the question with more details per your suggestionsLissie
P
5

First of all I want to say that it is really vega package fault - I think it is a bad way to deliver untranspiled code via npm. For example Angular Package Format guarantee that you will get es5 valid code, if you need it. But vega is not a clear angular dependency so let's solve it.

Why is this happening?

Because some developers deliver packages in es6+ standard and it is OK until you need es5 compatible application. In my opinion library developers should build and deliver es5 and es6 bundles, or it will be a headache for their users (like your case with vega).

Why does it only have an impact on some, and not all, devices?

To be honest I have very limited experience with native mobile development - all I can say here is that for example mobile Chrome and desktop Chrome have some differences in their engines. It means that there is no guarantee that using the same software will provide the same result. Sometimes you can find the bug in mobile browser and can't reproduce it in desktop browser.

I think in your case some devices with some browser engines can use es6 code - and some just can't.
Also in first version of your question there was useragent strings - i think advanced mobile developers can say more using than me.

Is there a way that I can work around the problem without switching to a different graphing library?

Yes. I created a repo with setup very similar to yours - simple ionic@4 project based on angular@8.

Your bundle now is es5 and es6 mixed. Let's do it fully es5 compatible to work in any browser (I tested this project even in ie11).
Steps to get the job done:

  1. Install dependencies. We will need them in further steps.
npm i -S regenerator-runtime
npm i -D @angular-builders/custom-webpack babel-loader @babel/core @babel/preset-env
  1. Change target property to es5 in tsconfig. "target": "es5"
  2. We will transpile async/await so we need regenerator-runtime polyfill to be added to polyfills.ts as import 'regenerator-runtime/runtime'
  3. The main step. Change builders in angular.json and add path to webpack.config.js to use custom webpack configuration for build and serve:
       "build": {
          "builder": "@angular-builders/custom-webpack:browser",
          "options": {
            "customWebpackConfig": {
                 "path": "./webpack.config.js"
              },
...
        "serve": {
          "builder": "@angular-builders/custom-webpack:dev-server",
  1. Create webpack.config.js in root folder with rules to transpile vega and it's dependencies. I found them in very imperative way.
// these dependencies are es6!!!
const transpileList = ['node_modules/vega', 'node_modules/d3', 'node_modules/delaunator'];

module.exports = function(base) {
    return {
        ...base,
        module: {
            ...base.module,
            rules: [
                ...base.module.rules,
                {
                    test: function(fileName) {
                        return transpileList.some(name => fileName.includes(name)) && fileName.endsWith('.js');
                    },
                    use: {
                        loader: 'babel-loader',
                        options: {
                            presets: ['@babel/preset-env']
                        }
                    }
                }
            ]
        }
    }
}

After these steps I hope your application will work in any es5 environment. I tried in desktop ie11 and tablet Samsung A with default Samsung browser.

Provincetown answered 11/1, 2020 at 15:24 Comment(8)
Just to shed light on this or example mobile Chrome and desktop Chrome have some differences in their engines any browser on iOS is using the Safari Engine :( Thats why there are differences.Shipboard
Cool, I will give this a try. By "imperative" did you mean "empirical" as in by trial-and-error? Like how did you actually determine what should go in the transpileList?Lissie
@Lissie yes you're right, I meant trial-and-error way. First syntax error I found in d3/array. Then in delaunator. Then in d3/delaunay and vega-dataflow as you described it in issue. I think a good practice here will be to use regexp or exclusion list as in my solution and to not point single dependencies, because count of vega package dependencies will grow after each version update.Provincetown
@Provincetown I'm happy to report that this fixed the problem. Thank you so much! I wish there was a way to more "automatically" detect and apply babel than by analysis of the names of the files, but this works, so I can't complain!Lissie
@Lissie glad to hear. I tried to use webpack's rule.issuer with value /vega/, but it doesn't work recursively (deep to to the last level) :( only first level imports were matched.Provincetown
do you know what mods are needed to get source maps in dev mode? the async / await code is pretty hard to follow as generated by this configuration...Lissie
@Lissie maybe "sourceMap": {"scripts": true,"vendor": true, "styles": false} in angular.json projects.<projectname>.architect.serve, but it can seriously increase your build timeProvincetown
@Provincetown Would you be so kind to look at this or this post please?Tompion

© 2022 - 2024 — McMap. All rights reserved.