Render 3D Objects in ARCore using GPS location?

Question

I trying to build augmented reality navigation using arcore in android. Is AR navigation direction is available in india? I just followed this link to develop ar navigation with core loaction using arcore. https://github.com/appoly/ARCore-Location/tree/legacy.

Is it possible to do ar navigation using ARCore in android? Any help much appreciated pls....

Dammar

Answer 1

I also worked on this and According to me, it is possible. I haven't finished it yet due to some other works but I can tell you the steps you can follow to achieve this.

Initially, you need to find out how to place an object without tapping on the screen as ARCore sample won't allow this. I asked this question on StackOverflow and finally getting a satisfactory answer which actually worked. How to place a object without tapping on the screen. You can refer this to achieve that.

After this, you need to find out the direction to which you need to place your ar object showing as the direction. for this, find out the heading using this formula

let lat1:Double = latSource/180 * .pi
let lng1:Double = lngSource/180 * .pi
let lat2:Double = latDestination/180 * .pi
let lng2:Double = lngDestination/180 * .pi

let y = sin(lng2 - lng1)*cos(lat2)
let x = cos(lat1) * sin(lat2) - sin(lat1) * cos(lat2) * cos(lng2 - lng1)

let tan2 = atan2(y, x)
let degre = tan2 * 180 / .pi
if degre < 0 {  return degre+360  }
else {  return degre  }

You need to pass the latitude and longitude of source and destination then this will return you the angle from TrueNorth of your destination. This is written is Swift, I hope you can convert it into your desired language. Java or Kotlin

Then you need to match this angle with your device compass angle and when it both matches use the method of without placing an object. I already shared the link. This will give you the object towards your destination.

After that, make a loop and place object at different z-axis. This is the code which I wrote the do this

@Override
public void onUpdate(FrameTime frameTime) {



        frame = fragment.getArSceneView().getArFrame();

        if (frame != null) {

            for (Object o : frame.getUpdatedTrackables(Plane.class)) {

                plane = (Plane) o;

                if (plane.getTrackingState() == TrackingState.TRACKING) {
                    fragment.getPlaneDiscoveryController().hide();

                    Iterator iterableAnchor = frame.getUpdatedAnchors().iterator();

                    if (!iterableAnchor.hasNext()) {
                        method(plane, frame);
                    }
                }
            }
        }

}

public void makeAr(Plane plane, Frame frame ) {

    for (int k = 0; k <10 ; k ++) {
        if (this.degree >= 160 && this.degree <= 170) {
            Toast.makeText(this, "walk", Toast.LENGTH_SHORT).show();
            List<HitResult> hitTest = frame.hitTest(screenCenter().x, screenCenter().y);

            Iterator hitTestIterator = hitTest.iterator();

            while (hitTestIterator.hasNext()) {
                HitResult hitResult = (HitResult) hitTestIterator.next();

                modelAnchor = null;
                modelAnchor = plane.createAnchor(hitResult.getHitPose());

                AnchorNode anchorNode = new AnchorNode(modelAnchor);
                anchorNode.setParent(fragment.getArSceneView().getScene());

                TransformableNode transformableNode = new TransformableNode(fragment.getTransformationSystem());
                transformableNode.setParent(anchorNode);
                transformableNode.setRenderable(MainActivity.this.andyRenderable);

                float x = modelAnchor.getPose().tx();
                float y = modelAnchor.getPose().compose(Pose.makeTranslation(0f, 0f, 0)).ty();

                transformableNode.setWorldPosition(new Vector3(x, y, -k));

            }
        }
    }
}

private Vector3 screenCenter() {
    View vw = findViewById(android.R.id.content);
    return new Vector3(vw.getWidth() / 2f, vw.getHeight() / 2f, 0f);
}

This OnUpdate method draw the Ar towards the destination. After getting this, you need to arrange everything with the Google Map Api data.

According to me, this is the best possible way to do navigation in AR using ARCore because there are very limitation in ARCore may be in future there will be some other libraries which will available to make it easy.

I also tried the ArCore Location but this won't help you beacuse it places the 2D image not the 3D as well as it is not stable when you are walking on the road.

Good Luck, Hope you'll finish this.

Hyohyoid

what does it mean: "if (this.degree >= 160 && this.degree <= 170)"? Actually, you skip the most interesting piece of code. You say correctly that the author need to match north bearing for device and (current/target) location, but it `s the most hard part as there is a lot of noice for android sensors. Can you share your experience, please? – Proximal

(this.degree >= 160 && this.degree <= 170), In This line, 160 & 170 actually depends upon the heading. (heading is the angle of two lats long from true North if you open compass in your phone and pick it vertical to the earth surface, the angle you'll see is actually the heading between the line of your back camera Facing and True North.) – Hyohyoid

you can find heading from this link, igismap.com/…. I already provided the for it. then you'll use my method like "if (this.degree >= heading-10 && this.degree <= heading+10)" you need to add this condition in your compass update, when your device compass angle matches with this condition then go with the for loop code that I provided to you and it will draw a path of continous object. It needs work and a lot of work to make it happen in Android. whereas in iOS you can do it very easily. – Hyohyoid

yep. That`s true about IOS. This is a very ugly thing that ARKit could return xyz coordinates already matched with true north, while the ARCore users should determine heading themselves considering the very noisy output from compass. Any thoughts how can we get more stable data from android compass? – Proximal

Android compass is ok but ARCore is not strong enough to handle this type of action for now. I created this project in ios with a different approach and that works very well. check it out here: drive.google.com/open?id=1cOhs_DH25Z3TCPDWZeiztMzvAs61iiGH – Hyohyoid

That`s exactly what I am doing now. Everything is ok untill I try to merge device heading & heading between current/target coordinates and to determine in such way rotation angle. Can we contact somewhere? I have a few questions. It will take not more than 15 min. – Proximal

Thanks. I have sent invitation. – Proximal

@AkashMishra I tried this code but not getting success. Can you describe further? – Propinquity

Answer 2

here there are three functions

one is adding a node using x,y and z in real world.

second is calculating bearing. more information on: https://www.sunearthtools.com/tools/distance.php#top

third is calculating x,y and z based on the location given.

private fun addPointByXYZ(x: Float, y: Float, z: Float, name: String) {
    ViewRenderable.builder().setView(this, R.layout.sample_layout).build().thenAccept {
        val imageView = it.view.findViewById<ImageView>(R.id.imageViewSample)
        val textView = it.view.findViewById<TextView>(R.id.textViewSample)

        textView.text = name

        val node = AnchorNode()
        node.renderable = it
        scene.addChild(node)
        node.worldPosition = Vector3(x, y, z)

        val cameraPosition = scene.camera.worldPosition
        val direction = Vector3.subtract(cameraPosition, node.worldPosition)
        val lookRotation = Quaternion.lookRotation(direction, Vector3.up())
        node.worldRotation = lookRotation
    }
}

private fun bearing(locA: Location, locB: Location): Double {
    val latA = locA.latitude * PI / 180
    val lonA = locA.longitude * PI / 180
    val latB = locB.latitude * PI / 180
    val lonB = locB.longitude * PI / 180

    val deltaOmega = ln(tan((latB / 2) + (PI / 4)) / tan((latA / 2) + (PI / 4)))
    val deltaLongitude = abs(lonA - lonB)

    return atan2(deltaLongitude, deltaOmega)
}

private fun placeARByLocation(myLocation: Location, targetLocation: LatLng, name: String) {
    val tLocation = Location("")
    tLocation.latitude = targetLocation.latitude
    tLocation.longitude = targetLocation.longitude

    val degree = (360 - (bearing(myLocation, tLocation) * 180 / PI))
    val distant = 3.0

    val y = 0.0
    val x = distant * cos(PI * degree / 180)
    val z = -1 * distant * sin(PI * degree / 180)
    addPointByXYZ(x.toFloat(), y.toFloat(), z.toFloat(), name)

    Log.i("ARCore_MyLat", myLocation.latitude.toString())
    Log.i("ARCore_MyLon", myLocation.longitude.toString())
    Log.i("ARCore_TargetLat", targetLocation.latitude.toString())
    Log.i("ARCore_TargetLon", targetLocation.longitude.toString())
    Log.i("ARCore_COMPASS", azimuth.toString())
    Log.i("ARCore_Degree", degree.toString())
    Log.i("ARCore_X", x.toString())
    Log.i("ARCore_Y", y.toString())
    Log.i("ARCore_Z", z.toString())
    Toast.makeText(this@LatLngActivity, "Compass: $azimuth, Degree: $degree", Toast.LENGTH_LONG).show()
}

the azimuth you're seeing here is the compass degree from sensors.

the full code is available in my gist.

https://gist.github.com/SinaMN75/5fed506622054d4247112a22ef72f375

Fatidic

@ SinaMN75 I tried this but not getting an object in the proper direction. Can you explain about the direction? – Propinquity

Answer 3

I recently developed a functionality to get this working without using any external system.

Here's the link to my repository: https://github.com/bhaskarblur/ARCore-Navigation-Anchors-using-LatLng-in-AR-World---Android

Read the Readme of repository to understand how it works.

This app uses ARCore with Sceneform to support AR & render objects in AR World. The current location of user is continuously calculated and also the yaw(azimuth) aka the rotation of the device around it's -z axis which is respective to north. Now, we have our current Latitude, Longitude & our rotation so we assume that we are an origin point on graph (0,0) and we added a Latitude & Longitude of a coordinate into the graph say it's (3,3). So, we convert this (3,3) via a CoordinateHelper class which converts LatLng into local coordinates using our current location, yaw and the target Location. Now we got the local coordinates, we set this as a LocalPosition to our AnchorNode in AR World.

Do leave an upvote if it helped you!

Catton

Answer 4

Is AR navigation direction is available in india?

Yes, the only sensible place today is China, as you can see here

Is it possible to do ar navigation using ARCore in android?

Not sure if I get, but yes, you can use AR Core to navigate with your app, like walk, and show AR Objects on the screen or get pose/frame informations from the device movement. You can put anchor in the world and connect this anchors with GPS locations, save this anchors on cloud/database and load in others cellphones. Or direct objects renders and save they on DB. When a user walk close to a place you load this in his cellphone.

Clanton