How can I get the color of a screen pixel THROUGH ADB

Question

I need to get the color information of a specific point on screen of my android phone.

Is there a way to do that through ADB?

I am now using the build-in command screencap to capture the whole screen and then read the color of the specific point. However, it is too slow.

Plum

Answer 1

I will post an answer to my own question. The answer maybe device-specified (nexus7 2013), and you can adjust it to your own devices.

1.Firstly, I find out that the command screencap screen.png is quite slow because it take most of its time converting to png file type. So, to save time, the first step is dump screen to a raw data file. adb shell screencap screen.dump

2.Check the file size. My screen resolution is 1920*1200, and the file size is 9216012 byte. Noticing that 9216012=1920*1200*4+12, I guess the data file use 4 byte to store every pixel information, and use another 12 byte to do some mystery staff. Just do some more screencaps and I find the 12 byte at the head of each file are the same. So, the additional 12 byte is at the head of the data file.

3.Now, things are simple by using dd and hd. Assuming that I want to get the color at (x,y): let offset=1200*$y+$x+3 dd if='screen.dump' bs=4 count=1 skip=$offset 2>/dev/null | hd

I get output like 00000000: 4b 73 61 ff s 21e sum 21e The 4b 73 61 ff is my answer.

Plum

Nice solution, but why i can't perform the command when i add "| hd" to the end on the phone, but i can perform it when executing from adb.exe console? What does "| hd" do? – Gonococcus

hd or hexdump is used to display the binary data. For example, you has "123" in a text file, and hd THE-FILE gets 31 32 33. What's your command? – Plum

hi, I don't understand the coordinate well, why $y is multiply by 1200 but not 1920? – Tungstate

There are 1200 pixels in X direction, and thus you should multiply $y by 1200. – Plum

What does 4b 73 61 ff translate too? RGB colors? Why 4 bytes? – Plusch

Replying to myself. It's Red, Green, Blue, Alpha, in this order. – Plusch

why i get only "sum 0", no display like "00000000: 4b 73 61 ff" – Harwill

android.googlesource.com/platform/frameworks/base/+/master/cmds/… – Inhalator

Answer 2

If your phone is rooted and you know its framebuffer format you could use dd and hd (hexdump) to get the pixel representation directly from the framebuffer file:

adb shell "dd if=/dev/graphics/fb0 bs=<bytes per pixel> count=1 skip=<pixel offset> 2>/dev/null | hd"

Usually <bytes per pixel> = 4 and <pixel offset> = Y * width + X but it could be different on your phone.

Dewitt

I type adb shell "dd if=/dev/graphics/fb0 bs=4 count=1 skip=1000 2>/dev/null | hd", It returns Usage: hd [-b base] [-c count] [-r delay] file. Seems that the 'hd' is executed in my bash shell rather than the adb shell. I try the following: I So – Plum

most likely you don't have root so dd does not produce any output for hd to process – Dewitt

I try the following: adb shell su dd if=/dev/graphics/fb0 bs=4 count=1 skip=1000 2>/dev/null | hd It returns something like 00000000: 00 00 00 00 s 0 sum 0 – Plum

Those 4 bytes 00 00 00 00 is the data you are looking for. – Dewitt

I am afraid not, because every skip like skip=0 skip=32 skip=1000 skip=10000 skip=32000 returns 00 00 00 00. BTW, my screen is not black. – Plum

Now I pull the whole fb0 to a binary file by adb pull /dev/graphics/fb0 androidFB.raw. I check the content of the file using hexedit androidFB.raw, and it's truly all 00 00 00 00. So, the problem is in /dev/graphics/fb0. Any ideas? Thanks for your reply! – Plum

I find out. Someone has the same problem, it's relate to modern GPUs. [ #17305111 ] – Plum

i got C6 7A 25 62 , how to change the value to RGB? – Tungstate

I tried everything but couldn't get the right pixel color using this method. I had to go with the accepted answer using a screencap file. Weird. – Charger

Answer 3

Based on previous accepted answer, i wrote a SH function able to calculate buffer and etc to work out of the box on my phone.

usage:

GetColorAtPixel X Y

GIST : https://gist.github.com/ThomazPom/d5a6d74acdec5889fabcb0effe67a160

widthheight=$(wm size | sed "s/.* //")
width=$(($(echo $widthheight | sed "s/x.*//g" )+0))
height=$(($(echo $widthheight | sed "s/.*x//g" )+0))
GetColorAtPixel () {
    x=$1;y=$2;
    rm ./screen.dump 2> /dev/null
    screencap screen.dump
    screenshot_size=$(($(wc -c < ./screen.dump)+0));
    buffer_size=$(($screenshot_size/($width*height)))
    let offset=$width*$y+$x+3
    color=$(dd if="screen.dump" bs=$buffer_size count=1 skip=$offset 2>/dev/null | hd | grep -Eo "([0-9A-F]{2} )" |sed "s/[^0-9A-F]*\$//g" | sed ':a;N;$!ba;s/\n//g' |cut -c3-8)
    echo $color;
}

And an alternative version, (the first one does not work for me when i embed it in an sh file for some unkown default hexdump behaviour issue)

widthheight=$(wm size | sed "s/.* //")
width=$(($(echo $widthheight | sed "s/x.*//g" )+0))
height=$(($(echo $widthheight | sed "s/.*x//g" )+0))
GetColorAtPixel () {
        x=$1;y=$2;
        rm ./screen.dump 2> /dev/null
        screencap screen.dump
        screenshot_size=$(($(wc -c < ./screen.dump)+0));
        buffer_size=$(($screenshot_size/($width*height)))
        let offset=$width*$y+$x+3

        color=$(dd if="screen.dump" bs=$buffer_size count=1 skip=$offset 2>/dev/null | /system/xbin/hd | awk '{ print toupper($0) }' | grep -Eo "([0-9A-F]{2})+" | sed ':a;N;$!ba;s/\n//g' | cut -c9-14 )
        echo $color;
}

Onomastics

Have you tried using hexdump -C instead of hd in your first example? Maybe that would fix the embed problem. – Rivalee

Answer 4

Correction for dd method (by Alex P. and superwijiafeng), if you want to know pixel in ($x,$y) coordinates:

offset=1200*($y-1)+($x-1)+3

(where 1200 is the width of your screen).

Babettebabeuf