How to test Android toast messages in Appium ( selenium Java)

Question

I am using Selenium with Java to run scripts on android (thru Appium server). I see that it is not possible to locate a toast by using selenium's

driver.findElement(By.LinkText("User not logged in")

in Appium

But can be used in Selendroid to capture toast messages.

I there a way I can use both Selendroid and Appium in the same script?

Clementius

Answer 1

Finally, we are able to read the toast message without the need of taking screenshots and performing OCR. I have tested this on Appium 1.15.1.

Toast messages comes under com.package.system.

Normally, Xpath for this will be "/hierarchy/android.widget.Toast". And, Class Name will be "android.widget.settings"

You can confirm this by refreshing element inspector screen when toast message is displayed.

WebDriverWait waitForToast = new WebDriverWait(driver.25);

waitForToast.until(ExpectedConditions.presenceOfElementLoacted(By.xpath("/hierarchy/android.widget.Toast")));

String toastMessage = driver.findElement((By.xpath("/hierarchy/android.widget.Toast")).getText();

System.out.println(toastMessage);

Orle

Yeah, I think this is the easiest solution. Tried with Appium 1.17.1 - works. One thing though, visibilityOfElementLocated does not work with toast notifications. Had to use presenceOfElementLoacted as in the example above. – Litman

Yes, this method saves lot of time when compared to OCR method(Screenshot method). – Orle

this doesn't work with appium inspector though right?, I couldn't select android toast messages using inspector using version 1.20 – Abirritant

Try to expand the xml hierarchy and see. Toast message will be normally and the end of hierarchy. I have not used 1.20 yet. It was working fine with 1.15 inspector. – Orle

Answer 2

Method 1: from Appium version 1.6.4 supports toast messages, for that you need to use automationName:'uiautomator2'.

toast = driver.find_element(:xpath, "//android.widget.Toast[1]")
if toast.text == "Hello" 

But i don't recommend this because uiautomator2 is not stable yet.

Method 2:

1. Trigger text message on the screen
2. Capture screenshots

3. Convert image to text file

   def assettoast(string)
     sname = (0...8).map { (65 + rand(26)).chr }.join
     $driver.driver.save_screenshot("#{sname}")
   
     # Make sure tesseract is installed in the system. If not you can install using "brew install tesseract" in mac
     system ("tesseract #{sname} #{sname}")
   
     text_file="#{sname}.txt"
     var= get_string_from_file(string, text_file)
     raise if var != true
   end
   

4. Check whether toast message is there in text file

   def get_string_from_file(word, filename)
     File.readlines(filename).each do |line|
     return true if line.include?(word)
     end
   end
   

Duster

Answer 3

Looks like you can't switch driver type within same session. If you trying to switch to Selendroid only for toast verification - you could use OSR image recognition engine.

Check this answer w/ Ruby bindings

Idea is quite simple:

• make toast message to appear
• take few screenshots
• iterate over taken screenshots and look for required text

Here is nice and simple example of OCR usage in Java: tess4j example (make sure that Tesseract engine installed)

Minus

Thanks!! So i downloaded Tess4J 2.0 from tess4j.sourceforge.net, I added all the jar files to my project in eclipse I also had given the below command to include the dll files gsdll32, libtesseract303 and liblept170 stored in a folder dllfiles in the code System.setProperty("jna.library.path","C:\\Tess4J\\dllfiles); On running the sample code. It is throwing an error "Exception in thread "main" java.lang.UnsatisfiedLinkError: The specified module could not be found" – Clementius

I was able to resolve the error. The code used doOCR function which was not working. It was Microsoft VC++ which was creating this error. I installed VC++ 2013 and the code ran but now the problem is tesseract is not converting the image properly. It is missing a lot of text and some which it recognised are incorrect. Is there any other good OCR that i can use? – Clementius

@Clementius actually you should play with image first to make it 'readable' img.contrast.normalize.negate.posterize(3).adaptive_resize(3) as you can see I use ImageMagick to contrast, normalize, negate, posterize and resize it x3. thats what I found to work well in my project – Minus

Answer 4

Step 1:
File scrFile=null;
String path1 = null;
BufferedImage originalImage=null;
BufferedImage resizedImage=null;
System.out.println("Starting\n\n\n\n");
scrFile = ((TakesScreenshot) appiumDriver).getScreenshotAs(OutputType.FILE);
System.out.println("after scrfile\n\n\n\n");
originalImage = ImageIO.read(scrFile);
System.out.println("after originalFile\n\n\n");
BufferedImage.TYPE_INT_ARGB : originalImage.getType();
resizedImage = CommonUtilities.resizeImage(originalImage, IMG_HEIGHT, IMG_WIDTH);
ImageIO.write(resizedImage, "jpg", new File(path + "/"+ testCaseId + "/img/" + index + ".jpg"));
Image jpeg = Image.getInstance(path + "/" + testCaseId + "/img/"+ index + ".jpg");
Step 2: 
BufferedImage pathforToast= original image;
Step 3:
System.setProperty("jna.library.path","C:/Users/Dell/workspace/MOBILEFRAMEWORK/dlls/x64/");
Tesseract instance = Tesseract.getInstance();
`enter code here`ImageIO.scanForPlugins();
String result=null;
result = instance.doOCR(pathforToast);`enter code here`
System.out.println(result);`enter code here`

Davita

Answer 5

Appium 1.6.4@beta latest version supports toast messages

Nigel

Answer 6

Take screen shot of Toast Message page and try to convert the image file in to Text and verify the text using the below code.

  public void imageconversion(String filePath) throws IOException,          
    {    
                ITesseract instance = new Tesseract();
    //file path is the image which you need to convert to text
                File imageFile = new File(filePath);  
                BufferedImage img = null;
                img = ImageIO.read(imageFile);
                BufferedImage blackNWhite = new BufferedImage(img.getWidth(),img.getHeight(), BufferedImage.TYPE_BYTE_BINARY);
                Graphics2D graphics = blackNWhite.createGraphics();
                graphics.drawImage(img, 0, 0, null);
                //path where your downloaded tessdata exists
                instance.setDatapath("E://ocr//data"); 
              //What language you required to convert,( e.g. English)
                instance.setLanguage("eng");        
                String result = instance.doOCR(blackNWhite);


                System.out.println(result);

            }

Mink

Answer 7

Appium Directly does not give any API to read toast message we need to do it using tess4j jar. First we need to take screen shot and then we need to read the text from screen shot using tess4j API.

 static String scrShotDir = "screenshots";
  File scrFile;
  static File scrShotDirPath = new java.io.File("./"+ scrShotDir+ "//");
  String destFile;
  static AndroidDriver driver = null;

 public String readToastMessage() throws TesseractException {
String imgName = takeScreenShot();
  String result = null;
  File imageFile = new File(scrShotDirPath, imgName);
  System.out.println("Image name is :" + imageFile.toString());
  ITesseract instance = new Tesseract();

  File tessDataFolder = LoadLibs.extractTessResources("tessdata"); // Extracts
                   // Tessdata
                   // folder
                   // from
                   // referenced
                   // tess4j
                   // jar
                   // for
                   // language
                   // support
  instance.setDatapath(tessDataFolder.getAbsolutePath()); // sets tessData
                // path

  result = instance.doOCR(imageFile);
  System.out.println(result);
  return result;
 }

 /**
  * Takes screenshot of active screen
  * 
  * @return ImageFileName
  */
 public String takeScreenShot() {
  File scrFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); 

  SimpleDateFormat dateFormat = new SimpleDateFormat("dd-MMM-yyyy__hh_mm_ssaa");
  new File(scrShotDir).mkdirs(); // Create folder under project with name
          // "screenshots" if doesn't exist
  destFile = dateFormat.format(new Date()) + ".png"; // Set file name
               // using current
               // date time.
  try {
   FileUtils.copyFile(scrFile, new File(scrShotDir + "/" + destFile)); // Copy
                    // paste
                    // file
                    // at
                    // destination
                    // folder
                    // location
  } catch (IOException e) {
   System.out.println("Image not transfered to screenshot folder");
   e.printStackTrace();
  }
  return destFile;
 }

For more details Refer this video - https://www.youtube.com/watch?v=lM6-ZFXiSls

Bellied

Answer 8

I found three ways to capture a Toast message and verify them.

1. To get the page source and verify the toast message from it.

public void verifyToastMessageUsingPageSource(String toastmsg) throws InterruptedException {
    boolean found = false;
    for(int i =0 ; i <8; i++){
      if(getDriver().getPageSource().contains("class=\"android.widget.Toast\" text=\""+toastmsg+"\"")){
        found = true;
        break;
      }
      Thread.sleep(300);
    }
    Assert.assertTrue(found,"toast message "+toastmsg+" is present");
  }

Similar can be found using Xpath: //android.widget.Toast[1]

2. Using the grep command, wait for a toast message in uiautomator events. run the command before clicking and toast message will be varified.

   adb shell uiautomator events | grep "ToastMessgae"

3. This is tricky and needs more code to run.

   • start capturing screenshots thread.
   • perform click action
   • stop screen capturing thread.
   • extract text from the captured images using OCR and verify the toast message is present in the captured images.

I prefer the 1st and 2nd option, it provides validation in less time with less code.

comment if you need code for 2nd and 3rd point.

Contain

Answer 9

Appium with version number>=1.6.4 supports toast notification with UiAutomator2. In Javascript with webdriver you can do like this

let toast=await driver1.elements("xpath","/hierarchy/android.widget.Toast");
let data=await toast[0].text();
console.log(data)

Ronen

Answer 10

Solutions for Appium + Javascript

Create a function in helper class and call it in test file with actual text param

Helperclass.js
-----------------------------
    async isToastMessageDisplayed(toastText) {
            const numberOfAttempts = 2; // Adjust the number of attempts as needed
                  const waitTime = 200; // Adjust the wait time between attempts in milliseconds
            
                  for (let i = 0; i < numberOfAttempts; i++) {
                      await driver.pause(waitTime);
                      const pageSource = await driver.getPageSource();
                      if (pageSource.includes(toastText)) {
                          console.log(`Toast message displayed ${i + 1}: ${toastText}`);
                          return true;
                      }
                  }
                  return false; }

Test.js
----------------------------------------------------------
    const toastText = "Wrong OTP entered. Please enter the correct OTP";
            const isToastDisplayed = await HelperClass.isToastMessageDisplayed(toastText);
            expect(isToastDisplayed).to.be.true;

Leukas