It seems that tesseract isn't detecting words when they are inside of a border/line like inside of a highlighted button. In my case it cannot detect "Done" with a login prompt when it is highlighted as shown in the attached file:
You can see the tesseract data frame found here:
x=0, y=0, w=2560, h=1600
yres=1600, xres=2560
======================================================
level page_num block_num ... height conf text
0 1 1 0 ... 1.000000 -1.000000 NaN
1 2 1 1 ... 0.017500 -1.000000 NaN
2 3 1 1 ... 0.017500 -1.000000 NaN
3 4 1 1 ... 0.017500 -1.000000 NaN
4 5 1 1 ... 0.016875 95.101944 Log
5 5 1 1 ... 0.013750 95.101944 in
6 5 1 1 ... 0.013750 95.915131 online
7 5 1 1 ... 0.013750 96.001770 with
8 5 1 1 ... 0.013750 96.001770 another
9 5 1 1 ... 0.013750 95.663040 device
10 2 1 2 ... 0.192500 -1.000000 NaN
11 3 1 2 ... 0.192500 -1.000000 NaN
12 4 1 2 ... 0.192500 -1.000000 NaN
13 5 1 2 ... 0.192500 95.000000
14 2 1 3 ... 0.019375 -1.000000 NaN
15 3 1 3 ... 0.019375 -1.000000 NaN
16 4 1 3 ... 0.019375 -1.000000 NaN
17 5 1 3 ... 0.013750 0.295380 master.
18 5 1 3 ... 0.019375 0.295380 keycloak.test:8443/auth/realms/...
19 2 1 4 ... 0.017500 -1.000000 NaN
20 3 1 4 ... 0.017500 -1.000000 NaN
21 4 1 4 ... 0.017500 -1.000000 NaN
22 5 1 4 ... 0.017500 95.878555 Login
23 5 1 4 ... 0.013750 92.476372 code:
24 5 1 4 ... 0.013750 92.043373 UBDY-RZDK
[25 rows x 12 columns]
In my specific case, I can work around this with kb_send("enter").
One thing that did help was changing the threshold in the models/gui.py from using threshold of 120 to 140. Making that parameter an optional variable we can pass in from the click_on() or assert_text() functions would help. It would also need to be available from the command line at least for the click-on sub-command.
Another option discussed was having the threshold automatically adjusted after some number of screenshot attempts. Maybe halfway through start incrementing the threshold by 10? Not sure what might work best for us here. This might also affect/fix the check_home_screen issue we're having now that "tosearch" is being detected as two separate words "to search".
It seems that tesseract isn't detecting words when they are inside of a border/line like inside of a highlighted button. In my case it cannot detect "Done" with a login prompt when it is highlighted as shown in the attached file:
You can see the tesseract data frame found here:
In my specific case, I can work around this with
kb_send("enter").One thing that did help was changing the threshold in the models/gui.py from using threshold of 120 to 140. Making that parameter an optional variable we can pass in from the
click_on()orassert_text()functions would help. It would also need to be available from the command line at least for theclick-onsub-command.Another option discussed was having the threshold automatically adjusted after some number of screenshot attempts. Maybe halfway through start incrementing the threshold by 10? Not sure what might work best for us here. This might also affect/fix the check_home_screen issue we're having now that "tosearch" is being detected as two separate words "to search".