We present a new technique to locate mathematical expressions in document images using images of handwritten queries, without the use of optical character recognition. In our approach, two types of X-Y cutting are used to construct document region indexes that hold potential matches . Candidates are selected based on a set of tolerance parameters and then are ranked using image similarity, with the top 10 matches being returned to the user. The ranking of candidates relies on decreasing sum-of-squares distance between the upper and lower image contours for the candidate region and query image; Dynamic Time Warping is used to compute this distance. We evaluate our system using the maximum region match between a candidate and the region from which a test query is taken. In addition, ten users are invited to participate in the human evaluation, in which they are asked to rank the similarity between each returned candidate and the original query region subjectively. Two types of queries are used for the experiments and comparisons: the original queries are extracted from the documents directly while their handwritten versions are obtained by asking the ten participants to draw queries following original ones. On average the precision of using original queries is about twice that of handwritten queries. The main contribution of this thesis is the demonstration of the feasibility of this novel query-by-expression method.
Library of Congress Subject Headings
Mathematical symbols (Typefaces)--Classification; Information retrieval; Image processing--Digital techniques; Optical pattern recognition
Computer Science (MS)
Department, Program, or Center
Computer Science (GCCIS)
Yu, Li, "Image-Based Math Retrieval Using Handwritten Queries" (2010). Thesis. Rochester Institute of Technology. Accessed from
RIT – Main Campus