RE: TPAMI-2007-06-0371.R1, "Tracking by an Optimal Sequence of Linear Predictors" Manuscript Type: Regular Dear Mr. Karel Zimmermann, We have completed the review process of the above referenced paper for the IEEE Transactions on Pattern Analysis and Machine Intelligence. Enclosed are your reviews. Associate Editor Dr. Patrick Perez has recommended to the Editor-in-Chief that your paper undergo a minor revision. If you should choose to revise your paper, please prepare a separate document describing how each of the reviewers' comments are responded to in your revision and send it to us in one month, 11-APRIL-2008. To revise your manuscript, log into https://mc.manuscriptcentral.com/tpami-cs and enter your Author Center, where you will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a Revision." Your manuscript number has been appended to denote a revision. Once the revised manuscript is prepared, you can upload it and submit it through your Author Center. When submitting your revised manuscript, you will be able to respond to the comments made by the reviewer(s) in the space provided. You can use this space to document any changes you make to the original manuscript. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the reviewer(s)’ questions and comments. You may also upload your responses as separate files for review along with your revision. If you choose to do this, please choose “Summary of Changes” as the file designation. IMPORTANT: Your original files are available to you when you upload your revised manuscript. Please delete any redundant files before completing the submission. When the submission process is complete, you will receive an automated confirmation email immediately. If you did not receive that email, your submission is not yet complete. The journal’s publication coordinator will contact you should we have any concerns or questions regarding your revision. Otherwise, your revision will be forwarded to the assigned associate editor with a request a final decision. Please be mindful when making your revisions that you still need to maintain the size limitations for papers submitted to TPAMI. Our manuscript types and submission length guidelines (including the main text, the abstract, index terms, illustrations and references) are as follows: TPAMI manuscript types and submission length guidelines are as follows: • Regular papers* – 14 double column pages • Comments paper – 2 double column pages • Survey papers** – 20 double column pages While double column is preferred, authors wishing to submit a single column manuscript must adhere to the following page limits: • Regular papers* – 35 single column pages • Comments paper – 4 single column pages • Survey papers** – 45 single column pages Please note that double column will translate more readily into the final publication format. Our peer review double column templates can be found on http://www.computer.org/portal/site/transactions/menuitem.eda2ca84d8d67764cfe79d108bcd45f3/index.jsp?&pName=transactions_level1&path=transactions/tpami/mc&file=author.xml&xsl=article.xsl&#templates Please do not hesitate in contacting us should you have any questions about our process or are experiencing technical difficulties. You can reach me at tpami@computer.org. Thank you for your contribution to TPAMI, and we look forward to receiving your revised manuscript. Sincerely, Ms. Elaine Stephenson on behalf of Dr. Patrick Perez IEEE Transactions on Pattern Analysis and Machine Intelligence IEEE Computer Society 10662 Los Vaqueros Circle Los Alamitos, CA 90720 USA Voice: 714.821.8380 Fax: 714.821.9975 tpami@computer.org ************** Editor Comments Editor: 1 Comments to the Author: The three reviewers are globally satisfied with the answers of the authors and the modifications made in this revised version of the manuscript. Apart from a number of minor points that should easily be taken into account, there are however two remaining issues: one concerns the important problem of tracking confidence measure, which should be discussed since the proposed method does not permit such a measure as opposed to existing competitors; the second one concerns the clarity, the extent and the fairness of the experimental comparisons. I urge the authors to take carefully into account the detailed criticisms associated to these two issues in the final version of their manuscript. ******************** Reviewer Comments Reviewer: 1 Recommendation: Author Should Prepare A Minor Revision Comments: This paper deals with off-line learning trackers, in particular an optimal sequence of linear predictors is determined. The papers presents the necessary math and experiments for this problem. This is a revision of a submitted paper. The paper has now been substantially improved, especially the readability is now much better. There are only some minor issues that should be considered: 1. Maybe you think about the name of the method NoSLLiP does not sound attractive especially the No infront. 2. Last sentence of the abstract, Jurie's tracker is in this context not understandable, maybe you reformulate the sentence. 3. There are still some articles missing, in particular it should be The set of ... 4. Def. 2 the term performance is guaranteed is vague in this context and should be defined. 5. Prop. 2 Border Predictor should be defined 6. Def. 8 I suggest to call it sequential predictor of order m 7. p. 31 should read for some reason ================= 1. Which category describes this manuscript?: Research/Technology 2. How relevant is this manuscript to the readers of this periodical? If you answer Not very relevant or Irrelevant please explain your rating under Public Comments below.: Very Relevant 1. Please evaluate the significance of the manuscript’s research contribution.: Excellent 2. Please explain how this manuscript advances this field of research and/or contributes something new to the literature. : The precise formulation of the optimization problem of sequential linear predictors for tracking. 3. Is the manuscript technically sound? In the Public Comments section, please provide detailed explanations to support your assessment: Yes 4. How thorough is the experimental validation (where appropriate)? Please discuss any shortcomings in the Public Comments section.: Compelling experiments; clearly state of the art 1. Are the title, abstract, and keywords appropriate? If not, please comment in the Public Comments section.: Yes 2. Does the manuscript contain sufficient and appropriate references? Please comment and include additional suggested references in the Public Comments section.: References are sufficient and appropriate 3. Does the introduction state the objectives of the manuscript in terms that encourage the reader to read on? If not, please explain your answer in the Public Comments section.: Yes 4. How would you rate the organization of the manuscript? Is it focused? Please elaborate with suggestions for reorganization in the Public Comments section.: Satisfactory 5. Please rate the readability of the manuscript. Explain your rating under Public Comments below. : Readable - but requires some effort to understand 6. How is the length of the manuscript? If changes are suggested, please make explicit recommendations in the Public Comments section.: About right Please rate the manuscript overall. Explain your choice.: Excellent ************************************************************** Reviewer: 2 Recommendation: Author Should Prepare A Minor Revision Comments: I am general satisfied with the answers to my comments. However, the answer to my comment way about the way that the method deals with occlusions is not very convincing and raises the more general issue of confidence to the observations and to the estimation. More specifically, there is no measure on the confidence/reliability of the estimation. In the case that multiple sequences of predictors are used in combination (when tracking for example the state of a 3D object) RANSAC does provides some help. However, it cannot be used to decide whether its final estimation is correct or not and it has been shown that versions in which a reliability measure is used (e.g. pRANSAC, probabilistic RANSAC) perform much better that the original RANSAC. In general, reliability issues are important for automatically detecting tracking failures, dealing with occlusions and for recovery from tracking failures. I think that this should be made clear in the manuscript. In addition, methods that address the issue of the reliability in a probabilistic framework, should be included at least in the state of the art section (such references were included in the original submission but were removed from the revised manuscript). In particular, references such as [15][16] that define the discriminative tracking in a probabilistic framework and [19] that explicitly addresses the problem of observation relevance/reliability should be retained. Both are important for the assessment of the reliability of the estimation and the later very relevant for occlusion handling. Regarding the experimental section, an explanation of what the loss-of-locks means would be necessary (I assume that the meaning is that the tracking is lost, but how is that detected?), as well as an explanation of how the tracking is reinitialized. ===================== 1. Which category describes this manuscript?: Research/Technology 2. How relevant is this manuscript to the readers of this periodical? If you answer Not very relevant or Irrelevant please explain your rating under Public Comments below.: Relevant 1. Please evaluate the significance of the manuscript’s research contribution.: Good 2. Please explain how this manuscript advances this field of research and/or contributes something new to the literature. : The main contribution is the use of a sequence of regressors. At each step the uncertainty area is reduced . The sequence at which the regressors are applied is chosen in a way that minimizes the computational complexity. 3. Is the manuscript technically sound? In the Public Comments section, please provide detailed explanations to support your assessment: Appears to be - but didn't check completely 4. How thorough is the experimental validation (where appropriate)? Please discuss any shortcomings in the Public Comments section.: Compelling experiments; clearly state of the art 1. Are the title, abstract, and keywords appropriate? If not, please comment in the Public Comments section.: Yes 2. Does the manuscript contain sufficient and appropriate references? Please comment and include additional suggested references in the Public Comments section.: Important references are missing; more references are needed 3. Does the introduction state the objectives of the manuscript in terms that encourage the reader to read on? If not, please explain your answer in the Public Comments section.: Yes 4. How would you rate the organization of the manuscript? Is it focused? Please elaborate with suggestions for reorganization in the Public Comments section.: Satisfactory 5. Please rate the readability of the manuscript. Explain your rating under Public Comments below. : Readable - but requires some effort to understand 6. How is the length of the manuscript? If changes are suggested, please make explicit recommendations in the Public Comments section.: About right Please rate the manuscript overall. Explain your choice.: Good **************************************************************** Reviewer: 3 Recommendation: Author Should Prepare A Major Revision For A Second Review Comments: The paper has been improved but is still not completely clear. In the introduction (state of the art) one could understand that trained trackers completely solve the problems of gradient descent techniques: convergence to a local minimum, an unknown number of iterations and an unknown basin of convergence. This is not the case of the proposed trained tracker, as confirmed by the fact that a compromise must be done between the the precision (i.e. the convergence to the true solution) and the the computational complexity (i.e. number of SLLiPs, number of RANSAC iterations, ...). The experimental results are still not completely convincing and should be improved. My point here is not to prove that trained trackers can have larger basin of attraction than gradient based trackers. This has been proved before. My concern is about a fair, clear and repeatable comparison between methods. This also apply to the comparison with standard trained trackers. Concerning gradient-based methods the experimental results need major improvements. For example, concerning the computational complexity, why you compare your method with KLT and not with another more efficient gradient-based method ? Also, the authors do not take into account the fact that subset selection of pixel can be performed also for gradient descent approaches. The comparison with the KLT approach is not clear. The figure 12(a) shows the error rate when the accuracy is worse than 5 pixels. I guess that the accuracy is between the ideal translation and the measured translation. The range of the ideal translations is from 5 to 100 pixels. The authors should show the image they used in the experiment, the points selected for the KLT methods (how many points ?) and the points selected for the LP and LP+DP methods. For a correct comparison it would be better to select the same points for all methods. It is not clear why in figure 12(a) the error rate for LP and LP+DP methods starts from 20 pixels. Which is the error rate for LP and LP+DP methods when the translation is 5, 10 or 15 pixels ? Should I deduce that the LP and LP+DP methods give no results in that cases ? Concerning the comparison with trained trackers some minor improvements are needed. Can you explain better the sentence: "Position is measured in each corner as a percentage of a current size of the object upper edge". Since you have the ground truth of each corners why you do not simple measure the difference between the true coordinates of each corner and the measured coordinates after tracking ? Concerning robustness, what do you mean by "the accuracy was worse than 25 %" ? ================== 1. Which category describes this manuscript?: Research/Technology 2. How relevant is this manuscript to the readers of this periodical? If you answer Not very relevant or Irrelevant please explain your rating under Public Comments below.: Relevant 1. Please evaluate the significance of the manuscript’s research contribution.: Good 2. Please explain how this manuscript advances this field of research and/or contributes something new to the literature. : The main contribution of the paper is the idea that the training stage should take into account the computational complexity of the tracking. The computational complexity of the tracking is proportional to the number of pixels of the support. 3. Is the manuscript technically sound? In the Public Comments section, please provide detailed explanations to support your assessment: Appears to be - but didn't check completely 4. How thorough is the experimental validation (where appropriate)? Please discuss any shortcomings in the Public Comments section.: Insufficient; clearly inferior to state of the art, or necessary tests are absent 1. Are the title, abstract, and keywords appropriate? If not, please comment in the Public Comments section.: Yes 2. Does the manuscript contain sufficient and appropriate references? Please comment and include additional suggested references in the Public Comments section.: Important references are missing; more references are needed 3. Does the introduction state the objectives of the manuscript in terms that encourage the reader to read on? If not, please explain your answer in the Public Comments section.: Could be improved 4. How would you rate the organization of the manuscript? Is it focused? Please elaborate with suggestions for reorganization in the Public Comments section.: Satisfactory 5. Please rate the readability of the manuscript. Explain your rating under Public Comments below. : Readable - but requires some effort to understand 6. How is the length of the manuscript? If changes are suggested, please make explicit recommendations in the Public Comments section.: About right Please rate the manuscript overall. Explain your choice.: Good