Answer Validation in Question-Answering System using Support Vector Machine

Document Type : Persian Original Article

Authors

1 Department of Electrical and Computer Engineering, University of Hormozgan, Bandar Abbas, Iran.

2 University of Hormozgan

3 Department of Multimedia, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Malaysia.

Abstract

Question-answering system is a special type of information retrieval system. Considering a set of documents (such as Web or local set of documents), the system should answer the questions asked in the natural language form. Question-answering systems generally consist of question processing and analysis, keyword generation, information retrieval, answer extraction, and answer validation. In such systems, selecting proper answers for user question is an important task which influences performance of the whole system. An appropriate answer validation technique can increase the performance of the question-answering system. In this study, Support Vector Regression (SVR) is employed in order to perform answer validation. SVR eliminates the risk of getting stuck in local minima by reducing the operational risk. The proposed system is applied on some of the TREC and Wikipedia questions repositories. In order to evaluate performance of the proposed system, following measures are calculated: Mean Reciprocal Ranking (MRR) and F-measure. Based on experimental results, the proposed system achieved 81% MRR and 49.7% F-measure which shows higher performance than systems with no answer validation and also systems using neural network-based answer validation.

Keywords


[1]     سمیرا بختیار قلعه تکی، میر حسین دزفولیان و محرح منصوری زاده، ،"ارائه روشی ترکیبی جهت امتیاز دهی پاسخ در سیستم­های پرسش و پاسخ"،کارشناسی ارشد، دانشگاه بوعلی سینا، همدان، صفحه­ی5-25، بهمن 92.
[2]     راحیل  قبادی، میر حسین دزفولیان و محرم  منصوری زاده ، "سامانه هوشمند سیستم پرسش و پاسخ چند سطحی در محیط­های یادگیری الکترونیکی"، کارشناسی ارشد، دانشگاه بوعلی سینا، همدان، صفحه­ی 10- 45، بهمن 92.
 
  [3]     M. R. Kangavari, S. Ghandchi, M. Golpour , “A New Model for Question Answering Systems.” Proceedings of World Academy of Science:Engineering & Technolig, pp. 3-5,2008.
  [4]     O. Kolomiyets and M. F. Moens, “A  survey  on  question answering  technology  from  an  information  retrieval perspective”, Information Sciences., vol.181,  pp. 5412-5434,2011.
  [5]     M. A.  GreenWood, “Open-Domain Question- Answering”,Ph.D. Thesis, University of Sheffield, UK, pp. 23-156, 2005.
  [6]     P. Gupta and V. Gupta,   “A  survey  of  text  question answering techniques” , International Journal of Computer Applications., vol. 53, no. 4, pp. 1–8, 2012.
  [7]     C. D. Manning, P. Raghavan and H. Schütze, “Introduction to information retrieval”, Cambridge  University  press, 2008.
  [8]     S. Tellex, B. Katz, J. Lin, A. Fernandes and G.  Marton, “Quantitative  evaluation  of passage retrieval  algorithms for question answering” ,In Proceedings of the 26th Annual International ACM SIGIR Conference, pp. 41–47, 2003.
  [9]     G. Lee, J.  Seo, S. Lee and H. Jung, “SiteQ:  Engineering  High  Performance  QA  System  Using  Lexico-Semantic Pattern Matching and Shallow NLP”,in TREC, 2001.
[10]     U.  Hermjakop, ”Parsing and question classification for question answering”, In Proceeding of the workshop on Open domain question answering; Vol. 12, pp. 1-6; 2001.
[11]     S. K.  Ray , S. Shailendra  and P. Bhagwati, “ A Semantic Approach for Question Classification using WordNet and Wikipedia.” Pattern Recognition Letters, vol. 31, pp. 1935-1943, 2010.
[12]     K.  Hacioglu and W. Ward, “ Question Classification With Support Vector Machine and Error Correcting Codes” , In Proceeding of the 2003 conference of the North American Chapter of Association for Computational Linguistics on Human Language Technology: comparing volume of the Proceeding of HLT-NAACL,  pp. 28-30, 2003.
[13]     P. Moreda, H. Lioners, E. Saquete and M. Palomar, “Combining Semantic Information in Question Answering System”,  Information Processing & Management , vol.47 , pp. 870-885, 2011.
[14]     A.  Moschitti and S. Quarteroni, “ Liguistic Kernel for Answer Ranking in Question Answering ”; Information Proccessing & Mnagement , vol.47, pp. 825-842, 2011.
[15]     J. Ko, L.  Si and E. Nyberg, “Combining  Evidence With a Probabilistic Framework for Answer Ranking and Answer Merging in Question Answering”, Information Processing & Management, vol.46, pp.  541-554, 2010.
[16]     M. Umamehaswari, M. Ramprasath and H. Hariharan, “Improved Question  Answering System by Semantic Reformulation”, In Advanced Computing(ICOAC), 40th  International Conference on,  pp. 1-4, 2012.
[17]     H.H. Matthias, W.D.W.  Edward and F.  Sadaoki, "Question  Answering  Using  Statistical  Language  Modeling", Computer Speech and Language , vol.26, pp. 193–209, 2012.
[18]     W.  Wang, C.  Men and W. Lu, “Online  Prediction  Model  Based  on  Support  Vector  Machine”,  Neurocomputing, vol.71, pp. 550-558, 2008.
[19]     B. Loni, M. long and D. MJ.  Tax, “Enhanced Question  Classification  with  Optimal Combination of Features”, Master of Sceience Thesis,  Department  of  Media  and  Knowledge Engineering, Delft  university  of  Technology, 2011.
[20]     Z.  Huang, M. Thint and Z. Qin, “Question Classification  Using  Headwords  and  their Hypernyms”,  In  Proceedings  of  the Conference  on  Empirical  Methods  in Natural Language Processing, pp. 927–936, 2008.
[21]     P.  Blunsom, K.  Kocik and J.R.   Curran, “Question Classification With Loglinear Models”;  In  Proceedings  of  the  29th  annual international  ACM  SIGIR  conference on Research  and  development  in  information retrieval, pp. 615–616, 2006..
[22]     V.  Cherkassky and Y. Ma, “Practical Selection of  SVM  Parameters  and  Noise  Estimation  for SVM Regression”, Neural Network, vol.17,  pp. 113–126, 2004.
[23]     S. J.  Yen, Y. C. Wu, J. C.  Yang, Y. S.  Lee, C.  J. Lee and J.J. Liu, “A Support Vector Machine Based Context Ranking  Model  for Question Answering.”  Information Sciences, vol.224, pp. 77-87, 2013.
[24]     H. J.  Jang, K.Y. Sung, M.G. Jang and S.H. Myaeng, “Compositional question answering: A divide and conquer approach”, Information  Processing & Management,vol.47,  pp.  808–824, 2011.
[25]     A. Tahri, and O. Tibermancine “DBPEDIA  Based  Factoid  Question Answering  System”,  International  Journal  of Web & Semantic Technology ,vol.4, pp. 1-16 ,2013.