Mathematics is recognized as an important subject in the school curriculum in Malaysia. It is a compulsory subject for many courses in matriculation, private colleges and universities. The purpose of this study is to identify the factors that influence the matriculation students in mathematical problem solving. Bayesian Network, a data mining technique, is used in this study to analyse the causal relationships. Bayesian network is a probabilistic graphical model which converts variables and their dependent relationships into nodes and arcs respectively. We compare the resultant networks using the different constraint and score based algorithms to identify the main factors affecting students in problem solving of mathematics. We found that students in Penang Matriculation College faced problem solving in mathematics owing to their problem with mathematical symbols. Hence, the students have no confidence in answering mathematics problems especially in questions related to their understanding of mathematical symbols.