How to Cite
Abstract
In this study a tool was developed which marks automatically computer programs written by novices in Java. Existing open source static analysers were evaluated in the development. Metrics and marking schemes were developed to mark and grade the programming assignments automatically. A sample of 59 first-year programming students’ projects was selected for marking. The projects were marked manually and automatically. The results showed that there is a high correlation between manual and automatic marking for all criteria. The mean Kendall’s tau is over 0.75 indicating a very high level of correlation between manual and automatic marking. However, depending on the marking criteria, non-systemic variations were found.
References
Berry, R. E., & Meekings, B. A. E. (1985). A style analysis of C programs. Communications of the Association for Computing Machinery, 28(1), 80–88.
Blumenstein, M., Green, S., Nguyen, A., & Muthukkumarasamy, V. (2004). An experimental analysis of GAME: A generic automated marking environment. In Proceedings of the 9th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE ’04) (pp. 67–71). UK, Leeds.
Burkhardt, J. M., Détienne, F., & Wiedenbeck, S. (2002). Object-oriented program comprehension: Effect of expertise, task, and phase. Empirical Software Engineering Journal. 7(2), 115–156.
Christopher, C. N. (2006, May 10). Evaluating static analysis frameworks. Retrieved from http://www.cs.cmu.edu/~aldrich/courses/654/toolschristopher -analysis-frameworks-06. pdf
Conover, W. J. (1980). Practical Non-Parametric Statistics (2nd ed.). New York: John Wiley and Sons.
Dreyfus, H. L., Dreyfus, S. E., & Athanasiou, T. (1986). Mind over machine: The power of human intuition and expertise in the era of the computer. New York: The Free Press.
Du Boulay, B. (1986). Some difficulties of learning to program. Journal of Educational Computing Research, 2(1), 57–73.
Fagan, M. E. (1999). Design and code inspections to reduce errors in program development. IBM Systems Journal, 38(2–3), 258–287.
Fenton, N.P., & Pfleeger, S. (1997). Software metrics: A rigorous and practical approach. (2nd ed.). Boston: PWS Publishing, International Thompson Computer Press.
Higgins, C. A., Gray, G., Symeonidis, P., & Tsintsifas, A. (2005). Automated assessment and experiences of teaching programming. Journal of Educational Resources in Computing, 5(3), 5–25.
Hovemeyer, D., & Pugh, W. (2004). Finding bugs is easy. SIGPLAN Notices, 39(12), 92–106.
Hsu, A., Jagannathan, S., Mustehsan, S, Mwmufiya, S, & Novakouski, M. (April, 24, 2007). Analysis Tool Evaluation: PMD Final Report. School of Computer Science, Carnegie Mellon University. Retrieved from: http://www.cs.cmu.edu/~aldrich/courses/654/tools/hsu-pmd-07.pdf
Hung, S, L., Kwok, L. F., & Chan, R. (1993). Automatic programming assessment. Computers and Education, 20(2), 183–190.
Joy, M., Griffiths, N., & Boyatt, R. (2005).The BOSS online submission and assessment system. Journal on Educational Resources in Computing, 5(3), 1–28.
JUNIT(2010). Retrieved from http://junit.sourceforge.net/
Lahtinen, E., Ala-Mutka, K., & Järvinen, H. (2005). A study of the difficulties of novice programmers. In ITiCSE ’05: Proceedings of the 10th Annual SIGCSE
Conference on Innovation and Technology in Education (pp.14–18), New York: ACM Press.
Lister, R., & Leaney, J. (2003). First year programming: Let all the flowers bloom. In Greening, T. and Lister, R., (Eds.) Proceedings of the Fifth Australasian Computing Education Conference (ACE 2003) (pp. 221–230). Adelaide: ACSm .
Mengel, S., & Yerramilli, V. (1999). A case study of the static analysis of the quality of novice student programs. Paper presented at Thirtieth SIGCSE Technical
Symposium on Computer Science Education, New York.
Moha, N., Gueheneuc, Y. G., Duchien, L., & Meur, A. F. L. (2010). DECOR: A method for the specification and detection of code and design smells. IEEE
Transactions on Software Engineering, 36, 20–36.
Nagappan, N., Williams, L., Hudepohl, J., Snipes, W., & Vouk, M. (2004, November). Preliminary results on using static analysis tools for software inspection. Paper presented at Fifteenth IEEE International Symposium on Software Reliability Engineering (ISSRE 2004), (pp. 429–439), St. Malo, France.
Olan, M. (2003). Unit testing: Test early, test often. Journal of Computing Sciences in Colleges, 19(2), 319–328.
Reek, K. A. (1989). The TRY system or how to avoid testing student programs. Paper presented at the Twentieth SIGCSE Technical Symposium on Computer
Science Education (SIGCSE ‘89) (112-116).
Rist, R. (1996). Teaching Eiffel as a first language. Journal of Object-Oriented Programming, 9, 30–41.
Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13(2).137–
Seo, B., Kim, D., Kim, Y. G., & Lee, S. (2009). Mini-Project: Tool or analysis practicum. Retrieved from http://www.cs.cmu.edu/~aldrich/courses/654/tools/
theCruX-PMD-2009.pdf
Spinellis, D. (2006). Code quality: The open source perspective. Boston, MA: Addison-Wesley.
Tremblay, G. and Labonte, E. (2003, July). Semi-automatic marking of Java programs using JUunit. In the Proceedings of the International Conference on
Education and Information Systems: Technologies and Applications (EISTA’03) (pp.42–47). Orlando, FL: International Institute of Informatics and Systemics.
Tremblay, G., Guerin, F., Pons, A., & Salah, A. (2008). Oto, a generic and extensible tool for marking programming assignments. Software: Practice and Experience, 38(3), 307-33.
Vashishtha, S., & Gupta, A. (November 25, 2008). Automated code reviews with Checkstyle Part 1. Retrieved from: http:// www.javaworld.com/javaworld/jw-11-2008/jw-11-checkstyle.html.
Wiedenbeck, S., Ramalingam, V., Sarasamma, S. & Corritore, C. L. (1999). A comparison of the comprehension of object-oriented and procedural programs
by novice programmers. Interacting with Computers, 11, 255–282.
Winslow, L. E. (1996). Programming pedagogy - A psychological overview. SIGCSE Bulletin, 28(3), 17–22.