Statistical Learning (STAT3040) Final Project - Semester 1, 2025
• INSTRUCTIONS:
1. The project must be typed (not handwritten). You should produce a reproducible document through Quarto in RStudio [https://rmarkdown.rstudio.com] for the project. The report should be no longer than 16 A4 pages [single sided] with a font size no smaller than 11 point. This does not include an appendix.
2. The project due date is listed on the Wattle (Turn-it-in) site. Upload the project through Wattle using Turn-it-in. You should submit your project in three different parts. Note that there are three tabs when submitting to Turnitin.
(a) A pdf file or Word file of your report (this will likely include important R code to highlight what you have done).
(b) A ‘.qmd’ file [a quarto file].
(c) A ‘.r’ file [provide all the R code from the ‘.qmd’ and any additional R code used that was done outside of the ‘.qmd’ file.]
To get the ‘.r’ file from an ‘.qmd’ file use:
> library(knitr)
> purl("xxx.qmd")
3. When putting together your report, write clearly and succinctly. Make sure to utilise ideas from throughout the course. Use appropriate graphs, tables, mathematics, and code to aid in describing your point or thinking process. Do not just “print” a set of results. Every result should be discussed and have a reason for being presented. No points will be awarded unless you clearly discuss what you are doing.
4. As this is a final project you may not discuss the project (questions, solutions, code, etc.) with your classmates or other individuals. You can discuss these with me or your tutor during our consultation times. You must independently write your own solutions. This includes all computer code, written language, and mathematics. Please see the university resources on Academic Integrity https://www.anu.edu.au/ students/academic-skills/academic-integrity for more details. For this assignment, you do not have to cite material from either of the two textbooks or class slides. Any other material should be appropriately cited, including websites. You must also cite any use of Artificial Intelligence (AI). Additionally provide a single paragraph discussing your use of AI for the assignment.
5. No late projects will be accepted. Note that this final project is considered similar to a final exam.
6. You may post questions to the discussion board, however please let myself or Mr. Zheng Xu answer the questions.
7. Have fun with the exploration!
The project consists of analysing and statistically modelling data on U.S. medical insurance claims over a several year period. Beyond the prediction competition, you should determine and discuss important co- variates (statistically and scientifically) and also appropriately account for and discuss uncertainty.
Data fields:
• claim (Y1 ) - claims in USD.
• comp (Y2 ) - medical complications (1 = yes, 0 = no).
• age - in years.
• proc - number of medical procedures (Low, Medium, High).
• drugs - number of prescribed drugs the individual is taking.
• emerg - number of emergency room visits.
• comorb - number of comorbidities.
• duration - number of days spent hospitalized.
The project should be written as report. Within the report there will be two data modelling and prediction components. Each one of these is described below. For each of the two components, you should consider at least 5 different classes of models/algorithms. Each of the models should be clearly outlined and compared (possible items to consider: k-fold cross-validation, uncertainty, predictive rank, etc). Also outline, discuss, and compare several naive predictions. Your best predictive model for each of the two components should be discussed in terms of statistically and scientifically important covariates. Additionally you should discuss the limitations. If you feel your best predictive model is not the best model for discussing the relationship between the covariates and a response, you may discuss another model (or combination of models) as well.
• Modeling Components:
1. claims: Based on any of the other variables in the training data, build models to predict and understand claims. The criterion for the predictions is Mean Squared Error. Make sure that all predictions are justified by a model. If a perfect score (or even top 10% score) is achieved without justification, a high penalty will be applied.
— The private link for the competition (which should not be shared outside of the class) is:
https://www.kaggle.com/t/f837aa93ce3c46bbb41e8537aec64811
2. comp: Based on any of the other variables in the training data, build models to predict and understand the classification of comp. The criterion for the predictions is the Correct Classification Rate. Make sure that all predictions are justified by a model. If a perfect score (or even top 10% score) is achieved without justification, a high penalty will be applied.
— The private link for the competition (which should not be shared outside of the class) is:
https://www.kaggle.com/t/5370523fef184651b969957b8945cf10
Marking Components:
1. Introduction and conclusion/discussion sections [15 points]
2. Exploratory data analysis [15 points]
3. Modelling component 1 [30 points]
4. Modelling component 2 [30 points]
5. Top predictions [10 points] - For each of the two prediction competitions the top 10% of individuals on the Private Leaderboard (when the competition closes) will be awarded 2.5 points while the top individual will be awarded 5 points. The predictions must be justified by a model and its discussion. Note that there is a Public Leaderboard which you are able to see and a Private Leaderboard which you cannot see until the competition closes. The evaluation of the test data are randomly and approximately evenly split between the public and private leaderboards.
Your writing, organisation, and presentation will be considered when grading.