Submit a single Jupyter Notebook based on the given template, which must contain • Point-by-point answers of the following issues; • If applicable, corresponding codes that are reproducible (i.e., they produce output consistent with your answers), and well documented (in the form of comments and markdown cells). You need to address the following issues. The percentages below add up to 100%, but they contribute to 8% of the overall grade of the course.
a. Summarize the background of the dataset. b. State the size of the dataset. c. For each variable, describe what it represents and its data type (numerical or categorical).
a. For each variable, determine the percentage of missing data. For any column with missing data, describe how you resolve the issue. Clearly state any assumption you made. b. For each variable, identify outliers (if any) and describe how you resolve the issue. Clearly state any assumption you made. c. For categorical variables, perform the necessary encoding.
a. For each variable, provide relevant summary statistics. b. For each variable, provide an appropriate visualisation depicting the distribution of its values, and summarize any key observation(s) you made. c. Perform bi-variate analyses on the variables. You do not need to analyse every pair; only focus on the pairs you believe are worth investigating and explain your choices. For each pair, describe the relationship between the two variables. Use appropriate statistical methods and/or visualization.
Submit a single Jupyter Notebook based on the given template, which must contain • Include Part I on EDA as well to produce a self-contained notebook. You may fine-tune Part I findings and/or codes based on feedback received on Part I submission; • Point-by-point answers of the following issues; • If applicable, corresponding codes that are reproducible (i.e., they produce output consistent with your answers), and well documented (in the form of comments and markdown cells).
You need to address the following issues.
a. Formulate one regression problem and one classification based on the dataset. b. State which problem (regression or classification) you would be investigating and why. c. Clearly specify the dependent variable you are predicting, and its significance.
a. Describe the steps taken to split the dataset into train and test sets b. State the model(s) you will train on, and explain your choice(s). Please limit yourself to no more than three models—Grading is based on the validity and soundness of your model, rather than the quantity. c. For each model, perform the training, and report the trained parameters and the training scores, if applicable.
a. For each model, predict the response variable on the test set. b. Describe the metric you use to evaluate your model(s). Report the test scores for each model. c. If you trained more than one model, identify the final model you would choose for the prediction task, and explain your choice.
a. Interpret what your model is implying, and summarize any insight you have drawn from the project. Explain if it is consistent with intuition, and if not, provide a plausible justification. b. Share any lesson you have learned from the project.
a. Describe the contribution of each member, including both the tangible (e.g., implementation and writing) and intangible (e.g. generating ideas, planning).
a. List any sources you have cited.