{ "cells": [ { "cell_type": "markdown", "id": "5eb8c31b", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Tutorial on Classification II: Dimension reduction\n" ] }, { "cell_type": "markdown", "id": "d15f67cd", "metadata": {}, "source": [ "
\n", " Context\n", " \n", "- Fisher Linear discriminant analysis (dimension reduction)\n", "- Classification by month\n", " \n", "
" ] }, { "cell_type": "markdown", "id": "fc4d03f7", "metadata": {}, "source": [ "In this tutorial, we are going to use the same data set as in the previous tutorial except that we are now going to explore the concept of dimension reduction. We still consider the same problem of classifying the days into rainy and dry days. In the original data set, there are 10 input features and we already saw that pressure is a good predictor. What about the other variables? " ] }, { "cell_type": "markdown", "id": "834f3c96", "metadata": {}, "source": [ "Let's first prepare the data set as we did in the previous tutorial: we load meteorological variables for paris for between 2000 and 2009 (subset of the ERA5 data set). As before, we assign each day to a class \"rainy day\" or \"dry day\". We also split this data set into a training data set and a test data set with the method `train_test_split` of the scikit-learn library." ] }, { "cell_type": "code", "execution_count": 1, "id": "eab2de3b", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 2, "id": "2203d00e", "metadata": {}, "outputs": [], "source": [ "df = pd.read_csv(\"data/era5_paris_sf_2000_2009.csv\", index_col='time', parse_dates=True)\n", "df_norm = (df - df.mean()) /df.std()\n", "df_day = df_norm.resample(\"D\").mean()\n", "\n", "# normalized threshold\n", "precip_th = -0.2\n", "# add tag \n", "df_day['tag'] = df_day['tp'].where(df_day['tp']> precip_th, 0)\n", "df_day['tag'] = df_day['tag'].where(df_day['tp']<= precip_th, 1)\n", "\n", "from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(df_day.drop(columns=['tp', 'tag']),df_day['tag'],test_size=.3,random_state=0)" ] }, { "cell_type": "markdown", "id": "c9822881", "metadata": {}, "source": [ "In the previous tutorial, we used the LDA method to predict the class of the data set. If you assume that the data set is Gaussian, Fisher-LDA is actually equivalent to the LDA and so we core functions are the same in Scikit-learn. There is however one extra interesting feature that we derived in Fisher-LDA because it can be used as a dimension reduction method. These new dimensions are the one that maximize the *between-class* variance and minimize the *within-class* variance." ] }, { "cell_type": "code", "execution_count": 3, "id": "1fa8c3ea", "metadata": {}, "outputs": [], "source": [ "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n", "lda = LinearDiscriminantAnalysis()" ] }, { "cell_type": "markdown", "id": "4768d3fc", "metadata": {}, "source": [ "Fit the train data set with the LDA model?" ] }, { "cell_type": "code", "execution_count": 4, "id": "e8116c90", "metadata": {}, "outputs": [], "source": [ "# your code here" ] }, { "cell_type": "markdown", "id": "7108be07", "metadata": {}, "source": [ "Make a prediction for the train and test data sets." ] }, { "cell_type": "code", "execution_count": 5, "id": "8088ffa4", "metadata": {}, "outputs": [], "source": [ "# your code here" ] }, { "cell_type": "markdown", "id": "897782d1", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "You can visualize a summary of the results with `classification_report`" ] }, { "cell_type": "code", "execution_count": 6, "id": "877cb863", "metadata": {}, "outputs": [], "source": [ "#from sklearn.metrics import classification_report\n", "# your code here" ] }, { "cell_type": "markdown", "id": "dd0ef9db", "metadata": {}, "source": [ "> ***Question***\n", "> - Do you feel like this model with more input features is doing better perdiction than the previous model that had only 2 input features?" ] }, { "cell_type": "markdown", "id": "70603b67", "metadata": {}, "source": [ "So maybe one (or more) of these extra features is helping doing a better prediction. F-LDA can help us shrink the data set to only 1 dimension that is the best dimension for classification. This dimension no longer has a physical meaning but we can use it to compact a data set while still retaining the cluster separation.\n", "\n", "> ***Question***\n", "> - With the function `lda.transform`, project the test data onto that particular dimension." ] }, { "cell_type": "code", "execution_count": 7, "id": "0b77c814", "metadata": {}, "outputs": [], "source": [ "# your code here\n", "# X_lda = " ] }, { "cell_type": "markdown", "id": "623d83d6", "metadata": {}, "source": [ "> ***Question***\n", "> - What is the dimension of X_lda? Why?\n", "> - Add the `X_lda` variable to `X_test` DataFrame." ] }, { "cell_type": "code", "execution_count": 8, "id": "473afe10", "metadata": {}, "outputs": [], "source": [ "# your code here" ] }, { "cell_type": "markdown", "id": "84c91b25", "metadata": {}, "source": [ "In the same figure, do a boxplot for each class. As before, you can use the `by=y_train.values` argument in order to separate the data set into the rainy class and dry class. You can remove the labels below the figure with `plt.xlabel(\"\")`" ] }, { "cell_type": "code", "execution_count": 9, "id": "c294f5dc", "metadata": {}, "outputs": [], "source": [ "# your answer here" ] }, { "cell_type": "markdown", "id": "112263e6", "metadata": {}, "source": [ "> ***Question***\n", "> - Plot the histogram of this projected data for each class.\n", "\n", "Hint: In order to do this plot, you'll have to to first `.groupby(y_train)` and then use `.plot.hist(...)`" ] }, { "cell_type": "code", "execution_count": 10, "id": "374902f8", "metadata": {}, "outputs": [], "source": [ "# your answer here" ] }, { "cell_type": "markdown", "id": "1606c34d", "metadata": {}, "source": [ "So we shrinked our 10-dimensional data set into a 1d data set... and this dimension is even better than pressure only to perform a classification. If we need to represent the data and highlight the class separation, we probably need to use this dimension." ] }, { "cell_type": "markdown", "id": "c19b4211", "metadata": {}, "source": [ "### Fisher LDA in 2D" ] }, { "cell_type": "markdown", "id": "b47b0011", "metadata": {}, "source": [ "By design, the maximum number of dimensions that you can get is $K-1$, where $K$ is the number of classes. We are going to explore how 3 different months really belong to 3 distinct categories.\n", "\n", "Adjust the parameters in the code below to select 3 months: you can pick months that are close in time or instead far apart." ] }, { "cell_type": "code", "execution_count": 11, "id": "917eaafc", "metadata": {}, "outputs": [], "source": [ "#mo1 = 1\n", "#mo2 = 7\n", "#mo3 = 8\n", "\n", "#X = df_day[df_day.index.month.isin([mo1,mo2, mo3])]\n", "#y = df_day[df_day.index.month.isin([mo1,mo2, mo3])].index.month" ] }, { "cell_type": "markdown", "id": "d7b7d530", "metadata": {}, "source": [ "Follow the exact same methodology as before and do a scatter plot of the projected data in 2d. Use a different color code for each class." ] }, { "cell_type": "code", "execution_count": 12, "id": "ece89e33", "metadata": {}, "outputs": [], "source": [ "# your code here" ] }, { "cell_type": "markdown", "id": "e6c22a42", "metadata": {}, "source": [ "In this 2d space, you can visualize two elements:\n", "- The between class spread (how classes are separated from each other)\n", "- The within class spread (how points within a class are separated from each other)." ] }, { "cell_type": "markdown", "id": "bb609ef3", "metadata": {}, "source": [ "Of course months that are well separated in time will exhibit a large between class gap compared to months that are close.\n", "\n", "> ***Questions***\n", "> - Which variables are responsible for that separation?\n", "> - try to remove these variables from your model, and see if you can you still separate months appart? " ] }, { "cell_type": "markdown", "id": "5e186998", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "***\n", "## Credit\n", "\n", "[//]: # \"This notebook is part of [E4C Interdisciplinary Center - Education](https://gitlab.in2p3.fr/energy4climate/public/education).\"\n", "Contributors include Bruno Deremble and Alexis Tantet.\n", "\n", "
\n", "\n", "
\n", " \n", "\"Logo\n", "\n", "\"Logo\n", "\n", "\"Logo\n", "\n", "\"Logo\n", "\n", "\"Logo\n", "\n", "\"Logo\n", "\n", "\"Logo\n", " \n", "
\n", "\n", "
\n", "\n", "
\n", " \"Creative\n", "
This work is licensed under a   Creative Commons Attribution-ShareAlike 4.0 International License.\n", "
" ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.12" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autoclose": true, "autocomplete": false, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 5 }