PatternRecognition

Project files for pattern recognition group assignment

Files

Currently contains the following files:

data/WikiEssentials_L4.7z: output file of the WikiVitalArticles program. Each document is included in its entirety (but split by paragraph).
preprocess_utils.py: preprocessing functions for Wiki data.
model_utils.py: various utility functions used for modeling (e.g. loading embeddings).
1_preprocess_raw_data.py: preprocessing of raw input data. Currently shortens each article to first 8 sentences.
2_baseline_model.py: tokenization, vectorization of input data and baseline model (1-layer NN with softmax classifier).

conda env create -f environment.yml

conda activate VitalWikiClassifier
conda install pytorch torchvision cudatoolkit=9.2 -c pytorch -c defaults -c numba/label/dev

install.packages("reticulate")

Check the .Rprofile file to ensure that R knows where to find your anaconda distribution.

Name		Name	Last commit message	Last commit date
Latest commit History 103 Commits
.idea		.idea
.vscode		.vscode
data		data
embeddings		embeddings
img		img
model_cnn		model_cnn
models		models
predictions		predictions
results		results
.Rprofile		.Rprofile
.gitignore		.gitignore
1_preprocess_raw_data.py		1_preprocess_raw_data.py
2_baseline_model.py		2_baseline_model.py
3_cnn_model.R		3_cnn_model.R
4_lstm_model.py		4_lstm_model.py
5_han_model.py		5_han_model.py
6_statistical_test.R		6_statistical_test.R
CNN_results.csv		CNN_results.csv
Final_calibrated.rds		Final_calibrated.rds
HAN.py		HAN.py
LSTM.py		LSTM.py
PatternRecognition.code-workspace		PatternRecognition.code-workspace
Pattern_Recognition_Project__Wikipedia_Article_Classification__Full_Paper_.pdf		Pattern_Recognition_Project__Wikipedia_Article_Classification__Full_Paper_.pdf
README.md		README.md
WikiFilter.Rproj		WikiFilter.Rproj
environment.yml		environment.yml
evaluate_HAN_results.R		evaluate_HAN_results.R
evaluate_baseline_results.R		evaluate_baseline_results.R
model_utils.py		model_utils.py
preprocess_utils.py		preprocess_utils.py