COMPARATIVE COMPLAINT TEXT CLASSIFICATION USING SVM, CNN, AND BERT WITH PRIORITY SCORING FOR NYC 311 SERVICE REQUESTS

Abstract

Abstract

Public complaint management systems like NYC 311 generate millions of service requests annually, posing significant challenges regarding manual categorization and resource prioritization. This study has two objectives: (1) to compare the classification performance of Support Vector Machine (SVM), Convolutional Neural Network (CNN), and Bidirectional Encoder Representations from Transformers (BERT) on a large-scale city complaint dataset, and (2) to develop a complaint prioritization framework that combines normalized complaint frequency with average resolution time. The NYC 311 Service Request dataset, comprising 1,203,418 cleaned records, was utilized, with a sample of 20,000 records selected via stratified sampling. Pre-processing steps included converting text to lowercase, removing numbers and special characters, and normalizing whitespace. All models were evaluated without class weighting using a shared test set of 4,000 records. The results indicate that BERT achieved the highest macro F1-score (0.9167), outperforming SVM (0.8861) and CNN (0.8900), while all three models recorded similar aggregate accuracy rates (95.60%–95.83%). Both SVM and CNN yielded a recall of zero for the "Noise – Street/Sidewalk" category, whereas BERT was able to partially recognize it demonstrating that contextual language understanding is essential for distinguishing complaint categories with semantic ambiguity. Priority score analysis identified "Illegal Parking" (0.6007) and "HEAT/HOT WATER" (0.5437) as the highest-priority complaint types, while geographic analysis revealed that Brooklyn and the Bronx bear a disproportionately higher burden of complaints compared to other boroughs. These findings indicate that transformer-based classification combined with composite priority assessment can support data-driven municipal complaint management that accounts for geographical differences.