LegalEval Large Language Model

Developed a custom PyTorch data loader that enhanced LLaMA2 model accuracy by 15% for log probability analysis, showcasing expertise in optimizing deep learning workflows. Leveraged Hugging Face Accelerate and DeepSpeed ZeRO 3 to implement model parallelism, reducing training time by 30% on A100 GPUs. Applied these advancements to legal text classification tasks for SemEval 2023 LegalEval, including Rhetorical Roles Labeling and Court Judgment Prediction, delivering impactful results in natural language processing.

Key Contributions

Data Management and Model Optimization

• Developed a custom PyTorch data loader to efficiently process and manage large-scale datasets.
• Optimized LLaMA2 model performance for log probability analysis, achieving a 15% increase in accuracy.

Advanced Model Training

• Implemented model parallelism techniques using:
• Hugging Face's Accelerate.
• DeepSpeed ZeRO 3.
• Achieved a 30% reduction in training time on A100 GPUs.

Model Fine-Tuning and Experiment Tracking

• Engineered and fine-tuned a Hugging Face decoder model for NLP tasks.
• Utilized Weights & Biases for experiment tracking, leading to:
• 20% improvement in Precision, Recall, and F1 scores.

Distributed Computing and Deployment

• Set up and configured a distributed computing cluster, enhancing:
• Data processing capabilities.
• Seamless model deployment across multiple nodes.

Legal Text Classification

• Executed tasks for SemEval 2023 LegalEval, including:
• Rhetorical Roles Labeling.
• Court Judgment Prediction.
• Contributed to a top 10% performance among 26 participating teams.