WA3486

Comprehensive Natural Language Processing (NLP) Training

This Natural Language Processing (NLP) training course teaches students the applications and techniques for developing NLP models, including large language models (LLMs). Attendees learn how to build and evaluate NLP models for tasks such as text generation, text classification, image synthesis, and image classification in use cases in insurance, finance, and other sectors.

Through interactive, hands-on labs and exercises, attendees reinforce their theoretical knowledge of the fundamental concepts such as types of neural network architecture, embeddings, multimodality, fine-tuning, and transfer learning while exploring ethical considerations and ensuring responsible AI practices.

Course Details

Duration

3 days

Prerequisites

  • Extensive prior Python development experience
  • Core Python Data Science skills, including the use of NumPy and Pandas
  • Foundational Knowledge in AI and Machine Learning

Target Audience

  • Programmers
  • Software Engineers
  • Computer Scientists
  • AI and Machine Learning Practitioners
  • Data Scientists
  • Data Engineers
  • Data Analysts

Skills Gained

  • Understand the basics of Natural Language Processing, including large language models (LLMs), and its applications
  • Understand the use cases for different Natural Language Processing architectures such as RNNs, LSTMs, and transformers
  • Build neural networks using deep learning frameworks such as TensorFlow and Keras
  • Learn about different techniques and algorithms used in Natural Language Processing
  • Investigate the semantic aspect of embeddings and their role in representing data in a lower-dimensional space, including techniques such as implementing word embeddings for natural language processing tasks
  • Review the capabilities and applications of large language models (LLMs), including BERT, GPT-3, and LLaMA, investigating their role in natural language processing, creative text generation, and code development
  • Develop skills to design and implement Natural Language Processing models
  • Explore the relationship between foundation models, fine-tuning, and transfer learning
  • Gain proficiency in evaluating and optimizing Natural Language Processing models
  • Apply Natural Language Processing models to real-world problems
  • Understand the importance of prompt engineering in NLP models
  • Learn the different types of prompts and their use cases
  • Develop skills to design and refine prompts for NLP models
  • Understand the importance of machine learning interpretability
  • Explore different types of ML interpretability models
  • Analyze standard techniques and methods for explainability
  • Evaluate the effectiveness of interpretability methods
  • Discuss ethical considerations and responsible AI practices
  • Understand regulatory compliance in AI systems
Course Outline
  • Introduction to Natural Language Processing
    • History of Natural Language Processing and Generative AI
    • What are Natural Language Processing models?
    • Understanding generative models
    • Contrasting generative and discriminative models
  • NLP Use Cases and Tasks
    • NLP use cases in insurance, finance, and consulting
    • Examples of NLP products
  • Tensorflow and Keras
    • What is TensorFlow and Keras?
    • Tensors and Python API
    • TensorFlow Lite
    • TFX (TensorFlow Extended)
    • TensorFlow Toolkit Stack
    • TensorBoard
    • Core Keras data structures
    • Components of a Keras model
    • Creating neural networks in Keras
    • The strengths and weaknesses of the functional API
  • Neural Networks and Deep Learning
    • What is a neural network?
    • Types of neural networks
    • Deep Learning
    • Navigating neural network layers
    • Neurons
    • Perceptrons and multi-layer perceptrons
    • Neural network training
  • Traditional NLP Models: RNNs and LSTMs
    • What are RNNs and LSTMs?
    • Feedforward neural networks vs. RNNs
    • Mathematical foundations of RNNs and LSTMs
    • Training and inference on RNNs and LSTMs
    • Limitations of RNNs and LSTMs
  • Text Representations
    • The semantic aspect of word embeddings
    • Word embeddings in NLP
    • Word embeddings in transformers
    • Word embedding similarity metrics
  • Transformers
    • What is a transformer?
    • Transformer use cases
    • Encoders and decoders
    • Self-attention mechanism
    • Multi-head attention
    • Tokenization
    • Transformer architecture
  • Foundation Models
    • What are foundation models or Large Language Models (LLMs)
    • Multimodality of foundation models
    • Model training techniques
    • LLM capabilities vs. size
    • Model quantization
    • Options for accessing LLMs
    • Context window and prompts
    • Foundation model architecture
    • Diffusion models and the diffusion process
  • Prompt Engineering and Retrieval-Augmented Generation (RAG)
    • Prompt engineering basics
    • Prompt types
    • Prompt context
    • Prompt pitfalls
    • One-shot prompting
    • Few-shot prompting
    • Chain-of-Thought prompting
    • Automated prompt engineering
    • What is RAG?
    • Vector indexing
    • LlamaIndex indexes: List, Vector Store, Tree, Keyword
    • LangChain: Model I/O, Retrieval, Chains, Memory, Agents
  • Pre-training and Fine-tuning Large Language Models
    • Building the pre-trained LLM
    • Benefits and challenges of pre-training
    • Pre-training phases
    • Fine-tuning over pre-training
    • Data preparation and processing
    • Data augmentation: text, image, and signal data
    • When to use fine-tuning
    • Domain adaptation
    • Multi-task learning
  • Evaluating LLMs
    • Diversity Metrics
    • Likelihood
    • Perplexity
    • Inception Score
    • BLEU
    • ROUGE
    • Human evaluation
    • LLM-on-LLM evaluation
  • Security and Privacy
    • What is AI cybersecurity?
    • Threats and challenges in AI cybersecurity
    • Adversarial attacks
    • Model inversion and extraction
    • Data poisoning
    • Robustness techniques
    • Differential privacy
    • Federated learning
    • Best practices in secure AI development
  • Responsible AI
    • What is an AI system?
    • Principles of AI Ethics
    • Fairness
    • Accountability
    • Transparency
    • Privacy and autonomy
    • Reliability
    • Explainable AI (XAI)
    • Model-Agnostic Visual Analytics (MAVA)
    • Human-AI Collaborated Evaluation (HACE)
    • AI ethics in practice
    • Regulatory compliance in AI systems

Lab Exercises

    • Learning the Colab Jupyter Notebook Environment
    • Basics of Tensorflow and Keras
    • Build a neural network to classify insurance fraud
    • Dive into RNNs and LSTMs
    • Dive into word embeddings
    • Implement transformer models for finance tasks
    • Compare LLM models for insurance tasks
    • Prompt engineering and RAG for insurance question-answering
    • Fine-tune an LLM for insurance topic classification
    • Evaluating LLMs
    • Nemo Guardrails for finance tasks
    • Design an LLM audit process