I will implement generative artificial intelligence on the Databricks platform

Service Description

The session will cover Generative AI, specifically Large Language Models (LLMs), and their practical application to solve real-world issues. The focus will be on natural language processing (NLP) utilizing widely used libraries like Hugging Face transformers and LangChain. I will guide you through understanding the details of pre-training, fine-tuning, and prompt engineering, and how to use this knowledge to construct a personalized chat model using the RAG method. We can also explore methods for assessing the performance and bias of LLMs.

SUBJECTS TO BE COVERED:

Standard NLP assignments

Prompt Engineering techniques

Retrieval Augmented Generation (RAG) principles

• Overall strategy
• Comparison of Vector Library and Vector Database

Advanced Reasoning with LLMs

• LangChain framework
• ReAct methodology

Model Optimization

• Fine-tuning process
• Fine-tuning with DeepSpeed acceleration
• Parameter-efficient fine-tuning (PEFT) methods
• Additive PEFT: Prompt Tuning strategies
• Re-parameterization PEFT: LoRA implementation

LLM Assessment and Evaluation

LLMOps practices

• Setting up a Hugging Face pipeline
• Monitoring LLM progress with Mlflow

Potential risks and difficulties associated with GenAI

Key considerations for deployment in production

EXAMPLES OF MODELS:

• DBRX (Databricks creation)
• Gemma (from Google)
• ChatGPT, GPT-3 (from OpenAI)
• LLaMa (developed by Meta)
• Dolly (Databricks creation)
• MPT (from MosaicML)