DSPy

1. Project/Product Overview

2. What does it mostly do?

DSPy's core innovation is to upgrade LLM calls from "handwritten strings" to "declarative compilers". This isn't another Prompt template library-it's a complete compiler framework.

2.1 core three elements

DSPy builds and optimizes LLM systems through three layers of abstraction:

2.2 Compiler Workflow

传统 Prompt 工程：
  写 Prompt 字符串 → 测试 → 效果不好 → 改字符串 → 再测试 → 换模型又得重来

DSPy 工作流：
  定义 Signature（声明任务） → 选 Module（执行策略） → Optimizer 编译（自动优化） → 保存为 JSON

Key difference: DSPy's optimization results in a savable, reusable, versioned JSON file, rather than a "by chance" piece of text. After changing the model, you only need to recompile it once, and the code does not need to be changed.

2.3 Core Concept: From Prompt to Program

DSPy takes LLM application development from "assembly language"(Prompt strings) to "high-level language" (declarative Python):

# 传统方式：脆弱的 Prompt 字符串
prompt = """You are a helpful assistant. Given the following email, extract
the event name and date. Return in JSON format. Email: {email}"""

# DSPy 方式：声明式类型化签名
class ExtractEvent(dspy.Signature):
    """Extract event details from an email."""
    email: str = dspy.InputField()
    event_name: str = dspy.OutputField()
    date: str = dspy.OutputField()

extract = dspy.Predict(ExtractEvent)
result = extract(email="Team offsite this Thursday at 2pm")
# Prediction(event_name="Team Offsite", date="Thursday")

When you need step-by-step reasoning, just replace the Module and the code does not change:

extract = dspy.ChainOfThought(ExtractEvent)  # 自动加推理步骤

When a tool call is required:

agent = dspy.ReAct("question -> answer", tools=[search, calc])

2.4 Advanced Competency

-Multimodal:'dspy. Image' as a field type, directly processing pictures, charts

-Assertions (assertion): runtime constraint check, automatic retry error correction

-PythonInterpreter: Security sandbox code execution, support for mathematical reasoning

-RLM (Recursive Language Model): Recursive Reasoning Module (new paper December 2025)

-Fine-tuning Integration: The optimizer not only adjusts Prompt, but also generates fine-tuning data (BetterTogether papers)

-Asynchronous support: Native 'async' execution, thread-safe, suitable for high-throughput scenarios

-Observability:MLflow Tracing (based on OpenTelemetry), each call can be traced

3. Applicable Scenario

4. Not quite the scene

5. Core Competence List

5.1 Signature (signature) ability

5.2 Module Matrix

5.3 Optimizer (optimizer) matrix

5.4 Enterprise and Engineering Capabilities

6. Architecture/deployment/integration approach

6.1 installation

# 稳定版
pip install -U dspy

# 最新开发版（从 main 分支）
pip install git+https://github.com/stanfordnlp/dspy.git

6.2 LM Configuration (Unified Model Interface)

DSPy via 'dspy. LM()'unified management of various LLM back-ends, code does not need to change due to model changes:

# OpenAI 模型
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

# Anthropic 模型
lm = dspy.LM("anthropic/claude-sonnet-4-20250514")

# 本地开源模型（通过 LiteLLM 或直接调用）
lm = dspy.LM("ollama/llama3.1")

# Azure OpenAI
lm = dspy.LM("azure/gpt-4o", api_key="...", api_version="...")

6.3 deployment mode

6.4 Integrated Ecosystem

-Observability:MLflow Tracing(OpenTelemetry standard)

-Deployment:MLflow Model Serving, PyPI release

-Model providers:OpenAI, Anthropic, Cohere, Mistral, Google Gemini, AWS Bedrock, Azure OpenAI, Ollama, on-premises HuggingFace models, and more

-Vector database: natively integrated with ColBERTv2 and other retriever, and can also access any vector library through tool functions.

-With other frameworks:LangChain DSPy official integration, DSPy module can be embedded in LangChain Pipeline;LlamaIndex can also be used complementary to DSPy

-Go Language: Community dspy-go project provides compatible implementation

How to use #7.

7.1 Minimum Complete Example

import dspy

# 1. 配置语言模型
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

# 2. 定义任务签名（替代手写 Prompt）
class SentimentClassifier(dspy.Signature):
    """Classify the sentiment of a sentence as positive, negative, or neutral."""
    sentence: str = dspy.InputField()
    sentiment: str = dspy.OutputField(desc="one of: positive, negative, neutral")

# 3. 选择执行策略
classify = dspy.ChainOfThought(SentimentClassifier)

# 4. 直接调用（零样本）
result = classify(sentence="The new feature is amazing and works flawlessly!")
print(result.sentiment)  # "positive"

7.2 complete training process with optimization

import dspy

# 准备训练数据
trainset = [
    dspy.Example(sentence="I love this product!", sentiment="positive").with_inputs("sentence"),
    dspy.Example(sentence="This is terrible.", sentiment="negative").with_inputs("sentence"),
    dspy.Example(sentence="It's okay, nothing special.", sentiment="neutral").with_inputs("sentence"),
    # ... 更多训练样本
]

# 定义评价指标
def sentiment_accuracy(example, pred, trace=None):
    return pred.sentiment.lower() == example.sentiment.lower()

# 创建分类器和优化器
classify = dspy.ChainOfThought(SentimentClassifier)
optimizer = dspy.GEPA(metric=sentiment_accuracy, auto="medium")

# 编译优化
optimized_classify = optimizer.compile(classify, trainset=trainset)

# 保存编译结果
optimized_classify.save("sentiment_classifier_v1.json")

# 加载使用
loaded = dspy.ChainOfThought(SentimentClassifier)
loaded.load("sentiment_classifier_v1.json")
result = loaded(sentence="Absolutely fantastic experience!")
print(result.sentiment)  # "positive"

7.3 Agent Example (ReAct Tool Call)

import dspy

lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)

# 定义工具（Python 函数即可）
def search(query: str) -> list[str]:
    """Search a knowledge base for relevant documents."""
    return knowledge_base.query(query, k=3)

def calculator(expression: str) -> float:
    """Evaluate a mathematical expression."""
    return dspy.PythonInterpreter({}).execute(expression)

# 创建 Agent
agent = dspy.ReAct("question -> answer", tools=[search, calculator])

# 使用 Agent
result = agent(question="What is the GDP per capita of France based on 2024 data?")
# Agent 自动推理：需要 GDP 数据和人口数据 → search → 计算除法
print(result.answer)  # "$46,029"

7.4 Multimodal Example

class AnalyzeChart(dspy.Signature):
    """Describe the trend and key data points in a chart."""
    chart: dspy.Image = dspy.InputField()
    title: str = dspy.OutputField()
    trend: str = dspy.OutputField()
    data_points: list[dict] = dspy.OutputField()

analyze = dspy.Predict(AnalyzeChart)
result = analyze(chart=dspy.Image("quarterly_revenue.png"))

8. What can I say before sales

8.1 a sentence positioning

• * "DSPy is Stanford's LLM compiler-turning your AI system from a handwritten Prompt 'handcrafted workshop 'into an automatically optimized 'industrial production line', Shopify saving 75 times the cost and 90 times the Databricks. "**

8.2 customer pain points → solutions

8.3 Differentiated Selling Points

vs LangChain (the most ecologically wide LLM framework):

-DSPy does not do orchestration and integration, but focuses on optimizing the quality of LLM calls themselves-complementary to LangChain

-LangChain the official integration of DSPy, DSPy module can be used as an optimization node in the LangChain Pipeline

-DSPy's optimization ability is not available in LangChain: LangChain helps you build Pipeline, and DSPy helps you tune Pipeline.

-Positioning: LangChain can be used as the framework DSPy as the optimization layer

vs LlamaIndex(RAG dedicated framework):

-LlamaIndex strong in data connection and index management, DSPy strong in LLM call compilation and optimization

-LlamaIndex is suitable for "help LLM find the right data",DSPy is suitable for "let LLM process the data correctly"

-DSPy's RAG optimization (retrieval strategy, build quality) works with LlamaIndex index layers

vs Handwritten Prompt (Traditional Way):

-Handwriting Prompt is art, DSPy is engineering-reproducible, versionable, automatable

-Handwritten Prompt relies on personal experience, DSPy relies on data and indicators-more scientific

-Handwritten Prompt model is equivalent to rewriting, DSPy recompile

-Research data: The optimized Prompt is not necessarily effective when used outside the framework (some scenes are degraded), indicating that DSPy is an overall programming model, not just a "Prompt generator"

vs domestic framework (Dify / Coze/MaxKB, etc.):

-DSPy is the underlying LLM compiler, the domestic framework multi-bias application layer.

-DSPy is more suitable for medium and large teams with ML engineering team and need fine control

-DSPy's academic endorsement (Stanford ICLR) has strong appeal to technology decision makers

8.4 Customer Value Story Line (5-Step Approach)

1. Cut in:"Is the effect stable after you launch the LLM application? How long did it take to debug the Prompt? Do you have to start all over again to change the model?"
2. Resonance :"Handwriting Prompt is like manually adjusting assembly code-you have to do it all over again for another CPU (model). The big model itself is evolving, but the approach to Prompt engineering is still stuck in 2023."
3. Demo : Define a classification task with 5 lines of Python on site → Change the ChainOfThought without changing the code → Automatically compile and optimize with GEPA → Improve the display effect → Save as JSON to prove "reproducible"
4. Advanced : Show Production Cases-Shopify from Single GPT-5 Call to Self-Hosting Qwen-3-9B Multi-Agent Architecture, 2x Quality Improvement and Cost Reduce from $5 million/Year to Zero
5. Heavy:"DSPy is not another Prompt template library-it is the compiler methodology for the ICLR 2026 Oral paper. Stanford, MIT Open Source, Shopify/Databricks/Dropbox/AWS are all in production."

9. Frequently Asked Customer Questions

10. PoC Recommendations

Recommended PoC Direction: From Handwritten Prompt to DSPy Compilation Optimization

Select an LLM task (classification/extraction/RAG) that the customer already has labeled data, and compare the compiled effect of handwritten Prompt vs DSPy.

Overall time: 3.5 days

Validation Metrics:

-Compiled Accuracy> Handwritten Prompt

-Post-Compile Stability (Multiple Run Variance)

-The effect of the compiled version after model migration is close to or higher than that of the original model handwritten version

-Compile time <30 minutes (200 samples)

-Compiled and saved as JSON, ready to load

PoC Success Criteria (Go/No-Go):

-✅Go: The compiled version is at least as effective as the handwritten version, and significantly reduces the workload of prompt maintenance.

-✅Go: The model migration experiment was successful, proving that the model only needs to be recompiled.

-❌No-Go: The customer has no annotation data at all and is unwilling to invest in annotation (at this time, the value of DSPy optimizer cannot be reflected)

11. Risks and Considerations

12. My Pre-Sales Judgment

Recommendation: Highly recommended (especially suitable for customers who have LLM applications but have unstable results or need large-scale cost reduction, especially medium and large enterprises with ML engineering capabilities)

Reason:

1. Methodology Leading :"Programming, not Prompting" is not a marketing slogan-it is a compiler paradigm verified by ICLR 2024 and ICLR 2026(Oral). In an era when AI infrastructure is becoming more and more mature, the manual workshop mode optimized by Prompt will eventually be eliminated, and DSPy is at the forefront of this paradigm shift.

1. Solid production cases :Shopify (cost reduction 75x, migration from GPT-5 to self-hosted Qwen-3-9B), Databricks (cost reduction 90x, open source model + optimization> Claude Opus 4.1), Dropbox(45% NMSE reduction)-these are not PoC experiments, but production systems with 10 million requests. Each case has an open technical blog/video detailing.

1. ROI narrative is clear : one-time compilation cost ($2-500) for long-term reasoning cost savings ($500000/year → fraction). This is a strong business argument in front of any CFO.

1. Engineering Friendly :MIT protocol open source, JSON save/load, MLflow Tracing(OpenTelemetry standard), asynchronous support, thread safety, MCP Server export-the infrastructure required for production is ready.

1. Academic endorsement business verification double insurance : Stanford team continuous output: 2023 DSPy → 2024 MIPROv2/BetterTogether → 2025 GEPA(ICLR 2026 Oral)/RLM. 523 PR and 109 Release per month prove that the community is extremely active.

Recommended Customer Persona:

-The LLM application has been launched, but the Prompt maintenance cost is high and the effect is unstable.

-ML/engineering team (1-2 people), Python ability

-Want to migrate from large to small models to reduce costs

-RAG/classification/extraction/Agent scenarios with labeled data (or willing to invest in labeling)

-Medium and large enterprises that value observability and engineering

-Use multiple model providers, requiring a unified LLM call layer

Not recommended situations:

-There is no labeled data at all and no plan to label it in the short term (Optimizer value cannot be reflected)

-Requires low-code/no-code platform (Dify/Coze/MaxKB recommended)

-The team has no Python capability at all (API direct or SaaS solutions are recommended)

-Only 1-2 simple Prompt, do not involve optimization (direct handwriting)

-Extremely cost-sensitive and unable to afford LLM calls during compilation

13. REFERENCE

-GitHub repository: https://github.com/stanfordnlp/dspy

-Official website: https://dspy.ai

-Installation Guide: https://dspy.ai/getting-started/installation/

-Getting Started Tutorial: https://dspy.ai/getting-started/program-dont-prompt/

-Production deployment: https://dspy.ai/production/

-Use Case Summary: https://dspy.ai/community/use-cases/

-DSPy Framework Paper (ICLR 2024):https://arxiv.org/abs/2310.03714

-GEPA Papers (ICLR 2026 Oral):https://arxiv.org/abs/2507.19457

-MIPROv2 papers: https://arxiv.org/abs/2406.11695

-BetterTogether(Fine-tuning Prompt Opt):https://arxiv.org/abs/2407.10930

-RLM (Recursive Language Model):https://arxiv.org/abs/2512.24601

-DSP Papers (Frame Origins):https://arxiv.org/abs/2212.14024

-Shopify production case: https://www.youtube.com/watch?v=bxToahwOVpY

-Dropbox Dash Case: https://dropbox.tech/machine-learning/optimizing-dropbox-dash-relevance-judge-with-dspy

-AWS Nova Migration Case: https://aws.amazon.com/blogs/machine-learning/improve-amazon-nova-migration-performance-with-data-aware-prompt-optimization/

-Databricks 90x cost reduction: https://www.databricks.com/blog/building-state-art-enterprise-agents-90x-cheaper-automated-prompt-optimization

-Databricks Genie:https://www.databricks.com/blog/pushing-frontier-data-agents-genie

-JetBlue case: https://www.databricks.com/blog/optimizing-databricks-llm-pipelines-dspy

-Replit Code Fix: https://blog.replit.com/code-repair

-MLflow integration: https://mlflow.org/docs/latest/llms/dspy/index.html

-Observability tutorial: https://dspy.ai/tutorials/observability/

-DeepWiki DSPy:https://deepwiki.com/stanfordnlp/dspy

-Discord Community: https://discord.gg/XCGy2WDCQB

-PyPI:https://pypi.org/project/dspy/

-dspy-go(Go language implementation):https://github.com/darwishdev/dspy-go

-GEPA Independent Library: https://github.com/gepa-ai/gepa

-GEPA Use Case Set: https://gepa-ai.github.io/gepa/guides/use-cases/

-Medical Imaging DSPy Application: https://arxiv.org/abs/2511.11898

-Multi-scene DSPy optimization study: https://www.alphaxiv.co/overview/2507.03620

• analysis date: 2026-07-02 | data aging: GitHub real-time access, official website content as of access date, v3.3.0b1 *