Skip to content

First-class support for few-shot examples in front matter #264

@sternj

Description

@sternj

Prerequisites

  • I have checked the latest documentationand this feature doesn't exist
  • I have searched for similar feature requests and found none
  • I have linked to related issues or discussions in the description (if any exist)

Type of Improvement

New Functionality

Proposed Solution

Related to #140 , but a more specific proposal-- I think examples should live somewhere in the front matter.

The schemata of prompts can change quite a bit over time, and the current paradigm for few-shot examples in Prompty (just to use an additional block) doesn't do a make any effort to keep examples in lock-step with the input and output schemas, which I think it should be able to do.

The best solution I can see off the top of my head is a new top-level field, examples, that looks like the following (using the inputs and outputs blocks from this test

inputs:
  firstName:
    type: string
    default: User
    sample: April
    description: The first name of the customer
  lastName:
    type: string
    sample: Kwong
    required: true
    strict: false
    description: The last name of the customer
  question:
    type: string
    description: The question to answer
    required: true
outputs:
  work:
    type: object
    json_schema: ${file:schema.json}
    description: The thing to output
examples:
  - inputs:
      firstName: Ordinary
      lastName: Name
      question: Is this a realistic name?
    outputs:
      work:
        answer: It's not really my place to say
        items:
          - http://creative.url

Checking each of the examples is feasible given the information provided in the input and output specs. I know there might be other design constraints, so I wanted to make the suggestion and get some feedback before PRing the Python runtime

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions