diff --git a/python/README.md b/python/README.md index 459cb82c..11571366 100644 --- a/python/README.md +++ b/python/README.md @@ -2,11 +2,19 @@ TypeChat is a library that makes it easy to build natural language interfaces using types. -Building natural language interfaces has traditionally been difficult. These apps often relied on complex decision trees to determine intent and collect the required inputs to take action. Large language models (LLMs) have made this easier by enabling us to take natural language input from a user and match to intent. This has introduced its own challenges including the need to constrain the model's reply for safety, structure responses from the model for further processing, and ensuring that the reply from the model is valid. Prompt engineering aims to solve these problems, but comes with a steep learning curve and increased fragility as the prompt increases in size. +Building natural language interfaces has traditionally been difficult. +These apps often relied on complex decision trees to determine intent and collect the required inputs to take action. +Large language models (LLMs) have made this easier by enabling us to take natural language input from a user and match to intent. +This has introduced its own challenges, including the need to constrain the model's reply for safety, +structure responses from the model for further processing, and ensuring that the reply from the model is valid. +Prompt engineering aims to solve these problems, but comes with a steep learning curve and increased fragility as the prompt increases in size. TypeChat replaces _prompt engineering_ with _schema engineering_. -Simply define types that represent the intents supported in your natural language application. That could be as simple as an interface for categorizing sentiment or more complex examples like types for a shopping cart or music application. For example, to add additional intents to a schema, a developer can add additional types into a discriminated union. To make schemas hierarchical, a developer can use a "meta-schema" to choose one or more sub-schemas based on user input. +Simply define types that represent the intents supported in your natural language application. +That could be as simple as an interface for categorizing sentiment or more complex examples like types for a shopping cart or music application. +For example, to add additional intents to a schema, a developer can add additional types into a discriminated union. +To make schemas hierarchical, a developer can use a "meta-schema" to choose one or more sub-schemas based on user input. After defining your types, TypeChat takes care of the rest by: @@ -24,7 +32,8 @@ Install TypeChat: pip install typechat ``` -You can also develop TypeChat from source, which needs [Python >=3.11](https://www.python.org/downloads/), [hatch](https://hatch.pypa.io/1.6/install/), and [Node.js >=20](https://nodejs.org/en/download): +You can also develop TypeChat from source, which needs [Python >=3.11](https://www.python.org/downloads/), +[hatch](https://hatch.pypa.io/1.6/install/), and [Node.js >=20](https://nodejs.org/en/download): ```sh git clone https://github.com/microsoft/TypeChat @@ -33,9 +42,13 @@ hatch shell npm ci ``` -To see TypeChat in action, we recommend exploring the [TypeChat example projects](https://github.com/microsoft/TypeChat/tree/main/python/examples). You can try them on your local machine or in a GitHub Codespace. +To see TypeChat in action, we recommend exploring the +[TypeChat example projects](https://github.com/microsoft/TypeChat/tree/main/python/examples). +You can try them on your local machine or in a GitHub Codespace. -To learn more about TypeChat, visit the [documentation](https://microsoft.github.io/TypeChat) which includes more information on TypeChat and how to get started. +To learn more about TypeChat, visit the +[documentation](https://microsoft.github.io/TypeChat/docs/python/basic-usage/) +which includes more information on TypeChat and how to get started. ## Contributing diff --git a/python/examples/sentiment/demo.py b/python/examples/sentiment/demo.py index 5273e20a..77b9371d 100644 --- a/python/examples/sentiment/demo.py +++ b/python/examples/sentiment/demo.py @@ -1,9 +1,12 @@ import asyncio - import sys + from dotenv import dotenv_values +from typechat import (Failure, TypeChatJsonTranslator, TypeChatValidator, + create_language_model, process_requests) + import schema as sentiment -from typechat import Failure, TypeChatJsonTranslator, TypeChatValidator, create_language_model, process_requests + async def main(): env_vals = dotenv_values() diff --git a/site/src/docs/python/basic-usage.md b/site/src/docs/python/basic-usage.md new file mode 100644 index 00000000..a44b294d --- /dev/null +++ b/site/src/docs/python/basic-usage.md @@ -0,0 +1,268 @@ +--- +layout: doc-page +title: Basic Python Usage +--- + +TypeChat is currently a small library, so we can get a solid understanding +just by going through the following example: + +```py +import asyncio +import sys + +from dotenv import dotenv_values +from typechat import (Failure, TypeChatJsonTranslator, TypeChatValidator, + create_language_model, process_requests) + +import schema as sentiment # See below for what's in schema.py. + +async def main(): + env_vals = dotenv_values() + model = create_language_model(env_vals) + validator = TypeChatValidator(sentiment.Sentiment) + translator = TypeChatJsonTranslator(model, validator, sentiment.Sentiment) + + async def request_handler(message: str): + result = await translator.translate(message) + if isinstance(result, Failure): + print(result.message) + else: + result = result.value + print(f"The sentiment is {result['sentiment']}") + + filename = sys.argv[1] if len(sys.argv) == 2 else None + await process_requests("😀> ", filename, request_handler) + +asyncio.run(main()) +``` + +Let's break it down step-by-step. + +## Providing a Model + +TypeChat can be used with any language model. +As long as you have a class with the following shape... + +```py +class TypeChatLanguageModel(Protocol): + + async def complete(self, prompt: str | list[PromptSection]) -> Result[str]: + """ + Represents a AI language model that can complete prompts. + + TypeChat uses an implementation of this protocol to communicate + with an AI service that can translate natural language requests to JSON + instances according to a provided schema. + The `create_language_model` function can create an instance. + """ + ... +``` + +then you should be able to try TypeChat out with such a model. + +The key thing here is providing a `complete` method. +`complete` is just a function that takes a `string` and eventually returns a +string (wrapped in a `Result`) if all goes well. + +For convenience, TypeChat provides two functions out of the box to connect to +the OpenAI API and Azure's OpenAI Services. +You can call these directly. + +```py +def create_openai_language_model( + api_key: str, + model: str, + endpoint: str = "https://api.openai.com/v1/chat/completions", + org: str = "" +): + ... + +def create_azure_openai_language_model(api_key: str, endpoint: str): ... +``` + +For even more convenience, TypeChat also provides a function to infer whether +you're using OpenAI or Azure OpenAI. + +```ts +def create_language_model( + vals: dict[str, str | None] +) -> TypeChatLanguageModel: ... +``` + +With `create_language_model`, you can populate your environment variables and +pass them in. +Based on whether `OPENAI_API_KEY` or `AZURE_OPENAI_API_KEY` is set, you'll get +a model of the appropriate type. + +The `TypeChatLanguageModel` returned by these functions has a few writable +attributes you might find useful: + +- `max_retry_attempts` +- `retry_pause_seconds` +- `timeout_seconds` + +Though note that these are unstable. + +Regardless of how you decide to construct your model, it is important to avoid committing credentials directly in source. +One way to make this work between production and development environments is to use a `.env` file in development, and specify that `.env` in your `.gitignore`. +You can use a library like [`python-dotenv`](https://pypi.org/project/python-dotenv/) to help load these up. + +```py +from dotenv import load_dotenv +load_dotenv() + +// ... + +import typechat +model = typechat.create_language_model(os.environ) +``` + +## Defining and Loading the Schema + +TypeChat describes types to language models to help guide their responses. +To do so, all we have to do is define either a [`@dataclass`](https://docs.python.org/3/library/dataclasses.html) or a [`TypedDict`](https://typing.readthedocs.io/en/latest/spec/typeddict.html) class to describe the response we're expecting. +Here's what our schema file `schema.py` look like: + +```py +from dataclasses import dataclass +from typing import Literal + +@dataclass +class Sentiment: + """ + The following is a schema definition for determining the sentiment of a some user input. + """ + + sentiment: Literal["negative", "neutral", "positive"] +``` + +Here, we're saying that the `sentiment` attribute has to be one of three possible strings: `negative`, `neutral`, or `positive`. +We did this with [the `typing.Literal` hint](https://docs.python.org/3/library/typing.html#typing.Literal). + +We defined `Sentiment` as a `@dataclass` so we could have all of the conveniences of standard Python objects - for example, to access the `sentiment` attribute, we can just write `value.sentiment`. +If we declared `Sentiment` as a `TypedDict`, TypeChat would provide us with a `dict`. +That would mean that to access the value of `sentiment`, we would have to write `value["sentiment"]`. + +Note that while we used [the built-in `typing` module](https://docs.python.org/3/library/typing.html), [`typing_extensions`](https://pypi.org/project/typing-extensions/) is supported as well. +TypeChat also understands constructs like `Annotated` and `Doc` to add comments to individual attributes. + +## Creating a Validator + +A validator really has two jobs generating a textual schema for language models, and making sure any data fits a given shape. +The built-in validator looks roughly like this: + +```py +class TypeChatValidator(Generic[T]): + """ + Validates an object against a given Python type. + """ + + def __init__(self, py_type: type[T]): + """ + Args: + + py_type: The schema type to validate against. + """ + ... + + def validate_object(self, obj: object) -> Result[T]: + """ + Validates the given Python object according to the associated schema type. + + Returns a `Success[T]` object containing the object if validation was successful. + Otherwise, returns a `Failure` object with a `message` property describing the error. + """ + ... +``` + +To construct a validator, we just have to pass in the type we defined: + +```py +import schema as sentiment +validator = TypeChatValidator(sentiment.Sentiment) +``` + +## Creating a JSON Translator + +A `TypeChatJsonTranslator` brings all these concepts together. +A translator takes a language model, a validator, and our expected type, and +provides a way to translate some user input into objects following our schema. +To do so, it crafts a prompt based on the schema, reaches out to the model, +parses out JSON data, and attempts validation. +Optionally, it will craft repair prompts and retry if validation fails. + +```py +translator = TypeChatJsonTranslator(model, validator, sentiment.Sentiment) +``` + +When we are ready to translate a user request, we can call the `translate` +method. + +```ts +translator.translate("Hello world! 🙂"); +``` + +We'll come back to this. + +## Creating a "REPL"` + +TypeChat exports a `process_requests` function that makes it easy to +experiment with TypeChat. +Depending on its second argument, it either creates an interactive command +line (if given `None`), or reads lines from the given a file path. + +```ts +async def request_handler(message: str): + ... + +filename = sys.argv[1] if len(sys.argv) == 2 else None +await process_requests("😀> ", filename, request_handler) +``` + +`process_requests` takes 3 things. +First, there's the prompt string - this is what a user will see before their +own input in interactive scenarios. +You can make this playful. +We like to use emoji here. 😄 + +Next, we take a text file name. +Input strings will be read from this file one line at a time. +If the file name was `None`, `process_requests` will work on standard input +and provide an interactive prompt (assuming `sys.stdin.isatty()` is true). +By checking `sys.argv`, our script makes our program interactive unless the +person running the program provided an input file as a command line argument +(e.g. `python ./example.py inputFile.txt`). + +Finally, there's the request handler. +We'll fill that in next. + +## Translating Requests + +Our handler receives some user input (the `message` string) each time it's +called. +It's time to pass that string into over to our `translator` object. + +```ts +async def request_handler(message: str): + result = await translator.translate(message) + if isinstance(result, Failure): + print(result.message) + else: + print(f"The sentiment is {result.value.sentiment}") +``` + +We're calling the `translate` method on each string and getting a response. +If something goes wrong, TypeChat will retry requests up to a maximum +specified by `retry_max_attempts` on our `model`. +However, if the initial request as well as all retries fail, `result` will be +a `typechat.Failure` and we'll be able to grab a `message` explaining what +went wrong. + +In the ideal case, `result` will be a `typechat.Success` and we'll be able to +access our well-typed `value` property! +This will correspond to the type that we passed in when we created our +translator object (i.e. `Sentiment`). + +That's it! +You should now have a basic idea of TypeChat's APIs and how to get started +with a new project. 🎉 diff --git a/site/src/docs/typescript/basic-usage.md b/site/src/docs/typescript/basic-usage.md index 1ed05cd2..9bf343b2 100644 --- a/site/src/docs/typescript/basic-usage.md +++ b/site/src/docs/typescript/basic-usage.md @@ -1,316 +1,324 @@ ---- -layout: doc-page -title: Basic TypeScript Usage ---- - -TypeChat is currently a small library, so let's take a look at some basic usage to understand it. - -```ts -import fs from "fs"; -import path from "path"; -import { createJsonTranslator, createLanguageModel } from "typechat"; -import { processRequests } from "typechat/interactive"; -import { createTypeScriptJsonValidator } from "typechat/ts"; -import { SentimentResponse } from "./sentimentSchema"; - -// Create a model. -const model = createLanguageModel(process.env); - -// Create a validator. -const schema = fs.readFileSync(path.join(__dirname, "sentimentSchema.ts"), "utf8"); -const validator = createTypeScriptJsonValidator(schema, "SentimentResponse"); - -// Create a translator. -const translator = createJsonTranslator(model, validator); - -// Process requests interactively or from the input file specified on the command line -processRequests("😀> ", process.argv[2], async (request) => { - const response = await translator.translate(request); - if (!response.success) { - console.log(response.message); - return; - } - console.log(`The sentiment is ${response.data.sentiment}`); -}); -``` - -## Providing a Model - -TypeChat can be used with any language model. -As long as you can construct an object with the following properties: - -```ts -export interface TypeChatLanguageModel { - /** - * Optional property that specifies the maximum number of retry attempts (the default is 3). - */ - retryMaxAttempts?: number; - /** - * Optional property that specifies the delay before retrying in milliseconds (the default is 1000ms). - */ - retryPauseMs?: number; - /** - * Obtains a completion from the language model for the given prompt. - * @param prompt The prompt string. - */ - complete(prompt: string): Promise>; -} -``` - -then you should be able to try TypeChat out with such a model. - -The key thing here is that only `complete` is required. -`complete` is just a function that takes a `string` and eventually returns a `string` if all goes well. - -For convenience, TypeChat provides two functions out of the box to connect to the OpenAI API and Azure's OpenAI Services. -You can call these directly. - -```ts -export function createOpenAILanguageModel(apiKey: string, model: string, endPoint? string): TypeChatLanguageModel; - -export function createAzureOpenAILanguageModel(apiKey: string, endPoint: string): TypeChatLanguageModel; -``` - -For even more convenience, TypeChat also provides a function to infer whether you're using OpenAI or Azure OpenAI. - -```ts -export function createLanguageModel(env: Record): TypeChatLanguageModel -``` - -You can populate your environment variables, and based on whether `OPENAI_API_KEY` or `AZURE_OPENAI_API_KEY` is set, you'll get a model of the appropriate type. - -```ts -import dotenv from "dotenv"; -dotenv.config(/*...*/); -import * as typechat from "typechat"; -const model = typechat.createLanguageModel(process.env); -``` - -Regardless, of how you decide to construct your model, we recommend keeping your secret tokens/API keys in a `.env` file, and specifying `.env` in a `.gitignore`. -You can use a library like [`dotenv`](https://www.npmjs.com/package/dotenv) to help load these up. - -## Loading the Schema - -TypeChat describes types to language models to help guide their responses. -In this case, we are using a `TypeScriptJsonValidator` which uses the TypeScript compiler to validate data against a set of types. -That means that we'll be writing out the types of the data we expect to get back in a `.ts` file. -Here's what our schema file `sentimentSchema.ts` look like: - -```ts -// The following is a schema definition for determining the sentiment of a some user input. - -export interface SentimentResponse { - sentiment: "negative" | "neutral" | "positive"; // The sentiment of the text -} -``` - -It also means we will need to manually load up an input `.ts` file verbatim. - -```ts -// Load up the type from our schema. -import type { SentimentResponse } from "./sentimentSchema"; - -// Load up the schema file contents. -const schema = fs.readFileSync(path.join(__dirname, "sentimentSchema.ts"), "utf8"); -``` - -Note: this code assumes a CommonJS module. If you're using ECMAScript modules, you can use [`import.meta.url`](https://nodejs.org/docs/latest-v19.x/api/esm.html#importmetaurl) or via [`import.meta.dirname`](https://nodejs.org/docs/latest-v21.x/api/esm.html#importmetadirname) depending on the version of your runtime. - -This introduces some complications to certain kinds of builds, since our input files need to be treated as local assets. -One way to achieve this is to use a runtime or tool like [`ts-node`](https://www.npmjs.com/package/ts-node) to both import the file for its types, as well as read the file contents. -Another is to use a utility like [`copyfiles`](https://www.npmjs.com/package/copyfiles) to move specific schema files to the output directory. -If you're using a bundler, there might be custom way to import a file as a raw string as well. -Regardless, [our examples](https://github.com/microsoft/TypeChat/tree/main/typescript/examples) should work with either of the first two options. - -Alternatively, if we want, we can build our schema with objects entirely in memory using Zod and a `ZodValidator` which we'll touch on in a moment. -Here's what our schema would look like if we went down that path. - -```ts -import { z } from "zod"; - -export const SentimentResponse = z.object({ - sentiment: z.enum(["negative", "neutral", "positive"]).describe("The sentiment of the text") -}); - -export const SentimentSchema = { - SentimentResponse -}; -``` - -## Creating a Validator - -A validator really has two jobs generating a textual schema for language models, and making sure any data fits a given shape. -The interface looks roughly like this: - -```ts -/** - * An object that represents a TypeScript schema for JSON objects. - */ -export interface TypeChatJsonValidator { - /** - * Return a string containing TypeScript source code for the validation schema. - */ - getSchemaText(): string; - /** - * Return the name of the JSON object target type in the schema. - */ - getTypeName(): string; - /** - * Validates the given JSON object according to the associated TypeScript schema. Returns a - * `Success` object containing the JSON object if validation was successful. Otherwise, returns - * an `Error` object with a `message` property describing the error. - * @param jsonText The JSON object to validate. - * @returns The JSON object or an error message. - */ - validate(jsonObject: object): Result; -} -``` - -In other words, this is just the text of all types, the name of the top-level type to respond with, and a validation function that returns a strongly-typed view of the input if it succeeds. - -TypeChat ships with two validators. - -### `TypeScriptJsonValidator` - -A `TypeScriptJsonValidator` operates off of TypeScript text files. -To create one, we have to import `createTypeScriptJsonValidator` out of `typechat/ts`: - -```ts -import { createTypeScriptJsonValidator } from "typechat/ts"; -``` - -We'll also need to actually import the type from our schema. - -```ts -import { SentimentResponse } from "./sentimentSchema"; -``` - -With our schema text and this type, we have enough to create a validator: - -```ts -const validator = createTypeScriptJsonValidator(schema, "SentimentResponse"); -``` - -We provided the text of the schema and the name of the type we want returned data to satisfy. -We also have to provide the type argument `SentimentResponse` to explain what data shape we expect (though note that this is a bit like a type cast and isn't guaranteed). - -### Zod Validators - -If you chose to define your schema with Zod, you can use the `createZodJsonValidator` function: - -```ts -import { createZodJsonValidator } from "typechat/zod"; -``` - -Instead of a source file, a Zod validator needs a JavaScript object mapping from type names to Zod type objects like `myObj` in the following example: - -```ts -export const MyType = z.object(/*...*/); - -export const MyOtherType = z.object(/*...*/); - -export let myObj = { - MyType, - MyOtherType, -} -``` - -From above, that was just `SentimentSchema`: - -```ts -export const SentimentSchema = { - SentimentResponse -}; -``` - -So we'll need to import that object... - -```ts -import { SentimentSchema } from "./sentimentSchema"; -``` - -and provide it, along with our expected type name, to `createZodJsonValidator`: - -```ts -const validator = createZodJsonValidator(SentimentSchema, "SentimentResponse"); -``` - -## Creating a JSON Translator - -A `TypeChatJsonTranslator` brings these together. - -```ts -import { createJsonTranslator } from "typechat"; -``` - -A translator takes both a model and a validator, and provides a way to translate some user input into objects within our schema. -To do so, it crafts a prompt based on the schema, reaches out to the model, parses out JSON data, and attempts validation. -Optionally, it will craft repair prompts and retry if validation failed.. - -```ts -const translator = createJsonTranslator(model, validator); -``` - -When we are ready to translate a user request, we can call the `translate` method. - -```ts -translator.translate("Hello world! 🙂"); -``` - -We'll come back to this. - -## Creating the Prompt - -TypeChat exports a `processRequests` function that makes it easy to experiment with TypeChat. -We need to import it from `typechat/interactive`. - -```ts -import { processRequests } from "typechat/interactive"; -``` - -It either creates an interactive command line prompt, or reads lines in from a file. - -```ts -typechat.processRequests("😀> ", process.argv[2], async (request) => { - // ... -}); -``` - -`processRequests` takes 3 things. -First, there's the prompt prefix - this is what a user will see before their own text in interactive scenarios. -You can make this playful. -We like to use emoji here. 😄 - -Next, we take a text file name. -Input strings will be read from this file on a per-line basis. -If the file name was `undefined`, `processRequests` will work on standard input and provide an interactive prompt. -Using `process.argv[2]` makes our program interactive by default unless the person running the program provided an input file as a command line argument (e.g. `node ./dist/main.js inputFile.txt`). - -Finally, there's the request handler. -We'll fill that in next. - -## Translating Requests - -Our handler receives some user input (the `request` string) each time it's called. -It's time to pass that string into over to our `translator` object. - -```ts -typechat.processRequests("😀> ", process.argv[2], async (request) => { - const response = await translator.translate(request); - if (!response.success) { - console.log(response.message); - return; - } - console.log(`The sentiment is ${response.data.sentiment}`); -}); -``` - -We're calling the `translate` method on each string and getting a response. -If something goes wrong, TypeChat will retry requests up to a maximum specified by `retryMaxAttempts` on our `model`. -However, if the initial request as well as all retries fail, `response.success` will be `false` and we'll be able to grab a `message` explaining what went wrong. - -In the ideal case, `response.success` will be `true` and we'll be able to access our well-typed `data` property! -This will correspond to the type that we passed in when we created our translator object (i.e. `SentimentResponse`). - -That's it! -You should now have a basic idea of TypeChat's APIs and how to get started with a new project. 🎉 +--- +layout: doc-page +title: Basic TypeScript Usage +--- + +TypeChat is currently a small library, so we can get a solid understanding just by understanding the following example: + +```ts +import fs from "fs"; +import path from "path"; +import { createJsonTranslator, createLanguageModel } from "typechat"; +import { processRequests } from "typechat/interactive"; +import { createTypeScriptJsonValidator } from "typechat/ts"; +import { SentimentResponse } from "./sentimentSchema"; + +// Create a model. +const model = createLanguageModel(process.env); + +// Create a validator. +const schema = fs.readFileSync(path.join(__dirname, "sentimentSchema.ts"), "utf8"); +const validator = createTypeScriptJsonValidator(schema, "SentimentResponse"); + +// Create a translator. +const translator = createJsonTranslator(model, validator); + +// Process requests interactively or from the input file specified on the command line +processRequests("😀> ", process.argv[2], async (request) => { + const response = await translator.translate(request); + if (!response.success) { + console.log(response.message); + return; + } + console.log(`The sentiment is ${response.data.sentiment}`); +}); +``` + +Let's break it down step-by-step. + +## Providing a Model + +TypeChat can be used with any language model. +As long as you can construct an object with the following properties: + +```ts +export interface TypeChatLanguageModel { + /** + * Optional property that specifies the maximum number of retry attempts (the default is 3). + */ + retryMaxAttempts?: number; + /** + * Optional property that specifies the delay before retrying in milliseconds (the default is 1000ms). + */ + retryPauseMs?: number; + /** + * Obtains a completion from the language model for the given prompt. + * @param prompt The prompt string. + */ + complete(prompt: string): Promise>; +} +``` + +then you should be able to try TypeChat out with such a model. + +The key thing here is that only `complete` is required. +`complete` is just a function that takes a `string` and eventually returns a `string` if all goes well. + +For convenience, TypeChat provides two functions out of the box to connect to the OpenAI API and Azure's OpenAI Services. +You can call these directly. + +```ts +export function createOpenAILanguageModel(apiKey: string, model: string, endPoint? string): TypeChatLanguageModel; + +export function createAzureOpenAILanguageModel(apiKey: string, endPoint: string): TypeChatLanguageModel; +``` + +For even more convenience, TypeChat also provides a function to infer whether you're using OpenAI or Azure OpenAI. + +```ts +export function createLanguageModel(env: Record): TypeChatLanguageModel +``` + +With `createLanguageModel`, you can populate your environment variables and pass them in. +Based on whether `OPENAI_API_KEY` or `AZURE_OPENAI_API_KEY` is set, you'll get a model of the appropriate type. + +Regardless, of how you decide to construct your model, it is important to avoid committing credentials directly in source. +One way to make this work between production and development environments is to use a `.env` file in development, and specify that `.env` in your `.gitignore`. +You can use a library like [`dotenv`](https://www.npmjs.com/package/dotenv) to help load these up. + +```ts +import dotenv from "dotenv"; +dotenv.config(/*...*/); + +// ... + +import * as typechat from "typechat"; +const model = typechat.createLanguageModel(process.env); +``` + + +## Defining and Loading the Schema + +TypeChat describes types to language models to help guide their responses. +In this case, we are using a `TypeScriptJsonValidator` which uses the TypeScript compiler to validate data against a set of types. +That means that we'll be writing out the types of the data we expect to get back in a `.ts` file. +Here's what our schema file `sentimentSchema.ts` look like: + +```ts +// The following is a schema definition for determining the sentiment of a some user input. + +export interface SentimentResponse { + sentiment: "negative" | "neutral" | "positive"; // The sentiment of the text +} +``` + +It also means we will need to manually load up an input `.ts` file verbatim. + +```ts +// Load up the type from our schema. +import type { SentimentResponse } from "./sentimentSchema"; + +// Load up the schema file contents. +const schema = fs.readFileSync(path.join(__dirname, "sentimentSchema.ts"), "utf8"); +``` + +Note: this code assumes a CommonJS module. If you're using ECMAScript modules, you can use [`import.meta.url`](https://nodejs.org/docs/latest-v19.x/api/esm.html#importmetaurl) or via [`import.meta.dirname`](https://nodejs.org/docs/latest-v21.x/api/esm.html#importmetadirname) depending on the version of your runtime. + +This introduces some complications to certain kinds of builds, since our input files need to be treated as local assets. +One way to achieve this is to use a runtime or tool like [`ts-node`](https://www.npmjs.com/package/ts-node) to both import the file for its types, as well as read the file contents. +Another is to use a utility like [`copyfiles`](https://www.npmjs.com/package/copyfiles) to move specific schema files to the output directory. +If you're using a bundler, there might be custom way to import a file as a raw string as well. +Regardless, [our examples](https://github.com/microsoft/TypeChat/tree/main/typescript/examples) should work with either of the first two options. + +Alternatively, if we want, we can build our schema with objects entirely in memory using Zod and a `ZodValidator` which we'll touch on in a moment. +Here's what our schema would look like if we went down that path. + +```ts +import { z } from "zod"; + +export const SentimentResponse = z.object({ + sentiment: z.enum(["negative", "neutral", "positive"]).describe("The sentiment of the text") +}); + +export const SentimentSchema = { + SentimentResponse +}; +``` + +## Creating a Validator + +A validator really has two jobs generating a textual schema for language models, and making sure any data fits a given shape. +The interface looks roughly like this: + +```ts +/** + * An object that represents a TypeScript schema for JSON objects. + */ +export interface TypeChatJsonValidator { + /** + * Return a string containing TypeScript source code for the validation schema. + */ + getSchemaText(): string; + /** + * Return the name of the JSON object target type in the schema. + */ + getTypeName(): string; + /** + * Validates the given JSON object according to the associated TypeScript schema. Returns a + * `Success` object containing the JSON object if validation was successful. Otherwise, returns + * an `Error` object with a `message` property describing the error. + * @param jsonText The JSON object to validate. + * @returns The JSON object or an error message. + */ + validate(jsonObject: object): Result; +} +``` + +In other words, this is just the text of all types, the name of the top-level type to respond with, and a validation function that returns a strongly-typed view of the input if it succeeds. + +TypeChat ships with two validators. + +### `TypeScriptJsonValidator` + +A `TypeScriptJsonValidator` operates off of TypeScript text files. +To create one, we have to import `createTypeScriptJsonValidator` out of `typechat/ts`: + +```ts +import { createTypeScriptJsonValidator } from "typechat/ts"; +``` + +We'll also need to actually import the type from our schema. + +```ts +import { SentimentResponse } from "./sentimentSchema"; +``` + +With our schema text and this type, we have enough to create a validator: + +```ts +const validator = createTypeScriptJsonValidator(schema, "SentimentResponse"); +``` + +We provided the text of the schema and the name of the type we want returned data to satisfy. +We also have to provide the type argument `SentimentResponse` to explain what data shape we expect (though note that this is a bit like a type cast and isn't guaranteed). + +### Zod Validators + +If you chose to define your schema with Zod, you can use the `createZodJsonValidator` function: + +```ts +import { createZodJsonValidator } from "typechat/zod"; +``` + +Instead of a source file, a Zod validator needs a JavaScript object mapping from type names to Zod type objects like `myObj` in the following example: + +```ts +export const MyType = z.object(/*...*/); + +export const MyOtherType = z.object(/*...*/); + +export let myObj = { + MyType, + MyOtherType, +} +``` + +From above, that was just `SentimentSchema`: + +```ts +export const SentimentSchema = { + SentimentResponse +}; +``` + +So we'll need to import that object... + +```ts +import { SentimentSchema } from "./sentimentSchema"; +``` + +and provide it, along with our expected type name, to `createZodJsonValidator`: + +```ts +const validator = createZodJsonValidator(SentimentSchema, "SentimentResponse"); +``` + +## Creating a JSON Translator + +A `TypeChatJsonTranslator` brings these together. + +```ts +import { createJsonTranslator } from "typechat"; +``` + +A translator takes both a model and a validator, and provides a way to translate some user input into objects following our schema. +To do so, it crafts a prompt based on the schema, reaches out to the model, parses out JSON data, and attempts validation. +Optionally, it will craft repair prompts and retry if validation fails. + +```ts +const translator = createJsonTranslator(model, validator); +``` + +When we are ready to translate a user request, we can call the `translate` method. + +```ts +translator.translate("Hello world! 🙂"); +``` + +We'll come back to this. + +## Creating the Prompt + +TypeChat exports a `processRequests` function that makes it easy to experiment with TypeChat. +We need to import it from `typechat/interactive`. + +```ts +import { processRequests } from "typechat/interactive"; +``` + +It either creates an interactive command line prompt, or reads lines in from a file. + +```ts +typechat.processRequests("😀> ", process.argv[2], async (request) => { + // ... +}); +``` + +`processRequests` takes 3 things. +First, there's the prompt prefix - this is what a user will see before their own text in interactive scenarios. +You can make this playful. +We like to use emoji here. 😄 + +Next, we take a text file name. +Input strings will be read from this file on a per-line basis. +If the file name was `undefined`, `processRequests` will work on standard input and provide an interactive prompt. +Using `process.argv[2]` makes our program interactive by default unless the person running the program provided an input file as a command line argument (e.g. `node ./dist/main.js inputFile.txt`). + +Finally, there's the request handler. +We'll fill that in next. + +## Translating Requests + +Our handler receives some user input (the `request` string) each time it's called. +It's time to pass that string into over to our `translator` object. + +```ts +typechat.processRequests("😀> ", process.argv[2], async (request) => { + const response = await translator.translate(request); + if (!response.success) { + console.log(response.message); + return; + } + console.log(`The sentiment is ${response.data.sentiment}`); +}); +``` + +We're calling the `translate` method on each string and getting a response. +If something goes wrong, TypeChat will retry requests up to a maximum specified by `retryMaxAttempts` on our `model`. +However, if the initial request as well as all retries fail, `response.success` will be `false` and we'll be able to grab a `message` explaining what went wrong. + +In the ideal case, `response.success` will be `true` and we'll be able to access our well-typed `data` property! +This will correspond to the type that we passed in when we created our translator object (i.e. `SentimentResponse`). + +That's it! +You should now have a basic idea of TypeChat's APIs and how to get started with a new project. 🎉