Class BaseSageMakerContentHandler<InputType, OutputType>Abstract

A handler class to transform input from LLM to a format that SageMaker endpoint expects. Similarily, the class also handles transforming output from the SageMaker endpoint to a format that LLM class expects.

Example:

class ContentHandler implements ContentHandlerBase<string, string> {
  contentType = "application/json"
  accepts = "application/json"

  transformInput(prompt: string, modelKwargs: Record<string, unknown>) {
    const inputString = JSON.stringify({
      prompt,
     ...modelKwargs
    })
    return Buffer.from(inputString)
  }

  transformOutput(output: Uint8Array) {
    const responseJson = JSON.parse(Buffer.from(output).toString("utf-8"))
    return responseJson[0].generated_text
  }

}

Type Parameters

  • InputType
  • OutputType

Constructors

constructor

Properties

accepts contentType

Methods

transformInput transformOutput

Constructors

constructor

Properties

accepts

accepts: string = "text/plain"

contentType

contentType: string = "text/plain"

Methods

AbstracttransformInput

  • transformInput(prompt, modelKwargs): Promise<Uint8Array>

  • Transforms the prompt and model arguments into a specific format for sending to SageMaker.

    Parameters

    • prompt: InputType

      The prompt to be transformed.

    • modelKwargs: Record<string, unknown>

      Additional arguments.

    Returns Promise<Uint8Array>

    A promise that resolves to the formatted data for sending.

AbstracttransformOutput

Settings

Member Visibility

On This Page

Constructors
Properties
Methods