Skip to content

Content Helpers

🚧 work in progress

Context construction

  • GenerateContextFromDocs generates the context content from a slice of documents: content.GenerateContextFromDocs(docs []string) string

Read and Write files

  • content.ReadTextFile(path string) (string, error)
  • content.WriteTextFile(path, content string) error

Parsing path

  • FindFiles searches for files with a specific extension in the given root directory and its subdirectories: content.FindFiles(dirPath string, ext string) ([]string, error)
  • Returns:
    • []string: A slice of file paths that match the given extension.
    • error: An error if the search encounters any issues.
  • ForEachFile iterates over all files with a specific extension in a directory and its subdirectories: content.ForEachFile(dirPath string, ext string, callback func(string) error) ([]string, error)
  • Returns:
    • []string: A slice of file paths that match the given extension.
    • error: An error if the search encounters any issues.
  • GetArrayOfContentFiles searches for files with a specific extension in the given directory and its subdirectories: content.GetArrayOfContentFiles(dirPath string, ext string) ([]string, error)
  • GetMapOfContentFiles searches for files with a specific extension in the given directory and its subdirectories: content.GetMapOfContentFiles(dirPath string, ext string) (map[string]string, error)

String interpolation

  • content.InterpolateString(str string, vars interface{}) (string, error)

Note

👀 you will find a complete example in:

Estimate the number of tokens in a text

  • content.CountTokens(text string) int
  • content.CountTokensAdvanced(text string) int
  • content.EstimateGPTTokens(text string) int

this could be useful to estimate the value of num_ctx

Note

👀 you will find a complete example in: