Improved scoring: Our scoring was previously too low for every document due to fused ranking methods. This is now much better, and you’ll see a better distribution of rankings.
Chunk and Document thresholds: You can now set your own thresholds in the search body for more control.
Smaller chunk sizes: Chunk sizes are now more sensible and generally smaller.
onlyMatchingChunks works as expected: This now removes all context chunks with a score of 0 (previously added for chatbot context). Let us know if you notice any issues!
Intra-document chunk querying: You can now query within the chunks of a single document for more focused results.
Completely revamped ingestion pipeline: Now works perfectly for images, videos, and PDFs—even if the links don’t end with .pdf or image extensions. We automatically fetch and parse the content!
Website ingestion: Website links are now much more stable and reliable.