Migrate the MVP into kotaemon (#108)

- Migrate the MVP into kotaemon.
- Preliminary include the pipeline within chatbot interface.
- Organize MVP as an application.

Todo:

- Add an info panel to view the planning of agents -> Fix streaming agents' output.

Resolve: #60
Resolve: #61 
Resolve: #62
This commit is contained in:
Duc Nguyen (john)
2024-01-10 15:28:09 +07:00
committed by GitHub
parent 230328c62f
commit 5a9d6f75be
31 changed files with 273 additions and 92 deletions

15
tests/test_ingestor.py Normal file
View File

@@ -0,0 +1,15 @@
from pathlib import Path
from kotaemon.indices.ingests import DocumentIngestor
from kotaemon.indices.splitters import TokenSplitter
def test_ingestor_include_src():
dirpath = Path(__file__).parent
ingestor = DocumentIngestor(
pdf_mode="normal",
text_splitter=TokenSplitter(chunk_size=50, chunk_overlap=10),
)
nodes = ingestor(dirpath / "resources" / "table.pdf")
assert type(nodes) is list
assert nodes[0].relationships