Module 05 · enterprise context
Chunk, embed, retrieve
Retrieval quality is decided long before the model sees anything. Change the chunking strategy and watch the ranking move — same corpus, same query, different answer. Embeddings here are a deterministic hashed bag-of-words projection so the lab runs offline.
$ python -m fde_toolkit rag --strategy sentence
Query
Sentence packed — coherent, variable size · 7 chunks, avg 50 tokens each
Retrieved context
hybrid: 0.6 dense + 0.4 keyword
- rank 1score 0.502 · dense 0.456 · keyword 0.57
Enhanced due diligence must be refreshed every twelve months. Transaction monitoring runs continuously.
- rank 2score 0.152 · dense 0.254 · keyword 0.00
The report is drafted by the investigator, reviewed by the compliance officer, and filed with the financial intelligence unit. No report is filed without a named human approver.
- rank 3score 0.126 · dense -0.076 · keyword 0.43
Enhanced due diligence applies whenever a customer is classified as high risk. High risk classifications include politically exposed persons, customers in sanctioned jurisdictions, and customers whose source of funds cannot be evidenced.
Strategy comparison
top hit for this query
fixed6 chunks · 0.196The onboarding policy requires every corporate customer to complete identity verification before the first transfer is released. Verification covers the legal entity, its registered address, and the ultimate beneficial owners. Enhanced due
overlap8 chunks · 0.367persons, customers in sanctioned jurisdictions, and customers whose source of funds cannot be evidenced. Enhanced due diligence must be refreshed every twelve months. Transaction monitoring runs continuously. Alerts are generated for tran
sentence7 chunks · 0.502Enhanced due diligence must be refreshed every twelve months. Transaction monitoring runs continuously.
paragraph6 chunks · 0.345Enhanced due diligence applies whenever a customer is classified as high risk. High risk classifications include politically exposed persons, customers in sanctioned jurisdictions, and customers whose source of funds cannot be evidenced. Enhanced due diligence must be refreshed every twelve months.
Lost in the middle
positional recall of assembled context
- pos 11
- pos 20.55
- pos 31
Put the strongest chunk first or last. Anything buried in the middle of a long context is the least likely to be used, which is why more retrieved chunks is not the same as a better answer.