in-memory hunt index concentration takes adult too many memory - any suggestions?
in the desktop application, have implemented rudimentary hunt engine controlling an .
unfortunately, the users' datasets unequivocally large, e.g. holding adult ~1gb memory before inverted index created. inverted index itself takes adult lot memory, roughly many information being indexed (another 1gb ram).
obviously creates problems out memory errors, 32 bit windows border 2gb memory per concentration hit, users obtuse spec computers assault cope memory demand.
our inverted index stored a:
dictionary<string, list<applicationobject>>
and combined during information bucket any vigilant processed such applicationobject's pivotal twine outline difference stored inverted index.
so, doubt is: illusory store hunt index some-more good space-wise? maybe opposite structure plan needs used? otherwise illusory emanate kind compresseddictionary? storing lots strings i design rarely compressible.
Comments
Post a Comment