Item11043: MongoDBPlugin's address fields needs to be indexed. Can we re-use _id?

pencil
Priority: Enhancement
Current State: Closed
Released In: n/a
Target Release: n/a
Applies To: Extension
Component: MongoDBPlugin
Branches:
Reported By: PaulHarvey
Waiting For: SvenDowideit
Last Change By: PaulHarvey
Imports are sometimes excruciatingly slow (perhaps when CPU is starved on the VM). Example, I get constant output looking like this:
Mon Aug 15 16:32:26 [conn928] update web_2eaf1699486c1f99ffe33f74035a63f2.current  query: { address: "Marine/Sponges/Specimens.G304789@1" } nscanned:1 168ms
Mon Aug 15 16:32:26 [conn928] update web_2eaf1699486c1f99ffe33f74035a63f2.current  query: { address: "Marine/Sponges/Specimens.G304790@1" } nscanned:1 251ms
Mon Aug 15 16:32:27 [conn928] update web_2eaf1699486c1f99ffe33f74035a63f2.current  query: { address: "Marine/Sponges/Specimens.G304791@1" } nscanned:1 230ms
Mon Aug 15 16:32:27 [conn928] update web_2eaf1699486c1f99ffe33f74035a63f2.current  query: { address: "Marine/Sponges/Specimens.G304792@1" } nscanned:1 230ms
Mon Aug 15 16:32:27 [conn928] update web_2eaf1699486c1f99ffe33f74035a63f2.current  query: { address: "Marine/Sponges/Specimens.G304793@1" } nscanned:1 171ms

When I switch to the web and do ensureIndex({address: 1}), the logs are instantly quieter, and the web seems to import much quicker.

Can we rename the address field as _id? I'm quietly concerned about memory usage, so it'd be nice to avoid an unnecessary index.

I notice there's a comment warning against this due to an ObjectID's usefulness for sharding support, but really, we've killed sharding support in much bigger ways than this. Is it an easy one?

-- PaulHarvey - 15 Aug 2011

ok, _id is a bad idea all round - it will become important later - i hope.

mind you, if there is no index set on address, that is bad. and should be changed. I'll see if there is some minimisation we cna do otherwise - clearly, the 'types' feature should have more impact tho

-- SvenDowideit - 15 Aug 2011

added index, going back to store and types work :/

-- SvenDowideit - 22 Aug 2011

ta

-- PaulHarvey - 22 Aug 2011
 

ItemTemplate edit

Summary MongoDBPlugin's address fields needs to be indexed. Can we re-use _id?
ReportedBy PaulHarvey
Codebase trunk
SVN Range
AppliesTo Extension
Component MongoDBPlugin
Priority Enhancement
CurrentState Closed
WaitingFor SvenDowideit
Checkins MongoDBPlugin:74d537b5ca54
TargetRelease n/a
ReleasedIn n/a
Topic revision: r5 - 22 Aug 2011, PaulHarvey
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy