Database Specialist

**Achild** · November 20th, 2009, 09:28 AM

Anyone in here knows what is the best way to store terabytes of data and search it quickly. The tricky part is there may be 4 different references in different tables for one thing, how can i make it so that the searches are faster.e.g. google searches any clarification needed pm me please.

**MOJO** · November 20th, 2009, 09:47 AM

Terabytes??? Really????

Are you sure you're not working for the NSA?

**MOJO** · November 20th, 2009, 10:21 AM

Seriously though....that's a loaded question that would depend on numerous variables. But, you should check out:

1. Oracle 10g
2. IBM DB2
3. SQL Server

Oh, yeah....drink heavily during the build...it helps!

**Achild** · November 20th, 2009, 02:13 PM

lol if i was working for NASA i doubt i'll be able to ask here lol, well i said terabytes to give the idea of the size i am thinking about. I am really looking for a suitable file structure or paging system, and since google is pretty efficient at searching i gave them as an example. "ok yea i exaggerated" Hhmm algorithms would best explain what is am thinking about, imaging 20 10GB tables with a lot of necessary repeated data.

**MOJO** · November 20th, 2009, 03:56 PM

NSA = National Security Administration (you know....wiring tapping and such)

I think you're still looking for a relational db. Any of the three would work, but you'll need to put a lot of thought into the user interface. Sounds like you'll need something web-based. Also, you said "necessary repeated data", so my usual speech about data normalization will fall on deaf ears ;-).

Google is really doing a search for Meta Tags embedded in the site coding. Your UI should have the ability to search through the tables and return the desired recordsets.

**Dave72** · November 20th, 2009, 07:19 PM

Ya, you cant have copies of data ('repeated data') .. way too many issues with that situation. You are probably correct in imagining multiple tables, but then as Mojo hinted at, you have to normalize it. Picture taking that common item out and into its own table (then it only exists once, and any edits/etc dont corrupt the whole db).

A good book for building up a working grasp of relational db's is The Practical SQL Handbook (refs Oracle, MS, Sybase and Informix systems)

**Achild** · December 18th, 2009, 10:01 AM

hey guys thanks for those responses, we worked it out. freetext, contains and isNull combination ftw! you may think it's google :P lol. but thanks guys