database - Store large number of data points? -
what best way store large number of data points?
for example temperature values measured every minute on lots of locations?
sql databases 1 row per data points doesn't seem efficient.
i know why reckon "not efficient". need explain data model , schema give better context of scenario.
storing multiple data points single row, when not related each other, , should indeed stand on own, not approach. meshing result in counter-intuitive , quirky query statements pull out correct data points need given scenario.
we have done work in power station before, collecting data various systems , metering equipment wide variety of gas , electrical parameters need monitored , aggregated. can come in every 3-5 minutes 30-60 minutes depending on type of parameters. these naturally results in millions of records per month.
the key indexing tables physical order tied sequence in records came in. (clustered index) new pages , extents created , filled sequentially incoming data. should prevent massive page splits , reshuffling.
Comments
Post a Comment