Showing posts with label aggregate. Show all posts
Showing posts with label aggregate. Show all posts

Monday, March 12, 2012

Performance Advice

Hi,

The app I work on has a few complex queries (selects with 5-9 tables joined, data volumes of 200,000 rows +). These typically tend to aggregate info from the tables in question. They work on large volumes of data. We also have some stored procs that works on several rows of data (again in large volumes) and perform writes else where.

Overall the tables that these two types of queries run on contain millions of rows of data, and they normally work on a subset that is also substancially large - 100, 000 rows for example.

The app I work on is some sort of a tool(decision support type) and we dont expect that these queries be used like typical transaction processing systems. They will probably only be used by a few people at a time (1-3).

I've noticed that running this feature on the dev db box (2 GHZ Desktop PC with 1 GB ram) the CPU spikes to 100% and the DB server starts refusing connections to other users. Overall untill the long running queries are completed the DB may not be used by any other user.

I've tried to run the same app on a multi proc server config box. What I find is that such queries still use one of the CPUs 100%. But connections are not refused and other users can still use the database probably because of the other procs on the box.

Although this is a somewhat odd question - what does this mean ? does it mean that the query is poorly written ? Yes it is complex and we've tried to tune it to the extent possible - indexes, join sequences, intermediate materilization etc.

is it wrong to throw hardware at such a problem ? How much more might the query be tweaked ? and what will be the threshold level after which the query may not be tweaked anymore and needs better hardware ? How does one determine this ?

What I'm hoping to get with the above questions are just pointers for me to explore further and learn how I should proceed.

Any suggestions will be great.

Thanks,

Avinash

Hello,

I am not sure whether you are using MAX DOP option in your query running in production on multiple CPU's. Please look up "Degree Of Paralellism" in BOL.

I hope this helps....

Thanks.....

|||

Do any other applications use the database(s) on this server? Are they affected by your complex queries?

If performance turns out to be a serious problem, and if you tend to frequently aggregate your data, then you could consider using Analysis Services as part of your solution.

Chris

|||

Hello Chris,

Many thanks for your response. I dont have an exact answer to your question because what we build is a product and it may very well be installed on a shared database server.

But what I have seen in general is that the complex queries eat up a 100% of just one CPU per user and the number of CPUs being used 100% goes up with additional users performing the complex query. In my scenario, we dont expect any more than 1 or 2 users running such queries. If the app is running on a quad board machine, other database users will not be absolutely stranded. That is assuming that the other database users dont have similar demands from the database.

I've seen other Transaction Processing kind of apps work allright on another database on the same database server. Each complex query chews up one processor from what I've seen.

Honestly - I'm not conversant with analysis services. I will look into it. But I was hoping to get some ideas on capacity planning and determining what may be the base hardware requirement to service our needs knowing/having some idea of the volume we work with.

Any ideas will be appreciated.

Thanks,

Avinash

|||hi,
firstly i think if you use long running transactions you should consider Windows Workflow Foundation.by coincidence i have searched about this condition. i have a database which has a table with an average of 3 million rows and a relative table which has a 1,5 million rows and lastly a relative table has a 2 million rows. so i thought about what can i do. despite i have a server which has lower system capacities than yours, my server's cpu never spikes to 100%. i think there is a problem in your sql statement. once i had 2 table with 100 000 and 250 000 rows and as yours my server crashed too, i looked at task manager and saw that this query increased memory to 7 GB! anyway i suggest you to use index tuning wizard. if you didnt index your tables correctly you should do that right now. but if you indexed your tables then you should consider partitioning on tables. as i said before i execute long queries and i get response between 0-1 second! if you clarify your queries functionality then i can help you.(e.g are they insert, update or select queries?)
|||

Hello Ibrahim,

My queries are mostly complex selects with a whole bunch where conditions. They typically aggrregate data across several thousands of rows and they need to sift through millions of rows to find the subset of data.

But also I have some inserts - but they have additional read logic which again means selects with complex conditions and aggregations.

The point that I seem to have accepted is - when you say "as i said before i execute long queries and i get response between 0-1 second! if " that cannot be an absolute statement right ? In the sense that at some point your needs will require better hardware. It cannot be that any need can be satisfied by existing hardware. What I'm trying to understand is how do you determine that for your needs, X type of hardware is non negotiable.

Thanks,

Avinash

|||In my past experience, I split a complex join statement to multiple small statement. That will reduce CPU usage. Also you can review your SQL statement to find out whether there has some logic error to reduce number of temp records generated.

Performance 500 million rows

Ok here goes,
500 Million Rows about 10 cols about 16 bytes each col.
What is the best way to approach multiple aggregate functions against
this data ?
I have pretty much any option available to me including 8 Dual 3 ghz
Zeons on GB eth with 2gb ram that are at my disposal
What is the best scheme for handling data of this size with expensive
queries (aggregares etc)
Partitioned Table structure , (horizontal or vertical)
Federated Servers ?
Any point in the right direction is appreciated right now my plan is
as follows
8 Servers in 4 pairs , each pair load balanced and containing the
same Data as it mate in the pair,
Partitioning the table between the 4 pairs
Is this reasonable ?
P.S. there is no concern on redundancy at this point.
Thanks
Chris
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
Analysis Services?
A
|||"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.c om...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris
Pre-aggregating your data and using analysis services is probably the
fastest method, but you may also take a look at Indexed Views in the BOL.
There are some assumptions and requirements with Indexed views however and
you should read those carefully. Things to consider are how often the data
is modified, how often it is queried etc. An indexed view can be helpful
if the data is relatively static and computations are deterministic. It is
also helpful if the queries will be using the views on a regular basis. (As
opposed to once a month or once a week).
50 million rows however is quite a lot, so I'm not sure whether or not
Indexed Views will be the most helpful.
HTH
Rick Sawtell
MCT, MCSD, MCDBA
|||"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.c om...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris
You could also an consider indexed view with the aggregated data in view
definition.
http://snodland.blogspot.com
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.788 / Virus Database: 533 - Release Date: 01/11/2004
|||Well I jacked up my test data to 1 billion rows (I feel like Dr. Evil
every time I say that) for testing, Im going to partition it against
the 8 servers for now and Ive come up with a fairly clean partitioning
scheme (I think
Someone mailed me a link to a rather lengthy and detailed MS run
trhrough on Federated Servers and it does indeed look like its going
to be my ticket, I (because of size) am going to seperate the table
between 16 partitions (2 on each server) and (about 10 or so of the
partitions probably wont be even selected on more than once a week) so
I am thinking this is the route
The only thing I am a little fuzzy about is replication from my MAIN
DB (it has a 1b rows) out to my federation, Anyone have any
suggestions on replication to my federation.
I should have also mentioned my data is read only on the federation
and partition table, It is just being reported on, to that end it does
need to be current.
Thanks for the quick responses.
Any help on pointers to replication with partitioned tables would be
great.
Chris
cwertman@.webchamps.com (WertmanTheMad) wrote in message
news:<13990782.0411091224.45f30ed4@.posting.google. com>...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris
|||Since your initial post was talking about multiple aggregates, instead of
looking up some small subset of detailed rows, you are making a mistake with
your implementaion... You are setting up a huge OLTP database structure for
a problem that looks like an OLAP problem..
If you are talking about showing product sales, by time period, by store,,,
some sort of aggregates like that - OLAP is definitely the answer... You
will find responses almost instantaneous....
I don't mean to be pushy and arrogant, but please take another look at
Analysis Services for this... You'll have fewer licenses, fewer machines,
faster response - about the only downside is that you'll have to get
familiar with another set of tools etc, (and that might not be bad either..)
Good luck on your project.
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.c om...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris
|||Also, it looks like you are willing to throw a lot of hardware at this
problem, but you're only going to put 2 GB on each server? That doesn't
make sense.
You should consider using the Analysis Services, but pump one of those, or a
couple of them, up to 8 GB!
Sincerely,
Anthony Thomas

"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.c om...
Ok here goes,
500 Million Rows about 10 cols about 16 bytes each col.
What is the best way to approach multiple aggregate functions against
this data ?
I have pretty much any option available to me including 8 Dual 3 ghz
Zeons on GB eth with 2gb ram that are at my disposal
What is the best scheme for handling data of this size with expensive
queries (aggregares etc)
Partitioned Table structure , (horizontal or vertical)
Federated Servers ?
Any point in the right direction is appreciated right now my plan is
as follows
8 Servers in 4 pairs , each pair load balanced and containing the
same Data as it mate in the pair,
Partitioning the table between the 4 pairs
Is this reasonable ?
P.S. there is no concern on redundancy at this point.
Thanks
Chris

Performance 500 million rows

Ok here goes,
500 Million Rows about 10 cols about 16 bytes each col.
What is the best way to approach multiple aggregate functions against
this data ?
I have pretty much any option available to me including 8 Dual 3 ghz
Zeons on GB eth with 2gb ram that are at my disposal
What is the best scheme for handling data of this size with expensive
queries (aggregares etc)
Partitioned Table structure , (horizontal or vertical)
Federated Servers ?
Any point in the right direction is appreciated right now my plan is
as follows
8 Servers in 4 pairs , each pair load balanced and containing the
same Data as it mate in the pair,
Partitioning the table between the 4 pairs
Is this reasonable ?
P.S. there is no concern on redundancy at this point.
Thanks
Chris> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
Analysis Services?
A|||"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.com...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris
Pre-aggregating your data and using analysis services is probably the
fastest method, but you may also take a look at Indexed Views in the BOL.
There are some assumptions and requirements with Indexed views however and
you should read those carefully. Things to consider are how often the data
is modified, how often it is queried etc. An indexed view can be helpful
if the data is relatively static and computations are deterministic. It is
also helpful if the queries will be using the views on a regular basis. (As
opposed to once a month or once a week).
50 million rows however is quite a lot, so I'm not sure whether or not
Indexed Views will be the most helpful.
HTH
Rick Sawtell
MCT, MCSD, MCDBA|||"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.com...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris
You could also an consider indexed view with the aggregated data in view
definition.
http://snodland.blogspot.com
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.788 / Virus Database: 533 - Release Date: 01/11/2004|||Well I jacked up my test data to 1 billion rows (I feel like Dr. Evil
every time I say that) for testing, Im going to partition it against
the 8 servers for now and Ive come up with a fairly clean partitioning
scheme (I think
Someone mailed me a link to a rather lengthy and detailed MS run
trhrough on Federated Servers and it does indeed look like its going
to be my ticket, I (because of size) am going to seperate the table
between 16 partitions (2 on each server) and (about 10 or so of the
partitions probably wont be even selected on more than once a week) so
I am thinking this is the route
The only thing I am a little fuzzy about is replication from my MAIN
DB (it has a 1b rows) out to my federation, Anyone have any
suggestions on replication to my federation.
I should have also mentioned my data is read only on the federation
and partition table, It is just being reported on, to that end it does
need to be current.
Thanks for the quick responses.
Any help on pointers to replication with partitioned tables would be
great.
Chris
cwertman@.webchamps.com (WertmanTheMad) wrote in message
news:<13990782.0411091224.45f30ed4@.posting.google.com>...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris|||Since your initial post was talking about multiple aggregates, instead of
looking up some small subset of detailed rows, you are making a mistake with
your implementaion... You are setting up a huge OLTP database structure for
a problem that looks like an OLAP problem..
If you are talking about showing product sales, by time period, by store,,,
some sort of aggregates like that - OLAP is definitely the answer... You
will find responses almost instantaneous....
I don't mean to be pushy and arrogant, but please take another look at
Analysis Services for this... You'll have fewer licenses, fewer machines,
faster response - about the only downside is that you'll have to get
familiar with another set of tools etc, (and that might not be bad either..)
Good luck on your project.
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.com...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris|||Also, it looks like you are willing to throw a lot of hardware at this
problem, but you're only going to put 2 GB on each server? That doesn't
make sense.
You should consider using the Analysis Services, but pump one of those, or a
couple of them, up to 8 GB!
Sincerely,
Anthony Thomas
"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.com...
Ok here goes,
500 Million Rows about 10 cols about 16 bytes each col.
What is the best way to approach multiple aggregate functions against
this data ?
I have pretty much any option available to me including 8 Dual 3 ghz
Zeons on GB eth with 2gb ram that are at my disposal
What is the best scheme for handling data of this size with expensive
queries (aggregares etc)
Partitioned Table structure , (horizontal or vertical)
Federated Servers ?
Any point in the right direction is appreciated right now my plan is
as follows
8 Servers in 4 pairs , each pair load balanced and containing the
same Data as it mate in the pair,
Partitioning the table between the 4 pairs
Is this reasonable ?
P.S. there is no concern on redundancy at this point.
Thanks
Chris

Performance 500 million rows

Ok here goes,
500 Million Rows about 10 cols about 16 bytes each col.
What is the best way to approach multiple aggregate functions against
this data ?
I have pretty much any option available to me including 8 Dual 3 ghz
Zeons on GB eth with 2gb ram that are at my disposal
What is the best scheme for handling data of this size with expensive
queries (aggregares etc)
Partitioned Table structure , (horizontal or vertical)
Federated Servers ?
Any point in the right direction is appreciated right now my plan is
as follows
8 Servers in 4 pairs , each pair load balanced and containing the
same Data as it mate in the pair,
Partitioning the table between the 4 pairs
Is this reasonable ?
P.S. there is no concern on redundancy at this point.
Thanks
Chris> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
Analysis Services?
A|||"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.com...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris
Pre-aggregating your data and using analysis services is probably the
fastest method, but you may also take a look at Indexed Views in the BOL.
There are some assumptions and requirements with Indexed views however and
you should read those carefully. Things to consider are how often the data
is modified, how often it is queried etc. An indexed view can be helpful
if the data is relatively static and computations are deterministic. It is
also helpful if the queries will be using the views on a regular basis. (As
opposed to once a month or once a week).
50 million rows however is quite a lot, so I'm not sure whether or not
Indexed Views will be the most helpful.
HTH
Rick Sawtell
MCT, MCSD, MCDBA|||"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.com...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris
You could also an consider indexed view with the aggregated data in view
definition.
http://snodland.blogspot.com
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.788 / Virus Database: 533 - Release Date: 01/11/2004|||Well I jacked up my test data to 1 billion rows (I feel like Dr. Evil
every time I say that) for testing, Im going to partition it against
the 8 servers for now and Ive come up with a fairly clean partitioning
scheme (I think:)
Someone mailed me a link to a rather lengthy and detailed MS run
trhrough on Federated Servers and it does indeed look like its going
to be my ticket, I (because of size) am going to seperate the table
between 16 partitions (2 on each server) and (about 10 or so of the
partitions probably wont be even selected on more than once a week) so
I am thinking this is the route
The only thing I am a little fuzzy about is replication from my MAIN
DB (it has a 1b rows) out to my federation, Anyone have any
suggestions on replication to my federation.
I should have also mentioned my data is read only on the federation
and partition table, It is just being reported on, to that end it does
need to be current.
Thanks for the quick responses.
Any help on pointers to replication with partitioned tables would be
great.
Chris
cwertman@.webchamps.com (WertmanTheMad) wrote in message
news:<13990782.0411091224.45f30ed4@.posting.google.com>...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris|||Since your initial post was talking about multiple aggregates, instead of
looking up some small subset of detailed rows, you are making a mistake with
your implementaion... You are setting up a huge OLTP database structure for
a problem that looks like an OLAP problem..
If you are talking about showing product sales, by time period, by store,,,
some sort of aggregates like that - OLAP is definitely the answer... You
will find responses almost instantaneous....
I don't mean to be pushy and arrogant, but please take another look at
Analysis Services for this... You'll have fewer licenses, fewer machines,
faster response - about the only downside is that you'll have to get
familiar with another set of tools etc, (and that might not be bad either..)
Good luck on your project.
--
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.com...
> Ok here goes,
> 500 Million Rows about 10 cols about 16 bytes each col.
> What is the best way to approach multiple aggregate functions against
> this data ?
> I have pretty much any option available to me including 8 Dual 3 ghz
> Zeons on GB eth with 2gb ram that are at my disposal
> What is the best scheme for handling data of this size with expensive
> queries (aggregares etc)
> Partitioned Table structure , (horizontal or vertical)
> Federated Servers ?
> Any point in the right direction is appreciated right now my plan is
> as follows
> 8 Servers in 4 pairs , each pair load balanced and containing the
> same Data as it mate in the pair,
> Partitioning the table between the 4 pairs
> Is this reasonable ?
> P.S. there is no concern on redundancy at this point.
> Thanks
> Chris|||Also, it looks like you are willing to throw a lot of hardware at this
problem, but you're only going to put 2 GB on each server? That doesn't
make sense.
You should consider using the Analysis Services, but pump one of those, or a
couple of them, up to 8 GB!
Sincerely,
Anthony Thomas
"WertmanTheMad" <cwertman@.webchamps.com> wrote in message
news:13990782.0411091224.45f30ed4@.posting.google.com...
Ok here goes,
500 Million Rows about 10 cols about 16 bytes each col.
What is the best way to approach multiple aggregate functions against
this data ?
I have pretty much any option available to me including 8 Dual 3 ghz
Zeons on GB eth with 2gb ram that are at my disposal
What is the best scheme for handling data of this size with expensive
queries (aggregares etc)
Partitioned Table structure , (horizontal or vertical)
Federated Servers ?
Any point in the right direction is appreciated right now my plan is
as follows
8 Servers in 4 pairs , each pair load balanced and containing the
same Data as it mate in the pair,
Partitioning the table between the 4 pairs
Is this reasonable ?
P.S. there is no concern on redundancy at this point.
Thanks
Chris

Saturday, February 25, 2012

Perform aggregate functions on uniqueidentifiers

For some reason, [on sql2k] one cannot perform "Count(X)" where X is of type
uniqueidentifier. Will future versions of sql server suffer from this
limitation? 2003 or 2005?
We came across this problem when we had to execute a query with multiple
table joins.
Hasani,
The workaround that I use is to store them as BINARY(16).
"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:%233sMM$flEHA.3564@.TK2MSFTNGP14.phx.gbl...
> For some reason, [on sql2k] one cannot perform "Count(X)" where X is of
type
> uniqueidentifier. Will future versions of sql server suffer from this
> limitation? 2003 or 2005?
> We came across this problem when we had to execute a query with multiple
> table joins.
>
|||clever, i'll tell my supervisor tomorrow.
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:Oc23mQglEHA.592@.TK2MSFTNGP11.phx.gbl...
> Hasani,
> The workaround that I use is to store them as BINARY(16).
>
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:%233sMM$flEHA.3564@.TK2MSFTNGP14.phx.gbl...
> type
>
|||"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:%23gbFEiglEHA.2892@.tk2msftngp13.phx.gbl...
> clever, i'll tell my supervisor tomorrow.
If you want to get even tricker, you can experiment with doing something
like this when you store the GUID:
SELECT CONVERT(BINARY(6), GETDATE()) + CONVERT(BINARY(10), NEWID()) AS
DateGUID
This reduces the uniqueness a bit (removes 6 of the 16 bytes), but not
too much because there are only so many rows you can insert every 3
milliseconds. The upside is that you can now cluster on your GUID column
without destroying INSERT performance.
|||Will sql server allow binary columntypes as primary keys?
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:eb$oqkglEHA.3712@.TK2MSFTNGP15.phx.gbl...
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:%23gbFEiglEHA.2892@.tk2msftngp13.phx.gbl...
> If you want to get even tricker, you can experiment with doing
> something
> like this when you store the GUID:
> SELECT CONVERT(BINARY(6), GETDATE()) + CONVERT(BINARY(10), NEWID()) AS
> DateGUID
> This reduces the uniqueness a bit (removes 6 of the 16 bytes), but not
> too much because there are only so many rows you can insert every 3
> milliseconds. The upside is that you can now cluster on your GUID column
> without destroying INSERT performance.
>
|||"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:eP7D5uglEHA.712@.TK2MSFTNGP09.phx.gbl...
> Will sql server allow binary columntypes as primary keys?
Yes. When I have used GUIDs as primary keys (rarely, I don't think it's
a great idea most of the time), I have used the BINARY(16) technique. More
recently I've used the date concatenation technique in a project and it
worked out very well.
|||What are you reasons for not using a guid as a primary key?
We currently use integers as a primary key, but we use a stored procedure to
generate a unqiue random non-sequential integer, and we store this value in
a table to stop duplicates. In that scenario, I'm arguing that we should
just use uniqueidentifier types because we seem to just be reinventing the
wheel, but then someone mention the aggregate function thing with
uniqueidentifier types. I'm not aware of any penalties associated with using
uniqueidentifier types though, other than, it will require more bytes per
column, than an int.
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:%23OvcnxglEHA.596@.tk2msftngp13.phx.gbl...
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:eP7D5uglEHA.712@.TK2MSFTNGP09.phx.gbl...
> Yes. When I have used GUIDs as primary keys (rarely, I don't think
> it's
> a great idea most of the time), I have used the BINARY(16) technique.
> More
> recently I've used the date concatenation technique in a project and it
> worked out very well.
>
|||Hasani (remove nospam from address) wrote:
> What are you reasons for not using a guid as a primary key?
> We currently use integers as a primary key, but we use a stored
> procedure to generate a unqiue random non-sequential integer, and we
> store this value in a table to stop duplicates. In that scenario, I'm
> arguing that we should just use uniqueidentifier types because we
> seem to just be reinventing the wheel, but then someone mention the
> aggregate function thing with uniqueidentifier types. I'm not aware
> of any penalties associated with using uniqueidentifier types though,
> other than, it will require more bytes per column, than an int.
You're right in that it's a lot more bytes per row using a UID as
opposed to an INT IDENTITY. Four times the storage, which translates to
a much larger index when using a uniqueidentifier. And as Adam
eloquently mentioned, using a UID as a clustered key does not work well
because you get a lot of page splitting and head movement on the drives.
Adding a date component as a prefix to the UID prevents much of th epage
splitting, increasing insert performance. However, using a UID as
clustered key means propagating that key to all non-clustered indexes,
making them much larger as well.
If you can, I would stick with an INT IDENTITY column for a PK.
David G.
|||"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:OUyrc5glEHA.2892@.tk2msftngp13.phx.gbl...
> What are you reasons for not using a guid as a primary key?
> We currently use integers as a primary key, but we use a stored procedure
to
> generate a unqiue random non-sequential integer, and we store this value
in
> a table to stop duplicates. In that scenario, I'm arguing that we should
> just use uniqueidentifier types because we seem to just be reinventing the
> wheel, but then someone mention the aggregate function thing with
> uniqueidentifier types. I'm not aware of any penalties associated with
using
> uniqueidentifier types though, other than, it will require more bytes per
> column, than an int.
I think David G pointed out most of the issues in his post, so I'll
instead refer to the only times I have had to use a GUID, which is when the
application itself was responsible for creating the key. Applications
cannot reliably create unique integers, so GUIDs are pretty much the only
choice (or natural primary keys, if there's one available).
Also, why would you want to use a non-sequential random integer instead
of an IDENTITY?
|||Maybe I contradicted myself when I said non-sequential random...
We essentially need a random number generator to use as a primary key value.
I don't know if sql supports it. All I've seen is a unique number generator
that increments by one on every insert. It's unique but not random. The
problem is is, this value is going to be made public and we don't want to
make it obvious that it's just an incrementing value (think cookies and
websessions).
What we currently do (sometimes) is have 2 columns, I that's an
autoincrementing int that's a primary key, and the other is a
uniqueidentifer column that isn't a primary key (but may have a constraint
to make sure there are no duplicates), and we would make the uniqueidentifer
value public so in a cookie, it would always look random.
I don't feel comfortable in the scenario because you have 2 columns that are
doing the same thing (preserving/ensuring uniqueness). So I'm trying to look
at all the tradeoffs of using a uniqueidentifier instead of an int, and vice
versa.
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:%23IUly9mlEHA.1652@.TK2MSFTNGP09.phx.gbl...
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:OUyrc5glEHA.2892@.tk2msftngp13.phx.gbl...
> to
> in
> using
> I think David G pointed out most of the issues in his post, so I'll
> instead refer to the only times I have had to use a GUID, which is when
> the
> application itself was responsible for creating the key. Applications
> cannot reliably create unique integers, so GUIDs are pretty much the only
> choice (or natural primary keys, if there's one available).
> Also, why would you want to use a non-sequential random integer instead
> of an IDENTITY?
>

Perform aggregate functions on uniqueidentifiers

For some reason, [on sql2k] one cannot perform "Count(X)" where X is of type
uniqueidentifier. Will future versions of sql server suffer from this
limitation? 2003 or 2005?
We came across this problem when we had to execute a query with multiple
table joins.Hasani,
The workaround that I use is to store them as BINARY(16).
"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:%233sMM$flEHA.3564@.TK2MSFTNGP14.phx.gbl...
> For some reason, [on sql2k] one cannot perform "Count(X)" where X is of
type
> uniqueidentifier. Will future versions of sql server suffer from this
> limitation? 2003 or 2005?
> We came across this problem when we had to execute a query with multiple
> table joins.
>|||clever, i'll tell my supervisor tomorrow.
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:Oc23mQglEHA.592@.TK2MSFTNGP11.phx.gbl...
> Hasani,
> The workaround that I use is to store them as BINARY(16).
>
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:%233sMM$flEHA.3564@.TK2MSFTNGP14.phx.gbl...
>> For some reason, [on sql2k] one cannot perform "Count(X)" where X is of
> type
>> uniqueidentifier. Will future versions of sql server suffer from this
>> limitation? 2003 or 2005?
>> We came across this problem when we had to execute a query with multiple
>> table joins.
>>
>|||"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:%23gbFEiglEHA.2892@.tk2msftngp13.phx.gbl...
> clever, i'll tell my supervisor tomorrow.
If you want to get even tricker, you can experiment with doing something
like this when you store the GUID:
SELECT CONVERT(BINARY(6), GETDATE()) + CONVERT(BINARY(10), NEWID()) AS
DateGUID
This reduces the uniqueness a bit (removes 6 of the 16 bytes), but not
too much because there are only so many rows you can insert every 3
milliseconds. The upside is that you can now cluster on your GUID column
without destroying INSERT performance.|||Will sql server allow binary columntypes as primary keys?
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:eb$oqkglEHA.3712@.TK2MSFTNGP15.phx.gbl...
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:%23gbFEiglEHA.2892@.tk2msftngp13.phx.gbl...
>> clever, i'll tell my supervisor tomorrow.
> If you want to get even tricker, you can experiment with doing
> something
> like this when you store the GUID:
> SELECT CONVERT(BINARY(6), GETDATE()) + CONVERT(BINARY(10), NEWID()) AS
> DateGUID
> This reduces the uniqueness a bit (removes 6 of the 16 bytes), but not
> too much because there are only so many rows you can insert every 3
> milliseconds. The upside is that you can now cluster on your GUID column
> without destroying INSERT performance.
>|||"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:eP7D5uglEHA.712@.TK2MSFTNGP09.phx.gbl...
> Will sql server allow binary columntypes as primary keys?
Yes. When I have used GUIDs as primary keys (rarely, I don't think it's
a great idea most of the time), I have used the BINARY(16) technique. More
recently I've used the date concatenation technique in a project and it
worked out very well.|||What are you reasons for not using a guid as a primary key?
We currently use integers as a primary key, but we use a stored procedure to
generate a unqiue random non-sequential integer, and we store this value in
a table to stop duplicates. In that scenario, I'm arguing that we should
just use uniqueidentifier types because we seem to just be reinventing the
wheel, but then someone mention the aggregate function thing with
uniqueidentifier types. I'm not aware of any penalties associated with using
uniqueidentifier types though, other than, it will require more bytes per
column, than an int.
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:%23OvcnxglEHA.596@.tk2msftngp13.phx.gbl...
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:eP7D5uglEHA.712@.TK2MSFTNGP09.phx.gbl...
>> Will sql server allow binary columntypes as primary keys?
> Yes. When I have used GUIDs as primary keys (rarely, I don't think
> it's
> a great idea most of the time), I have used the BINARY(16) technique.
> More
> recently I've used the date concatenation technique in a project and it
> worked out very well.
>|||Hasani (remove nospam from address) wrote:
> What are you reasons for not using a guid as a primary key?
> We currently use integers as a primary key, but we use a stored
> procedure to generate a unqiue random non-sequential integer, and we
> store this value in a table to stop duplicates. In that scenario, I'm
> arguing that we should just use uniqueidentifier types because we
> seem to just be reinventing the wheel, but then someone mention the
> aggregate function thing with uniqueidentifier types. I'm not aware
> of any penalties associated with using uniqueidentifier types though,
> other than, it will require more bytes per column, than an int.
You're right in that it's a lot more bytes per row using a UID as
opposed to an INT IDENTITY. Four times the storage, which translates to
a much larger index when using a uniqueidentifier. And as Adam
eloquently mentioned, using a UID as a clustered key does not work well
because you get a lot of page splitting and head movement on the drives.
Adding a date component as a prefix to the UID prevents much of th epage
splitting, increasing insert performance. However, using a UID as
clustered key means propagating that key to all non-clustered indexes,
making them much larger as well.
If you can, I would stick with an INT IDENTITY column for a PK.
David G.|||"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:OUyrc5glEHA.2892@.tk2msftngp13.phx.gbl...
> What are you reasons for not using a guid as a primary key?
> We currently use integers as a primary key, but we use a stored procedure
to
> generate a unqiue random non-sequential integer, and we store this value
in
> a table to stop duplicates. In that scenario, I'm arguing that we should
> just use uniqueidentifier types because we seem to just be reinventing the
> wheel, but then someone mention the aggregate function thing with
> uniqueidentifier types. I'm not aware of any penalties associated with
using
> uniqueidentifier types though, other than, it will require more bytes per
> column, than an int.
I think David G pointed out most of the issues in his post, so I'll
instead refer to the only times I have had to use a GUID, which is when the
application itself was responsible for creating the key. Applications
cannot reliably create unique integers, so GUIDs are pretty much the only
choice (or natural primary keys, if there's one available).
Also, why would you want to use a non-sequential random integer instead
of an IDENTITY?|||Maybe I contradicted myself when I said non-sequential random...
We essentially need a random number generator to use as a primary key value.
I don't know if sql supports it. All I've seen is a unique number generator
that increments by one on every insert. It's unique but not random. The
problem is is, this value is going to be made public and we don't want to
make it obvious that it's just an incrementing value (think cookies and
websessions).
What we currently do (sometimes) is have 2 columns, I that's an
autoincrementing int that's a primary key, and the other is a
uniqueidentifer column that isn't a primary key (but may have a constraint
to make sure there are no duplicates), and we would make the uniqueidentifer
value public so in a cookie, it would always look random.
I don't feel comfortable in the scenario because you have 2 columns that are
doing the same thing (preserving/ensuring uniqueness). So I'm trying to look
at all the tradeoffs of using a uniqueidentifier instead of an int, and vice
versa.
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:%23IUly9mlEHA.1652@.TK2MSFTNGP09.phx.gbl...
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:OUyrc5glEHA.2892@.tk2msftngp13.phx.gbl...
>> What are you reasons for not using a guid as a primary key?
>> We currently use integers as a primary key, but we use a stored procedure
> to
>> generate a unqiue random non-sequential integer, and we store this value
> in
>> a table to stop duplicates. In that scenario, I'm arguing that we should
>> just use uniqueidentifier types because we seem to just be reinventing
>> the
>> wheel, but then someone mention the aggregate function thing with
>> uniqueidentifier types. I'm not aware of any penalties associated with
> using
>> uniqueidentifier types though, other than, it will require more bytes per
>> column, than an int.
> I think David G pointed out most of the issues in his post, so I'll
> instead refer to the only times I have had to use a GUID, which is when
> the
> application itself was responsible for creating the key. Applications
> cannot reliably create unique integers, so GUIDs are pretty much the only
> choice (or natural primary keys, if there's one available).
> Also, why would you want to use a non-sequential random integer instead
> of an IDENTITY?
>|||"Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com> wrote
in message news:%23Aar%23RnlEHA.1356@.TK2MSFTNGP09.phx.gbl...
> Maybe I contradicted myself when I said non-sequential random...
> We essentially need a random number generator to use as a primary key
value.
> I don't know if sql supports it. All I've seen is a unique number
generator
> that increments by one on every insert. It's unique but not random. The
> problem is is, this value is going to be made public and we don't want to
> make it obvious that it's just an incrementing value (think cookies and
> websessions).
If you're only generating one at a time, why not just use RAND()?|||We'll there's a stored procedure someone created that uses RAND to create a
unique integer, by storing all values created by the stored proc in a table,
to stop duplicates, but, unfortunately, when a record is deleted that has a
value generated by the stored procedure, it doesn't remove the generated
value from the lookup table used by the stored procedure. That's the only
reason why I'm was RAND, but I can modify the code to make sure deleted
records 'release' the generated RAND value. But I do like the uid because
it's alphanumeric, which is secure more in a cookie, well in cracking time,
than an all numeric cookie.
"Adam Machanic" <amachanic@.hotmail._removetoemail_.com> wrote in message
news:OGss9inlEHA.748@.TK2MSFTNGP15.phx.gbl...
> "Hasani (remove nospam from address)" <hblackwell@.n0sp4m.popstick.com>
> wrote
> in message news:%23Aar%23RnlEHA.1356@.TK2MSFTNGP09.phx.gbl...
>> Maybe I contradicted myself when I said non-sequential random...
>> We essentially need a random number generator to use as a primary key
> value.
>> I don't know if sql supports it. All I've seen is a unique number
> generator
>> that increments by one on every insert. It's unique but not random. The
>> problem is is, this value is going to be made public and we don't want to
>> make it obvious that it's just an incrementing value (think cookies and
>> websessions).
> If you're only generating one at a time, why not just use RAND()?
>|||Hasani (remove nospam from address) wrote:
> We'll there's a stored procedure someone created that uses RAND to
> create a unique integer, by storing all values created by the stored
> proc in a table, to stop duplicates, but, unfortunately, when a
> record is deleted that has a value generated by the stored procedure,
> it doesn't remove the generated value from the lookup table used by
> the stored procedure. That's the only reason why I'm was RAND, but I
> can modify the code to make sure deleted records 'release' the
> generated RAND value. But I do like the uid because it's
> alphanumeric, which is secure more in a cookie, well in cracking
> time, than an all numeric cookie.
>
You could add a computed column to the table to do the same thing (which
would eliminate the overhead of using a uniqueidentifier altogether).
And you can start the identity value higher if you don't want it to
start at 0.
Something like:
Create Table #test (
ID INT IDENTITY NOT NULL,
SomeText nvarchar(10),
"CookieID" as N'ALPHA-STUFF' + RIGHT(N'0000000000' + CAST(ID as
NVARCHAR(10)), 10))
Insert into #test values ('ABC')
Insert into #test values ('ABC')
Insert into #test values ('ABC')
Insert into #test values ('ABC')
Insert into #test values ('ABC')
Select * from #test
David G.

perform aggregate function & group by

Hello, anyone can help?
below SQL works in sybase but fail in sql2000. sql2000 show error on the sum
of netweight (cannot perform aggregate function) and group by commodity_code
(invalid column name !!)
SELECT
(select HTS
from bur_inv_item_list
where item = material
and plantcode = plant) as commodity_code,
(select DESCRIPTION
from bur_inv_item_list
where item = material
and plantcode = plant) as description,
uom as uom,
(select Orig
from bur_inv_item_list
where item = material
and plantcode = plant) as coo,
netweight =
sum(case when plant = '0014' or plant = '0024' then
round(qty * (select weight
from bur_inv_item_list
where item = material
and plantcode = plant), 2)
else
round(qty * (select weight
from bur_inv_item_list
where item = material
and plantcode = plant) /2.20462, 2)
end),
sum(ext_cost) as cost
from bur_inv_cntr_list
group by
commodity_code,
description,
uom,
coo
order by
commodity_code,
description,
uom,
coo
;The logical evaluation order of a SELECT statement is (top to bottom):
FROM
WHERE
GROUP BY
HAVING
SELECT
ORDER BY
TOP
As you see, the SELECT hasn't happened yet when the GROUP BY is performed. T
his mean that you cannot
refer to any column alias name in the GROUP BY clause. Some product diverts
from the ANSI SQL
standard behavior, SQL Server does not (in this regard). So push the express
ions in a derived table
and work against that.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"frankie lo" <frankiecblo@.hotmail.com> wrote in message
news:emgGyQOaGHA.1196@.TK2MSFTNGP03.phx.gbl...
> Hello, anyone can help?
>
> below SQL works in sybase but fail in sql2000. sql2000 show error on the s
um of netweight (cannot
> perform aggregate function) and group by commodity_code (invalid column na
me !!)
>
> SELECT
> (select HTS
> from bur_inv_item_list
> where item = material
> and plantcode = plant) as commodity_code,
> (select DESCRIPTION
> from bur_inv_item_list
> where item = material
> and plantcode = plant) as description,
> uom as uom,
> (select Orig
> from bur_inv_item_list
> where item = material
> and plantcode = plant) as coo,
> netweight =
> sum(case when plant = '0014' or plant = '0024' then
> round(qty * (select weight
> from bur_inv_item_list
> where item = material
> and plantcode = plant), 2)
> else
> round(qty * (select weight
> from bur_inv_item_list
> where item = material
> and plantcode = plant) /2.20462, 2)
> end),
> sum(ext_cost) as cost
> from bur_inv_cntr_list
> group by
> commodity_code,
> description,
> uom,
> coo
> order by
> commodity_code,
> description,
> uom,
> coo
> ;
>|||hi tibor,
thanks for your info. do you have any sample/case. I want to see the sample
to modify below script.
"Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote in
message news:u3R1jsPaGHA.504@.TK2MSFTNGP03.phx.gbl...
> The logical evaluation order of a SELECT statement is (top to bottom):
> FROM
> WHERE
> GROUP BY
> HAVING
> SELECT
> ORDER BY
> TOP
> As you see, the SELECT hasn't happened yet when the GROUP BY is performed.
> This mean that you cannot refer to any column alias name in the GROUP BY
> clause. Some product diverts from the ANSI SQL standard behavior, SQL
> Server does not (in this regard). So push the expressions in a derived
> table and work against that.
>
> --
> Tibor Karaszi, SQL Server MVP
> http://www.karaszi.com/sqlserver/default.asp
> http://www.solidqualitylearning.com/
>
> "frankie lo" <frankiecblo@.hotmail.com> wrote in message
> news:emgGyQOaGHA.1196@.TK2MSFTNGP03.phx.gbl...
>|||Here's a very simple example where a derived table is used so you don't have
to repeat the DATEPART
expression:
USE pubs
SELECT DATEPART(mm, pubdate) AS pub_month, COUNT(*) AS no_of_titles
FROM titles
GROUP BY DATEPART(mm, pubdate)
SELECT pub_month, COUNT(*) AS titles
FROM
(
SELECT DATEPART(mm, pubdate) AS pub_month
FROM titles
) AS i
GROUP BY pub_month
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"frankie lo" <frankiecblo@.hotmail.com> wrote in message
news:OqUKBdRaGHA.2368@.TK2MSFTNGP03.phx.gbl...
> hi tibor,
> thanks for your info. do you have any sample/case. I want to see the sampl
e to modify below
> script.
> "Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote i
n message
> news:u3R1jsPaGHA.504@.TK2MSFTNGP03.phx.gbl...
>|||Hi Tibor,
Many Thanks.
"Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote in
message news:ebF1yHSaGHA.1196@.TK2MSFTNGP03.phx.gbl...
> Here's a very simple example where a derived table is used so you don't
> have to repeat the DATEPART expression:
> USE pubs
> SELECT DATEPART(mm, pubdate) AS pub_month, COUNT(*) AS no_of_titles
> FROM titles
> GROUP BY DATEPART(mm, pubdate)
> SELECT pub_month, COUNT(*) AS titles
> FROM
> (
> SELECT DATEPART(mm, pubdate) AS pub_month
> FROM titles
> ) AS i
> GROUP BY pub_month
>
> --
> Tibor Karaszi, SQL Server MVP
> http://www.karaszi.com/sqlserver/default.asp
> http://www.solidqualitylearning.com/
>
> "frankie lo" <frankiecblo@.hotmail.com> wrote in message
> news:OqUKBdRaGHA.2368@.TK2MSFTNGP03.phx.gbl...
>

Perform aggregate against group value

Is there a way that I can perform my aggregate function agains the value in
group row instead of details row?
Because I do not show numeric values in the details but group row.
--
SevDer
http://www.sevder.com
A new .NET Source For .NET DevelopersDid you try the Previous aggregate function? Just place it into the group
header like =Previous(Fields!Country.Value) and it should work. Note: the
previous function has only one argument.
-- Robert
This posting is provided "AS IS" with no warranties, and confers no rights.
"SevDer" <sevder@.newsgroup.nospam> wrote in message
news:uEOFqzm2FHA.1188@.TK2MSFTNGP12.phx.gbl...
> Is there a way that I can perform my aggregate function agains the value
> in group row instead of details row?
> Because I do not show numeric values in the details but group row.
> --
> SevDer
> http://www.sevder.com
> A new .NET Source For .NET Developers
>
>|||Hi Robert,
I tried Previous as you suggested but this time I endup with empty
datacell..
However, please excuse me that I was not clear enough previously, I want to
perform this aggregate against the group in the footer. So I tried to use
the full previous function as described in the help "Previous(Expression,
AggFunction, PreviousScope, AggScope)" but it basically fails as you've
mentioned.
Anyway, do you have a solution for me?
I would like to sum my group values in the footer.
--
SevDer
http://www.sevder.com
A new .NET Source For .NET Developers
"Robert Bruckner [MSFT]" <robruc@.online.microsoft.com> wrote in message
news:uzeghIq2FHA.3244@.tk2msftngp13.phx.gbl...
> Did you try the Previous aggregate function? Just place it into the group
> header like =Previous(Fields!Country.Value) and it should work. Note: the
> previous function has only one argument.
> -- Robert
> This posting is provided "AS IS" with no warranties, and confers no
> rights.
>
> "SevDer" <sevder@.newsgroup.nospam> wrote in message
> news:uEOFqzm2FHA.1188@.TK2MSFTNGP12.phx.gbl...
>> Is there a way that I can perform my aggregate function agains the value
>> in group row instead of details row?
>> Because I do not show numeric values in the details but group row.
>> --
>> SevDer
>> http://www.sevder.com
>> A new .NET Source For .NET Developers
>>
>|||> I would like to sum my group values in the footer.
Maybe I'm missing something, but assuming you group on
=Fields!FieldName.Value, just adding an expression like
=Sum(Fields!FieldName.Value) in the table footer should sum the group
values.
-- Robert
This posting is provided "AS IS" with no warranties, and confers no rights.
"SevDer" <sevder@.newsgroup.nospam> wrote in message
news:%23rqJeUw2FHA.1184@.TK2MSFTNGP12.phx.gbl...
> Hi Robert,
> I tried Previous as you suggested but this time I endup with empty
> datacell..
> However, please excuse me that I was not clear enough previously, I want
> to perform this aggregate against the group in the footer. So I tried to
> use the full previous function as described in the help
> "Previous(Expression, AggFunction, PreviousScope, AggScope)" but it
> basically fails as you've mentioned.
> Anyway, do you have a solution for me?
> I would like to sum my group values in the footer.
> --
> SevDer
> http://www.sevder.com
> A new .NET Source For .NET Developers
>
> "Robert Bruckner [MSFT]" <robruc@.online.microsoft.com> wrote in message
> news:uzeghIq2FHA.3244@.tk2msftngp13.phx.gbl...
>> Did you try the Previous aggregate function? Just place it into the group
>> header like =Previous(Fields!Country.Value) and it should work. Note: the
>> previous function has only one argument.
>> -- Robert
>> This posting is provided "AS IS" with no warranties, and confers no
>> rights.
>>
>> "SevDer" <sevder@.newsgroup.nospam> wrote in message
>> news:uEOFqzm2FHA.1188@.TK2MSFTNGP12.phx.gbl...
>> Is there a way that I can perform my aggregate function agains the value
>> in group row instead of details row?
>> Because I do not show numeric values in the details but group row.
>> --
>> SevDer
>> http://www.sevder.com
>> A new .NET Source For .NET Developers
>>
>>
>