Not more graphs! ;-)

As soon as I’ve taken the darned kids to daycare –i can hear them arguing about a monkey right now.

I know, I know, I find myself in that situation all the time - arguing about a monkey.

We can then look at what variations might occur and theorise what effect that might have. eg.

* some reads requesting the same data as others, and hitting disk/controller cache.
* some index range scans issuing a very large number of small read requests, and each of those requests being delayed in favour of multiblock reads and hence suffering from performance degradation.

I’ll post a chart or two, Doug, which show good support for the way that the anticipatory scheduler keeps the disks reading at close to their theoretical capacity even in this type of multiuser environment. As soon as I’ve taken the darned kids to daycare –i can hear them arguing about a monkey right now.

“I have some preliminary results, by the way. The anticipatory scheduler is showing much better performance and, maybe as importantly, stability of performance with variations in degree of parallelism, even up to degrees that represent gross over-parallelism (DoP of 30 for data on 4 disks).”

I think the major possible failing in this type of test if it’s based on a single user because once those other tricky users start wanting to do stuff too, it’s going to undermine the purity of the tests. i.e. contiguous storage and the best scheduler might be very important to the importance of improving a single job but the blasted things tend to be multi-user ;-) (Oh, and I don’t see that as the same as PX being ‘multi-user’ because there’s co-ordination of the activity towards a single end-goal)

One of the issues I’m also looking at is whether it is possible to change the scheduling algorithm for a device dynamically — ie. without unmounting/mounting. It would anable the devices to be put into OLTP and batch-sympathetic i/o scheduling modes at different times of the day/month.

I have some preliminary results, by the way. The anticipatory scheduler is showing much better performance and, maybe as importantly, stability of performance with variations in degree of parallelism, even up to degrees that represent gross over-parallelism (DoP of 30 for data on 4 disks).

I’ve been idly wondering if using RMAN as a benchmark for sequential read operations v. parallelism might be a reasonable idea. For example, to test whether 30 2G files v. 1 60G file make a difference because of round-robinning - imp into each, then see how long RMAN takes to backup each with various numbers of channels.

Wish I had a hobby! :-)

I don’t have to worry about it myself, I just have to worry about what algorithm is managing it. I know that disk head movement represents an inefficiency that can theoretically be reduced by the choice of the correct scheduler, so all I need to do is choose the right one and provide the conditions for it to perform optimally (ie. large sequential reads, reasonably contiguous data to read).

With regard to a SAN, I suggest that you make it very clear to the administrators that they n0ow have an application with predicatable access patterns — say large sequential reads of logically contiguous data. If their answer to problems is that they have enough disks and cache to cope with anything that can be thrown at it then consider how much better performance could be if they would actually optimise the storage in line with the application’s use of it. Recall also how the attitude to RAID5 has changed over the years — no longer is the “enough disks and cache” thought to be an adequate response to the problems inherent in the technology.

By the way, this is my hobby! :D Sad but true.

If ASM scatters blocks around in a way that promotes excessive head movement, in comparison to the regular management, then I’d expect to see a performance issue. If there was a performance issue then that would be a black mark against its use. however, I don’t think it does (though I may be wrong).

Does the issue of the slowing of one read matter when other process are being equally served at the same time? That’s a question at the very heart of i/o scheduling (and queueing theory, which is what alll this is about) — what is meant by “equally served”? If it means that the scheduler finishes with one read request and then immediately moves the disk heads to satisfy another, then moves the heads back to satisfy another request from the first process then that might seem equitable, but it’s exactly analagous to a single check-in handling multiple check-ins at the airport at the same time.

Consider passenger A and passenger B, both waiting to be served. To check in each passenger takes five minutes, so passenger A is checked in in five minutes and passenger B waits for five minutes then gets checked in and is gone after a total wait of ten minutes. If, in an effort to be equitable to both parties, the check-in agent flits between the two then the total time to check them both in is now eleven minutes (taking into account a total latency of one minute due to walking between the desks), and they both wait the full eleven minutes to be finished. Not equitable at all!

For general purpose/OLTP databases I think that your points are very valid, but don’t neglect the context — data warehousing, where jumping through hoops to achieve goals that make no sense in other worlds is practically a way of life. This is by no means a “one size gfits all” topic — everything here is qualified by “Data Warehousing only”

The scan doesn’t have to be 100% contiguous through the entire segment, we just have to avoid the kind of excessive disk head movement that would harm performance. From the Oracle side that means allocating reasonably large extents — at least 32-64 Mb I’d think, although in practice we can easily go larger than that, into the Gb range for large data sets (the practical upper limit would be the least of the partition size and the ETL single load volume, divided by the number of devices over which they are spread, off the top of my head).

We also want to avoid ASSM — in the data warehouse we almost certainly want to promote a physical row order to either promote efficient compression or to lower the clustering factor for some set of columns indexes. ASSM does not provide a solution to any problem we encounter in DW, so I would not use it. PCTFREE 0, PCTUSED 99, COMPRESS, and use direct path to load the data.

I don’t think that parallel query is such a problem either — Oracle will attempt to allocate ranges to each PQ slave based on datafile if possible, and in any case will allocate ranges of rowid’s that make each slave’s target data set logically contiguous. Whatever mysterious algorithms govern the allocation for PQ slaves must surely allocate ranges with a view to logical contiguity (if that is a word) or the result would be terribly messy.

Multiple data files ought also to be no problem. Large extents and allocating new extents in multiples of the number of data files appears to take care of that (on LMT), although I haven’t run a rigourous test on it. Maybe that’s next :D

I don’t think that freelists, freelist groups and ASSM are relevant to PQ.

Now it could well be that it’s difficult to just say, “Scheduler X is the best for databases” (in fact I’m sure it is), but if we characterise databases by access patterns (”small random access with reads and writes mixed”, “large sequential access with reads during the day and writes at night”) then there ought to be a theoretical best choice, and we ought to be able to run a test to demonstrate that. In the “real world” we also ought to be able to benchmark applications and look at mean response times over the course of a week ora month when running different schedulers — maybe our theory gets supported and shows us and average response time 30% lower with scheduler X than scheduler Y. The important thing I think is to be aware that there are these options and to understand what the choices imply in terms of how they interact with our applications’ access patterns.

Here’s something else to consider — in 2.6.18 the default scheduler changes from anticipatory to CFQ http://linux.inet.hr/cfq_to_become_the_default_i_o_scheduler.html What impact is that going to have on your application? Maybe it’ll be an improvement, maybe it’ll counteract benefits experienced elsewhere. Maybe when you upgrade through that boundary you want to start passing “elevator=as” to the kernel at boot time to prevent the scheduler change.