I’ll look further into this and see how it scales against a real multimillion row data set.

Thanks very much indeed Bob. I’ll post back with results.

SELECT SRC_ID, TGT_ID
FROM (
SELECT SRC_ID, TGT_ID, MATCH
FROM SS_SET
MODEL
DIMENSION BY (SRC_ID, TGT_ID)
MEASURES (SRC_DATE, TGT_DATE, 0 MATCH)
RULES (
MATCH[ANY,ANY] ORDER BY SRC_DATE DESC, TGT_DATE ASC = CASE WHEN sum(MATCH)[ANY,CV()] = 0 AND sum(MATCH)[CV(),ANY] = 0 THEN 1 ELSE MATCH[CV(),CV()] END
)
)
WHERE MATCH = 1

I’m not sure how fast this will work with the type and volume of your data set, but it was fun trying to figure out how to do it in SQL.

You were talking about two (hence independent) data sets with, seemingly, a many-to-many relationship which needed resolving. It wasn’t obvious there was also a pool of candidate connections … your sample data set seemed to be just that, sample data … and not sample data with a rule built into it.

Also, with the old data set you were ordering and pushing into the memory stack the same pair of attributes. Now you’re ordering by a pair of attributes but push into the memory stack a different pair of attributes. Not exactly the same thing.

Even with your new test case there seem to be built-in assumptions which may or may not be relevant. For example, update src_date and tgt_date, for all rows, to the same date. With that, the result of your query (and algorithm for that matter) becomes non-deterministic … but why that would not constitute a real data set, I don’t know.

Having said that, I think your approach is quite valid and elegant. The query that I did was not substantially faster than your approach and, if anything, would get - given the extra rules required - only more complicated and slower.

I cannot think of a linear sql solution … a solution here appears to require a recursive approach. To do that without pl/sql would need some sort of hierarchical query (that a connect by query is straight sql is questionable anyway). Given your volume, the hierarchical query would be slooooowww! Hence I would not even attempt that.

Which brings me back to … someone may come up with a liner sql solution and I would be curious to see it; but I know it won’t be faster than what you have. As long as you don’t blow/page the user memory with those pl/sql arrays I think you’re good. Hopefully this matching is a one-time affair since any insert/update/delete has the potential to destabilize the whole thing (change the result and/or make it non-deterministic).

The only thing I would do differently is to pipe the results and do away with the need to reset those global arrays, etc. It was also faster in my tests for 1 million rows though it may be different for your real data:


create or replace type typ as object ( s number, t number );
/
show errors

S T
---------- ----------
5 1
1 3
4 4
2 2


Here’s a crack at a better data set:

drop table ss_set;

create table ss_set (src_id number, src_date date, tgt_id number, tgt_date date);

insert into ss_set values (1,date ‘2007-06-01′,1,date ‘2007-06-01′);
insert into ss_set values (1,date ‘2007-06-01′,2,date ‘2007-06-10′);
insert into ss_set values (1,date ‘2007-06-01′,3,date ‘2007-06-03′);
insert into ss_set values (2,date ‘2007-05-01′,1,date ‘2007-06-01′);
insert into ss_set values (2,date ‘2007-05-01′,2,date ‘2007-06-10′);
insert into ss_set values (2,date ‘2007-05-01′,4,date ‘2007-05-03′);
insert into ss_set values (3,date ‘2007-07-01′,1,date ‘2007-06-01′);
insert into ss_set values (4,date ‘2007-05-20′,2,date ‘2007-06-10′);
insert into ss_set values (4,date ‘2007-05-20′,4,date ‘2007-05-03′);
insert into ss_set values (5,date ‘2007-08-01′,1,date ‘2007-06-01′);
commit;

So the ordered data set looks like this:

select src_id,tgt_id
from ss_set
where rownum > 0
order by src_date desc, tgt_date asc
/

SRC_ID TGT_ID
———————- ———————-
5 1
3 1
1 1
1 3
1 2
4 4
4 2
2 4
2 1
2 2

And the final result should look like this:

exec ss.reset;

select * from
(select src_id,tgt_id
from ss_set
where rownum > 0
order by src_date desc, tgt_date asc)
where ss.is_new(src_id,tgt_id) = 1
/

SRC_ID TGT_ID
———————- ———————-
5 1
1 3
4 4
2 2

Note that source 3 has no valid connection because its only candidate connecton was to target 1, and source 5 already connected to that because it has a later source date.

Hopefully this isn’t too simple a data set …

Elapsed: 00:00:00.01


WITH src_data AS (
SELECT ROWNUM val
FROM DUAL
CONNECT BY LEVEL <= 1000
), tgt_data AS (
SELECT ROWNUM + 77 val
FROM DUAL
CONNECT BY LEVEL <= 1000
)
SELECT *
FROM (
SELECT
sd.VAL src_val,
td.VAL tgt_val,
ROW_NUMBER() OVER ( ORDER BY sd.VAL DESC ) src_num,
ROW_NUMBER() OVER ( ORDER BY td.VAL ) tgt_num
FROM SRC_DATA sd, TGT_DATA td
) L1
WHERE L1.src_num = L1.tgt_num