xml queueing design help
- From: Chris Stephens <cstephens16@xxxxxxxxx>
- To: "oracle-l@xxxxxxxxxxxxx" <Oracle-L@xxxxxxxxxxxxx>
- Date: Fri, 29 Jul 2005 10:28:08 -0400
9.2 on windows (development)...moving to Linux in a month. Production
will be linux.
I've just begun work on a document submission application. There
will be 3 seperate types of documents submitted. We are going to make
use of AQ. The documents need to be processed in a certain order so
we are going to use priority queueing. One of the parameters into our
submit_queue procedure will tell us what type of file is being
submitted. 2 of the file types are highly variable in structure but
small in size. The third is simple measurement data that is pretty
constant but is very large. The people originally working on the
system envisioned that all 3 documents will be xml because they will
be passed around among various platforms. I don't have a problem with
that on the 2 smaller, more variable documents but i'm worried about
processing performance of the larger file.
Currently, the plan is to submit the file into the queue as a clob and
then we'll use xquery to move the data into relational tables where it
will be evaluated by a qa engine developed in .net (no control over
that, they want to reuse code already in place). We'll be using clobs
in the queue because there are fairly strict time constraints for
notifications that the submission was received and storing it as an
xmltype will slow that process down.
once it is in the queue, i'm worried that moving the xml data to the
qa area is going to be s-l-o-w. my initial thought was to test that
against placing the submission files on the server and then just using
the queue as a notification that the file is there. We would then use
external tables to move the data. The only problem is that i've been
told we flat out cannot do that. The files must go directly into the
database. The next option is strip out the xml tags on the app server
and submit the file as a fixed field text document with an
automatically generated xml document describing the format of that
text document. This will Greatly reduce the size of the file but i'm
not sure how fast i can get at clob data.
So, i guess my questions are:
Any other options i'm not thinking of?
Anyone go through a similar project? If so how was the performance?
How slow is xquery? (i can answer this myself but i thought i'd throw
it out there and possibly save some time)
How slow is sql access to fixed lenght clob data?(i can answer this
myself but i thought i'd throw it out there and possibly save some
time) ..oracle text would help me here correct?
Thanks for any help/comments!
chris
--
//www.freelists.org/webpage/oracle-l
Other related posts:
