Strange Oracle XMLType.getClobVal() result
Asked Answered
E

4

9

I use Oracle 11g (on Red Hat). I have simple regular table with XMLType column:

CREATE TABLE PROJECTS
(
  PROJECT_ID NUMBER(*, 0) NOT NULL,
  PROJECT SYS.XMLTYPE,
);

Using Oracle SQL Developer (on Windows) I do:

select T1.PROJECT P1 from PROJECTS T1 where PROJECT_ID = '161';

It works. I get one cell. I can double click and download whole XML file.

Then I tried to get result as CLOB:

select T1.PROJECT.getClobVal() P1 from PROJECTS T1 where PROJECT_ID = '161';

It works. I get one cell. I can double click and see whole text and copy it. BUT there is a problem. When I copy it to clipboard I get only first 4000 characters. It seems that there is 0x00 character at position 4000 and the rest of CLOB is not copied.

To confirm this, I wrote check in java:

// ... create projectsStatement
Reader reader = projectsStatement.getResultSet().getCharacterStream( "P1" );
BufferedReader bf = new BufferedReader( reader );
char buffer[] = new char[ 1024 ];
int count = 0;
int globalPos = 0;
while ( ( count = bf.read( buffer, 0, buffer.length ) ) > 0 )
    for ( int i = 0; i < count; i++, globalPos++ )
        if ( buffer[ i ] == 0 )
            throw new Exception( "ZERO at " + Integer.toString(globalPos) );

Reader returns full XML but my exception is thrown because there is null character at position 4000. I could remove this single byte but this would be rather strange workaround.

I don't use VARCHAR2 there but maybe this problem is related to VARCHAR2 limitation (4000 bytes) somehow ? Any other ideas ? Is this an Oracle bug or am I missing something ?

-------------------- Edit --------------------

Value was inserted using following stored procedure:

create or replace
procedure addProject( projectId number, projectXml clob ) is
  sqlstr varchar2(2000);
begin

  sqlstr := 'insert into projects ( PROJECT_ID, PROJECT ) VALUES ( :projectId, :projectData )';
  execute immediate sqlstr using projectId, XMLTYPE(projectXml);

end;

Java code used to call it:

try ( CallableStatement cs = connection.prepareCall("{call addProject(?,?)}") )
{
    cs.setInt( "projectId", projectId );
    cs.setCharacterStream( "projectXml", new StringReader(xmlStr) , xmlStr.length() );
    cs.execute();
}

-------------------- Edit. SIMPLE TEST --------------------

I will use all I learned from your answers. Create simplest table:

create table T1 ( P XMLTYPE );

Prepare two CLOBs with XMLs. First with null character, second without.

declare
  P1 clob;
  P2 clob;
  P3 clob;
begin

  P1 := '<a>';
  P2 := '<a>';
  FOR i IN 1..1000 LOOP
    P1 := P1 || '0123456789' || chr(0);
    P2 := P2 || '0123456789';
  END LOOP;
  P1 := P1 || '</a>';
  P2 := P2 || '</a>';

Check if null is in the first CLOB and not in the second one:

DBMS_OUTPUT.put_line( DBMS_LOB.INSTR( P1, chr(0) ) );
DBMS_OUTPUT.put_line( DBMS_LOB.INSTR( P2, chr(0) ) );

We will get as expected:

14
0

Try to insert first CLOB into XMLTYPE. It will not work. It is not possible to insert such value:

insert into T1 ( P ) values ( XMLTYPE( P1 ) );

Try to insert second CLOB into XMLTYPE. It will work:

insert into T1 ( P ) values ( XMLTYPE( P2 ) );

Try to read inserted XML into third CLOB. It will work:

select T.P.getClobVal() into P3 from T1 T where rownum = 1;

Check if there is null. There is NO null:

DBMS_OUTPUT.put_line( DBMS_LOB.INSTR( P3, chr(0) ) );

It seams that there is no null inside database and as long as we are in the PL/SQL context, there is no null. But when I try to use following SQL in SQL Developer ( on Windows ) or in Java ( on Red Hat EE and Tomcat7 ) I get null character at position 4000 in all returned CLOBs:

select T.P.getClobVal() from T1 T;

BR, JM

Enmity answered 23/11, 2012 at 18:35 Comment(6)
Can you write it to a file using utl_file and see how the contents look like? Can you also try select XMLType.getClobVal(PROJECT) from PROJECTS;? (Nothing functionally different though)Predella
How was the columm populated? Are you sure the problem is with retrieval - sounds unlikely if different clients see the same thing. You could also select a subtring of the value and see if the null char is still in there.Goldplate
I ran utlfile.sql and prvtfile.plb, but I still can not use utl_file (ORA-06521: PL/SQL: Error mapping function), sorry.Enmity
I used XMLType.getClobVal(PROJECT) and null character is there. Then I used T1.PROJECT.getBlobVal(nls_charset_id('AL32UTF8')) and there is NO null character in downloaded blob (using SQL Developer or Java).Enmity
I added inserting procedure in my post above.Enmity
@Alex I edited my post. It seems that indeed null is not there, because it is not possible to populated XMLTYPE column in such a way. It seems that problem is with retrieval.Enmity
T
7

It's not an Oracle bug (it stores and retrieves the \0 just fine. It's a client/windows bug (Different clients behave differently in regards to "NUL" as does windows)

chr(0) is not a valid character in non-blobs really (I'm curious how you ever get the XMLType to accept it in the first place as usually it wouldn't parse).

\0 is used in C to denote the end of a string (NUL terminator) and some GUIs would stop processing the string at that point. For example:

![SQL> select 'IM VISIBLE'||chr(0)||'BUT IM INVISIBLE'
  2  from dual
  3  /

'IMVISIBLE'||CHR(0)||'BUTIM
---------------------------
IM VISIBLE BUT IM INVISIBLE

SQL>

yet toad fails miserably on this: TOAD

sql developer fares better, as you can see it:

SQL Developer

but if you copy it, the clipboard will only copy it up to the nul character. this copy paste error isn't SQL developers fault though, it's a problem with windows clipboard not allowing NUL to paste properly.

you should just replace(T1.PROJECT.getClobVal(), chr(0), null) to get round this when using sql developer/windows clipboard.

Traipse answered 23/11, 2012 at 23:52 Comment(4)
Thank you for response. Of course clipboard is problem of Windows, but this is not the main problem, it is just a way to test it. I want to know why null is there :). And I think this is not a problem of client. When I download single cell from this table (using SQL Developer, or Blob in Java) there is no null character. It only appears when I use PROJECT.getClobVal() (both in SQL Developer and in Java). So for me it seams that getClobVal() inserts null character in position 4000 of it's result.Enmity
no it does not do this (or shouldn't) and if it is, on every single LOB, i'd recommend you file an SR with Oracle. I use XMLType extensively on files well over 1MB and its never dropped a chr(0) in there (10.2.0.4 + 11.2.0.2).Traipse
Thanks. I also use 11.2.02 (Express Edition). I edited my post with simplest test. It seems that problem occurs with every getClobVal() result when it is transfered outside the PL/SQL.Enmity
i've tested with 11.2.0.2.0 enterprise client + 11.2.0.2.0 enterprise db, and cant replicate, given your testcase, the 4000 chr(0) issue. sql developer version used was 3.2.20.09. have you tried using another machine or reinstalling your client s/ware to see if that addresses it? i cant see anything on oracle support site either, but as your db side test showed no chr(0), i'd start with client side (ie your java drivers and oracle client install)Traipse
B
3

I also was experiencing this same issue exactly as described by Mikosz (seeing an extra 'NUL' character around the 4000th character when outputting my XMLType value as a Clob). While playing around in SQLDeveloper I noticed an interesting workaround. I was trying to see the output of my XMLType, but was tired of scrolling to the 4000th character, so I started wrapping the Clob output in a substr(...). Much to my surprise, the issue actually disappeared. I incorporated this into my Java app and confirmed that the issue was no longer present and my Clob could be retrieved without the extra character. I know that this isn't an ideal workaround, and I'm still not sure why it works (would love if someone could explain it to me), but here's an abbreviated example of what I've currently got working:

// Gets the xml contents
String sql = "select substr(x.xml_content.getClobVal(), 0) as xml_content from my_table x";
ps = con.prepareStatement(sql);
if(rs.next()) {
  Reader reader = new BufferedReader(rs.getCharacterStream("xml_content"));
  ...
}
Bobsledding answered 21/2, 2013 at 17:31 Comment(2)
Also, it is worth mentioning that we use the XMLType column in a couple places. In the places where we have registered a schema (structured xml), I do not see this issue. For the XMLType column not using a registered schema we do see this issue.Bobsledding
Thanks a lot for your suggestion. It solved my issue too. This was the only place I could find a suitable solution.Raceway
E
3

Bug:14781609 XDB: XMLType.getclobval() returns a temporary LOB when XML is stored in a CLOB. fix in patchset 11.2.0.4

and another solution if read as blob, then no error like

T1.PROJECT.getBlobVal(nls_charset_id('UTF8'))
Elsewhere answered 18/2, 2014 at 15:32 Comment(0)
F
2

Easy enough to verify if it's the .getClobVal() call or not - perform an INSTR test in PL/SQL (not Java) on your resultant CLOB to see if the CHR(0) exists or not.

If it does not, then I would point the finger at your Oracle client install.

Foot answered 27/11, 2012 at 0:59 Comment(2)
Thanks. You are right that there is no null when in PL/SQL. It appears when the result is transfered outside. It is strange because I use two different clients ( SQL Developer on Windows and Java on Red Hat ).Enmity
You could be suffering from a JDBC bug; seeing as both of your clients are Java-based.Foot

© 2022 - 2024 — McMap. All rights reserved.