This blog is on permanent hold

Instead of being rude and not concluding something, I'm closing this blog. It'll remain as is for as long as Blogger exists, I decide to remove it permanently, or the Earth dies.

In other words, it's sticking around for information's sake.

Enjoy the historical blogs.

-John

Archive_Schema

This procedure (which by the way can EASILY be converted to a script) is to arcive a set of selected schemas. To do this, the user you login as (you shouldn't use SYS if you can avoid it... similar principle as root in the *NIX world) has to have the SELECT_CATALOG_ROLE granted. Note the AUTHID CURRENT_USER in the CREATE OR REPLACE line. This is also part of the security features for this script to allow viewing the DDL of other schema objects provided by Oracle and PL/SQL.

CREATE OR REPLACE PROCEDURE Archive_Schema( fileDirectory IN VARCHAR2, schemaList IN VARCHAR2, delimiter IN VARCHAR2 DEFAULT ',' ) AUTHID CURRENT_USER IS
    mCLOB               CLOB;
    mOutputFile         UTL_FILE.FILE_TYPE;
    mOwner              VARCHAR2(30);
    mObjectType         VARCHAR2(18);
    mObjectName         VARCHAR2(30);
    mDDLCursorID        INTEGER;
    mReturnValue        INTEGER;
    mErrCode            NUMBER;
    
    mDBHostName         VARCHAR(64);
    mDBInstanceName     VARCHAR(16);
    mDBInstanceNumber   NUMBER;
    mDBVersion          VARCHAR2(17);
    mDBStartupTime      DATE;
    
    mTypeSchemaArchive       typeSchemaArchive := typeSchemaArchive();
    
    PROCEDURE GetDBInfo IS
        BEGIN
            DBMS_SQL.PARSE( mDDLCursorID, 'SELECT instance_number, instance_name, host_name, version, startup_time  FROM v$instance', DBMS_SQL.NATIVE );
    
            DBMS_SQL.DEFINE_COLUMN( mDDLCursorID, 1, mDBInstanceNumber );
            DBMS_SQL.DEFINE_COLUMN( mDDLCursorID, 2, mDBInstanceName, 16 );
            DBMS_SQL.DEFINE_COLUMN( mDDLCursorID, 3, mDBHostName, 64 );
            DBMS_SQL.DEFINE_COLUMN( mDDLCursorID, 4, mDBVersion, 17 );
            DBMS_SQL.DEFINE_COLUMN( mDDLCursorID, 5, mDBStartupTime );
            
            mReturnValue := DBMS_SQL.EXECUTE_AND_FETCH( mDDLCursorID );
                
            DBMS_SQL.COLUMN_VALUE( mDDLCursorID, 1, mDBInstanceNumber );
            DBMS_SQL.COLUMN_VALUE( mDDLCursorID, 2, mDBInstanceName );
            DBMS_SQL.COLUMN_VALUE( mDDLCursorID, 3, mDBHostName );
            DBMS_SQL.COLUMN_VALUE( mDDLCursorID, 4, mDBVersion );
            DBMS_SQL.COLUMN_VALUE( mDDLCursorID, 5, mDBStartupTime );
        END;
    --END GetDBInfo
        
    PROCEDURE OpenOutputFile( fileDirectory IN VARCHAR2, schemaList IN VARCHAR2, delimiter IN VARCHAR2 DEFAULT ',' ) IS
        filename         VARCHAR2(255);
        
        BEGIN
             filename :=  mDBInstancename || '_' || REPLACE( schemaList, delimiter, '_' ) || '_' || TO_CHAR( SYSDATE, 'yyyymmddhh24miss' ) || '.sql';
             mOutputFile := UTL_FILE.FOPEN( fileDirectory, filename, 'w', 32767 );
        END;
    --END OpenOutputFile
    
    PROCEDURE ParseSchemaList( schemaList IN VARCHAR2, delimiter IN VARCHAR2 DEFAULT ',' ) IS
        -- declare and initialize PL/SQL table based on pre-created type
        -- variables needed for parsing
        offset            NUMBER DEFAULT 1;
        overallLength     NUMBER DEFAULT LENGTH( schemaList );  -- The length of the input string.
        stringLength      NUMBER;                               -- The length of the current chunk.
    
        BEGIN
            -- load temp table type from delimited list
    
            WHILE offset < overallLength LOOP
                stringLength := INSTR( schemaList, delimiter, offset );
                IF stringLength = 0 THEN
                    stringLength := overallLength + 1;
                END IF;
    
                mTypeSchemaArchive.EXTEND;
                mTypeSchemaArchive( mTypeSchemaArchive.COUNT ) := SUBSTR( schemaList, offset, stringLength - offset );
    
                offset := stringLength + 1;
            END LOOP;
        END;
    --END ParseSchemaList;
    
    PROCEDURE PrintOwnerHeader( p_Owner IN VARCHAR2 ) IS
        BEGIN
            UTL_FILE.NEW_LINE( mOutputFile, 2 );
            UTL_FILE.PUT_LINE( mOutputFile, '/*==================================' );
            UTL_FILE.PUTF( mOutputFile, 'SCHEMA: %s', p_Owner );
            UTL_FILE.NEW_LINE( mOutputFile, 1 );
            UTL_FILE.PUT_LINE( mOutputFile, '==================================*/' );
        END;
    -- END PrintOwnerHeader;
    
    PROCEDURE PrintObjectHeader( p_owner IN VARCHAR2, p_objectType IN VARCHAR2 ) IS
        BEGIN
            UTL_FILE.NEW_LINE( mOutputFile, 2 );
            UTL_FILE.PUT_LINE( mOutputFile, '/*==================================' );
            UTL_FILE.PUTF( mOutputFile, '%s SCHEMA OBJECT: %s', p_owner, p_objectType );
            UTL_FILE.NEW_LINE( mOutputFile, 1 );
            UTL_FILE.PUT_LINE( mOutputFile, '==================================*/' );
        END;
    -- END PrintObjectHeader;
       
    PROCEDURE PrintFileHeader IS        
        BEGIN
            UTL_FILE.PUT_LINE( mOutputFile, '/*==================================' );
            UTL_FILE.PUTF( mOutputFile, 'Select database schemas as of: %s', TO_CHAR( SYSDATE, 'dd-mon-yyyy hh24:mi:ss' ) );
            UTL_FILE.NEW_LINE( mOutputFile, 1 );
            UTL_FILE.PUTF( mOutputFile, 'HOST NAME:        %s', mDBHostName );
            UTL_FILE.NEW_LINE( mOutputFile, 1 );
            UTL_FILE.PUTF( mOutputFile, 'INSTANCE NAME:    %s', mDBInstanceName );
            UTL_FILE.NEW_LINE( mOutputFile, 1 );
            UTL_FILE.PUTF( mOutputFile, 'INSTANCE NUMBER:  %s', TO_CHAR( mDBInstanceNumber ) );
            UTL_FILE.NEW_LINE( mOutputFile, 1 );
            UTL_FILE.PUTF( mOutputFile, 'DB VERSION:       %s', mDBVersion );
            UTL_FILE.NEW_LINE( mOutputFile, 1 );
            UTL_FILE.PUTF( mOutputFile, 'DB STARTUP TIME:  %s', TO_CHAR( mDBStartupTime, 'dd-mon-yyyy hh24:mi:ss' ) );
            UTL_FILE.NEW_LINE( mOutputFile, 2 );
    
            UTL_FILE.PUT_LINE( mOutputFile, 'Selected Schemas:' );
            
            FOR schemas IN ( SELECT COLUMN_VALUE FROM TABLE(CAST(mTypeSchemaArchive AS typeSchemaArchive)) ) LOOP
                UTL_FILE.PUTF( mOutputFile, '    %s', schemas.COLUMN_VALUE );
                UTL_FILE.NEW_LINE( mOutputFile, 1 );
            END LOOP;
    
            UTL_FILE.NEW_LINE( mOutputFile, 1 );
            UTL_FILE.PUT_LINE( mOutputFile, '==================================*/' );
            UTL_FILE.NEW_LINE( mOutputFile, 2 );
        END;
    -- END PrintFileHeader;
    
    BEGIN
        mDDLCursorID := DBMS_SQL.OPEN_CURSOR;
        
        GetDBInfo;
        
        OpenOutputFile( fileDirectory, schemaList, delimiter );
    
        ParseSchemaList( schemaList, delimiter );        
    
        PrintFileHeader;
        
        DBMS_METADATA.SET_TRANSFORM_PARAM( DBMS_METADATA.SESSION_TRANSFORM, 'STORAGE', FALSE );
        
        FOR schemas IN ( SELECT owner, object_type, object_name FROM ALL_OBJECTS WHERE  owner IN ( SELECT COLUMN_VALUE FROM TABLE(CAST(mTypeSchemaArchive AS typeSchemaArchive)) ) ORDER BY owner, object_type, object_name ) LOOP
            BEGIN
                IF mOwner IS NULL THEN
                    mOwner      := schemas.owner;
                    mObjectType := schemas.object_type;
    
                    PrintOwnerHeader( mOwner );
                    PrintObjectHeader( mOwner, mObjectType );
                ELSE
                    IF mOwner <> schemas.owner THEN
                        mOwner      := schemas.owner;
                        mObjectType := schemas.object_type;
    
                        PrintOwnerHeader( mOwner );
                        PrintObjectHeader( mOwner, mObjectType );
    
                    ELSIF mObjectType <> schemas.object_type THEN
                        mObjectType := schemas.object_type;
    
                        PrintObjectHeader( mOwner, mObjectType );
    
                    END IF;
                END IF;
    
                mCLOB := DBMS_METADATA.GET_DDL( mObjectType, schemas.object_name, mOwner );
    
                UTL_FILE.PUT( mOutputFile, DBMS_LOB.SUBSTR( mCLOB, DBMS_LOB.GETLENGTH( mCLOB ), 1 ) );
                UTL_FILE.FFLUSH( mOutputFile );
            EXCEPTION
                WHEN OTHERS THEN                
                    UTL_FILE.PUTF( mOutputFile, 'Error on %s: %s.%s: %s', mObjectType, mOwner, schemas.object_name, SQLERRM(SQLCODE));
                    UTL_FILE.NEW_LINE( mOutputFile, 1 );
                    UTL_FILE.FFLUSH( mOutputFile );
            END;
        END LOOP;
    
        UTL_FILE.FCLOSE( mOutputFile );
    
        DBMS_METADATA.SET_TRANSFORM_PARAM( DBMS_METADATA.SESSION_TRANSFORM, 'DEFAULT', TRUE );
        
        DBMS_SQL.CLOSE_CURSOR( mDDLCursorID );    
    END;
/

Here's a sampling of the output from this:

/*==================================
Select database schemas as of: 11-aug-2005 15:52:32
HOST NAME:        gilligan
INSTANCE NAME:    gilligan
INSTANCE NUMBER:  1
DB VERSION:       9.2.0.5.0
DB STARTUP TIME:  10-aug-2005 14:33:41

Selected Schemas:
    PSG
    PERFSTAT

==================================*/




/*==================================
SCHEMA: PERFSTAT
==================================*/


/*==================================
PERFSTAT SCHEMA OBJECT: INDEX
==================================*/

  CREATE UNIQUE INDEX "PERFSTAT"."STATS$BG_EVENT_SUMMARY_PK" ON "PERFSTAT"."STATS$BG_EVENT_SUMMARY" ("SNAP_ID", "DBID", "INSTANCE_NUMBER", "EVENT") 
  PCTFREE 10 INITRANS 2 MAXTRANS 255 
  TABLESPACE "TOOLS" 
 
  CREATE UNIQUE INDEX "PERFSTAT"."STATS$BUFFER_POOL_STATS_PK" ON "PERFSTAT"."STATS$BUFFER_POOL_STATISTICS" ("SNAP_ID", "DBID", "INSTANCE_NUMBER", "ID") 
  PCTFREE 10 INITRANS 2 MAXTRANS 255 
  TABLESPACE "TOOLS" 
 
  CREATE UNIQUE INDEX "PERFSTAT"."STATS$DATABASE_INSTANCE_PK" ON "PERFSTAT"."STATS$DATABASE_INSTANCE" ("DBID", "INSTANCE_NUMBER", "STARTUP_TIME") 
  PCTFREE 10 INITRANS 2 MAXTRANS 255 
  TABLESPACE "TOOLS" 
 
  CREATE UNIQUE INDEX "PERFSTAT"."STATS$DB_CACHE_ADVICE_PK" ON "PERFSTAT"."STATS$DB_CACHE_ADVICE" ("SNAP_ID", "DBID", "INSTANCE_NUMBER", "ID", "BUFFERS_FOR_ESTIMATE") 
  PCTFREE 10 INITRANS 2 MAXTRANS 255 
  TABLESPACE "TOOLS" 
 
  CREATE UNIQUE INDEX "PERFSTAT"."STATS$DLM_MISC_PK" ON "PERFSTAT"."STATS$DLM_MISC" ("SNAP_ID", "DBID", "INSTANCE_NUMBER", "STATISTIC#") 
  PCTFREE 10 INITRANS 2 MAXTRANS 255 
  TABLESPACE "TOOLS"

UNDO stats watching...

After we had some UNDO woes here at work, I had to go through the ugly process of figuring out how much UNDO space we could use and how to set our UNDO_RETENTION parameter. Oh, is this a confusing and frustrating process.

Here's some statistics of our current systems without the gory detail:

17 GB of UNDO tablespace and no room to grow.
18000 for the UNDO_RETENTION parameter.
A 3K+ second MAX(V$UNDOSTAT.MAXQUERYLEN)
Data from 8 days ago (rolling) to current in V$UNDOSTAT.
We periodically get ORA-01555 (snapshot too old) and ORA-30036 (unable to extend undo segment) errors.

Here's what I have discovered so far:

1. Never read too far in posts on the internet. Flame wars distract you too much. No matter how entertaining it is to read Howard J. Rogers argue with Joel Garry, find what you need and get out. Time's 'a wastin'.


2. UNDO is an inexact science.


3. Oracle has a handle on things quite well.


4. Oracle has a funky way of figuring out how much UNDO space you will need for a given UNDO_RETENTION parameter.


5. Everyone has interesting ways to calculate UNDO space and the "optimal" UNDO_RETENTION parameter.

From Swiss company Akadia AG, I was able to derive some interesting statistics using some of the scripts they provided.
There formula for calculating optimal UNDO_RETENTION is this:

OPTIMAL UNDO RETENTION = ACTUAL UNDO SIZE / ( DB_BLOCK_SIZE * UNDO_BLOCK_PER_SEC )

This script will show the actual undo size:

SELECT SUM(a.bytes) "UNDO_SIZE"
FROM v$datafile a,
     v$tablespace b,
     dba_tablespaces c
WHERE c.contents = 'UNDO'
     AND c.status = 'ONLINE'
     AND b.name = c.tablespace_name
     AND a.ts# = b.ts#;

In my case, this is the result:

   UNDO_SIZE
  ----------
  1.7826E+10

(I think that equals 17GB...)

Next we get our UNDO blocks per second:

SELECT MAX(undoblks/((end_time-begin_time)*3600*24))
      "UNDO_BLOCK_PER_SEC"
FROM v$undostat;

And again, my results:

UNDO_BLOCK_PER_SEC
------------------
           651.855

And finally, the DB_BLOCK_SIZE:

SELECT TO_NUMBER(value) "DB_BLOCK_SIZE [KByte]"
FROM v$parameter
WHERE name = 'db_block_size';

And of course, my results one more time:

DB_BLOCK_SIZE [KByte]
---------------------
                 8192

Now using their formula, I get (I fixed the UNDO_SIZE number for this calculation):

 17825792000 / ( 651.855 * 8192 ) = 3338.17

Here's the script to do all of this at once:

SELECT d.undo_size/(1024*1024) "ACTUAL UNDO SIZE [MByte]",
       SUBSTR(e.value,1,25) "UNDO RETENTION [Sec]",
       ROUND((d.undo_size / (to_number(f.value) *
       g.undo_block_per_sec))) "OPTIMAL UNDO RETENTION [Sec]"
FROM ( SELECT SUM(a.bytes) undo_size
       FROM v$datafile a,
               v$tablespace b,
               dba_tablespaces c
       WHERE c.contents = 'UNDO'
         AND c.status = 'ONLINE'
         AND b.name = c.tablespace_name
         AND a.ts# = b.ts#
     ) d,
       v$parameter e,
       v$parameter f,
     ( SELECT MAX(undoblks/((end_time-begin_time)*3600*24))
              undo_block_per_sec
       FROM v$undostat
     ) g
WHERE e.name = 'undo_retention'
  AND f.name = 'db_block_size'
/

And the output from this:

ACTUAL UNDO SIZE [MByte]
------------------------
                   17000

     UNDO RETENTION [Sec]     
-------------------------
                    18000                    

OPTIMAL UNDO RETENTION [Sec]
----------------------------
                        3338

Now, after all that, I still am nowhere. If I set the UNDO_RETENTION to 3338, we will get ORA-01555 errors. If I leave it at 18000, eventually we will get ORA-30036 as the UNDO tablespace fills up.

I would like to know how I can associate the current AND active data in V$UNDOSTAT with the current ACTIVE and UNEXPIRED data in DBA_UNDO_EXTENTS. How can I tell what is active and what is expired in V$UNDOSTAT? Or am I looking in the wrong place?

I'm giving this to Tom...

CreateROUserLike.sql - Create read only user like

With this PL/SQL script I created, you can now create a read only version of a database user. This script will create the user with the extension "_RO" on the read/write user name. After it creates the user, it will grant EXECUTE on FUNCTION, PROCEDURE, PACKAGE and PACKAGE BODY object types and SELECT on others. Because a DATABASE LINK can't have a synonym or special grants, it recreates these in the read only user's schema.

Project idea: Open source performance tuning software

My idea has these features:

• Use STATSPACK queries to gather and review statisitcs.


• Store results locally for quicker review.


• Realtime performance stats gathering.


• More to come...

This project would be done in .NET with possible porting to MONO later on. This would mean that it would need to be limited in what it uses in the way of Microsoft based assemblies. This should be exciting plus free! Just gotta check with Oracle about the STATSPACK queries. But in thinking, this tool would rely on STATSPACK to be installed. So using the queries form the report shouldn't be a problem. Plus, being OSS there shouldn't be a licensing issue.

Next question, what should I use for a local data store? DB4O is an OSS non-relational database used by numerous companies, but will it work for this application? Are there other options similar to Access or DB4O?