Tag Archives: Standard Edition

#Oracle cutting in inspiration and new business?


Over the many years Oracle has been leading the database world, I guess they are now taking something of a wrong turn.
Let me briefly fill you in on my thoughts.

Basically I see two “minor” shifts that are significantly indicative of this:

  1. Oracle Standard Edition 2
  2. Oracle ACE Program

Okay, so you might think I am crazy, but let me try to explain.

Oracle Standard Edition 2

Sometime last year, the long expected, anticipated…, dreaded perhaps even, change to the Oracle database licensing strategy was there.

Oracle Standard Edition (SE) and Oracle Standard Edition One (SE1) licenses were addressed.
There was A LOT of debate on this, I mean, A LOT. Discussions which ran all the way back to HQ, and were driven by passionate people inside and outside of Oracle, inside and outside of the Oracle community… To no avail.

It had been very clear for quite a long time that the SE / SE1 strategy was nothing short of unsustainable inside the Oracle licensing realm. Even though, Oracle SE and SE1 enabled many projects and customers to adopt the phenomenal Oracle technology for their projects. It has some limitations, but with smart thinking and smart planning, a lot of projects could be run with Oracle SE(1). “I am such a good DBA, I can even do it with Oracle Standard Edition!”
Alas, we now have Oracle Standard Edition 2 (SE2) with a new and upgraded price of US 17k (!!) making this solution rather out of the question for many of the projects meant in the above. Please note that SE1 already was a significant investment for some of the projects I have learned to know over the years in regions as the Baltics and Africa.
Yes, of course, I know you can do all of this “In the cloud”. But with the limitation that there are hardly any CSPs (Cloud Service Providers 😉 that enable you to make use of the “cheaper” Oracle license. If you want to leverage your local cloud vendor (mind my word-choice here) it’s BYOL (Bring Your Own License) and, voila, you’re done in for anyway.

Hence, the first significant “shift” in Oracle’s span of attention for new business, creativity and growth…

Oracle ACE Program

More recently there was also a change in the Oracle ACE Program. Which has also led to much debate. But… that bit of the change I am not referring too, I am referring to the bit that does not affect me directly…

Oracle has a small number of very highly appreciated and “industry leading” community advocates called “Oracle ACE Directors”. These people not only have a deep knowledge of everything that is happening in this corner “of the industry”, but are also very passionate about sharing this knowledge. Sharing with Oracle Users, sharing with stakeholders within the Oracle organization, basically, with everyone with a hunger for knowledge around the technology.

For this, these Directors had a few privileges. When the invested their time and their energy in traveling this globe to share, Oracle would support them in some of their travel expenses. This always had the air of “wow, they are paid”. Believe me, it was bare minimal support, just a flying ticket and a hotel-bed to a previously approved conference, when they actually were accepted to do a talk. Nothing shiny, nothing business-classy…

Until now. With the changes to the system, also these modest privileges for the Directors have seized to be.

There was my second significant “shift” in Oracle’s span of attention for new business, creativity and growth…

It has me worried… I should not worry, as it does not affect my day-to-day business… yet.

Albeit we have this cool tech, with PL/SQL, with APEX, with all the features, options and what not, to create solutions that could really better the word (I also firmly believe this).

Oracle is just closing this door, and my toes were still in the doorway, so that hurts.

This was my rant, hope it helps.


#DOAG2016, definitely a crazy week.

#DOAG2016, the largest Oracle Community gathering in Europe. Taking place in Nuremberg, at the Nuremberg Convention Center NCC, one of the more impressive places to hold such a conference, towering 4 stories high, with a big central atrium!!
It is a huge effort to get all of this together!

In this blog-post I want to highlight some of the crazy things I experienced this week… And… I did try to follow my own schedule, but I wasn’t overly successful.

Young talent

One of the things that was somehow quite clear this week, is that we have a lot of young talent out there, eager to learn and share experiences. It is not just the #NextGen “movement” of DOAG, of which Carolin Hagemann made me aware, but just young people on the conference itself.

Discussing “Young PL/SQL” at the unconference session made us all aware that our part of the IT trade is no very sexy and popular with the youngsters. This all despite what was mentioned above. In universities we train SQL, but we don’t train to create real-life business applications, leveraging the power of the one language that keeps SQL close to the data it feasts on, PL/SQL. But, more on that below (Thick Database Paradigm).
To promote PL/SQL, basically two ground requirements were defined:

  1. Create a free ‘PDB as a Service’ for schools;
  2. Inspire teachers to talk about data centric computing

By finding somebody to be regionally or globally owner of this quest, it should be possible to get young professionals as familiar using PL/SQL for creating performant and business-ready applications as they were familiar using Microsoft Excel to do their accounting “back in the days”

ACE program

“There is a disturbance in the force!”

For everybody not acquainted with the Oracle ACE Program by the Oracle Technology Network… You should be!! Please read up, as it is an incredible cool initiative.

The disturbance, you ask?
Well, to retain your “status”, Oracle expects you to do “stuff” and this “stuff” is then evaluated on a yearly basis. Basically the initiative, the disturbance, is to get some transparency in “the stuff”. And, as always, everybody wants change, but few actually are good at “change”. There are rimples and things that change, but in the end; everything will be fine, unless, obviously, when it will not be fine.

Talks

I was honored to (co)host to talks at #DOAG2016:

Bad Boys of Replication – Changing everything…
With Oracle ACED and good friend Björn Rost, about an intense migration project we did some time ago. We were even offered to host our talk in Tokio, the biggest hall at DOAG!

Saving lives at sea at an industrial scale using Oracle Cloud Technology
An insightful (at least I like to think so) talk with my colleague Oliver Limberg. The talk is about the rapid development of a global portal for the maritime logistics branch.

I had a blast, and I hope you did too!

Community spirit

Oracle User Group conferences are about sharing and are about fun. Mr. Martin Widlake wrote a good post about that.

Apart from all the “more formal” things that happened, there were quite a few extracurricular activities, mostly involving an Irish Pub or a restaurant.

This all may sound quite funny and exciting, and, yes, it is alto talk with your co-workers: “Oh, hey, you are going to have fun and party all week!” Of course it is not a drag and a bore, but it has very profound function!
Whenever you run into trouble, these are the exact same people that are not only able, but probably also inclined to help you out, as you would help them out, as friends do among each other. In the end, they, you, your boss and your clients benefit. This is not to be underestimated too much.

The extra, special bit, that DOAG offers are the so called “unconference sessions”.
Not scheduled, no slides, nothing official, just getting together and discussing subjects of interest. Our “Young PL/SQL” was one of these “unconference session”, which turned out to be a great (and valuable) success!

Meeting people

Just to name a few, heroes of long and of yet to come for #DOAG2016:

Dietmar Neugebauer
Frank Dernoncourt
Joel Kallman
Johannes Ahrends
Kamil Stawiarski
Laurent Leturgez
Maja Veselica
Marcel Hofstetter
Piet de Visser
Sabine Heimsath
Stefan Kinnen
Stew Ashton
Uwe Hesse
Zoran Pavlovic
And alle the ones I forget to mention here!!

Thick Database Paradigm

Noting new in IT…

Well, no.

The Thick Database Paradigm (opposed to the “No PL/SQL Paradigm”) is nothing new. We have actually all been doing this since the eighties. Program your business rules, your constraints, everything that makes sure that your data is all that you want it to be, close to that data.
There are so many reasons that speak in favor of this approach that it is nearly overwhelming and deserves at least a book in itself. But, let me make a small attempt to highlighting a few here:

  • spare yourself network bandwidth, by not sending data all over your network to be processed
  • safeguard your data inside the (Oracle) database, so it can be protected by all that has been invented to do so
  • Transact data where it lives and combine and aggregate it there, you will be amazed by the efficiency
  • Remind yourself why you used to think “business logic in middle teer” was a good idea

If you leave possibile religious believes aside, there is no other conclusion possible then that the reinvention of “Thick Database” is the (re)discovery of 2016, right from the time when IT still made sense.

Yes, there are cases where an “Enterprise Service Bus” makes sense, but, as with every technology withing IT, it has a very specific area where it actually adds value or even makes sense. At best, a lot less than all the places where it is used currently!
Not to get carried away in this joyful blog-post, I will leave this topic at this.

The end

I hope to see you at the next Oracle User Group conference, somewhere… Please watch for the asterisk at his page for the conferences that I will attend.

SYSAUX LOB segment for auditing bug not released in Standard Edition

Last week we were struck by an issue, which turned out to be a bite from a bug!
SYSAUX table-space had quickly filled up to the “my data-file is full”-limit, which in the end was fixed by adding a data-file.

Strange thing though, that for a very small footprint database, we now have a very big SYSAUX table-space.

Some investigation brought me to the Unified Auditing being standard active in database 12c (you can read up on that background with my friend Ann Sjökvist here).
We are faced though with a different (and possibly a little more obscure) Bug 20077418 – RECLAIMING THE SECUREFILE LOB SEGEMENT IN 12.1 Standard Edition.
What this bug boils down to is the following:
There is a lot of audit data recorded by default, the ORA_SECURECONFIG profile is running out of the box. I haven’t taken the time to figure exactly out what is written, where and how, but I know it involves a LOB segment (SYS_LOB0000091833C00014$ by SYSAUD) which is, in our case in comparison to the total database size, HUGHE!! The management of this audit data, usually driven by DBMS_AUDIT_MGMT, has absolutely no effect on this segment (at least not on shrinking it).

Searching for the mentioned bug you just find to EE bugs (18109788 & 22272580) but they at least they give _some_ clues… The actual bug is undisclosed and in status 11 (being worked on).
In the end it means that auditing is fine, even in SE, but, for the moment, restrain yourself… The data you gather cannot be managed (yet). And for the rest:

If
select policy_name
from audit_unified_enabled_policies
;

yields any results, consider switching this auditing off (eg.SQL> noaudit policy ORA_SECURECONFIG;)

Hope this helps…

DBA_FEATURE_USAGE_STATISTICS and SE2

This blog post is inspired on work I have been doing on Standard Edition databases and the returning confusion about what is and what is not part of Standard Edition.

DBA_FEATURE_USAGE_STATISTICS is a tool in determining license usage for the Oracle database. It is good to understand the implications of each entry, know what is happening in your database and thus be able to have a substantial conversation about the usage of your license, being SE, SEO, SE2 or EE!

This list is the full list of DBA_FEATURE_USAGE_STATISTICS and I have found no source where there is a mapping of these features to database editions. As it is a lot of tedious work I call upon the community to help complete the list and make it as accurate as can be. So, if you have news, improvements, other bits of information, please send it to me and I will make sure it gets added!

WARNING: Still… with all the work that goes into these answers, it is not the law, it is a very serious interpretation of facts which will pay a part in helping you make the right decision when it comes to database licensing.

Feature Standard Edition
Active Data Guard – Real-Time Query on Physical Standby NO !
ADDM NO !
Advanced Replication NO !
Application Express YES
ASO native encryption and checksumming NO – EE option !
Audit Options NO !
Automatic Maintenance – Optimizer Statistics Gathering YES
Automatic Maintenance – Space Advisor YES
Automatic Maintenance – SQL Tuning Advisor NO !
Automatic Memory Tuning
Automatic Segment Space Management (system) YES
Automatic Segment Space Management (user)
Automatic SGA Tuning YES
Automatic SQL Execution Memory YES
Automatic SQL Tuning Advisor NO !
Automatic Storage Management
Automatic Undo Management
Automatic Workload Repository
AWR Baseline NO !
AWR Baseline Template NO !
AWR Report NO !
Backup BASIC Compression
Backup BZIP2 Compression
Backup Encryption
Backup HIGH Compression
Backup LOW Compression
Backup MEDIUM Compression
Backup Rollforward
Backup ZLIB Compression
Baseline Adaptive Thresholds
Baseline Static Computations
Bigfile Tablespace
Block Media Recovery NO !
Change Data Capture NO !
Change-Aware Incremental Backup
Character Semantics
Character Set
Client Identifier
Clusterwide Global Transactions
Compression Advisor
Crossedition Triggers
CSSCAN
Data Guard NO !
Data Mining NO – EE option !
Data Recovery Advisor
Database Migration Assistant for Unicode
Database Replay: Workload Capture NO ! 1
Database Replay: Workload Replay NO ! 1
DBMS_STATS Incremental Maintenance
Deferred Open Read Only
Deferred Segment Creation NO !
Direct NFS
Dynamic SGA
Editioning Views
Editions
EM Database Control
EM Grid Control
EM Performance Page
Encrypted Tablespaces
Exadata
Extensibility
File Mapping
Flashback Data Archive NO ! 2
Flashback Database NO !
GoldenGate NO – EE option ! 3
HeapCompression
Hybrid Columnar Compression NO !
Instance Caging NO !
Internode Parallel Execution
Job Scheduler
Label Security NO – EE option !
LOB
Locally Managed Tablespaces (system) YES
Locally Managed Tablespaces (user)
Locator YES
Logfile Multiplexing
Long-term Archival Backup
Materialized Views (User) NO !
Messaging Gateway NO !
MTTR Advisor
Multi Section Backup
Multiple Block Sizes
Object
OLAP – Analytic Workspaces NO – EE option !
OLAP – Cubes NO – EE option !
Oracle Database Vault NO – EE option !
Oracle Java Virtual Machine (system) YES
Oracle Java Virtual Machine (user)
Oracle Managed Files
Oracle Multimedia
Oracle Multimedia DICOM
Oracle Secure Backup
Oracle Text
Oracle Utility Datapump (Export)
Oracle Utility Datapump (Import)
Oracle Utility External Table
Oracle Utility Metadata API
Oracle Utility SQL Loader (Direct Path Load)
Parallel SQL DDL Execution NO !
Parallel SQL DML Execution NO !
Parallel SQL Query Execution NO !
Partitioning (system) YES
Partitioning (user) NO – EE option !
PL/SQL Native Compilation
Quality of Service Management NO !
Read Only Tablespace
Real Application Clusters (RAC) YES 4
Real-Time SQL Monitoring
Recovery Area
Recovery Manager (RMAN) YES
Resource Manager NO !
Restore Point
Result Cache NO !
RMAN – Disk Backup
RMAN – Tape Backup
Rules Manager
SecureFile Compression (system) YES
SecureFile Compression (user)
SecureFile Deduplication (system) YES
SecureFile Deduplication (user)
SecureFile Encryption (system) YES
SecureFile Encryption (user)
SecureFiles (system) YES
SecureFiles (user)
Segment Advisor (user)
Segment Shrink
Semantics/RDF NO !
Server Flash Cache
Server Parameter File
Services
Shared Server
Spatial NO – EE option !
SQL Access Advisor
SQL Monitoring and Tuning pages NO – EE option !
SQL Performance Analyzer NO !
SQL Plan Management NO !
SQL Profile
SQL Repair Advisor
SQL Tuning Advisor
SQL Tuning Set (system) YES
SQL Tuning Set (user)
SQL Workload Manager
Streams (system) YES 5
Streams (user)
Transparent Data Encryption
Transparent Gateway YES – option
Transportable Tablespace NO ! 6
Tune MView
Undo Advisor
Very Large Memory
Virtual Private Database (VPD) NO !  7
Workspace Manager
  1. Unless used for upgrade to Enterprise Edition.
  2. Unless used without history table optimization.
  3. Goldengate can also be used with Standard Edition, it is a separate product.
  4. RAC on Enterprise Edition is an option.
  5. No capture from redo.
  6. Import transportable tablespaces in all editions.
  7. Policies on XDB$ACL$xd_sp in sys.v_$vpd_policy are internal ( “out of the box”) policies that are used by XDB to control the access to certain internal tables. All the logic is implemented in the xdb.DBMS_XDBZ package and there is no way one can control / influence the way this is working.

Introducing FETCHER in a running replication process

This is no regular bit of work and it will probably (and hopefully) never hit you in a production setup…

The prerequisite is that you know how on-line data replication in general, and Dbvisit Replicate specifically, work.

The following case is true:
I had half of a replication pair running.
It means that the MINE process was running, converting REDO-log in PLOG-format. The APPLY process had not yet started because the target database was still being prepared.

dbvisit-replicate-logical-replication-made-easy-18-638-300x225The reason for this is that we needed to start converting redo-log information to PLOG information while we were setting up the target environment. The reason for that was that the setup (exporting source, copying dump to target and importing) was taking quite a bit of time, which would impact redo-log storage to heavily in this specific situation.

It was my suspicion that the MINE process was unable to get enough CPU-cycles from the production server to actually MINE more redo-log seconds than wall-clock seconds passed. In effect, for every second of redo-log information that was mined, between 1 and 6 seconds passed.

This means that the replication is lagging behind and will never be able to catch up.

To resolve this, the plan was to take the MINE process of the production server and placed on an extra server. On the production server, a process called FETCHER would be introduced. The task of this process is to act as a broker between the database and the MIN process, forwarding the requested on-line an archived redo log files.

Normally (!) you would use the nifty opportunities that Replicate offers with the setup wizard and just create a new setup. And actually, this is what I used to figure out this setup. And, if you can, please do use this…

Why didn’t I then, you would rightfully ask?

Well… The instantiation process would take to long, and did I say we were under time-pressure?

  • Setup wizard, 5 minutes
  • The famous *-all.sh script, ~ 1 hr.
  • Datapump Export, ~ 10 hrs.
  • Copy from DC old to DC new,  ~ 36 hrs.
  • Datapump Import, ~ 10 hrs.

So, totally we could spend 57:05 hrs. to try to fix this on the go…

Okay, here we go:

Note: cst-migration is the name of the replication project as you specified it in setup wizard when setting up Replication.

TIP: When setting up on-line replication, it is worth your effort to create separate tnsnames.ora entries for your project, like ‘repl-source’ and ‘repl-target’ acros all nodes.
It can get hellishly confusing if you have, as in this case, a database that is called <cst> and is called the same on the source and target server!

1. Step one:
We obviously had the ./cst-migration/config directory from our basic setup with just MINE & APPLY. This directory holds (among others) the ./cst-migration/config/cst-migration-ontime.ddc file. This file holds the Dbvisit Replicate Repository contents that is needed to run the processes.

From this setup, MINE is actually running. We actually concluded the fact that we were not catching up from this process.

2. Step two:
Now we run dbvrep -> setup wizard again and create a Replicate setup directory with FETCHER and isolate the ./cst-migration+fetcher/config/cst-migration+fetcher-onetime.ddc.

By comparing the two files, I was able to note the differences and therewith conclude the changes necessary to introduce a FETCHER process. It is a meticulous job to make sure all the paths on all the three servers are correct, that port numbers are correct and that all the individual steps are take in the right order. This is the overview.

Having these changes, it is all downhill from now.

3. Step three:
Using the Dbvisit Replicate console, the new entries and the changes were made to the DDC-information stored in the Replicate repository. You can enter these manually or execute your change-file by executing @<change-file-name> inside the console.

4. Step four:
Create the ./cst-migration directory on the system you will use for the relocated MINE process and copy the cst-migration-MINE.ddc and cst-migration-run-source-node.sh in this directory.
Rename the cst-migration-run-source-node.sh to cst-migration-run-mine-node.sh to reduce confusion.
Make sure that the paths mentioned in the cst-migration-MINE.ddc are correct for the system you are starting it on!

NOTE: Please make sure that you can reach both the source and the target database from this node using the tnsnames-entries you have created for the replication setup.

5. Step five:
Rename the cst-migration-MINE.ddc on the source node (!) to cst-migration-FETCHER.ddc and change the cst-migration-run-source-node.sh file to start the FETCHER process in stead of MINE process.

You are now ready to start your new replication processes!

NOTE: If you are running APPLY already, there are some additional things you need to be aware of.

Although it was not the case when I came across this challenge, I am happy to say that Dbvisit have verified and accepted this solutions as a supported action.

Hope this helps.

My picks, no, Agenda… for UKOUG_Tech15

I went over the agenda for UKOUG_Tech15 and took my picks & suggestions.
Then I thought, why not share these…

MONDAY

The Oracle Database In-Memory Option: Challenges & Possibilities
Christian Antognini – Trivadis AG

Standard Edition Something for the Enterprise or the Cloud?
Ann Sjökvist – SE – JUST LOVE IT

All about Table Locks: DML, DDL, Foreign Key, Online Operations,…
Franck Pachot – DBi Services

Silent but Deadly : SE Deserves Your Attention
Philippe Fierens – FCP
Co-presenter(s): Jan Karremans – JK-Consult (Having a link here would be silly, right)

Oracle SE – RAC, HA and Standby are Still Available. Even Cloud!
Chris Lawless – Dbvisit

SE DBA’s Life a Bed of Roses?
Ann Sjökvist – SE – JUST LOVE IT

Oracle Standard Edition Round Table
Joel Goodman – Oracle
Co-presenter(s): Ann Sjokvist, Philippe Fierens, Jan Karremans

TUESDAY

Watch out for #RepAttack… all day long!!
And earn your RepAttack badge-ribbon…

Advanced ASH Analytics: ASHmasters
Kyle Hailey – Delphix

Community Keynote – Dominic Giles

Oracle BI Cloud Service – Moving Your Complete BI Platform to the Cloud
Mark Rittman – Rittman Mead

Infiniband for Engineerd Systems
Klaas-Jan Jongsma – VX Company

Oracle Database In-Memory Option – Under the Hood
Maria Colgan – Oracle

Do an Oracle Data Guard Switchover without Your Applications Even Knowing
Marc Fielding – Pythian

Using Oracle NoSQL to Prioritise High Value Customers
James Anthony – RedStack tech

WEDNESDAY

HA for Single Instance Databases without Breaking the Bank
Niall Litchfield – Markit

Database Password Security
Pete Finnigan – PeteFinnigan.com

Connecting Oracle & Hadoop
Tanel Poder – PoderC LLC

Enterprise Use Cases for Internet of Things
Lonneke Dikmans – eProseed
Co-presenter(s): Luc Bors – eProseed

Bad Boys of On-line Replication – Changing Everything
Bjoern Rost – portrix Systems GmbH
Co-presenter(s): Jan Karremans – JK-Consult

RMAN 12c Live : It’s All About Recovery,Recovery,Recovery
René Antúnez – Pythian

Hopefully it will attend you to some interesting session for you!

Kscope15, a celebration of tech…

Kscope15 promised to be a brand new experience in more than one way.

Kscope15LogoAs I start to write this report, I am flying from Düsseldorf airport to Atlanta. It will be the first time flying to the United States with a stopover, and because of Erik van Roon, I came prepared. With just carry-on luggage, I should end up at my final destination, Fort Lauderdale, Florida, together with my ‘stuff’. I am flying Delta Airlines this time, and for an airline that promises just a ‘lunch’ during the 9 hours flight, they do come up with a lot of food…

My colleagues of FOEX have already arrived at the event-venue and are setting up our booth.

On arrival at Fort Lauderdale airport, I am scheduled to meet-up with distinguished product manager for PL/SQL and EBR, Bryn Llewellyn. From there, we would travel to Hallandale Beach to check into our hotels. This plan was only hindered by sheer force of wind shear at Atlanta International, which delayed my flight.

The first day, the Sunday, started off with a boiling walk to the Diplomat hotel. Upon registration I was pleasantly surprised that FOEX had graciously upgraded my conference pass to a full pass, which is cool as I get to attend sessions! And the kind ladies of ODTUG had even attached an ACE Associate ribbon to my name-tag, of which I am kind of proud.

I had so many cool meet-ups and run-ins at Kscope. Just to name a few new friends in no particular order:

Of course I spent most of my time in the APEX and database development tracks. If you look at the momentum that APEX is generating, I think we can safely say that we are making a difference… We can say with confidence: #LetsWreckThisTogether!

The “together” bit was beautifully expressed by Joel Kallman as you could hear a pin drop when Carl Backstrom and Scott Spadafore of the APEX team were remembered…

But still there is a lot of work that has to be done to further spread the word on APEX. I guess I have had at least 4 conversations where I had the opportunity to talk about and explain APEX to people who were still oblivious. That is one of the most rewarding this to do.

Nikki beachThe week passed so quickly and most experiences are becoming great memories very quickly now. The countless meet ups with friend and heroes from the Oracle world, the white party at Nikki Beach and the after party at The Mansion and of course the Oracle content which was dished out with great quality.

Just on more thing… Travelling Über is the best! I have been doing this in San Francisco and used the service here to get back to the airport. Why would you take a taxi with this service around? Because of the way it works, the drivers I have met, have been much more friendly than regular ‘cabbies’. I would recommend this any day.

So, now I am heading home, hanging in the sky somewhere between Fort Lauderdale and Atlanta. Thinking back on Roel’s blog post on his first Kscope… will this have changed my life? Quite possibly, but on the other hand things could not get much more crazy than they have been over the last 6 to 12 months!!

If you are looking to read up on the business side of things, please check out the FOEX blog!

Please also don’t forget to check out the #Kscope15 hash tag on Twitter and remember, when you are at an Oracle conference, also use the #orclconf as additional hash tag. This will help to make it even easier to follow your favorite tech-community on-line!

Register redo-log manually with Divisit Replicate

For those of you who haven’t been working with on-line data replication; in short, it is a way to copy data from a source database to a target database and do this on-line (both databases are active) and do it near-real-time.
This means that when you enter data in you source database, you can immediately query it from your target database. This makes on-line data replication ideal for numerous tasks, like moving and / or upgrading your database while it is being used, with almost no downtime at all.

This tale is of an actual project that I conducted. I used Dbvisit Replicate as my tool of choice.

dbvisit-replicate-logical-replication-made-easy-18-638Dbvisit Replicate can use a so-called FETCHER process to act as the “long-arm” for the MINE process. Mining extracts the information from the redo-log files, but, in specific situations, this can be too much of an overhead for the source database server. By moving the MINE to a proxy server, this overhead can be significantly reduced.

In some cases it can be useful to manually transfer redo-log files to the mining stage directory of Dbvisit.
I came across this requirement when catching up a lot of redo from a RAC database. In this case, the RAC cluster creates two streams of redo. When starting the replication processes, the first thread is transferred by FETCHER from the source server to the proxy, before the second thread is transferred. This means mining will pause until the second thread successful delivers the first redo-log file of the second thread. The redo-log information from the second stream is necessary to create consistent and chronologically ordered SQL-statements for the target database. In effect, the SCN’s from first redo-log information of the first stream need to line up with the SCN’s of the second redo-log information.

In this case, this meant having to wait a day or more before mining can start. This is why I decided to copy a number of redo-log files from the source server to the proxy server, where the MINE process is running, manually.
After the copy, the files need to be registered with in the dbvrep-repository. Without this information, the MINE process has no knowledge of the files that are present and about what their contents are.

The update is an easy insert statement, but it should be handled with care, as this needs to be quite precise and it needs a bit of specific information about the redo-log files being added.
You can use the following insert statement to register the files:

insert into dbvrp.dbrsmine_redo_log_history
       (
       ddc_id
     , mine_process_name
     , sequence
     , thread
     , resetlogs_id
     , first_scn
     , next_scn
     , online_name
     , arch_name
     , read_count
     , from_fetcher
     , last_mine_start
     , last_mine_end
     , create_date
     , last_change_date
       )
values
       (
       1
     , ‘MINE’
     , 128779 -- sequence number of the copied file;
     , 2 -- assuming you are updating this thread.
     , 804864915 -- the reset-logs id from the redo-log file
     , 199910296688 -- the first scn from the redo-log file
     , 199911476897 -- the next scn from the redo-log file
     , null
     , ‘/u01/app/oracle/some-big-storage/dbvrep-mine/mine-stage/thread_2_seq_128719.1485.804864915’
       -- full path and name of the file
     , 0
     , ‘Y'
     , null
     , null
     , sysdate
     , sysdate
       )
;

And you can get the information you need about the files here:

select lh.sequence#
     , di.resetlogs_id
     , lh.first_change#
     , lh.next_change#
  from v$log_history lh
 inner join v$database_incarnation di
 using (resetlogs_change#)
 where sequence# = 128779
;

After registering the first file for the second thread, in the Replicate-console, you can watch the MINE process kick off. This process will then again halt after the first file of the second stream is processed in parallel with the first file of the first stream.

Schermafbeelding 2015-05-31 om 21.23.11

I kept adding files until the FETCHER process was able to take over, or you could do this until you test-case or PoC is over.

OUGN15, The “boat conference” revisited

Jan at shipsport
Reflections on OUGN

Sometimes things in life can change quickly! It is only two years ago that I came to Oslo for the first time to join the Scandinavian Oracle crew on a boat trip to Kiel.
At that time I had never actually participated in this kind of experience and I wasn’t into presenting either. Together with my good friend Philippe Fierens I discovered a whole new world back then. You could have read about these experiences in some blogpost, but this was lost in the move to my own site, sorry!

And this trip couldn’t have been more different though! With three presentations accepted the two days at sea will be a reunion with the friends I made over the last years, as well as a way to contribute to one of the most tight knit tech communities I know. And this will be in a scene that I remember vividly from being a newbie… And this is somewhat strange, believe me

After a quick and pleasant flight I touched down in Oslo, flying from Amsterdam with a decent sized crew of Dutch Oracle enthusiasts, including my good friends Patrick Barel and Alex Nuijten. Waiting in the Oslo airport for Luís Marques, I catches up with Gurcan Orhan, which was a great surprise.
Later that day we found ourselves in the Oslo harbor for the speakers dinner. You can imagine the collective amount of Oracle knowledge packed into that one restaurant!

frits
Enkitek’s Frits Hoogland on Ansible

After a somewhat restless night we arrived, on Thursday morning, at the ship Color Fantasy with the Heli Helskyaho-company, just in time for the keynotes. It was good to see Mark Rittman and James Morle made it on board too. Especially as James was up for the delivery of version 2.0 of his vibrant keynote! Next we proceeded to bring our luggage to our cabins and grab a spot of lunch on the exhibition floor down in the belly of the ship. The setup of the exhibition was quite nice and gave a good opportunity to mix and mingle.
The afternoon was spent on sessions, where I visited Frits Hoogland with the Ansible talk, and preparation for my own session at 18:00. This is the last run of this APEX presentation, as I have retired it after OUGN15. The slides will be archived here.
After finalizing the preparation for the third edition of the Standard Edition Round Table (aka “slide polishing”) with the #orclSERT team, comprising of Ann Sjökvist, Philippe and myself, it was time for the souree and for diner in the grand restaurant on board. It has been a good first day!

Diner
Dinner with the international crew on board the Color Fantasy.
Gin-tonic
Warm reception at Kiel port.

The second day of OUGN15 started with a multitude of sessions including the third edition of the Oracle Standard Edition Round Table, which was actually quite busy and interactive. We had some good discussions, and that at 09:00, so thank you, everybody.
Of course, as was declared a tradition, Björn Rost was present in the Kiel harbor. With the famous “Basil smash Gin & tonic” and sandwiches we were welcomed on German soil.
My afternoon comprised 3 sessions, starting with my own called “Okay, and now my database server crashed…” which was quite nicely received. Next Alex Nuijten on 12c new features for developer, topped off with Tim Gorman who taught us to be CSI people, in finding issues in the database.
After an enjoyable evening in the various bars and discotheques of the ship we retired the official part of the Oracle User Group Norway Vårseminar 2015, thanking the board and of course especial Øyvind Isene, for their hard work.

If you want to catch up further on the unconference communications surrounding this event, please do checkout the Twitter hashtag #OUGN15. This will also include a great set of snapshots and pictures taken along the way…

Oslo, until the next time!

A new form of on-line data protection

In the last few years I have been active with data replication solutions in the Oracle realm as you may know. This data replication field is one that has many angels, so there is something new to learn every day and sometimes there even are really new possibilities!

Take heed…

The first and most familiar form of the data replication forms is ‘physical data replication’, also known as ‘Standby Database‘.
In this form of replication, both source and target database are binary identical. Changes are propagated by copying the archived redo logfile from the source database to the environment for the standby database lives. Most often this is another server, preferably in another building in another town, far enough away to not be struck by the same havoc.

There are basically 3 ways to accomplish this;

  1. Use Oracle Data Guard (in Enterprise Edition Oracle database)
  2. Use Dbvisit Standby (in all Oracle database Editions)
  3. Write your own scripting (not recommended in any case)

The second and more emerging form of data replication is ‘Logical Data Replication’.
In this form of replication, there is not real relationship between the source and the target database, other than that the target database houses data coming from the source database. They can live on different systems, be from different database version, a different operating system or even be from a different vendor.
Data is harvested from the source database, converted and copied over to the target database / system. On the target system this data is being applied, in the native speech of the the target database.

There are a few ways to accomplish this, but basically every vendor has the same technique. It is more a matter of pricing, basically.

  1. Oracle Golden Gate (expensive, complex)
  2. Dell Shareplex (somewhat expensive)
  3. IBM Infosphere (ComPlex, expensive)
  4. Dbvisit Replicate (easy, affordable)

So, having discussed this, as this is not new, why this blogpost?

Well…

A Standby database is more or less closed. You can open it occasionally to query some data, but that interrupts the apply-process.
On-line data replication does what it says, you have an active database, where data is continuously added. This way you can, for example query, the same data on two sources to spread load.

The case I mean to discuss is the following:

“I have 10 source database and I want one target database (ah, presto, on-line data replication) and I want to backup 5 tables from each source to the target database (again, on-line data replication, but wait, backup?) so I can easily copy back specific data to the source (eeeuhm, yes…) whenever a user messes up the source tables (aï…) and I want the target to be update each day at 23:00 (so… okay!)

This reeks after somewhat of a hybrid approach!

We cannot do regular on-line data replication, for this is aimed at being real-time.
And we cannot leverage Standby database, since it needs to be centralized in one database and not 10. Next to that it would take some administration to open up the standby database in read-only mode, take the copy, and close the database again.

Working with Dbvisit, we came up with “Pause Apply” and “Resume Apply”, which we combine to form “Delayed Apply“.
This delayed apply would neatly answer the question posed.

  • By “delaying” the application of changes to the data, we could make sure the requested tables are only updated from 23:00 on;
  • We can combine the 50 tables (10 databases x 5 tables) in one single target database, since it is a logical approach to the matter;
  • We can easily restore or copy back corrupted data, since both the source and the target database remain continuously open.

Using Dbvisit Replicate, having this kind of protection for your “logical test-cases”, what this company was doing to require this solution, is really affordable.
It can help in dynamically and quickly resetting specific data-sets or test-cases while remaining much more flexible than creating scripts to reset a specific data-set or test-case! And, of course, there are many more ways to use this neat feature…