SIRCA Technology
Latest Initiatives
Case Study - ASX/SIRCA data cleaning
As you may or may not know, we work very closely with the ASX to
provide you with quality SEATS, CLICK and CHESS data. Late last
year, the SIRCA Team at Australian Technology Park, Eveleigh, implemented
a Data Cleansing System (DCS) that validates ASX data as it is received
from the ASX. The DCS validates for completeness, integrity, and
also validates using a variety of hashing algorithms to ensure the
new data integrates with existing data held by SIRCA on its databases.
Outliers are manually scrutinized and missing data re-requested
from the ASX.
As a mark of the success of this project, the ASX now uses SIRCA’s
data as a backup to its existing databases. Another great example
of SIRCA proactively identifying and resolving data issues. Our
intention, this year, is to further enhance our relationship with
the ASX to provide SIRCA members with quality data whilst providing
valuable assistance to our data providers.
What is RASP?
Everyone knows about it but what does it all mean? RASP stands
for Research Application Service Provider. Essentially the RASP
is a web site that will allow member university researchers to access
a wide variety of financial markets data on-line. The site will
provide researchers with access to intra-day data supplied by Reuters
covering over 240 markets.
Reuters Database
The “Application Service Provider” element means that
software applications can be built over the top of the RASP data
set. These applications will also be made available to researchers.
Researchers will be able to build research tools, metrics, statistics
and other functions over the database. Also, data miners will be
able to implement data mining applications over the data.
Progress on the Systematic Infrastructure Grant
The infrastructure at SIRCA is currently being upgraded to take
advantage of newer, faster, more efficient technologies. We are
implementing a brand new processing infrastructure that clusters
together multiple processing servers that will outperform our current
infrastructure. We have also implemented a Storage Area Network
that improves SIRCA’s data retrieval response times and allows
us to expand our processing capability with the minimum of fuss
and at a lower cost to members. All our data is also backed up to
our archive. We are also looking at GRID computing which will allow
us to join other high performance computing facilities in a grid
similar to an electricity grid. All you have to do is get on the
grid and buy your processing power. This is all very new so watch
out for big developments in this area.
Cluster Computing
You may or may not have heard about clustered computing so here
we go. Clustered computing allows you to join multiple servers together
so they act as one. At the front of the cluster server is a load
balancer that manages the jobs as they are submitted by researchers.
The load balancer finds the most available server in the cluster
and submits the job to that server without interrupting any other
researcher’s job. It also allows us to dedicate servers to
extract data from the huge databases we have available. This will
provide faster data request turnaround times and also will be the
heart of the RASP. When we finish this upgrade, we will have 9 servers,
all attached to SIRCA’s databases.
SII Grant, short term disruption, long-term benefits
Earlier this year, SIRCA took a huge step forward for the benefit
of its members by undertaking a migration of all data and programs
to the new cluster described above. It is a monumental task to which
SIRCA Technology has been assigned to complete in a very short space
of time. We are very pleased with progress and have put all our
resources into the project to minimise disruption to your requests.
Considerably improved processing speed will allow our team to provide
your data requirements sooner in the very near future.
|
